digg

Join DiggAbout

Discover the best of the web!
Learn more about Digg by taking the tour.

Storing data for the next 1000 years

tgdaily.com — A team of researchers claims to have come up with a power-efficient, scalable way to reliably store data with regular hard drives for an estimated 1400 years.

Bury
show profanity settings
expand all
  • kmgreen, on 04/23/2008, -17/+7I hear these guys are really, really smart.
    • googooly, on 04/23/2008, -10/+1and you sound like an idiot
    • kksurf, on 04/23/2008, -4/+2Anything coming out of Santa Cruz has got to be good :)
    • CyberSol, on 04/23/2008, -0/+3Do you talk in third person a lot?

      Nice work btw
      • kmgreen, on 04/23/2008, -0/+7Actually, we didn't think it would go this far. I jokingly put that up last night when a friend mentioned that it had 12 diggs... Looking back, I should have done a rick roll.
        • specialK16, on 04/24/2008, -0/+1So he is not only a scientist, but also a funny one. Lol.
  • speel, on 04/23/2008, -2/+15In 1400 years we will have electrical current flowing through our finger prints that will be capable of holding onto data.
    • Harbinger67, on 04/24/2008, -0/+10That's assuming we don't collectively pull a Roman Empire and plunge ourselves into centuries of more Dark Ages, or even worse, devolve into an Orwellian dystopia with no way to break the cycle.

      I'd say we have a 50/50 shot at either.
      • VinceNoir, on 04/24/2008, -2/+1I like the way you think. One more scenario for you... I believe that human consciousness is reaching the maximum limits of scalability to keep human civilization on Earth going. The only answer is the migration of consciousness into machines. It's coming. But at what price? All the singularity fans out there might think that nirvana will be in some sort of machine/human hybrid or virtualized space for our consciousness. But, for the humans who actually like being biological, that brings with it whole bunch of problems. Humanity would no long be human. Especially if the next step in evolution is to adapt to the polluted Earth. Machines would see the easy solution to air pollution: Simple. Stop needing clean air to breathe. In fact modify the entire atmosphere to something more conducive to machines. OK... so we lose what remaining biological organisms that there are, big deal! The human race will continue in it's new form with nary a problem. But you know what? The cockroaches will likely STILL survive.
  • wejmahtin, on 04/23/2008, -1/+58Yay. I can rest assured that my great great great great great grandchildren will get to enjoy my pr0n.
    • nakile, on 04/23/2008, -2/+13But they won't even care about it. In the future they'll use a brain-computer interface and experience it first hand.
      • megaton, on 04/23/2008, -1/+32I experience porn first hand.
        • itsgotyou, on 04/23/2008, -0/+37Until I get cramps and then I get to experience the 2nd hand.
        • 1legend, on 04/24/2008, -0/+6A digg for each of you!
        • janzendavi, on 05/05/2008, -0/+1I have a good feeling we will be seeing this in a screenshot titled "Why I Love Digg" in about 24 hours.
    • hollyminkowski, on 04/23/2008, -1/+1If Ray Kurzweil is right then in 1400 years you might still enjoy you own pr0n.
    • inverselogic, on 04/23/2008, -0/+2Brazillian fart fetish...mmm
  • ePuck, on 04/23/2008, -5/+3It uses current technology, harddrives
  • ams876, on 04/23/2008, -6/+1Woohoo, lets start with the porn!
  • megaton, on 04/23/2008, -4/+66Storing data isn't nearly as challenging as ensuring future generations will be able to RETRIEVE that data.

    1,400 years doesn't mean ***** if nobody knows how to interface with an SATA port.
    • skmice2, on 04/23/2008, -0/+8Agreed - and add proprietary binary formats on top of that as well!
      On the other hand, they may be able to reverse-engineer the "high-tech toys" we are using nowadays simply by pressing a button (or making a thought).
    • sdub, on 04/23/2008, -0/+26If we can decipher Mayan glyphs, I'm confident my heirs will be able to reverse engineer the SATA standard.
      • howea, on 04/23/2008, -6/+0Unless your heirs die out.
      • SkippyDoorknob, on 04/23/2008, -6/+2So you're saying that SATA hard drives store data as Mayan glyphs?
      • megaton, on 04/24/2008, -5/+1You're right. I imagine deciphering the inner workings of million-transistor integrated circuits is, in fact, just as easy as figuring out Mayan glyphs.
      • calenlas, on 04/24/2008, -0/+2The real problem is that a person in the distant future must know that there is data there to even begin the task of decoding it. If one looks at Mayan or Egyptian hieroglyphics, one can see order and the implication that the order represents data. Hard drives platters, CDs, magnetic tape, etc. all suffer from the same problem that there is no hint that this hunk of metal or plastic is anything more interesting than other hunks of metal or plastic. Ever looked at a naked hard drive platter? Nothing special there, looks like a blank for minting a really big coin. More likely in 1400 years the only value hard drives will have is in being melted down for the metals.

        So far only two ways have been found to preserve data for thousands of years. The first is to carve the data in stone and store it in the desert. Even then, all those hieroglyphics would have meant nothing without the Rosetta Stone. The second, and more reliable is to maintain the data as living history to be transmitted to each new generation as is the case with mythology, religion, etc. Oral histories have a wonderful way of changing with each generational transmission, but sacred texts have been maintained verbatim for thousands of years as long as they're protected and duplicated as need be.

        This brings me to my final point, which is that since the only reliable way to preserve our data for the future is to continually re-archive it, transfer to new storage formats, etc. the human race will need more and more archivists to take care of our data. In a few eons we will become a species of librarians endlessly generating and archiving and retrieving our data.

        Maybe not, but it does sound like an interesting premise for a sci-fi novel.
    • dafunkmonster, on 04/23/2008, -1/+1well, given that we aren't exterminating all of our current computer engineers, I'm pretty sure it won't be difficult to maintain accessibility to SATA drives in the future.
      • megaton, on 04/24/2008, -1/+5The Dark Ages were obviously not a part of your Earth's timeline.
        • robbh66, on 04/24/2008, -2/+1Yes. With the dark ages everyone on earth forgot how to read and write. That's why there is no recorded history prior to the renaissance. Moron.
          • megaton, on 04/24/2008, -1/+1The point is that the information regarding any digital recording standard could be destroyed during a period of unsettle.
    • CyberSol, on 04/23/2008, -5/+1The Hard drives are connected via IP so as long as ip is still around, things will be fine. Also what makes you think the system won't be upgraded as time goes by?
    • expert01, on 04/24/2008, -0/+2It's okay, the array will contain information on how to access it.

      /sarcasm
  • Nhmarine, on 04/23/2008, -8/+1What data would I want to store for 1400 years? Hmm... -thinks deeply, then pontificates to the people hanging around-
    "Hmm.. I dont know, I dont have anything of that importance," says person number one. (a decent response)
    "I would store my brother so everyone could see how miserable he makes me," says person number two, or may we classify him as idiot number one.
    "I said storing data, not people," I say. "People are data in a way," is the rebuttal of idiot number one. "Omg, shutup"
    This was the actual conversation I had about this... wow
    • GeekyGerge, on 04/23/2008, -0/+6What is conversation like?
      • ScottoGato, on 04/23/2008, -0/+2sudo apt-get conversation?
        • expert01, on 04/24/2008, -0/+1If you're using KDE, make sure you use sudo apt-get install konversation
  • hiabex, on 04/23/2008, -10/+1Wow! Data storage that will exceed the longevity of the species that created it. The answer must be '42'.
  • raydeen, on 04/23/2008, -13/+4It's a hard drive thats never gonna give you up or let you down.
  • jimthenglishman, on 04/23/2008, -2/+15i don't think anything in 1000 years will be able to read the old data we have now
  • googooly, on 04/23/2008, -1/+151000 years from now mankind will rewind back our history thro magnificent asian porn
  • Novion76, on 04/23/2008, -10/+4"640K ought to be enough for anybody. " - Bill Gates, 1981

    We'll see 50 years from now if it still holds :P Or if we're even still using hard drives
    • Providence, on 04/23/2008, -0/+7I'm sure he meant for the time...You know...like in 1981. I'm sure he didn't mean that 640k would be enough for everyone forever.
    • czeman, on 04/23/2008, -4/+3Bill Gates also said that Vista was their best OS yet.
  • xptweakerntn, on 04/23/2008, -1/+13What would this even be useful? If someone gave you one of those hard drives that kept popping up on digg that were only FORTY years old (pic of the huge thing being lifted out of a plane), would you be able to view what was on it? What connection interface did it use? What file system? Heck, I'll even go this far, whatever data was on it, what file type was it? Imagine that even if the data on these "power-efficient" hard drives lasted 100 years instead of 1000, what are the chances of finding a Sata controller that will read the ext3 (GO LINUX) formatted drive with the USConstitution.txt? Instead of working on drives that will retain data for 1400 years, bring me a drive that will actually transfer data a 3GB/s.
    • hollyminkowski, on 04/24/2008, -0/+6The idea is to retain the data long enough that it can be quickly transferred to a new, more reliable and cheaper media.
      The drives used in a backup scheme like this will be ground up in trash in a decade or so...but the data will still exist on newer media.
      There are old silent movies that were sent to the Library of Congress as photographic images on long paper rolls. For most of these the paper rolls are the only surviving copy the celluloid films having long ago disintegrated. people are transferring these rolls to DVD now and in the near future they will be again transferred to more stable media. A point will be reached when they will become for all intents eternal on some futuristic ultra-reliable storage system. The idea is too keep your data alive long enough for it to become eternal.
      • expert01, on 04/24/2008, -0/+1"Quickly" is relative - to me, 2MB/s is not very quick, especially not with petabytes of storage.

        As far as usefulness (op), I can see archive.org using this.
  • angusm, on 04/23/2008, -2/+1Let me guess - you have it laser-etched onto the metal casing of the hard drive?
  • f4nt0m4s, on 04/23/2008, -3/+2It's kind of cool knowing that all the mutants with 7 dicks, cyborgs and robots of the future will be able to look back and watch our greatest Internet achievement: porn.
    • itsgotyou, on 04/23/2008, -0/+2They'll feel right at home when they see my Urotsukidoji collection.
  • hocusposus, on 04/23/2008, -3/+2just wait for google storage
  • mCanada, on 04/23/2008, -0/+26 We still have documents from 1400 years ago and it we can probably make paper more resilient with new technology. Paper is proven to last centuries and at least one millennium. It can hold .jpg's .gif's .txt .doc which is most of the important stuff anyways. .avi's are a problem, but that's the price you pay for archival.
    • daemonx, on 04/24/2008, -0/+1and data retrieval is not an issue..mebbe some translation might be required if the present languages die out.
  • f4nt0m4s, on 04/23/2008, -1/+11I'm totally going to archive meatspin, tubgirl, and 2 girls 1 cup, so I can psyche out people from the future!
    • KMartSheriff, on 04/23/2008, -0/+12I can see it now...

      *scientist presenting a 2008 .avi file to a room full of people* "Behold! The past!..."

      *a small dancing man singing "We're no strangers to love..." is seen. The room becomes confused and thousands of long-dead diggers/4channers rofl in their graves*
    • ats314, on 04/24/2008, -0/+4oh yeah, well i'm going to rickroll the future.
  • Stevanoski, on 04/23/2008, -0/+1In the medical field the problem is how to recover data 10 years from now much less 1400 years. When we look at reducing our document load we look at the history of how data is stored and how could we access that data as storage tech evolves. As an example if documents had been moved to 51/4 floppy origianlly we would have to keep computers online that still have that tech.

    • ShempRider, on 04/23/2008, -0/+5I see your point, but data on a 5.25" disk can be moved to a hard drive...given the equipment and patience. And, while I appreciate tech advances, I still can't buy a commercial drive with a warranty beyond, what? 5 years? Maybe a little more.?
      That doesn't mean a drive can't be sealed up and not used, but I'd bet many people here have experienced, or know someone who's experienced, a drive crash.

      Maybe I just don't see the necessity of their find. Important stuff can be moved from medium to medium as advances are made, and that doesn't necessarily include kids doing faceplants on youtube.
  • majordanger, on 04/23/2008, -0/+7in 1400 years the data will be unreadable because the naked and dirty IT operators will be too busy looking for grubs to eat and trying to stay warm huddling around the cow dung fire .
  • Somedude2137, on 04/23/2008, -2/+5What happened to pen and paper?
    • SkippyDoorknob, on 04/23/2008, -0/+5You know how long it takes to write out all those 1's and 0's by hand?
  • DaneArden, on 04/23/2008, -1/+13Save data for 1400 years? No problem print it out on non acid paper but it in sealed pottery jars and put it in a cave in the desert good for 2000 years easy. There are some good cave I can suggest at Qumran.
  • ShempRider, on 04/23/2008, -0/+1I look forward to verifying their claim.
  • budsstud26, on 04/23/2008, -2/+3Storing data in the traditional sense of how we think about it is likely to be obsolete within 50 years. The amount of data in the world is increasing exponentially, and in the short term, it will continue to live where it does today: the Internet. Distributed, redundant, and gradually migrated from format to format as technologies change. In the longer-term future (30 - 50 years), we'll likely use atomic and quantum methods to store and compute massive amounts of data, and having a bunch of hard drives from even 50 years ago that hold hundreds of petabytes of information will be pretty useless, as that will represent less than a trillionth of a percentage of the information we would need to store for the next 1400 years for whatever reason.
    • drgmdp, on 04/23/2008, -4/+2please shut up.
      you must be one of those who promised me flying cars for the XXI century.
      ***** you, where is my flying car?
  • kmgreen, on 04/23/2008, -0/+23The linked article does not clearly articulate our ideas. Our goals are to create an evolvable, reliable, power-managed system for archival storage. The system itself is made of intelligent storage devices (which can be can utilize ANY storage media). Our current model presents disks as cost effective compared to tape, while providing capabilities that tapes cannot (i.e. advanced erasure coding techniques). We do not assume that any data can survive on a particular device, but rather the system as a whole is survivable and data can migrate.

    This is still an active, experimental project. While the results are promising, it is not ready for production.

    PLEASE refer to the following accurate references :

    The UCSC press release : http://www.ucsc.edu/news_events/text.asp?pid=2130

    Our paper : http://www.ssrc.ucsc.edu/Papers/storer-fast08.pdf

    Kevin Greenan and Mark W. Storer

  • Rafer, on 04/23/2008, -0/+1This will never be more popular than the tried and true method of inscribing data on rocks and blasting them into space.
  • SteelTallon, on 04/23/2008, -0/+1One more reason to freeze myself for the future..
    Mental Note: Take computer with SATA support.
  • farkis, on 04/23/2008, -3/+1What's the big deal? I just print everything out.
  • stanleyford, on 04/23/2008, -0/+1"Archiving data, which means that you don’t just backup data, you also want to have fast access to it..." -- Doesn't archiving usually imply that the archived data is off-line, i.e., you don't have fast access to it?
  • argylesocks, on 04/23/2008, -0/+5I'm sure many people are closed minded enough that they believe that Word will still be in in use 1400 years from now. If we are ready to talk about storing data for 1400 years we better start talking about truly open standards that are created and will be maintained by the community. Not ones that are just faux open ones like OOXML that are only intended for interfacing with MS products. MS and their lock-in strategies are the wost enemy to long term data storage. This is no joke and not something to be brushed off, people are storing critical government documents, archiving historical documents, creating digitally exclusive literature in the form of e-books and much more on computers now. Don't make retrieving this data something that future generations have to struggle with, just do it right the first time and make them fully open. For your own future and future generations demand fully open standards. If you don't believe me that files linked to one company products are a pain go and open that .WPD file on your backup drive.... oh wait...
  • bluepass, on 04/23/2008, -1/+4I doubt the human race will still exist in 1400 years...
  • hollyminkowski, on 04/23/2008, -0/+2This is an interesting idea for long-term storage. But I doubt that it will seem very up to date in 15 years. In the 2020s storing several pb of data reliably will likely cost a few dollars. Dugg for this systems use of ARM controllers running at 400mhz....my favorite chips to program :-)
    My short-term (30yrs) data storage is done using data dvd's secured by error-correction files made using DVDISASTER http://dvdisaster.net/en/index.php Two copies of each DVD are made (2 different manufacturers) and error correction files are likewise doubly stored on DVD...and error correction files for the main correction DVD's are stored on 2 HD's. so I'm covered :-)
    I can recover a single DVD that has lost 25% of its data...and with 2 copies it is unlikely I will ever
    lose a disk. BTW the 2 copies are kept in different locations...one at home and one at a friends home. I'd just die if I lost all my digital home videos, my movie collection...all my old email and program source code :-(
    • CyberSol, on 04/23/2008, -0/+1If only we all had the time + energy + drive to do what you've done. I really wish more people knew how dangerous not backing things up could be.
      +1
      • hollyminkowski, on 04/23/2008, -0/+1I lost a VERY important DVD that had lots of source code and other stuff on it. It had become discolored around the outside edge...this brought me to my NEVER AGAIN moment! I am just finishing up my 680th DVD using my backup plan. I do about 2 DVD's of data a week...then I transfer the error correction files to the backup location at a friends house about once a month using a 16gb thumb drive. http://i31.tinypic.com/1scgu8.jpg
  • Flashman, on 04/23/2008, -0/+3His name is Mark Storer? How appropriate.
    • xXcooXx2, on 04/23/2008, -2/+0LOL
    • mwstorer, on 04/24/2008, -1/+3Sometimes I really wish my last name was Rockstar, but unfortunately I don't think my name dictates my work.
      Mark W. Storer
  • xXcooXx2, on 04/23/2008, -1/+0this is cool but why would we need to keep our data for that long?
  • anononon, on 04/23/2008, -1/+4A couple of slabs of granite and a few lightning strokes from the almighty and your data can last forever.
  • mwstorer, on 04/24/2008, -0/+5I'd like to briefly comment on a common question that comes up in any discussion of archival storage. Application preservation (actually interpreting bits), is an active, and interesting research area, but I'm afraid it's outside the scope of this particular project. We admit that we're punting on that issue because you can't answer every question in a single paper. In the FAST 2008 paper this article is referring to (and that my colleague Kevin linked to earlier), we're strictly dealing with preserving bits.

    Thanks for all your interest. It's a very active area of research so there's inevitably a lot lively discussions on the topics.

    Mark W. Storer
  • locojones, on 04/24/2008, -0/+2Google must be salivating at this development. Now they can not only store ALL your personal information, they can store ALL your personal information for 1400 years!
  • LinuxGalore, on 04/24/2008, -0/+1I had a chat with some of my fellow engineers about just this problem and we came up with the following.

    1. Store the data on glass etched tablets
    2. Part of every plate has a simple easy to translate guide on how to read the data and file formats.

    Its not perfect but its pretty close
  • editthis, on 04/24/2008, -0/+1Your data is not going anywhere kids. The idea that anyone won't have everyone's information in the blink of an eye in 1400 years from now is a joke. Like most of you I started backing up on floppies, then to CD, then to DVD and now we'll move that same .txt email from 1995 onto blue-ray or whatever new media comes at us. It's easy to see why nothing would get lost. So this whole article pisses me off. Don't even get me started on online redundancy. The data we have will quickly be lost in the ocean of storage space we'll have access to. We're just at the start of it all, more data then media to place it on.

    Also, if you're data isn't important enough for you to be backing up, no one in 1400 years from now will probably care. Threw a little Darwin in there for you. Yeah, eat that up.
  • caponumen, on 04/24/2008, -0/+1Does it have a cockroach friendly interface?
  • mintblogger, on 04/24/2008, -0/+0Such long durability will enable to keep generations of information. But what about the cost? May be storage device 10 times costly than computer :-)
  • omikun, on 04/24/2008, -0/+1Um, this is obviously for large organizations with a large set of data to protect for ages like government or science agencies, not your average geek.
  •  

    Login or register to add a comment

    Create a new account or login to join in the conversation on Digg. You'll also be able to Digg stories to help promote things you like.


© Digg Inc. 2008 — Content posted by Digg users is dedicated to the public domain.
DIGG, DIGG IT, DUGG, DIGG THIS, Digg graphics, logos, designs, page headers, button icons, scripts, and other service names are the trademarks of Digg Inc.