What is Digg?Sponsored by Best Buy
Maybe The Best Tech Deal EVER--Give Computers To Whole Family view!
bestbuy.com - Geek Squad(r) installation, HP desktop, monitor, laptop & netbook, wireless router--all for $1,199.99!
81 Comments
- inactive, on 10/12/2007, -1/+43Do you know how long people have been saying that?
Let me put it this way. Ask your average Joe if he wants 100 GB or 200 GB for the same price. Do you think he cares about these "raids" or whatever? No, he just wants the most megabytes on his processor! - bitterg, on 10/12/2007, -1/+19The day is coming when RAID or something very RAID-like will be common in PCs, for this very reason. With data protection (mirroring or parity, it doesn't matter), hard drives will become true throwaway commodities.
- jackhole, on 10/12/2007, -2/+15RAID is not a backup method. The first letter in RAID stands for Redundant, not Resilient. RAID ensures sustained availability of data resources in event of individual disk failure. It is a poor choice for protecting data because every disk in the array is still susceptible to data corruption at the array controller, and hardware failure due to power spikes and environmental factors such as vibration or moisture. If you want to protect your data you must back it up to unpluggable and firesafe-able external data storage: tape drives, external hard drives, DVD/CD-Rs, etc. Don't use a square peg for a round hole.
EDIT: rmxz kind of beat me to it. - rmxz, on 10/12/2007, -5/+16RAID is not the answer, for a number of reasons:
* Raid doesn't protect against raid controllers dying.
* Raid doesn't protect against some types of disk errors (where a write claims to have completed, but writes an incorrect value which goes undetected/uncorrected)
* Raid doesn't protect against user error.
* Raid doesn't protect against incorrect data written due to problems like bad memories; bad cables; bad DMA controllers etc
While RAID is nice for the most common kinds of disk failures; after a couple unrecoverable errors I've put a lot more faith in incremental backups done daily to a computer in a different room/office/facility than RAID in one box. And it's sad to see that the most popular raid (RAID5) is uniquely poor for recovery: http://www.miracleas.com/BAARF/RAID5_versus_RAID10.txt
At home, having automated nightly backups to an old computer in my garage (google for the articles on using rsync to do incremental backups) I can recover from most any failure - including user error; and the total loss of my desktop machine. Having weekly rsyncs of that to an encrypted external hard drive in my car, I can even recover from the whole house going down to a flood/fire/etc.
http://www.mikerubel.org/computers/rsync_snapshots/
At work we do similar; with full redundancy for everything except databases; and replication/hot-standby databases. The only RAID most of these machines is using is RAID0 for performance. Data that is especially important gets mirrored/replicated to more than 2 servers at more than 1 location. - inactive, on 10/12/2007, -1/+11Average Joe might care and even understand when Dell offers it, as they do right now in domestic pc's, with words like
250gb (security) 2 x 250gb hard drives
500gb (speed) 2 x 250gb hard drives - dougsnell, on 10/12/2007, -0/+8Until customers make something other than "How big?" their first question, that's all that matters.
What's the non-sense about hard drive manufacturers not wanting to make better hard drives because they need to sell hard drives? They're too focused on "How big" to worry about anything else because every few years the home user is out of room and gets a new computer. Once manufacturers can out-pace the home user's rate of consumption of space, they'll move on to focus on another selling point (like reliability ... or, more likely, cost). - TortfeasorG, on 10/12/2007, -5/+13Experts: Tautological Headlines Validate Themselves
- ungamedplayer, on 10/12/2007, -0/+8Its like I've always said.
Drives exist in three states,
About to die.
On fire.
Dead.
Consider this when you think "should I consider backups for this computer... ?" - ray901, on 10/12/2007, -3/+10FTA
"Two conclusions stood out: self-monitoring data isn't useful for predicting individual drive failures, and temperature and activity levels don't correlate well with drive failure"
In other words the drives do not fail due to normal testable working conditions, they fail because of design and/or materials failure. i.e. we are still bad at making hard drives (except for capacity - that is where all the r&d is). So have two or more instead.
The same approach was used for dual core chips - we were not good enough to come up with a truly outstanding OS that could perform multitasking properly - so the solution is to provide two cores, the multitasking efficiency of the OS is then irrelevant.
/not saying hardware failover is a bad thing but we should now why it is being touted and sold to us - darkstar949, on 10/12/2007, -0/+7@an0nymous - Google uses consumer grade parts due to the fact that they are cheaper and just uses a massively redundant system to ensure the reliability of the system as a whole. It has been mentioned a few times before, but Google doesn't bother trouble shooting device failures - they just replace the parts with another box that is already sitting idle waiting to take the load.
- austindkelly, on 10/12/2007, -2/+9@baalzebub: yeah but where do you keep your pr0n?
- jackhole, on 10/12/2007, -0/+6@ fkr2
Fine. "By its very definition redundancy is a form of backup." It is also the worst possible form of backup, for the reasons I and rmxz outlined.
A hard drive + external caddy is no more expensive than a hard drive + dedicated raid chip. Additionally, because it is portable, it is also more useful on the consumer level than a redundant internal disk, not to mention it is infinitely easier to set up. The best backup plan for "Average Joe" is to get a usb pen drive and/or external hard disk and copy "My Documents" to it every week. - ThatsNotPudding, on 10/12/2007, -0/+6So what kind of hard drives do they use?
- jackhole, on 10/12/2007, -0/+6@ willjc
You are assuming that just because the rotational speed of the platter is constant performance does not increase. Most of the R&D required for these larger hard drives goes towards making platter sizes larger, via making data "size" smaller. You can see this in the new perpendicular storage methods now coming to market. As data density goes up, so does performance, because (1) there are less platters for the drives to juggle and (2) the head needs to cover a smaller space to read/write data. Space and performance are correlated in hard drives. Quality, as mentioned, is not. - an0nymous, on 10/12/2007, -3/+9@ipod
Let me begin by saying you are a tremendous dick. A huge phallus.
Now then:
**
""bulletproof" what a douche."
What's the problem wiith using the term bulletproof? I meant of greater reliability. Did you not understand?
**
**
Why don't you go buy all enterprise SCSI drive you idiot, or maybe you could read that Google found that they are no more reliable than regular SATA drives.
**
On my home computer I actually run a seagate scsi3 raid5 array (80 to 68 pin adaptors) attached to a adaptec 19160. Silly to use it for a desktop system, but my OS is on a Raptor 75gb. The reason I run them is that they provide redundancy for my data and that they were free (outmoded kit from my job. At a large data center, that is.)
I hadn't read that Google found no bonus for consumer vs enterprise grade storage. Interesting. Is there a citation?
"You really have no concept of scale or stuff on the scale that Google is using, so shut up and go back to doing something 1337 like burning CDs."
Maybe not. I have been to the EMC Research Triangle Park facility and I found that quite impressive. They run SCSI 320 connected to SCA2 in their test lab, for those that are interested. But I am asking questions to learn. I cede the mantle of 1337ness to you, though "ipod".
In summary:
Go ***** Yourself, you ***** child.
Blocked. - inactive, on 10/12/2007, -0/+6@rslc
Wait... did you just say that a company that stands to save/lose millions of dollars on hard drives just threw together some numbers? - TonyCubed, on 10/12/2007, -0/+5Would be nice to see what manufacturer failed the msot.
- ScottyMo, on 10/12/2007, -1/+6The trends are already in place. The next selling point is not reliability or cost, but size (not capacity, but actual dimensions). Unfortunately, they're moving toward getting larger capacity in smaller spaces (look at iPod/iPod nano, cell phones, other portable devices) and while cost may come down as a side-effect of the size R&D money, unless something big happens, quality is still coming up short in the priority scale.
- skyfire1, on 10/12/2007, -0/+5I love google. If google was a girl she would be the hottest chick ever. Now in the odd chance that the ESA comes knocking on my door, I'll have something that will explain the hard drive loss.
- lpmiller, on 10/12/2007, -1/+6Who is this Average Joe, and why don't we just punch him in the crotch repeatedly?
- willjc, on 10/12/2007, -1/+6there's no real reason to have a single drive that holds 1 terabyte+ if the standard is still the equivalent of 7200RPM and modest read/write speeds, and will likely fail in a few years.
- an0nymous, on 10/12/2007, -0/+5all of which were consumer-grade serial and parallel ATA units with.."
Consumer rather than enterprise grade? @ google? In their production environment?
Is this a strategy, "less reliable but cheaper" vs "bulletproof but expensive?" - computergod, on 10/12/2007, -0/+5It depends if average Joe has lost a drive before. That has a way of making people care about the integrity of their data. Naturally this means that people will end up buying a RAID 1 system, losing a hard drive, then the other, then complaining that the RAID didn't work.
Link to /. article on this paper:
http://hardware.slashdot.org/article.pl?sid=07/02/18/0420247 - mwsherman, on 10/12/2007, -1/+6My Firefox live bookmark had the Digg headline as, "Experts: No cure in sight for unpredictable hard..." .
- spliznork, on 10/12/2007, -1/+5"and temperature and activity levels don't correlate well with drive failure"
Not exactly. The Google report says temperature *does* correlate with drive failure, just that the correlation is inverse with expectation -- drives that were not actively cooled and thus ran hotter failed less than drives that were cooled. - toxonix, on 10/12/2007, -0/+4Smaller, denser, faster, cheaper, lower power NVRAM. If I could get a 80GB NVRAM chip to replace my hard drive with, I'd jump on it.
- Namco, on 10/12/2007, -0/+4I haven't had a Maxtor fail in..... well, I've never had one fail. Western Digital and Toshiba on the other hand....
Maxtors ran hot and were noisy back in the day, but they're pretty damn reliable now. - mitrovarr, on 10/12/2007, -0/+4Actually, I'd say the reason that processor companies are moving to multiple cores is because it's much easier two make two cores than it is to make a single core processor that's twice as fast. Dual cores actually rely on intelligent software design to make them work correctly (applications must be properly multithreaded.) Poor programming can reduce their benefit.
- inactive, on 10/12/2007, -0/+3I could pull up countless stories since the earliest times of computing when they said the exact same thing. Yet here we are with 1TB hard drives.
- sagemane, on 10/12/2007, -0/+3There should be no debate about whether backups are better than RAID: backups and RAID are for completely different purposes. Backups are for data protection, RAID (except 0) is for data availability. In most cases you can rebuild your RAID array and you won't need to use your backup, but that is a just nice side benefit of RAID; you still need something to restore from when your users delete files or something goes wrong that RAID can't fix.
EVERY system that contains non-disposable data needs to be backed up (in two locations); systems that contain non-disposable data AND need to have high availability need to be backed up AND RAIDed. And maybe have redundant RAID controllers/NICs/PSUs/etc., diverse network connections, and a mirror system in a different location. And any availability you add to the live system needs to be in the backup too; what good is having an array that survives a disaster if the only remaining copy of your data is on one set of tapes that will take a month to restore? Which is why this isn't to say that you shouldn't RAID most everything: you should because it will let you recover from most issues quickly and with less service interruption, just don't make it the last line of defense.
ZFS has good solutions to many of these problems because you can quickly make arrays with variable levels of parity, easily mirror them, and take snapshots as often as you want, giving you protection (mirrored snapshots) and availability (parity and mirroring) at the filesystem level, and these things are available in many other solutions as well.
What trends like that mean is that there won't be a big push toward more reliable drives (not that drives won't get more reliable as a function of improved technology, but that we won't focus on reliability at the expense of large amounts of price and capacity). Especially in Google's case their infrastructure philosophy is based around lots of cheap, redundant parts, not fewer, more expensive and slightly more reliable parts. If companies like Google weren't looking at innovative ways to get the most bang for the buck out of these affordable drives and decided to keep using top-of-the-line enterprise drives for everything because they're more reliable, we'd likely still be using 100MB email accounts. - benitojuarez, on 10/12/2007, -1/+4Best way to avoid uppredictable hard drive loss is to not buy that dirt cheap hard drive (most likely a maxtor) thats 70 dollars or so less then all comparable models. Its cheap for a reason.
- Joe_rigby, on 10/12/2007, -0/+3Spinning magnetic discs and light bulbs will give way to solid state and led diodes.
- bradleyland, on 10/12/2007, -0/+3"Is this a strategy, 'less reliable but cheaper' vs 'bulletproof but expensive?'"
It appears that this may be their strategy, almost to the tee. Following is a SPECULATIVE article (capitalized as not to be confused for fact) about Google's storage strategy.
http://blog.topix.net/archives/000016.html
The important bit:
"Google has 100,000 servers. [nytimes] If a server/disk dies, they leave it dead in the rack, to be reclaimed/replaced later. Hardware failures need to be instantly routed around by software.
Google has built their own distributed, fault-tolerant, petabyte filesystem, the Google Filesystem. This is ideal for the job. Say GFS replicates user email in three places; if a disk or a server dies, GFS can automatically make a new copy from one of the remaining two. Compress the email for a 3:1 storage win, then store user's email in three locations, and their raw storage need is approximately equivalent to the user's mail size." - chijim70, on 10/12/2007, -0/+3Arg, didn't hit edit quick enough. For the last part there. RAID via a card does not increase write times and decreases read times.
- totorototoro, on 10/12/2007, -1/+4Wonder how much the Hard Drive manufacturers paid Google not to reveal which drive brand had the highest failure rate.
- superkendall, on 10/12/2007, -0/+3Get an external HD man. Less than $100 for an enclosure plus drive that will back up all those pictures in about an hour (or less).
Let me tell you what happens if you don't opt for a backup. You get to pay several hundred dollars to guys who disassemble the drive and try to get as much as they can off the platter. Then you get to hope not too many pictures were permanently mangled from the head hitting the platter on todays ultra-dense discs.
Your HD will die. The clock is ticking. Do you want it to die on its terms, or yours? - Namco, on 10/12/2007, -0/+3So my poorly ventilated Tivo with the 2 120GB Deathstars (still going strong after 3 years), are actually going to live longer due to heat buildup? Awesome!
- inactive, on 10/12/2007, -0/+3@jackhole
I have a raid 5 setup at home and you know how much extra it cost me? 1 extra disk. That's it. No raid controller, no fancy raid motherboard. Everything runs off the onboard sata. It also checksums every bit of data on the disks and performs a bi-mpnthly sweep of the data to validate that all bits match the checksums and that the parity disk matches everything. It will fix any errors that it can on the fly with the checksums and parity disks. If major errors start to appear, it will report it to me and I can swap out a disk with a simple reboot or if I hook up a hot spare, the system can start using the spare without an reboot. I also have snapshots to protect against erased files (although if you erase a file before a snapshot picks it up, you are sol). It is also a cached COW raid which makes for very impressive performance.
Point is this, much of what is being bitched about on this thread regarding raid is true for some raid setups, but certainly not all. Raid does not have to be expensive or require expensive controllers.
I will agree with most of the posters here that say that raid is not the perfect solution - it isn't. You need offsite backup for important files; however, a cheap raid setup can protect your data from 90 percent of failures which are usually simple drive failures. Things in the other 10 percent (fire, lightning, flood, burglary, etc) require something offsite. - bpwagner, on 10/12/2007, -0/+2Not sure if I am doing the correct thing, but I use raid 1 (mirror) for my main computer, I back up off site (Mozy) and use seagate 'ES' drives. I hope that is good enough.
- rslc, on 10/12/2007, -1/+3Good that google bring out the Quality issue.
Just like how PSU are so inefficient until Google bring out this issue (of cos global warming plays apart too) - dubbleenerd, on 10/12/2007, -0/+2as long as we have marketing folk who advertise storage media purely in terms of the number of copies of the Library of Congress that can be stored on it, we will always have people who would pick capacity over reliability.
- Jugalator, on 10/12/2007, -0/+2"Ever heard of DVDs?"
This is why I hope for HD-DVD / Blu-ray to drop in price sooner rather than later :-(
I care for the new backup possibilities with those at least as much as for the video quality...
DVD's are just so damn cumbersome with either cost inefficient dual-layer discs (somehow... after all these years) or small 5 GB discs when you have like 50 GB of 500 GB you want to back up regularly... I agree with the above poster in that *another* hard drive may be the best option now. Not raided to help against failing controllers or logic errors being written, but just with some simple scheduled backups taking the most important stuff. You'd have to be even more unlucky here to lose data than on a RAID. - martin308, on 10/30/2007, -1/+3Where does it say that RAID arrays are more risky? I would have thought a mirrored or level 5 array would be safer. Thats what I'm moving to
- superkendall, on 10/12/2007, -2/+4RAID mirrored with a third unit to swap out is simply the best way to go - even if both drives in the mirrored RAID array die at once, the swapped out one is probably going to be good long enough to at least make a copy. And you can keep the third drive offsite.
Five disk RAID arrays may be cool but as this study shows they really are more risky, especially for the home user. - mitrovarr, on 10/12/2007, -0/+2Wow, that's really good to know. Maybe I can take the obnoxiously loud aftermarket hard drive cooler off my drive.
- superkendall, on 10/12/2007, -0/+2The difference in RPM's would really have little to do with the likelihood of a crash - it's not the head that's moving at 5400 RPM, it's the platter. The platter is not what causes the crash, movements of the head into the platter cause the crash. At any RPM above 30 that's going to cause some damage and mess up the head.
It's not faster drives that are screwing you over. It's drives being made with cheaper components and worse QA. Even if you could buy a 5400 drive today for your application it would still probably fail as often. - mahmoodsdotjpg, on 10/12/2007, -0/+2google doesn't need their cash. there's gotta be some good reason for it though. possibly so X manufacturer can't say "endorsed by google" or somesuch.
- ahhell, on 10/12/2007, -1/+3Ever heard of DVDs? They have this new technology that lets you "burn" data on these plastic disks called DVDs. Most of the standard ones, you get fit 4.7GB on a disk.
You might want to get them a shot. - rmxz, on 10/12/2007, -0/+2@oni308 : "Where does it say that RAID arrays are more risky? I would have thought a mirrored or level 5 array would be safer. Thats what I'm moving to"
Here's a good article - short summary - for some forms of disk failure RAID5 silently returns garbage.
http://www.miracleas.com/BAARF/RAID5_versus_RAID10.txt
"The problem is that despite the improved reliability of modern drives and the improved error correction
codes on most drives... it is more than a little possible that a
drive will become flaky and begin to return garbage. This is known as
partial media failure. ...When a drive returns garbage, since RAID5 does not EVER check parity on
read (RAID3 & RAID4 do BTW and both perform better for databases than
RAID5 to boot) when you write the garbage sector back garbage parity will
be calculated and your RAID5 integrity is lost! " -
Show 51 - 82 of 82 discussions
© Digg Inc. 2009
— Content created and posted by Digg users is