232 Comments
- ghaltmann, on 03/14/2008, -2/+111One time while messing around in /public_html on a client's site I accidentally deleted index.htm.
I didn't have a backup of index.htm. His site was effectively destroyed. No index, no backup. Yeah, I thought I was dead.
I calmly remembered that I had visited it in IE for compatibility reasons, and that IE notoriously caches things. I unplugged the net, went to IE and navigated to the site. It came up. View Source> Copy>paste upload as index.htm and no one was the wiser.
Don't panic. - talonstriker, on 03/14/2008, -0/+77Thats the only good thing I've read about IE on digg.
- baalzebub, on 03/14/2008, -1/+71i would reply, if you press enter on that keyboard i will pound you with my fists until you bleed out your ears...
- neko, on 03/14/2008, -1/+63It was okay, they had an instance of Emacs running. Problem solved!
- Peepo, on 03/14/2008, -2/+55
"If any of you ever have the same problem, here's the hex for future reference:
070100002c000000000000000000000000000000000000000000000000000000
0000dd8fff010000dd8f27000000fb02ef07000000fb01ef070000000000bc8f
8800040000bc012f65746300
"
Yeah, we'll just ya know execute this binary and recover evry motha ***** thing.
***** ! I wish i have 1 % of his unix skill - yevkasem, on 03/14/2008, -6/+55it wouldn't suck that bad, and i don't know how it's happened to so many of you, because it's not a valid command. it's not even close to being right.
recursive is "-R"; has to be capital.
the options come before the mode.
you need to specify a file.
you must have been using british unix or something. - EarlOfLade, on 03/14/2008, -1/+41See.. This is why I love Linux.
With a Live-CD, I could have easily restored from a tape device whatever backups were available. Their experience is a great example of horrible sysadmins who has never done a disaster recovery in a test environment. I once administered over 30 UNIX servers remotely and had once a week, a random backup tape sent to my office and with said tape, I always did a full restore and recovery on a blank test server and compared the result with the live system.
If you never test your disaster strategy, how do you know it works and how do you get any practice in disaster recovery? - StaticSignal, on 03/14/2008, -8/+47You know what's just as bad?
# chmod 777 -r
...And you're one directory too high. Ouch. - inactive, on 03/14/2008, -0/+37So the geocities website was ok?
- Evolutuon, on 03/14/2008, -1/+37I think some of you overlooked that this happened in 1986..
- bxblox, on 03/14/2008, -0/+31You got lucky it wasn't there was no PHP. That could have made for some confusion.
- Chickenlip, on 03/14/2008, -5/+35This wouldn't be nearly so devastating if he followed the cardinal user's rule, and NEVER had a shell open as root. sudo is your friend, and imo, you should ALWAYS use that, and never su. The only time I use a root shell is when I need to boot into single-user mode to do some recovery oslt.
If you carouse around your filesystem with a root shell, and aren't ***** EXTREMELY careful with any command you issue, then your darwinified ass deserves to get his system hosed. - RogerStrong, on 03/14/2008, -0/+27We used to use Star multiports - which allowed us to run serveral terminals and printers through one serial line and modem. The software installed into the /star directory.
One of my co-workers was on the phone, walking a customer through removing the software for one of these from an NCR Tower. The command he gave over the phone to the customer was "rm -r star". It took a full fifteen minutes for him to figure out why it was taking so long..... - diggopolous, on 03/14/2008, -0/+25It's all fun and games until someone loses an eye-node.
- username7D8, on 03/14/2008, -0/+25once I put a logout command at the bottom of my friends login script. It was pretty funny. Took him a little bit to figure out what the hell is going on! ;-)
- Claude1971, on 03/14/2008, -18/+41Is Unix a part if Vista?
- image0434, on 03/14/2008, -2/+23 I think this is why girls don't like us
- inactive, on 03/14/2008, -1/+19Essentially it does: rm: remove -r:recursive /:the root folder of everything.
So: it erases the "whole computer".
It's short and lethal. - inactive, on 03/14/2008, -13/+30word of advice for linux users: if you want to advertise linux, never, I repeat, never let anyone read that thing.
"I owe a debt of thanks to David Korn for making echo a built-in of his shell; needless to say, /bin, together with /bin/echo, had been deleted. What transpired in the next few minutes was that /dev, /etc and /lib had also gone in their entirety; fortunately Neil had interrupted rm while it was somewhere down below /news, and /tmp, /usr and /users were all untouched."
Once I reached this paragraph, I seized to understand anything. It's like a chinese guy talking in binary code. - Evolutuon, on 03/14/2008, -0/+17Story Date: 1986. Epic.
- Tyr7BE, on 03/14/2008, -2/+18You could easily spout off the same technobabble for any windows or OS X box. Just because all that stuff exists doesn't mean you have to be intimately familiar with it to use your linux box to surf the web. Just like if someone trashed your windows box, you'd probably be up ***** creek if you didn't know all the gory details of the system internals. This is just the unix equivalent.
- RealmDown, on 03/14/2008, -0/+15I'm going back to my reading of BOFH now.
- smotpoker, on 03/14/2008, -0/+15hahahaha
- nbyn, on 03/14/2008, -0/+15It would be different, but you still have a 50/50 chance of guessing what the Chinese binary code is.
- Rikushix, on 03/14/2008, -2/+17Because it might go down any second:
Unix Recovery Legend
This classic article from Mario Wolczko first appeared on Usenet in 1986.
Have you ever left your terminal logged in, only to find when you came back to it that a (supposed) friend had typed "rm -rf ~/*" and was hovering over the keyboard with threats along the lines of "lend me a fiver 'til Thursday, or I hit return"? Undoubtedly the person in question would not have had the nerve to inflict such a trauma upon you, and was doing it in jest. So you've probably never experienced the worst of such disasters....
It was a quiet Wednesday afternoon. Wednesday, 1st October, 15:15 BST, to be precise, when Peter, an office-mate of mine, leaned away from his terminal and said to me, "Mario, I'm having a little trouble sending mail." Knowing that msg was capable of confusing even the most capable of people, I sauntered over to his terminal to see what was wrong. A strange error message of the form (I forget the exact details) "cannot access /foo/bar for userid 147" had been issued by msg. My first thought was "Who's userid 147?; the sender of the message, the destination, or what?" So I leant over to another terminal, already logged in, and typed
grep 147 /etc/passwd
only to receive the response
/etc/passwd: No such file or directory.
Instantly, I guessed that something was amiss. This was confirmed when in response to
ls /etc
I got
ls: not found.
I suggested to Peter that it would be a good idea not to try anything for a while, and went off to find our system manager.
When I arrived at his office, his door was ajar, and within ten seconds I realised what the problem was. James, our manager, was sat down, head in hands, hands between knees, as one whose world has just come to an end. Our newly-appointed system programmer, Neil, was beside him, gazing listlessly at the screen of his terminal. And at the top of the screen I spied the following lines:
# cd
# rm -rf *
Oh, *****, I thought. That would just about explain it.
I can't remember what happened in the succeeding minutes; my memory is just a blur. I do remember trying ls (again), ps, who and maybe a few other commands beside, all to no avail. The next thing I remember was being at my terminal again (a multi-window graphics terminal), and typing
cd /
echo *
I owe a debt of thanks to David Korn for making echo a built-in of his shell; needless to say, /bin, together with /bin/echo, had been deleted. What transpired in the next few minutes was that /dev, /etc and /lib had also gone in their entirety; fortunately Neil had interrupted rm while it was somewhere down below /news, and /tmp, /usr and /users were all untouched.
Meanwhile James had made for our tape cupboard and had retrieved what claimed to be a dump tape of the root filesystem, taken four weeks earlier. The pressing question was, "How do we recover the contents of the tape?". Not only had we lost /etc/restore, but all of the device entries for the tape deck had vanished. And where does mknod live? You guessed it, /etc. How about recovery across Ethernet of any of this from another VAX? Well, /bin/tar had gone, and thoughtfully the Berkeley people had put rcp in /bin in the 4.3 distribution. What's more, none of the Ether stuff wanted to know[work?] without /etc/hosts at least. We found a version of cpio in /usr/local, but that was unlikely to do us any good without a tape deck.
Alternatively, we could get the boot tape out and rebuild the root filesystem, but neither James nor Neil had done that before, and we weren't sure that the first thing to happen would be that the whole disk would be re-formatted, losing all our user files. (We take dumps of the user files every Thursday; by Murphy's Law this had to happen on a Wednesday). Another solution might be to borrow a disk from another VAX, boot off that, and tidy up later, but that would have entailed calling the DEC engineer out, at the very least. We had a number of users in the final throes of writing up PhD theses and the loss of a maybe a weeks' work (not to mention the machine down time) was unthinkable.
So, what to do? The next idea was to write a program to make a device descriptor for the tape deck, but we all know where cc, as and ld live. Or maybe make skeletal entries for /etc/passwd, /etc/hosts and so on, so that /usr/bin/ftp would work. By sheer luck, I had a gnuemacs still running in one of my windows, which we could use to create passwd, etc., but the first step was to create a directory to put them in. Of course /bin/mkdir had gone, and so had /bin/mv, so we couldn't rename /tmp to /etc. However, this looked like a reasonable line of attack.
By now we had been joined by Alasdair, our resident UNIX guru, and as luck would have it, someone who knows VAX assembler. So our plan became this: write a program in assembler which would either rename /tmp to /etc, or make /etc, assemble it on another VAX, uuencode it, type in the uuencoded file using my gnu, uudecode it (some bright spark had thought to put uudecode in /usr/bin), run it, and hey presto, it would all be plain sailing from there. By yet another miracle of good fortune, the terminal from which the damage had been done was still su'd to root (su is in /bin, remember?), so at least we stood a chance of all this working.
Off we set on our merry way, and within only an hour we had managed to concoct the dozen or so lines of assembler to create /etc. The stripped binary was only 76 bytes long, so we converted it to hex (slightly more readable than the output of uuencode), and typed it in using my editor. If any of you ever have the same problem, here's the hex for future reference:
070100002c000000000000000000000000000000000000000000000000000000
0000dd8fff010000dd8f27000000fb02ef07000000fb01ef070000000000bc8f
8800040000bc012f65746300
I had a handy program around (doesn't everybody?) for converting ASCII hex to binary, and the output of /usr/bin/sum tallied with our original binary. But hang on---how do you set execute permission without /bin/chmod? A few seconds thought (which as usual, lasted a couple of minutes) suggested that we write the binary on top of an already existing binary, owned by me...problem solved.
So along we trotted to the terminal with the root login, carefully remembered to set the umask to 0 (so that I could create files in it using my gnu), and ran the binary. So now we had a /etc, writable by all. From there it was but a few easy steps to creating passwd, hosts, services, protocols, (etc), and then ftp was willing to play ball. Then we recovered the contents of /bin across the ether (it's amazing how much you come to miss ls after just a few, short hours), and selected files from /etc. The key file was /etc/rrestore, with which we recovered /dev from the dump tape, and the rest is history.
Now, you're asking yourself (as I am), what's the moral of this story? Well, for one thing, you must always remember the immortal words, DON'T PANIC. Our initial reaction was to reboot the machine and try everything as single user, but it's unlikely it would have come up without /etc/init and /bin/sh. Rational thought saved us from this one.
The next thing to remember is that UNIX tools really can be put to unusual purposes. Even without my gnuemacs, we could have survived by using, say, /usr/bin/grep as a substitute for /bin/cat.
And the final thing is, it's amazing how much of the system you can delete without it falling apart completely. Apart from the fact that nobody could login (/bin/login?), and most of the useful commands had gone, everything else seemed normal. Of course, some things can't stand life without say /etc/termcap, or /dev/kmem, or /etc/utmp, but by and large it all hangs together.
I shall leave you with this question: if you were placed in the same situation, and had the presence of mind that always comes with hindsight, could you have got out of it in a simpler or easier way? Answers on a postage stamp to:
Mario Wolczko
------------------------------------------------------------------------
Dept. of Computer Science ARPA: miw%uk.ac.man.cs.ux@cs.ucl.ac.uk
The University USENET: mcvax!ukc!man.cs.ux!miw
Manchester M13 9PL JANET: miw@uk.ac.man.cs.ux
U.K. 061-273 7121 x 5699
------------------------------------------------------------------------ - Angostura, on 03/14/2008, -1/+15Well do *you* know the mandarin for "one" and "zero"? No? It would be worse, see.
- RealmDown, on 03/14/2008, -1/+14Having been involved in the DR business, and assisted in many tests and actual recoveries, I commend you sir.
- SmellyGeekBoy, on 03/14/2008, -4/+17What is it with Digg and obssessing over this command?
- techmonkey4u, on 03/14/2008, -0/+13Nerds make the websites you comment on.
- agimat, on 03/14/2008, -0/+13The year was 1992, on an NCR Unix server. I hit the return key too fast and it was all too late when i noticed the extra space after the '*'.
rm -f * .ext
I still find myself staring to double and triple check the the command before executing an rm operation up to this day. - RealmDown, on 03/14/2008, -1/+13Better not leave your terminal logged in.....
- Gryffydd, on 03/14/2008, -0/+11Don't you mean "ceased to understand"? I wouldn't normally pick on somebody for that, but when it's in the context of complaining about not being able to understand someone's written English I can't resist.
- stretch611, on 03/14/2008, -0/+11Actually in this case, it starts with ~/* and ~ will start at the user's home directory, not the root directory. Only the user's documents and settings are destroyed. The rest of the computer will work normally.
- cowninja, on 03/14/2008, -0/+11It's 1986 on a VAX. So unless you mean live tape..
- hyperair, on 03/14/2008, -0/+10You don't need sudo to thrash your entire home directory, which "rm -rf ~/*" will do.
- amneosis, on 03/14/2008, -0/+9Great story, pure heroism.
- gimmeslack12, on 03/14/2008, -0/+9I don't have any friends that are smart enough to know how to spell UNIX much rather type/know "rm -rf /"
- newwatch51, on 03/14/2008, -0/+9wait... you use Windows 98?!
- redcard, on 03/14/2008, -1/+10No, because I never logged in as root :)
And I never left an open console.
lockvt , lockout, and a lockit were and are my friends. - cowninja, on 03/14/2008, -0/+9This is on an old VAX. Unless the VAX was running BSD, which I doubt, I don't think sudo existed. http://www.gratisoft.us/sudo/history.html
- TyCage, on 03/14/2008, -1/+10I once did "sudo chmod -R 755 .*" to change the permissions on a bunch of files that were all in directories named with a leading dot.
It matched ..
It was not a good thing. - elementop, on 03/14/2008, -0/+8rm -rf
The -f flag says "don't ask me to confirm deletions, just do it." - embeemb, on 03/14/2008, -2/+9LOL.. damn.. that would actually hurt..
- RealmDown, on 03/14/2008, -0/+7And proud of it. It beats the hell out of politics.
- PsychoticDude85, on 03/14/2008, -0/+7~/* is within the home directory, it's the user's files, which they have the power to delete. For an individual user it can be just as bad.
- nxtwrld, on 03/14/2008, -0/+7what kind of lama uses unix leaving his terminal with root logged in.
- Narcism, on 03/14/2008, -2/+9I did that on my second install.
My first install, I used "root" as a username... oops. - ohcoaster, on 03/14/2008, -0/+7best wishes suzanne under neat that we will miss you
-
Show 51 - 100 of 231 discussions



What is Digg?
Browsing Digg on your phone just got easier with our enhancements to the