47 Comments
- Darth_tater, on 10/12/2007, -0/+26restoring from a backup would restore what ever vulnerability as well.
the first thing you should to is disable the net interface (or better yet, unplug the damn eithernet) - burtonbe, on 10/12/2007, -2/+20Not so. A clever hacker can always convince someone through social engineering to plug the computer back in and turn it on.
- garethevans, on 10/12/2007, -5/+20How to keep your server completely secure against all attacks, past present and future:
Step one: Unplug the power cables.
Congratulations, you're completely secure! - jlabs, on 10/12/2007, -4/+17"Only if the sysadmin is a complete moron.
A Linux server, when well taken care of, is rock solid"
This can be said about any operating system, even windows. - DuoPros, on 10/12/2007, -2/+14Seriously, STFU about dupes! If its a dupe, press the goddamn button!!
- NetJoe, on 10/12/2007, -1/+12The author makes it sound like this is a reliable and simple strategy. A compromised machine is forever untrustworthy. If an application was the entry point removing it or tightening the configuration may not be simple. Often the password file is compromised and the users use their passwords in multiple locations. You can restore the backups to a temporary partition and move the data back by hand to start.
This subject is so complicated you really should read what the big distributions have to say, as well as sans and cert. It's not something that can be summarized in a couple paragraphs or even a couple large books. I'm afraid this article may just give people a false sense of security. - Latka, on 10/12/2007, -5/+15Only if the sysadmin is a complete moron.
A Linux server, when well taken care of, is rock solid. - Klowner, on 10/12/2007, -1/+11IIRC, they've been local exploits. I'd much rather have to deal with local exploits than remote exploits.. Say for example, something in my dhcp client.... *ahem*
- elusive, on 10/12/2007, -1/+10The proper way to restore a hacked server would actually be to copy the contents of the hard drive (for analysis) and then re-image it, restoring data from clean backups. I have seen many people get into trouble by "cleaning" a previously compromised system and then brining it back online because they missed something or because the visible damage was there just to distract the sys admin and prevent them from noticing the attackers true intentions.
- kubudubudubuntu, on 10/12/2007, -0/+7Hooking them up to a separate syslog sever is always a good idea, and making it as transparent as possible, this will not prevent an attacker, but log the info on when it actually happened , so one can restore/secure a system right before the attack.
But the best idea is.... dont be a nublet, and secure it in the first place ... =) - Matt2k, on 10/12/2007, -0/+7Best practices dictate that you should always wipe a server and reinstall after a hack, but the advice here was suprisingly pragmatic. Digg to you, sir!
- DontSayFanboy, on 10/12/2007, -0/+6
Exactly. If you think you can clean a system after it's been hacked, you are betting that you can beat the attacker at his own game. Can you find every single bit of the rootkit? Are you smart enough to find every backdoor and plug every hole, even if the vulnerability hasn't been published yet? The fact that your server has been hacked is pretty strong evidence that you have already lost this bet.
The best advice I have read so far is that once you have identified that your system has been compromised, immediately login and use dd combined with netcat to shuttle a live hard disk image off to another trusted system for forensic investigation. The same can also be done with a core dump of the current contents of the system's memory. While you are looking over the dumps for the how, where, and when of the compromise, monitor the system's network traffic and be prepared to null route any malicious traffic at the router.
So much of this article is dumb. Find out who owns the attacker's files? Ok, here you assume that the attacker has never heard of the chown command. Use lsof on the running system to learn more about the attacker's processes? Ok, sure, as if lsof hasn't been compromised as well. YOU CAN NOT TRUST ANYTHING ON THE SYSTEM.
You need to do all of your forensics on another trusted system. Once you've identified a compromised system, it is toast. Do not put it back into production. Do not try and 'clean' it. Learn as much as you can from it, wipe it, and try and do better next time.
And what is this paranoia about the attacker destroying all of your data? Aren't you doing backups? I think this mindset says way more about the author than any of their supposed security tips does. If this is an important system that warrants such a careful and tedious restore from compromise, why do you have to pussyfoot around your data and hope that the attacker doesn't delete it for you? Back your ***** up, bring up a more secure OS and restore from your most recent snapshot. End of story. - mancat, on 10/12/2007, -0/+6Make sure to tunnel syslog data if at all possible, as if one machine is compromised, the attacker can easily capture any plaintext syslog packets being transmitted on the same network segment. System logs can provide very juicy, succulent information for attackers. I want a steak
- n0xie, on 10/12/2007, -0/+6"Reinstall Windows"
I thought you wanted to solve the problem, not making it worse? - greyghost487, on 10/12/2007, -0/+5Linux (or any OS) is only as secure as the poorly programmed PHP apps that run on top of it. A hole in a PHP app can compromise a whole system. ("OLDPWD" for any of you that had the recent ubbthreads exploit happen to them) and quite honestly once they are in they are in.
Sure you can patch the hole in the PHP software or in the OS. but honestly, once they are in, they are in. the walls have been breached. Step one for these people is to install a backdoor or three immeadiatly once they get on your system.
Then they carry on razing hell (Spam mailer scripts, fake bank websites). knowing that as soon as you delete their scripts and their sites, they can immeadiatly upload them again. IMO its always better to completly start from scratch, that is the only way to know for sure. - elusive, on 10/12/2007, -0/+5"restoring from a backup would restore what ever vulnerability as well."
Well obviously you should fix the problem (patch, reconfigure, whatever) after you re-image it and before you bring it online. I was just making a quick comment, not writing a formal HOWTO. - WeeBull, on 10/12/2007, -0/+5I say we dust off and nuke it from orbit.
It's the only way to be sure. - mancat, on 10/12/2007, -1/+5Sorry, meant to say "SSH tunnel" if it wasn't obvious.
- neko, on 10/12/2007, -0/+3Regardless of OS, if a server has been hacked, the only way to be -sure- is to take off and nuke the site from orbit. It's the only way to be sure. Once something is compromised, the attacker could have a boatload of rootkits auto-installed very quickly.
Unless you're doing something like running your system off a CDR Knoppix disk, or keeping your /bin, /usr/bin/, /sbin etc on a hard drive that has been jumpered to ignore writes, there's really nothing you shouldn't be paranoid about. Paranoia is a good thing for system security.
Just because you're paranoid doesn't mean they're not probing your network ;) - pdiddle, on 10/12/2007, -0/+3The person who wrote the first comment on TFA had it right on. Discovering every exploit can be majorly time consuming and nearly impossible. Backups and reinstalls are the only sure fire way to eliminate future attacks from the same people. Of course the vast majority of people who administrate UNIX/Linux servers (mainly web servers) haven't a clue about what they are doing, and it really pains me to see people making money with hosting companies which are quite vulnerable :
- Spec8472, on 10/12/2007, -0/+2Anyone who does this is being (to put it mildly) silly.
If there are no recent backups, or you need the latest lot of data (and it's still available) - pull the drive out and back it up to another machine as an ISO file or similar. Make sure the backup worked. (No, don't just make a backup from the compromised machine) Use that to pull out any files you need.
Then go and format the compromised drive and re-load the OS, patch it, etc.
On a machine which has been compromised to admin level rights, you cannot trust that they haven't installed other compromised services which will lie to any form of detection you might care to offer.
Anyone who is being paid to do a system restore for a client should never ever allow the machine to keep running after a compromise. - Atomic1fire, on 10/12/2007, -0/+2no script is perfect and when a script isnt perfect and on linux, linux isnt perfect becuase the scripts holes
was said before by someone else and was worth repeating
use any os you want but nothing is rocksolid enough time and work can break any software - galemathias, on 10/12/2007, -1/+3Intruders on a system most often want to "own" it. Crating a trigger that wipes the macine in case of a network disconnect would seriously damage his "property"
- kubudubudubuntu, on 10/12/2007, -0/+1An attacker would only be able to use that tunnel to own the log server,. It should instead be without an ip, on a hub.
But as 'Netjoe' pointed out, this is a VERRY big subject. - ViceVirtue, on 10/12/2007, -6/+7There have been a number of linux kernel exploits lately... Linux isn't rock solid... Kept up-to-date it's great though...
- Khabi, on 10/12/2007, -1/+2Seriously, I would never allow a machine thats been comprimised back on the network without reimaging it, with or without backups. With any luck the data that needs to be saved isn't executable and most likely safe to be backed-up even after the server was exploited (MySQL, XML, HTML, etc etc). In that case its resonably safe to put on the new system (tho should probably have someone at least do a quick audit to be sure).
And its completely possible to reimage a linux machine remotely without someone on the other end. Been there done that :) - rincebrain, on 10/12/2007, -0/+1buried because the article doesn't have any unique content, and the author's ability to proofread is obvious.
- pathfindertech, on 10/12/2007, -0/+1Most people probably don't even realize their system was hacked. If they don't know enough to secure their system, they probably don't know enough to realize their system is running spambots, rootkits, proxies, etc.
- error401, on 10/12/2007, -3/+4Thermite.
- lbrtuk, on 10/12/2007, -0/+1Use an old dot matrix printer.
You'll get to use up that huge pile of 'computer paper' that's been sitting in the bottom of the cupboard for fifteen years. - kubudubudubuntu, on 10/12/2007, -0/+1+ the syslog server could even have scripts so when triggered it could automatically 'block connections to external network=>mirror the partition(s)=>restore the system and everything to previous state'
- FKnight, on 10/12/2007, -1/+1"Only if the sysadmin is a complete moron.
A Linux server, when well taken care of, is rock solid."
Yeah. If a Linux system gets hacked, it's the admin's fault. If a Windows system gets hacked, it's Bill Gates' fault. - jer2eydevil88, on 10/12/2007, -2/+2I know its not a perfect preventative solution but running Clam AV can offer some protection.
http://www.clamav.net/
If your server does get taken over you can always scan the backups with this.. - streamx, on 10/12/2007, -0/+0Too many aliens!!!
Where is Ripley? - buldir, on 10/12/2007, -0/+0Hold the phone! A Linux server can be hacked?
- farr, on 10/12/2007, -3/+2This is retarded, in the time it would take to do his dumb restoring steps (which don't by any means ensure someone cleverer than you or this guy hasnt already thought of what youre doing) you could be up and running with an up to date installation and fresh new outlook on the importance of security. Marked as lame....
- plasmatic, on 10/12/2007, -16/+15Windows has much more security issues with being a server than linux has. Booting windows would only cause more problems.
- DontSayFanboy, on 10/12/2007, -6/+4This article is retarded.
The article says to not rush to action and kill an offending script because the attacker might notice this and clean the system for you. That is instruction #1.
Instruction #2 is to disconnect the network cable. There is a big assumption here that disconnecting the network cable won't trigger the same reaction from some hidden process you haven't caught yet. Who is to say that your attacker hasn't trojaned sshd and once sshd notices it can no longer talk to it's command and control system, it starts formatting the hard drive? I know if I were compromising systems, I would consider a down link to be a signal that the game is over and it's time to start wiping the system clean of any of my tracks. - twmsdude, on 10/12/2007, -7/+4Awesome Idea!! So...should we start with you as an editor? You obviously have brains. Sounds like a good idea to me. Dick.
- Computer_Kid, on 10/12/2007, -5/+1How to recover from a Hacked windows machine:
Get DBAN
Use DBAN
Reinstall Windows
Reconfigure servaces - streamx, on 10/12/2007, -5/+0It's easy - just install *BSD!
- jer2eydevil88, on 10/12/2007, -10/+3Lets start a new fade on digg, any post that starts with "This article is retarded." gets dugg down.
And feel free to digg me down too so no one has to read that line again! - i440, on 10/12/2007, -17/+3How can a Linux server possibly get hacked?
Sometimes, I can't even get /my own/ Linux system to do what I want it to do. Now that's security. - anonymousc0ward, on 10/12/2007, -16/+1how can such a useless stupid piece of ***** make it to a top rating. digg is ***** - bring back editors with brains.
- Disease, on 10/12/2007, -24/+5ZOMG, Linux breaking? That's impossible!
- V1ncent, on 10/12/2007, -30/+8Boot Windows. There. It's been finally said in response to the "problem with Windows boot linux" bs. Hehehe...
- AOHELL, on 10/12/2007, -31/+2Hmm. old digg from 2 days ago back on top of main page? Some bug?


What is Digg?
Digg is coming to a city (and computer) near you! Check out all the details on our