digg

Join DiggAbout

Call for questions
Submit and vote up questions you'd like to see answered by Kevin & Jay at the next Digg Townhall on 11/18.

Rogue San Francisco sysadmin coughs up passwords

out-law.com — San Francisco City Council regained access to its own computer network today after Mayor Gavin Newsom convinced network administrator Terry Childs to give them the passwords.

show profanity settings
expand all
  • AmyVernon, on 07/23/2008, -7/+14insane!
  • nafai, on 07/23/2008, -1/+32$5 million in bail? Really?
  • lesleye, on 07/23/2008, -6/+1OMG!
  • seowriter, on 07/23/2008, -7/+2Dugg !!
  • jabberwolf, on 07/23/2008, -19/+2What a piss ant *****!

    This is what happens when you give a little no one a little power. It becomes his life and only control over anything. The IT management's policies should have fired his ass the first day he did not release a password to them. That and the IT management should always check on completed jobs with verification of passwords and documentation.

    Give a power hungry nobody and a lazy management team, and this is what you get.
    • kaelyiesta, on 07/23/2008, -0/+13My, you make a lot of assumptions. This entire string of articles gives the man no chance to present his side of the story at all. Wait a bit longer until the facts are(seemingly) all in before condemning him so completely.
      • PopcornDave, on 07/23/2008, -0/+4Now, now, what's more fun than a witch hunt without facts?
        /s
    • treed, on 07/23/2008, -0/+11The dude's a CCIE. There are only about 16,000 of those in the world. He's hardly a "nobody".
    • PopcornDave, on 07/23/2008, -1/+10"This is what happens when you give a little no one a little power. It becomes his life and only control over anything."

      You've just described the TSA and a good portion of police officers currently employed today.
    • ashfish, on 07/23/2008, -0/+4Read this article: http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2 ...

      it actually gives his side of the story.

      I'd be pretty freaking pissed if I was getting edged out of my job of 5 years, especially if I built the damn system.
  • rvandy, on 07/23/2008, -1/+66According to an interview I read with someone who works within the dept, Childs basically built the entire thing himself, and served as a one man admin, tech support, manager, etc, was overworked, and saw the rest of his department as a bunch of incompetents. He was the only one with admin access for months, and then a new manager came into place, demanded Childs allow other admins access, and he refused. I think the charges will be dropped eventually.
    • mrblue182, on 07/23/2008, -1/+12I would be pissed too.
    • BitBurner, on 07/23/2008, -4/+15I wouldn't let other admins have access to a network i built from the ground up that runs a whole city. I would do every thing in my power to make sure that the "network" was protected and safe. I think this guy did the right thing, he protected his network. Why he is being punished so bad is beyond me. 6 million bail is unbelievable.
      • majordanger, on 07/24/2008, -0/+9uhhhh.... I think "His" network was paid for by the taxpayers of San Francisco.
      • mdoverkill, on 07/24/2008, -2/+4I could kind of understand that he probably put a lot of blood,sweat, and tears into that network. But he needs to be a ***** professional first and foremost. I don't care how good of an admin you are, act like a damn professional.

        No single admin should have control over the entire municipal network.

      • mithrasinvictus, on 07/24/2008, -0/+2@mdoverkill the fact that he had that exclusive control in the first place, unprofessional as it may seem, does prove that his superiors were somewhat incompetent.
      • PhilLesh69, on 07/24/2008, -0/+4You think this guy did the right thing?

        You've never been a system administrator, have you? The job title isn't "overlord of the system" or "owner of the system". This is no different than if the guy in charge of an Army motor pool changed all the keys on all the humvees and deuce and a half trucks. Sure, he's in charge of keeping them running, but they are owned by his employer.

        If you've ever been a system administrator, you should have read the document you signed when you took the job. The systems and software you administer, and any programs, scripts, or tools you write in order to maintain those systems remain the sole property of [whatever] company. Just like software developers sign documents that say the software they write belongs to the company, all copyrights and trademarks and patents belong to the employer.

        This guy broke the law. He denied access to the owner of computer hardware and the software systems running on it. He shut down services, and prevented the government from operating.

        What if part of his network managed the 911 emergency services? What if part of his network managed the distribution of vital services to support the hungry or sick?

    • mojonandha, on 07/23/2008, -4/+5U must really be a 5 year old. Its easy to establish a SOP or a Work instruction whereby when someone needs access, they read and understand the SOP/WI and signoff on that before being granted access. And auiditing is not a new thing. So its very clear the this dude is a total nutcase who doesnt want do all the things i have said above.
      • Kyrato, on 07/23/2008, -4/+5"U" need grammar school.
      • crzdmn, on 07/23/2008, -3/+9I beg to differ, first this guy did not do anything harmful to the network other than protecting it from other people screwing it up. Which ironically is exactly what his job was, to maintain and secure the cities network infrastructure.

        During the time he has been in prison, the network has run fine, no problems, no vandalism, nothing. Yet their treating him like a common cyber criminal who just hacked into the FBI HQ. Someone where he works is flexing some muscle to make this happen, probably to prove a point.

        In the end, when ***** comes tumbling down after someone screws up guess who'll they will be shelling cash out to on a consulting basis?

        Also, handing over any admin access to a network before someone has been trained on the network is pure stupidity. No IT admin worth his beans would do that, not without reassurance that nobody will ever touch it. Also on this same point the guy is/was the most experienced admin they had, as well as held the highest level of Cisco certification available and was the ONLY one to hold it.
      • PhilLesh69, on 07/24/2008, -1/+2crzdmn,

        From the article: "He is accused of blocking all access to the city's network and routers by resetting passwords"

        He wasn't just keeping someone from having root access. He was preventing the USE OF ALL servers and access to the ENTIRE network.

        This is no different than if a general manager of a retail store changed all the locks and refused to give the keys to the owners of the store.

        The guy felt threatened that they were bringing in people to replace him, and he overreacted. What he should have done is update his resume, and while he trained his apparent replacements, he should have been on monster.com looking for a new job.

        I've been through that before, when I was younger and complacent. I had the sense that I had a high level of job security, and I got complacent, and eventually my bosses decided that my ability to bail them out of every crisis didn't make up for me showing up for work at 1 pm. So they brought in new people. I had to train them. So I got a better job during the time it took me to train my replacements.

        This guy's failure was to not recognize that reality. He could have taken a 5 to 10% pay raise by moving on, but he decided to "take his ball and go home". Now he's in jail with a $5 million bail.
    • Eezyville, on 07/23/2008, -0/+4http://digg.com/security/Why_San_Francisco_s_netwo ...

      I read the same thing here. ^^^
      • FlyingSpaghetti, on 07/23/2008, -1/+1I did not. This is my first exposure to this data.
    • mllawso, on 07/24/2008, -1/+2I'd let them have the passwords.
      1. Laugh and be unhelpful when they break it.
      2. Demand a raise, and oversight of the idiots that screwed it up.
      3. Fix network.
      4. Remind them of 1 and 3 the next time they try to pull something like that again.
      • ifknot, on 07/24/2008, -1/+3But Childs was made of different stuff to you, he was more concerned, possibly neurotically overconcerned, about having a stable reliable network to ensure the really important people in society got their wages etc... He just wanted to protect something he saw as important from people he thought were idiots. Don't forget they were happy to pay one man to do 3-6 peoples jobs without any security SOPs, one man who never took a holiday so that he could be on call 24/7 365/yr. The tragedy is the way he was treated as a consequence.
  • fmtoffolo, on 07/23/2008, -4/+8pencil
  • hayzeus, on 07/23/2008, -2/+75He just wanted his stapler back...
    • PhilLesh69, on 07/24/2008, -0/+1they switched from the Swingline to the Boston stapler, but I kept my Swingline stapler because it didn't bind up as much, and I kept the staples for the Swingline stapler and it's not okay because if they take my stapler then I'll set the building on fire
  • bubbles19518, on 07/23/2008, -9/+2I don't understand why they needed him to give it to him. They should control everything about the network. If they can't reinstall the software they can swap out the hardware with a machine with a password THEY know.
    • johndoesovich, on 07/23/2008, -1/+4It's not that easy. For example.... Let's say you build your entire operation on active directory, passwords are everything. If you don't know your sys admin passwords, you are locked out, that simple. Wanna restore active directory to a point in time? Gotta have the directory restore password. It's not as easy as it seems plus all the lost hours for the IT guys having to restore their email which is another pain in the rear.

      You are talking about days, weeks or possibly longer for an effective restore of their system.

      I see it will be difficult for his side to be heard and without effective governing controls in place, things like this can happen. Although I knock Sarbanes Oxley and how horrible it is and how difficult it makes my job, it protects everyone involved and would guarantee things like this wouldn't happen. Maybe if government entities followed the same guidelines they are forcing corporations to follow, this could have been prevented.
      • ratsg, on 07/24/2008, -1/+1what does active directory have to do with the network?
      • teh_techie, on 07/24/2008, -0/+2@ ratsg - you missed the first part of his post that says "For Example".
    • teh_techie, on 07/23/2008, -0/+5..and have to rebuild the custom network configuration from scratch. It's not as simple as you make it out to be. This isn't a little d-link router on a 192.168.x.x network.
      • PhilLesh69, on 07/24/2008, -0/+1Yep. It will cost them millions of dollars, not to mention all the losses during the outage. If tax collection is involved, tens or hundreds of millions. If emergency services are involved, lives could be lost.

        Basically, this guy conducted an internal denial of service attack. He shut down a computer system owned by a city government.
  • Sarevok9, on 07/23/2008, -1/+9The mayor socially engineered the passwords out of the sysadmin?
    (Blatent) /s/
    • PopcornDave, on 07/23/2008, -0/+4Probably threatened to read him all his proposals for the rest of his term as mayor.
      /s
  • Totz83, on 07/23/2008, -5/+6SSBwd25lZCB5b3UgYWxs
  • billbugger, on 07/23/2008, -6/+4Waterboarding works :-
  • gaapgod, on 07/23/2008, -0/+10Childs is in jail until he can raise $5m in bail = until trial
    • PhilLesh69, on 07/24/2008, -0/+1considering his bail is five mil, and considering that he is obviously guilty of obstructing the use of systems owned by a city government (the same city whose judges are going to be presiding over his case), this guy is going to be doing a little bit of time.
  • kronzdigg, on 07/23/2008, -2/+1any twosome newsom
  • mrblue182, on 07/23/2008, -2/+19$5 million is a tad extreme.
    • PhilLesh69, on 07/24/2008, -0/+2Shutting an entire city government out of their computer network (not internet, network, meaning all their servers, and ability to communicate with all the various government agencies) is also a tad extreme.

      What if your grandmother couldn't get proper 911 service because the city had been ***** over by this little hack?
  • ObeseSnake, on 07/23/2008, -4/+3You would think the hardware vendors would jump in to swap out the routers, get some free press and move on.
  • punkrockscks, on 07/23/2008, -6/+6he is my hero.
    • IceUck, on 07/23/2008, -1/+3Why? Would you hire this 'hero' to install your home network?
  • sockpuppets, on 07/23/2008, -0/+41He reset the passwords to "password."

    City officials were crippled.
  • bloodguard, on 07/23/2008, -2/+3Someone needs to ask why they didn't have build and deploy scripts. I've worked on sites that had hundreds of routers and we could rebuild it from scratch almost as fast as we could unbox the routers.

    And with a proper back channel console network you can reset the whole network in a few min. if you think someone's compromised the routers.

    Gavin "Hair product ponce" Newsom needs to start swinging the axe starting at the top.
    • t0nic, on 07/23/2008, -0/+0That's great if you have the console password. The other security story linked said he wasn't saving configs to flash and also disabled password recovery in the bios. It sounds like he was straight up paranoid in many ways.

      If the other Network Engineers knew he was so paranoid as to not save configs and also to disable password recovery options in ROMMON then well they would be screwed which the other story said. http://digg.com/security/Why_San_Francisco_s_netwo ...

  • BitBurner, on 07/23/2008, -4/+18This guy was protecting the network!!!! He didnt comprimise it or files or anythign...he put it on lockdown to protect it....he used best pratices given the situation...WTF is up with 5 Million bail??? Murders get 1 million? He was protecting the network from people who were going to comprimise it if the passwords were "given out" I would have done the same thing. When new people come in that want access to something they know nothing about or have no business having such access what would you do? I gurantee the network will be hosed in a week if just anyone has access.
    • ssmith39, on 07/23/2008, -1/+3It *doesn't belong to you*.
    • adougherty, on 07/23/2008, -0/+4You got a lot more from the article than I did. I only read that he changed the passwords, and would tell anyone what they were. I never read anything about the competency of the new hires. Perhaps you're right, or perhaps he was a power hungry jerk that wanted to exert control over his employer.

      I think it's a little early to judge him, one way or the other, especially before trial.
      • BitBurner, on 07/24/2008, -0/+1No you read one article. He didn't change the passwords. It was set lock the account after 5 unsuccessful tries. That is standard procedure and policy on every network I admin. There is way more to the story and neither you nor I can see it all. Especially from one article.
  • LordSkywalker, on 07/23/2008, -1/+5He's GUILTY!*


    *until proven innocent. (Yeah, like we'll let that happen.)
  • axis, on 07/23/2008, -1/+2Ahh Ahh Ahh, you didn't say the magic word.

    My respect for the SF city council is gone, but not sure how I felt about them before this. Why was one guy holier than thou? What if dude gets hit by a bus? A small network doesn't even have just one guy knowing the password. Idiots. Maybe they should get their recovery floppy!
  • mattlohkamp, on 07/23/2008, -1/+5"The four most-used passwords are: love, sex, secret, and... god. So, would your holiness care to change her password?"
  • Deejster, on 07/23/2008, -3/+2SETEC ASTRONOMY
  • beingdevious, on 07/23/2008, -0/+12reminds me of the fat dude in Jurassic park.
  • RomeyRome, on 07/23/2008, -0/+6Rumor has it it was "letmein"
  • Exstatica, on 07/23/2008, -0/+4the reason his bail is so high is because they don't want him posting bail and then getting on a computer and tearing down the network or causing more problems.
  • fuzed, on 07/23/2008, -0/+5He should have negotiated for all charges to be dropped and an immediate release in order to get the passwords back.
    • Kyrato, on 07/23/2008, -0/+3This was San Fransisco.. it isn't that simple.
  • Brak710101, on 07/23/2008, -2/+2209 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
  • trollick, on 07/23/2008, -1/+2I bet it was "qwerty" or "12345"
  • United857, on 07/23/2008, -1/+1If this was encrypted data using public-key encryption, it's one thing.

    But apparently, this was the password for the router(s). Any router I've worked with, from the most basic Netgear to the top of the line Cisco, had a way to reset the admin password as long as you had physical access to the machine, via a jumper, DIP switch, whatever. Why was this such a problem for them?
    • joshblufs, on 07/23/2008, -0/+7http://www.networkworld.com/news/2008/072308-san-f ...

      But without access to either Childs' passwords or the backup configuration files, administrators would have to essentially re-configure their entire network, an error-prone and time consuming possibility, Chase said. "It's basically like playing 3D chess," he said. "In that situation, you're stuck interviewing everybody at every site getting anecdotal stories of who's connected to what. And then you're guaranteed to miss something."

      Without the passwords, the network would still continue to run, but it would be impossible to reconfigure the equipment. The only way to restore these devices to a manageable state would be to knock them offline and then reconfigure them, something that would take weeks or months to complete, disrupt service and cost the city "hundreds of thousands, if not millions of dollars," Ramsey claims.
    • t0nic, on 07/23/2008, -1/+1You know you can disable password recovery and lock out even rommon bios resets right?
    • ratsg, on 07/24/2008, -0/+1you either didn't RTFA, don't know anything about real routers or both.
  • philz, on 07/23/2008, -1/+6I believe you have my stapler...
  • highps3, on 07/23/2008, -0/+6Somehow I just dont think he "handed" them over. Also his bail is ridiculous. Ive seen less for people that assault & rape people.
    • NachoBusiness, on 07/24/2008, -0/+2yeah but they didn't assault & rape someone who knew the judge on a first name basis. American legal system strikes again...
  • buddyw, on 07/23/2008, -5/+3Good luck ever getting an IT job again Mr. Childs.
  • ipodman715, on 07/23/2008, -0/+5ok, this needs to be made into a made for tv movie now
    • bmatherlyjr, on 07/23/2008, -0/+4Here are some suggestions:

      Held Hostage - The Terry Childs Story
      Locked Out - The San Francisco City Network Fiasco

      How about a commercial?

      "Hi I am you might know me, I am Terry Childs and I like things secured so that's why when I am at work slaving over a city network, I eat sandwiches my wife makes and places into this new lock tight ziplock sandwich bag..." (assuming he is a family man of course)

      Hey I took a shot.
  • petaganayr, on 07/23/2008, -0/+4Why cough up the password if you're already in jail and unable to raise bail. Let them figure it out, or trade the password for a much lower bail. I would have held on to that password until I die, or let me out of prison.
    • lagannt, on 07/23/2008, -0/+2Yeah sure, just don't drop the soap...
      • petaganayr, on 07/23/2008, -0/+2I don't think they released him after giving up the password nor lowered his bail. So I don't see why he should help them.
  • mickin014, on 07/23/2008, -1/+2That's what they get for messing with him! Those are the same users who ask where the any key is on a keyboard.
    • majordanger, on 07/24/2008, -1/+1So if you pay someone to construct a room addition for you , that you are unable to build..
      but when the contractor refuses to hand over the keys because he built it .. That would be okay with you?
  • p51d007, on 07/24/2008, -1/+2Heck, they already had him in jail on a 5 million bond. Unless some kind of back channel deal was struck, I would have said screw you, figure it out yourself. The only reason I can think of him giving them the password is:
    A. They put him in a cell with "Bubba"
    B. He got some sort of signed deal that the charges would be dropped.
    C. Both A&B
  • slen7, on 07/24/2008, -2/+1http://www.youtube.com/watch?v=HLv6Jw-RsoY (not rick roll)
  • yacks, on 07/24/2008, -1/+1Oh let San Fran burn.. Gavin Newsom can bite my shiny metal butt... He believes the NRA is responsible for an illegal immigrant shooting and killing a family..instead of his sanctuary city policy or even the illegal immigrant himself. I bet he blames Linux Torvalds for this one..
  • vision777, on 07/24/2008, -0/+0He still has to follow the instructions of his superiors in the company. It is not his network it belongs to the city of San Francisco. I am sure the entire city government of San Francisco has others who can safely administrate a network, if not they can easily hire someone who can.
  • WoollyMittens, on 07/24/2008, -0/+1I suppose the guy got about *this* close to being flown to Syria by a CIA plane to have his toe-nails pulled out by Aghmed.
  • sleepless, on 07/24/2008, -0/+1All I can say is that that password probably triggered some sort of secret back door that probably started some doomsday countdown... good luck San Francisco!!!
  • tomarocco, on 07/24/2008, -0/+2Sellout
  • tokyoturnip, on 07/24/2008, -2/+2Buried as lame. As long as they have physical access to any of the devices they could take it back.
  • peppino, on 07/25/2008, -1/+1Login: admin
    Password: admin
  • AUi000, on 08/02/2008, -0/+0San Francisco IT Tech Terry Childs-Links Articles References

    http://digg.com/security/San_Francisco_IT_Tech_Ter ...
  •  

    Login or register to add a comment

    Create a new account or login to join in the conversation on Digg. You'll also be able to Digg stories to help promote things you like.


© Digg Inc. 2008 — Content posted by Digg users is dedicated to the public domain.
DIGG, DIGG IT, DUGG, DIGG THIS, Digg graphics, logos, designs, page headers, button icons, scripts, and other service names are the trademarks of Digg Inc.