Rogue Sys Admin Still Haunts San Francisco

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Terry Childs, the system administrator who is in jail awaiting trial for, in effect, holding San Francisco’s fiber-optic wide area network hostage back in July, continues to darken the lives of members of the city’s IT department.

Childs had installed equipment on the network without authorization and essentially taken it over, creating a super password, then refusing to hand it over until the city’s mayor, Gavin Newsom, visited him in jail a week after his arrest. Then, on Aug. 28, the IT department got a shock: It found yet another unauthorized device on the network.

That was a terminal server, and “it was probably pulled immediately,” Ron Vinson, chief administrative officer and deputy director of the San Francisco Department of Technology, told InternetNews.com.

The department is now scrutinizing the network even more closely in fears of getting yet another unpleasant surprise. “We don’t believe we’ve found all the devices, so we’re going to continue going through the network,” Vinson said. “Just this morning they came into my office and went through all the devices there,” he added.

His department is working with high-tech consultants Xtech, a minority/women business enterprise joint venture between two San Francisco-based companies that has a contract with the city and county of San Francisco for all technology hardware, software and services procurement. Xtech is partnering with Cisco, (NASDAQ: CSCO), which provided the networking infrastructure, to help with the remediation, Vinson said.

Why did a trusted systems administrator such as Childs suddenly turn rogue? The fiber-optic WAN he was working with connects all of San Francisco’s computers, handles city e-mail, payroll and other functions and also handles some of the systems of the city’s police department, and it would make sense to only provide access to a critical network like that to someone who can be trusted.

“When you get levels of access to things in the city, there’s protocols to be followed,” Vinson said. “If it’s anything to do with the police and fire departments, you may need to have specific background checks,” he said. “The computer department currently doesn’t have these protocols in place.”

Failed processes

It’s more than just a lack of protocols; the city’s processes and systems are in disarray. Childs, 43, had been convicted twice of aggravated robbery as a teenager and of misdemeanor weapons possession in 1995, when he was 30 according to the San Francisco Chronicle, facts that should have shown up on the employment application anyone applying for a job with the city has to fill in.

Apparently the process failed somehow, and he was hired in March 2003 by the City Department of Telecommunications and Information Services, now known as the Department of Technology, as a network engineer, the San Francisco Chronicle said.

Childs only came under suspicion earlier this year when the Department of Technology began beefing up security after getting funding from the city government. “We had hired a new security chief and were conducting inventory before implementing new security protocols for the network, and at that point certain things were discovered that looked to be suspicious,” Vinson said.

In May, Child’s managers found he had filled a room in the department’s Market Street offices with computer equipment nobody knew anything about, the San Francisco Chronicle said. They also realized Childs controlled access to the city network.

The rogue devices linked to the network were not discovered earlier because the San Francisco IT department’s change-management system is manual, not automatic. “When someone makes a change, like conducting maintenance on the network, it’s his job to put in that this is happening and it gets out to the stakeholders who are affected,” Vinson explained. If that change isn’t put in, “another system may pop up and say this system went down.”

San Francisco’s asset discovery and management processes were also antiquated, so Childs was able to work around them. The city is updating them now.

This article was first published on InternetNews.com.