Security: The expense that keeps on costing

It was the fourth of May 2000, and e-mails saying ILOVEYOU were arriving at the Green Bay, Wis., headquarters of Schneider National Inc., as they were elsewhere around the world. But Schneider’s response to the virus in the days spent recovering was different from that of other organizations.

“We acted extremely early and aggressively,” says Paul Mueller, vice president of communication technology services at the $3 billion trucking company. “We had some impact, but it was minimal.” Some files on a few servers were changed, but the company was able to isolate them and restore them with backed-up copies.

Schneider National Inc.’s Paul Mueller, VP, communication technology services Photo: Mark Derse 2000

Mueller’s group immediately wrote software to isolate the infected messages, put virus scanning software in place everywhere in the company, and started restoration procedures for affected files. As a result, Schneider’s servers stayed up, helped greatly by the fact that the company uses Notes from IBM Corp.’s Lotus Development Corp. and not Microsoft Corp.’s Outlook, whose address book the virus uses to spread.

The ending could have been far worse, as many organizations around the globe found out. Still, the outcome wasn’t totally painless: the Love Bug cost Schneider more than $10,000 in employee time and lost opportunities for productive work, according to Mueller’s estimates. It was money no one had expected to spend.

Even with the best plans, intentions, and preparations, computer security is never completely achieved–and is never fully paid for. Whether recovering from attacks, updating constantly changing software, training employees, or beefing up servers to handle the computing demands of security systems, companies find they must pay and pay and pay to keep safe–often in ways they never expected.

The cost of vigilance

Schneider, for example, would have been in serious trouble had it not invested in a plan for recovering from a disaster and hired in-house experts who could help it. When the Love Bug hit, the company was ready to act. Given the five-figure bill Schneider had to swallow, however, it becomes clear that paying for disaster recovery doesn’t stop with planning. There is a cost to vigilance as well.

“You have to work three shifts, and you have to buy [or outsource] the security infrastructure,” says Ron Newman, CEO of Enstar LLC, a security consulting company in Irving, Texas. That infrastructure includes personnel and monitoring software. It all costs money–sometimes a lot of it. But lack of care can be even more expensive.

“What happens if there is a problem and your [company’s] name gets splashed across a headline of The Wall Street Journal?” asks Newman. “The one thing you don’t want to do is say ‘no comment.'”

Unlike Schneider, Needham, Mass.-based SHYM Technology Inc. uses Microsoft Outlook, and a few employees opened the Love Bug. This not only raised some havoc with internal e-mail for a while, but also generated thousands of additional copies of the message. In fact, SHYM had to lock down its e-mail server to prevent these time bombs from transmitting, potentially causing problems for customers, vendors, and business partners alike, which could damage relationships.

A note of irony is that SHYM makes software that integrates Web, e-mail, and client/server applications with public key infrastructure (PKI) systems that use digital certificates, which are electronic signatures that authenticate the source of a message. Certificates might have eliminated the problem, but SHYM fell victim to another hidden security cost: the need for other companies to implement the same technology. Since relatively few organizations and individuals use PKI, it probably is unrealistic to insist that incoming e-mails include certificates.

SHYM’s experience also demonstrates how software must be updated to remain secure. Anti-virus software, for example, typically protects only against known viruses. When a new virus is released, there is a window of opportunity in which it can propagate and cause enormous damage. Despite high-placed news reports and fast-acting anti-virus software developers, the Love Bug cost billions of dollars worldwide. Even if a virus has been around for a while, it will be new to an anti-virus system unless you are using a version of that software that recognizes the virus. Similarly, browser, operating system, and application vendors all release periodic updates to shore up discovered security breaches.

“Usually the biggest thing that bites people in the leg is not dedicating the resources to do the job properly,” says David W. Ford, owner of Network Knowledge Inc., a Bozeman, Mont., security consulting firm, and a network security instructor for IT training provider Global Knowledge Network. “A great share of [break-ins] happen not because hackers found some clever new way of breaking in, but because the company didn’t have the resources to apply the long list of patches that were available over a year ago.” Having software that isn’t updated also creates a false sense of security, which might be more dangerous than no security.

Training trauma

Even when it is current, no mechanical protection is perfect. Employees need ongoing training to be on guard, but because people are fallible, they may forget or ignore what they have been taught.

SHYM, for example, trained employees how to recognize and handle questionable e-mails, like those with unidentified attachments. In contradiction to their training, though, a few people configured Outlook to automatically launch attachments. Had they followed instructions, the Love Bug would have remained unrequited.

“People will get sloppy, even people that know better,” says SHYM’s executive vice president, Mike Rothman. “With adequate training, you can mitigate some of those risks. [But] in the end, people are still going to make mistakes.” Those making the mistake, while not fired, received some additional training time, notes Rothman, and a possible lecture or two.

Another costly mistake is not taking the time to understand an application. Sysix Technologies LLC of Oakbrook Terrace, Ill., an enterprise software solution seller, was installing SAP AG’s R/3 enterprise resource planning software at a client. “They didn’t realize how much information this made public to all their users,” says Paul Melko, manager of integrated solutions at Sysix. The client was planning a reorganization and hadn’t told its employees, but did update the information in R/3. “Some industrious fellow realized that all the report-tos had changed,” remembers Melko. The information came tumbling out a full month before intended, damaging both employee morale and employer credibility.

The human factor drives most companies to try and make things as foolproof as possible. According to those who have been down that road, the scope of such efforts–and their associated expenses–expands quickly. Indeed, someone working at a company performs most security breaches, say security experts. To keep servers safe, for example, they need to be locked away from unauthorized personnel. Walls that stop short of a ceiling, or floors with crawl spaces that someone could use to enter a server room, have to be modified. That means modifying a building.

“I can do all I want to have good passwords and network security, but if I can’t prevent access to machines, someone can come in and unplug [servers],” says Rocky Johnson, senior network analyst at NEC USA Inc. in Irving, Texas.

Budget reel

NEC was “about average” when it came to security, says Johnson. Wanting to improve the company’s security, Johnson hired consultants to help spot weaknesses and develop a strategy, and suddenly found himself surrounded by problems. One of the more common security weaknesses NEC had to address was its use of analog telephone lines. Employees who work from home often use analog lines that aren’t protected by company communications systems. These employees don’t think anything can happen because no one knows their telephone number. They don’t realize that programs on the Internet let hackers dial 10,000 telephone numbers an hour and determine which are connected to computers.

The overall price of security startled Johnson. “Initially I was thinking this is going to cost a couple of bucks and some time,” he says. “Then it starts adding up very quickly and within two days of looking at the problem it gets into the millions [of dollars] easily.”

For example, software tools, such as the SAFEsuite family of products from Atlanta-based Internet Security Systems Inc., and other high-end products that check for potential weaknesses to different types of hacker attacks, can cost as much as six figures. Even a small company could easily pay $200,000 to establish a secure IT infrastructure, Johnson estimates.

PKI, touted as an up-and-coming security approach by vendors and analysts, can be even more expensive than these general security costs. Avon Products Inc., a Rye, N.Y.-based cosmetics and personal-care product company, had considered using PKI to provide corporate information access to many dispersed sales representatives. But the cost was prohibitive, says Matthew Lagana, IT manager for global information protection at the company.

“With all the costs combined, it was sizeable and probably two or three hundred thousand dollars to get it off the ground,” he says. “Those were just immediate costs.” Instead, the company is working with a password-protected system, Lagana adds, which is more cost-effective than PKI.

Less immediately pressing, but no less necessary, are upgrades to servers and networks. The heavy use of encryption puts a strain on CPUs, memory, and communications bandwidth.

“No one wants to sacrifice performance for additional security,” says Lagana. “That’s an issue that comes up on a daily basis when I’m on project review.” So while PKI is theoretically more secure than password-protected systems, it comes down to how much security your information needs. As an example, historic sales data might require one level of security, while major strategic plans would be quite another.

As a result, infrastructure costs will climb by 10% to 25%, depending on how extensively a company uses encryption, he estimates. “You’re never going to get the old performance. The game is trying to close that gap as much as possible. Otherwise, you will see performance degradation right up front, real fast,” Lagana notes.

Too much security?

Then there is the potential cost of having too much security. Sysix’s Melko remembers one client who owned a half dozen newspapers. Classified ads are a significant portion of a newspaper’s revenue, and the client decided to allow readers to place ads over the Web.

Concerned about security, the client’s network administrators decided to disable simple network management protocol (SNMP) on all the firm’s routers, because under many implementations of SNMP, command messages easily can be intercepted and modified. The act may have protected the network, but it also became impossible to monitor the Web servers–meaning part of the company’s production system could break down without warning, Melko says.

“It’s a matter of balancing” real security needs with the real need to conduct business, says Melko. “It was a little more zealous than they had to be.”

No matter what the balance, the only certainty is that doing everything necessary will cost. “I’d be hard pressed to tally up the cost of managing and [administering] security,” says Schneider’s Mueller. “Suffice it to say it’s in the hundreds of thousands of dollars a year.”

Erik Sherman ([email protected]) is a freelance writer and photographer in Marshfield, Mass. His latest book is Home Networking Visual Jumpstart and he is also the author of Home Networking! I Didn’t Know You Could Do That (Sybex, 2000).