The Open Source Fight Club

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

SAN FRANCISCO — The first rule of Fight Club is to not speak about Fight Club. That is
unless of course you’re an open source vendor that, by definition, aims to make
everything open and available.

Open source networking vendor Untangle wants to talk about its Fight Club,
an event, scheduled for LinuxWorld here this week, that will pit open source antivirus and gateway security solutions against their proprietary competitors. The event could highlight the validity of open source solutions in a
space that Untangle alleges has been held back in a conspiracy
by testing labs.

“We believe test labs get paid by proprietary vendors and have no motivation
to show a free and open source solution is better,” Dirk Morris, founder and
CTO of Untangle, said. “Because of their lack of transparency, we
are left to assume they are not performing fair tests or aren’t testing at
all because they fear the results.”

Morris explained that two years ago, Untangle decided to add antivirus
scanning to its network gateway solution. At the time they did a “bake off
to see which vendors are best from a technology fit and performance angle.

Morris figured that antivirus technology was fairly commodity and that
everyone would score fairly similarly, but it turns out his initial
assumption was wrong.

The methodology used by Morris was straightforward. He pulled a set of
viruses off the Web and out of his e-mail. None of the viruses were less
than two months old. His testing found that most vendors caught less than 50
percent of the viruses, and in general the whole group did very poorly.

“To our surprise, ClamAV [open source antivirus], which we thought sucked
at the time, caught the most viruses — all but one — and did so using the least amount of computing resources,” Morris said.

Untangle then integrated ClamAV into its solution and approached testing
labs to get various security certifications. According to Morris, some
refused to test the Untangle solution because of its use of ClamAV. At the
same time, Morris said that the testing labs claimed they had tested ClamAV
and it had done poorly and refused to give Untangle any details, results or
test sets.

Though the Fight Club event at LinuxWorld is a short test in terms of time
duration, Morris argues that it will still prove his point.

“We aren’t testing any bleeding-edge viruses and only viruses found in the
wild. Every vendor in the test should catch every virus,” Morris said.
“However, I think you’ll see that some solutions are distinctly poor despite
public claims, and that open source is one of the best, if not the best,
alternative.”

Untangle isn’t the only open source networking vendor that will be at
LinuxWorld, open source networking vendor Vyatta will be there, too. Though Vyatta also makes use
of ClamAV in its solution, Dave Roberts, vice president of strategy at
Vyatta, told internetnews.com that he might drop by for grins, but
they aren’t participating.

Roberts also does not see the same conspiracy against open source that
Untangle’s Morris sees.

“The fact is testing houses get paid to do testing. As a result, they work
with products where somebody can step up to fund that testing,” Roberts
explained.

“If an open-source project isn’t well organized and funded, the
testers won’t spend the time on a project that isn’t going to bring some
income. Many open source projects are a loose affiliation of people with
little organization structure. In some portions of the development model,
that’s a great strength; in the case of funding testing, it’s a weakness.”

Roberts admitted that there is a lot of open source that is under-tested.
That’s one reason why Vyatta is a big supporter of the commercial
open source model since commercial entities have the funding and resources
to sponsor deep testing of code.

“At Vyatta, we have sponsored testing of our code base at the University of
New Hampshire’s Interoperability Lab (UNH-IOL), and we published all the
results on our Web site,” Robert noted. “If the Untangle event results in
better testing for open source AV solutions, then the community wins, and
that’s a good thing for everybody.

Though it’s not participating in Untangle’s Fight Club, Vyatta has its own event
in which it is hoping to help out open source.

The Vyatta Open
Arcade Challenge will have bunch of classic arcade games — Asteroids,
Pac Man, Defender, Donkey Kong, and Centipede — for folks to play; Roberts said all the proceeds from the event will go to various open source projects.

In either case both events certainly sound a lot less violent than Brad
Pitt’s Fight Club.

This article was first published on InternetNews.com.