Open Source Storage: Disruptive Technology or Distant Dream?

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

The storage world is abuzz with the potential of open source storage projects such as Cleversafe and Amanda. What could they mean for enterprise storage? Could this spell the end for high-margin storage sales? Could they revolutionize backup and disaster recovery? The answer is maybe yes, and — once the vendor community wraps its wits around the threat — probably not.

Storage analysts, at least, are enthusiastic about the possibilities.

Andreas Antonopoulos, an analyst with New York City-based Nemertes Research, believes that Cleversafe and Amanda can change the way we do storage.

“This represents the natural evolution of storage virtualization,” says Antonopoulos. “Location becomes irrelevant and availability becomes a transparent feature.”

Steve Duplessie, founder and senior analyst at Enterprise Strategy Group, calls Cleversafe a bold initiative. The basic idea is to provide internet-based storage over vast geographical distances to mitigate the impact of natural disasters. It is similar to the storage grid concepts that have been appearing in recent years, but it takes it to another level.

“The idea isn’t new, but the economics are suddenly viable it seems, with the cost of capacity being so cheap,” says Duplessie. “The issue is the commercial applicability — is performance going to be good enough and will it be secure enough?”

To his mind, it makes a lot of sense in the MySpace/YouTube world, where there is tons of data being uploaded and downloaded. But such applications have personal impact rather than business impact. The problem, Duplessie says, is how to translate that potential into a financial model that works.

It’s been tried before, of course. Startups such as Scale8 came close but didn’t quite fly. And open source storage projects may well suffer the same fate in the end.

Amanda is an open source alternative to enterprise backup and recovery. There is also Zmanda, a commercially packaged version of the Amanda (Advanced Maryland Automatic Network Disk Archiver) backup utility, originally developed by the University of Maryland. It already has tens of thousands of customers.

“The real problem for Zmanda is that it is inevitable that Microsoft is going to take the backup market away,” says Duplessie. “Eight years ago this may have been a play, but my gut tells me this is a non-starter now.”

Clever and Safe

While Zmanda has its doubters, some feel Cleversafe is a concept whose time has come. The software is freely available, with more than 3,000 downloads to date. It is built around the Linux File System project, which allows Linux computers to store information on a dispersed Cleversafe Research Storage Grid. Data is distributed across the Internet to 11 grid storage hosting facilities in North America.

Via Cleversafe IDA (information dispersal algorithms) and the Cleversafe grid software, it offers the ability to build grids for dispersing and storing data. 25 developers are currently at work on Cleversafe, which has been available for about 18 months under the GNU General Public License (GPL) 2.0. It runs on Red Hat, CentOS, Debian and Fedora Linux.

“Cleversafe plans to be one of several companies in the dispersed storage market,” says Chris Gladwin, CEO of Cleversafe LLC of Chicago. “More than 3,000 licenses have been issued since last April.”

Right now, the company is testing its 11-node grid in North America and is building a global 40 to 50-node grid. On the commercial side, Gladwin says the company is focusing on technology for the moment. But he is planning to offer hosted storage services to businesses sometime in 2007.

“Imagine being able to plug any intelligent device into the Internet and access data securely from anywhere at anytime,” says Gladwin. “It also offers increased security, privacy, reliability and cost-effectiveness.”

How? By using open source dispersed storage, data can be stored on inexpensive, commodity hardware. In addition, the architecture is highly scalable and the data is disaster tolerant — even if half the data centers go down, the data is still intact.

“Cleversafe has the potential to become a hugely disruptive force in storage.”
— John Webster

“Cleversafe has the potential to become a hugely disruptive force in storage,” says John Webster, an analyst at Illuminata. “The only price one currently pays to register and log on is a 10 Mbps connection to the internet and 1 TB of capacity for each connected Linux server.”

In Webster’s opinion, open source has lagged far behind in storage to date. While it represents a significant threat to OS and database vendors, it hasn’t posed much of a menace to storage.

“What makes Cleversafe disruptive is the concept of global data persistence,” says Webster. “If your application’s data is always available from the grid, why back it up locally?”

“If you test an application using cleversafe.org storage and it works, I think you’ll exploit it further, perhaps for all it’s worth,” Webster continues. “That possibility alone could cost the current storage establishment billions.”

He admits, however, that there are still performance issues to be resolved; for example, it is not yet capable of supporting transaction-laden applications.

Webster sees Zmanda as being less disruptive and possibly of less long-term value, as it appears to be following an established open source route to market. Although he believes it will garner plenty of attention, he feels that models like Cleversafe and Amazon S3 are more likely to prevail.

This article was first published on Enterprise Storage Forum. To read the full article, click here.