Linux Cure for Exchange Storage Bloat

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

By Victor Chang

With e-mail rapidly becoming the principal business record of employee activity, e-mail systems need to provide massive scalability, high performance, and cost-effective ways to retain, search, and manage data. Yet today, in their efforts to deliver acceptable performance at an acceptable cost, most corporations require IT to severely limit the size of mailboxes in their e-mail servers. This tactic drives users to spend time managing their e-mail according to the arbitrary limits imposed by the e-mail server and its legacy database architecture.

Users spend precious work hours copying e-mails to desktop PST files – which may be backed up rarely or never – or other secondary e-mail storage systems, copying them to their hard drives, or just deleting these important business records. If an employee needs to reference an old e-mail, he has to remember what it was called and where he filed it, then reload the PST file and search for the message. The productivity hit can be huge, and there is no guarantee the employee will actually find the critical information he is looking for.

Today there is a new way to deal with e-mail storage issues – e-mail and collaboration servers that offer an alternative to Exchange and the legacy database storage architecture. This new type of e-mail server leverages the Linux file system, thus making it significantly faster, simpler, and more efficient to store e-mails. In addition to its operational simplicity, an open-source server can employ a wider range of commodity storage options, letting organizations take advantage of the precipitous drop in hard disk storage pricing to deliver unlimited e-mail scalability.

To gain this efficiency, these systems leverage new and more open message-storage capabilities based either on cost effective commercially packaged storage solutions or on modern Linux file systems. Both choices enable corporations to overcome the inefficiencies of legacy databases, simplify backup and restore, and reduce system-administration complexity. File-based storage lets these e-mail systems scale cost-effectively, maximizing user productive while minimizing IT costs.

To take full advantage of a file-system approach, an e-mail server needs to be designed from the ground up for storage efficiency. For example, the server may write data directly into the filing system using a one-file-per-message approach that allows visibility into disk operations and careful optimization of system performance.

At a technical level, a server designed to minimize disk reads, disk writes, and especially disk-head seeks can provide users with ‘bottomless’ mailboxes. A modern e-mail server designed with this in mind, and unconstrained by legacy database design, can deliver higher performance despite large e-mail stores and demanding applications such as archiving and handheld synchronization.

Modern Linux filing systems such as XFS and Ext3 leverage these optimizations and support features such as journaling (to ensure hierarchy integrity following a power cut), clustering and replication (DRBD), and snapshots (via LVM). With modern e-mail server architecture, IT can complete backup operations without requiring a freeze or a database snapshot.

The file system’s one-file-per-message architecture simplifies restore operations by removing database-synchronization issues. File-based storage eliminates the problem of database corruption because there is no intermediate database to fragment or become corrupted. Similarly, there is no need for database compaction – a high-risk operation that can double the storage requirements of legacy e-mail systems. Recovering from a disaster such as a disk crash is simpler using file-system architecture because it provides an easy way to build low-cost server clusters, or to implement geographically remote data replication, by eliminating database-synchronization issues.

As customers expand their use of file-system-based e-mail servers, they can also exploit the file system for their archiving and compliance needs. For instance, when searching back through old e-mail records, there is no need to mount stored data in a fully configured secondary e-mail server. Instead, users can search backed-up file stores directly for whatever they require, taking advantage of the file system’s natural searchability.

Similarly, the high throughput, scalability, cost-effectiveness, searchability, and open standards support of file-system-based e-mail servers make them ideal for configuration as archive servers (sometimes known as “Journal” servers) for those companies requiring continuous real-time compliance support.

If the goal of the IT department is to reduce costs, improve data manageability, increase performance, and give each user a significantly larger or even bottomless e-mail store, a file system fulfills that goal transparently to the user, making e-mail server storage easy to manage and maintain.

About the Author

Victor Chang is the Vice President of Engineering for PostPath, Linux-based, drop-in Exchange replacement. Prior to joining PostPath, Mr. Chang was Vice President of Engineering at Infoblox and held a variety of executive positions at RSA Security (later acquired by EMC). Before RSA Security, Mr. Chang held engineering and management positions at Apple Computer, Tsunami, Intel, and Xerox.