Coping with Chaotic Storage Growth

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Until relatively recently, corporate IT systems were often a hodgepodge of disparate devices and protocols that didn’t interoperate. Different vendors, operating systems, conflicting standards and varying approaches to implementation led to a landscape littered with IT islands and information silos. Thankfully, this has mostly been resolved with the exception of one key zone of operations — storage.

Storage remains, in many respects, a mess. On the storage technology side, organizations typically include a mix of Direct Attached Storage (DAS), Storage Area Networks (SANs), and Network Attached Systems (NAS). At the same time, they have to deal with multiple hardware systems each with built-in proprietary software to manage RAIDs, tape drives, and other storage management functions.

These systems, in turn, run on the gamut of operating systems — various versions of Windows, UNIX, Linux, NetWare, and Mainframe OSes, not to mention the fact that many have their own unique file systems.

Other sources of storage chaos include widely separated sites, out of control disk space requirements, and staffing levels that have not kept up with growth. Add to that tightening IT budgets, and you are faced with a significant challenge.

“With eBusiness-related storage requirements doubling annually, organizations are faced with three main challenges: 1) building storage environments that can scale to meet demand, 2) effectively managing these increasingly complex environments, and 3) coping with ever-shrinking back up windows,” says Bill North, research director for storage software at International Data Corp. (IDC).

One possible solution to the chaotic nature of enterprise storage is the storage portal. In this article we’ll take a look at the growth of storage, its repercussions in the enterprise, and how storage portals may simplify some of the issues facing the industry.

Page 2: Over-Provisioning

Over-Provisioning

According to IDC, storage needs grow an average of 50% to 100% annually. Other analysts put growth levels even higher. The usual answer has been to ensure enough capacity through over-provisioning — i.e. throw more servers at the problem. This results in storage resources averaging a utilization rate of only about 30%.

This situation is compounded by a growing management/personnel headache. An SRC (Strategic Research Corp.) study found that over the next three years enterprise storage demand would increase seven-fold while the management resources needed would remain flat.

“Complex storage systems with out-of-control management costs have created an intense need for better Enterprise Storage Management (ESM) tools,” asserts Michael Peterson, president of SRC.

The blame for this situation, though, can’t be conveniently attributed to storage hardware vendors. Products such as Compaq Insight Manager (CIM), Dell OpenManage, and Hitachi High Command do a great job of managing vendor specific systems and, in some cases, in interoperating with some types of generic storage hardware.

What ends up happening, though, is that storage administrators end up having to understand dozens of tools: they want to see how RAID is running on their Dell servers, so they open up the Dell OpenManage screen; they want to view their Compaq RAID arrays, which means it’s time to open up CIM; they want to check utilization, performance, and bandwidth levels on various systems, so they pull up numerous screens of various proprietary tools offered by BMC, EMC, and a host of others.

So to get anything done — never mind understand the overall storage environment — administrators must hop from console to console to configure, allocate, reallocate, and care for the various storage assets at their disposal. This often means going from building to building and system to system in order to back up, restore, mirror, or troubleshoot.

A seemingly basic upgrade in one department from DAS to NAS, for instance, can turn into a nightmare. Changes must be made to all backup, recovery, and business continuation/disaster recovery procedures. For backup alone, this means going to a different screen for every backup product in use in the enterprise — in a surprisingly large number of companies, this can run from two to a half-dozen.

“Storage management is often an ad-hoc collection of disparate tools and predominantly independent device management utilities,” reports Nancy Marrone, senior analyst at Enterprise Management Group.

Page 3: A Solution in Sight?

Storage Portals

Marrone is a strong advocate of Enterprise Storage Automation (ESA) as a means of streamlining storage management. One such ESA approach that may solve many of the above issues is the storage portal.

“What has been missing is a storage management layer that sits above existing applications and systems,” says Phil Triede, a storage management executive at Islandia, NY-based Computer Associates (CA). “A well designed storage portal can integrate and access disparate systems and allow control of all of them automatically from a single console.”

While CA’s BrightStor Portal is probably the most advanced storage portal currently available, other vendors aren’t too far behind. HP offers the Compaq Storage Management Portal, and AlphaNet recently announced their Managed Enterprise Storage Portal (which also utilizes BrightStor technology).

These tools, in greater or lesser degree, provide the following elements:

Broad platform support
Comprehensive application host and storage device discovery
Enterprise-wide reporting
Interaction with current management tools
Open standards
Vendor neutrality
Web-based single point of management

Page 4: Current Shortcomings in Storage Portals

Current Shortcomings in Storage Portals

Such portals, though, are not yet the finished article. There is still much ground to cover before we achieve true end-to-end ESA — where administrators monitor and manage the entire scope of the storage environment — including laptops, workstations, servers, switches and arrays — from one screen, and all routine tasks are automated.

“Instead of carrying out as many as 15 manual steps to add new storage to a SAN environment, policy-driven intelligence would automatically discover new storage capacity, configure it appropriately, and move data accordingly,” says Triede.

The key elements missing, then (though coming very soon, according to CA and other vendors), are:

Dynamic provisioning: The ability to automatically discover new storage capacity, configure it, and take care of moving the data. Thus, storage management applications can conduct error-free, timely, and automatic processing of storage needs without the administrator having to involve him or herself in the myriad details such as where the capacity is physically located or its low-level configuration.

Automated storage best practices: Defining storage best practices allows an administrator to map out the key processes and procedures, and incorporate them into a workflow engine that consolidates sequences of tasks such as backup and restore so that their many steps can be automated. Administrators can then focus on true storage management and troubleshooting.

Topology-guided management: Constant change is the order of the day in the land of storage. Of course, this makes it difficult for storage operators to keep current on all the latest updated needed to applications, processes, systems, and components that correspond to each change. Topology-guided management automatically modifies all aspects of the environment. This separates out the business of storage from the underlying technology. Thus, backup and recovery operations will be able to determine the best path, route, and procedures to accomplish the tasks based on current up-to-date storage topologies.

According to storage vendors, such additions to the storage administrator’s arsenal are less than a year away. When that day comes, new storage devices, platforms, and systems will be automatically discovered, allocated against enterprise-wide policies, provisioned based on best practices, and deployed where most needed. It will soon be possible to balance data across storage volumes, perform backup/recovery tasks, and optimize storage paths according to bandwidth availability and storage loads without administrator involvement.

ESA’s top-down approach to storage management is no pipe dream either. While it represents an evolution of storage management to a higher level, it is really only catching up to what is now regarded as the norm in systems and network management — where achieving a unified view across the enterprise, once a newsworthy subject, is now regarded as the norm.

This story originally appeared on Datamation.

»

See All Articles by Drew Robb