Combining Big Data and Cloud: Transforming IT and Business

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Sick of hearing and reading the buzzwords du jour: Big Data and cloud computing? Separately they are touted as solving all of the world’s ills, including curing cancer and hunger, though many warn that massive shifting of internal IT systems to the cloud will be a hazard to the career of IT pros in the enterprise.

Cloud Storage and Backup Benefits

Protecting your company’s data is critical. Cloud storage with automated backup is scalable, flexible and provides peace of mind. Cobalt Iron’s enterprise-grade backup and recovery solution is known for its hands-free automation and reliability, at a lower cost. Cloud backup that just works.

SCHEDULE FREE CONSULT/DEMO

However, several recent surveys I’ve been involved with indicate that Big Data projects and systems in the cloud – in other words, combining the technologies – genuinely have beneficial transformation impacts on the overall enterprise. Fortunately, the combination also provide numerous opportunities for IT pros.

The data on the impact of the cloud computing and Big Data together is particularly striking. Eight out of ten respondents to a Harvard Business Review Analytic Services survey earlier this year said the combination of cloud and Big Data was not only critical to their organizations, but has already had a transformational impact. While mobile devices and their access to enterprise systems also is clearly critical and transformational for a majority of respondents, it is important to note that cloud only becomes transformational for the majority of respondents when it is combined with mobile and/or Big Data (see table below).

how to chose a hard drive

Source: “Delivering on the Promise of Digital Transformation,” Harvard Business Review Analytic Services August, 2014

Of course, most of the Big Data and cloud-based transformation for the overall enterprise comes when employees have easier access to enterprise applications and data. Enterprise resource planning, customer relationship management, human capital management and other mission-critical apps taking advantage of Big Data information in the cloud will be the mother lode of business benefit within two years, according to the surveys and subsequent qualitative interviews.

Currently, 41% of the 303 respondents to a survey on cloud tools indicated their organizations had already migrated critical enterprise apps to a private cloud, and half said the shift would occur by the end of 2015. In addition, 38% said they were already using private cloud for delivering high performance apps such as analytics, via the private cloud now. Roughly half said their organizations would be relying on a private cloud for analytics within two years.

Notice that cost savings are not the biggest benefit cited from cloud and big data project veterans. While reducing some IT costs has been widely noted as a cloud benefit, the combination of Big Data in the cloud is primarily seen as a top-line mover – increasing revenues directly or indirectly, via the identification, launch and operation of new products and services.

While the survey sample sizes were not large enough to drill down into specific user segments, qualitative research indicates that retailers and several other industries are among the biggest combiners of Big Data and the cloud. Processing the torrent of marketing, point of sales, economic stats and other structured and unstructured data from external (cloud) and internal sources provides rich opportunities to increase sales and net income.

“Retailers love cloud-based customer experience applications because they help them know the customer and how to reach them,” notes Dain Hansen, director of integration product marketing for Oracle. “How customers are using social and tapping into their reviews and perspectives at every stage of buying cycle is very much a part of their everyday activities.”

The big opportunity for IT pros comes from that 360-degree view of their customers. Integrating Big Data with on-premise, private and public cloud systems that contain historical information about customers has become a major challenge. Indeed, eight out of 10 respondents to the cloud management survey cited integration and interoperability as important as they deploy apps in the cloud.

The crucial nature of integration and interoperability doesn’t become obvious to the non IT users of Big Data in the cloud right away. Initial excitement over the ease of use and lack of capital expenditures required to use salesforce.com or other SaaS tool usually gives way to painful reality after several months or years, and as more cloud-based apps are used.

“While cloud gets people excited due to the huge benefits at the outset, integration with legacy systems is needed to make it more valuable,” adds Hansen. There are several reasons for this. First is the differentiation challenge. He explains that SaaS adoption by an entire industry commoditizes that technology, and the various users in any industry won’t be able to differentiate themselves. “To compete, each retailer must differentiate their SaaS applications using integration customizations and extensions to their on-premise IT.”

Another key challenge to real transformation from Big Data in the cloud is that 360-degree view. Many large companies in financial services, retailing and healthcare are the products of extensive acquisition sprees. Unfortunately, many of those enterprises’ IT infrastructures consist of a multitude of acquired systems. Merger-based retailers, for example, have a difficult time trying to get a complete view of a customer’s purchases, especially if some of the subsidiaries had used cloud based apps and relied on Big Data. “Getting all that data consolidated and integrated is a real tough challenge,” Hansen notes.

A looming hurdle in the Big Data in the cloud world is the massive amount of unstructured data that is viewed as having tremendous value. Integrating and processing the vast number of events, social media feeds, videos and other unstructured sources of data is a monumental challenge.

Hansen and others recommend that organizations looking to be transformed via Big Data in the cloud first focus on augmenting the real-time processing and integration of structured data in their relational systems. In addition, companies should look at rethinking how they can combine these two worlds using tools that can discover, capture, integrate and analyze both unstructured and structured data together.

For example, take a batch-oriented data warehouse and move it to the cloud as a real-time resource, easily accessed and used by managers and professionals in the operating units. Augmenting existing systems by moving on premise batch apps to the cloud also has the appeal of being a lower cost approach that is sure to quickly provide positive returns.

That’s an example of how IT pros can not only embrace but promote the cloud combined with Big Data. And enjoy new career opportunities as their own organizations are transformed.

Here’s a copy of the report on cloud integration challenges and management tools.

Photo courtesy of Shutterstock.