Big Data for the Midmarket: Value In Six Areas

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Also see: Choosing a Big Data Solution: Seven Steps

Big returns are already being achieved by mid-market companies that have deployed Big Data projects. The Big Data package – heavy-duty storage, processors and analytics tools combing through huge volumes of varied data types in real time or near real time – has moved beyond early adopter phase. It’s quickly becoming a mainstream revenue and profit generator, according to a new survey and report project I participated in.

Late last year, around two thirds of survey respondents – from companies with 2,000 to 5,000 employees, from around the world – indicated that their Big Data initiatives had already benefited the organization. When we drilled down to determine where and how much of a difference Big Data was making for respondents, we found that half the organizations with Big Data systems in place gave themselves high grades for decision making, improving product quality, finding and taking advantage of new business opportunities, improving customer service and operations and other key strategic tasks. Only a quarter of organizations that had not put their Big Data programs into production gave themselves similarly high grades, as you can see from this table:

big data for midmarket

An important insight for IT professionals is the widespread use of Big Data among organizations. The conventional wisdom says that marketing is the only department that has taken the Big Data deep dive, but our survey and others are finding important applications in customer service, operations, supply chain management, financial analysis and human resources. As you can see in the following table, marketing systems are expected to be the sixth most important source of internal data for Big Data programs over the next few years:

big data for midmarket

Darin Bartik, Dell’s executive director of information management products, explains that while marketing and social media have become synonymous with Big Data in large companies, mid-market companies find Big Data opportunities in other areas since they may not have large social media activities underway. “Manufacturing firms have a tremendous opportunity to analyze sensor data and other manufacturing operations,” he says. “Supply chain data, such as partner behavior, is one example. There are many different aspects beyond sales and marketing.”

However, IT pros may want to think twice before approaching the CFO or the COO with a Big Data project for the finance or operations departments. Successful Big Data projects are typically based on a collaboration between the business units and the IT department, where the line of business takes the lead. Bartik and others warn that, like other big and important IT initiatives, a Big Data project needs executive sponsorship first and foremost.

“Senior executives in an organization know their big operational challenges, such as global competition or a tough sales environment, and should tackle those problems first,” Bartik says. Since solving those problems will be most likely to attract top executive interest, they are the most likely to be funded and supported. And once those Big Data solutions are successful they will naturally lead to addressing other corporate pain points.

While we all know that executive sponsorship is always a pre-requisite for a large successful IT project, other ingredients are quite important for Big Data success, too. As you can see from this list of Big Data challenges confronted by the early adopters, there is no one major sink hole, but lots of potholes. Here’s the top 10 list:

big data for midmarket

Survey respondents also noted a wide variety of explanations for Big Data project failures. Note that many of them are either not on the top 10 challenge list, or have widely different rankings:

big date for midmarket

The paucity of data scientists and others who are skilled in statistical analysis, business and IT issues is well known, but is especially acute for the mid market. “These are tough people to find,” notes Bartik. “I often recommend that companies create a data science team that is not located in one line of business or isolated in IT. Take your best and brightest from the technical, analytical and business areas and create a separate team. They need to exercise their Big Data muscles first.”

Closely related to the paucity of Big Data skills is the surplus of gut feel. Many Big Data projects fail because there isn’t a data-driven decision culture. Too many senior executives will insist that their prior experience trumps analytics based insights. While these cultural issues are way outside of the IT professionals’ control, they can capsize even the most well-funded and well-executed Big Data project.

Photo courtesy of Shutterstock.