Big Data Challenges: Business Analytics Pain Points Persist

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Amid all the hoopla over the power of Big Data to digest the gut-busting flow of data from web sites, social media, RFID sensors and mobile devices – with the glamour of bedazzled visualizations and real time insights delivered to your handheld – we often forget that the weaknesses of analytics tools continue to bedevil end users.

Data from a summer survey of more than 300 senior executives and managers from medium and large companies around the world highlights both the promise and the pain of current analytics tools. The vast majority of these respondents were in key functional areas outside of IT: finance, sales, marketing, customer service and operations like supply chain or sustainability.

As you can see in the table below, a surprisingly large percentage of the respondents are not exactly totally thrilled with their analytics tools.

Simply put, more than half say their tools don’t give them the information they need, don’t include key data points, aren’t available to all who need the info, aren’t fast enough or easy to use. And anyone who is surprised at those gripes has been hiding in the data center too long. I’ve heard the same (and more) complaints five or even 10 years ago.

big data, business analytics

Ironically, some of their frustrations come from heightened expectations. The users have gotten a great deal of value from these tools over the years, but they know they could get even more value if they were predictive, included unstructured data and were more broadly available.

These three “success problems” have been cited in other surveys over the past five years. But it is surprising they remain annoyances for two thirds of the survey respondents in 2012.

Even more surprising are gripes that are older than a Razr cellphone. Six years ago almost two thirds of the respondents to a survey I managed of a similar-sized group of business executives and managers, mostly outside of IT, complained the tools were too hard to use. Fast forward to 2012 and 55% complain the tools are too hard to use.

Not much progress, despite all the new fancy front end user interfaces, visualization and other “solutions” to the challenge. While the poor ease of use complaint also is tied up in lousy public and college education, poor training and other non IT issues, the industry still has a long way to go.

Another perennial gripe must be especially frustrating for IT pros, their managers and the CFOs who funded so many solutions: In 2006, six out of ten survey respondents complained that their overall systems performance was lousy. Fast forward to 2012, after untold billions have been spent on data warehouse appliances, in-memory processing, faster servers, gigabit Ethernet networks and a host of other hardware/software accelerators. The end result: 59% say the tools are too slow.

Given the margin of error of the two surveys and slight differences in demographics and the exact wording of the questions, I wouldn’t come to a definitive conclusion. But it is safe to say we haven’t moved the needle much in improving performance issues.

Of course I’ve heard what you would expect to hear from vendors asked to explain why so little progress after so much innovation and product sales. One senior marketing official who has been around the industry for more than a decade acknowledges that most of the tools out there are really focused on the past.

“While everybody has better visualization tools, none of them have focused on the predictive side,” noted Nobby Akiha, Actuate’s senior vice president of marketing. And the most common predictive analytics tools – think R, SPSS and others – are harder to use, he conceded.

However, improving the ease of use of predictive tools is high on the to-do list of many analytics vendors. Another key perspective on this is the availability of Big Data, Akiha added.

“Big Data is giving more confidence in the predictive model,” he explained. “In the past, predictive analytics depended on sampling and modeling, but now Big Data offers enough processing power to analyze the whole data set, not just sampling.”

And I’ll bet throughput and ease of use complaints will persist long after the next new hot trend after Big Data cools.