The Truth About Big Data

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Running late to the big data party? Don’t worry, says Gartner, a technology research firm. It’s just getting started.

Gartner has good news for IT and business managers who feel their organizations are missing the big data hype train. While interest is high, adoption rates indicate that the market for big data solutions is in its infancy.

Seventy-three percent of organizations surveyed by the research group said that they are investing or plan to invest in big data technologies. Yet, only 13 percent said that they had deployed related solutions.

Big data projects are stalling out in the planning stage, Gartner discovered. “The biggest challenges that organizations face are to determine how to obtain value from big data, and how to decide where to start,” said the firm in a statement. “Many organizations get stuck at the pilot stage because they don’t tie the technology to business processes or concrete use cases.”

Once big data initiatives get underway, Gartner recommends that organizations sweat the small stuff.

Another big data myth is that individual data quality flaws have little impact on big data analytics due to the “law of large numbers.” Not so, warned Ted Friedman, vice president and distinguished analyst at Gartner.

“In reality, although each individual flaw has a much smaller impact on the whole dataset than it did when there was less data, there are more flaws than before because there is more data. Therefore, the overall impact of poor-quality data on the whole dataset remains the same,” said Friedman in prepared remarks.

Gartner researchers also cleared up some misconceptions about big data and its impact on the data center.

Big data won’t eliminate the need for data integration. IT managers banking on “schema on read” approaches to enable flexible methods of processing information may be in for a shock. “In reality, most information users rely significantly on ‘schema on write’ scenarios in which data is described, content is prescribed, and there is agreement about the integrity of data and how it relates to the scenarios,” stated Gartner.

Gartner is also combating the notion that it is pointless to invest in a data warehouse for advanced analytics. And no, “data lakes,” vast repositories that store data in their native format, won’t replace data warehouses. “Data warehouses already have the capabilities to support a broad variety of users throughout an organization. [Information management] leaders don’t have to wait for data lakes to catch up,” said Nick Heudecker, research director at Gartner, in a statement.

Pedro Hernandez is a contributing editor at Datamation. Follow him on Twitter @ecoINSITE.

Photo courtesy of Shutterstock.