Big Data and Social Listening

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

The Greek Stoic, Epictetus, said, “We have two ears and one mouth so that we can listen twice as much as we speak.”

Through the power of the Internet, we now have approximately three billion mouths speaking and six billion ears listening. But even with twice as many ears as mouths, filtered listening is a science. Data scientists combine filters, artificial intelligence, natural language processing, machine-based learning, and text analysis to make sense of the passel of data facing them.

Social media data scientists use those same tools to process the data extracted from the Internet’s various social media platforms. Monitoring those platforms in real time for specific data and trends is known as social listening.

Social listening is a relatively new science that attempts to filter through the noise in order to extract meaningful, actionable data from blogs, reviews, online newspapers, articles, tweets, and posts. Anything posted as text can be scraped and analyzed against a set of data sieves to bring order out of chaos.

The purposes behind social listening are simple: To extract unsolicited opinion, to gather real world case studies, and to examine sentiment about products and services.

For example, a company that makes smartwatches wants to know what its competition has done correctly, where the flaws are in its products, and how consumers feel that the products could be improved upon. The company wants to gather opinions, ratings, reviews, and sentiments about competitor’s smartwatches before investing time and money into a product that has no advantages over current offerings. To gather this information manually would put the company out of the proper release cycle and perhaps make the product obsolete by the time it debuts.

Using big data analytics, social media scientists can gather and extract pertinent data from many sources simultaneously and produce the initial results within a few days and keep capturing data in real time to keep the reporting fresh and accurate.

A second social listening example is where a company released a product six months ago and, after the initial post release “honeymoon” period, sales figures have dropped dramatically. Company executives and shareholders want answers. Social media analysis and listening can help. By performing analytics on product related data, the company discovers that a member of the product’s creative team leaked an early specification that led to a competitive advantage by another company with a similar product. Chances are good that no amount of Google searching would have unearthed the same definitive conclusion.

Other than product or service feedback, what other insights can one gather from social media analysis? One of the important aspects of public relations is gaining recognition for a company through networking with influencers and though leaders. Social media analysis can pinpoint who those influencers and thought leaders are in specific spaces.

A startup storage company that developed a clever portable device that can upload data to a cloud account by simply plugging it into a USB slot in any computer without intervention doesn’t have a lot of money for big-ticket advertising. It can’t reach potential customers because it can’t put their product in front of customers via costly advertising. As an alternative, the company hires a public relations company to help promote the product by sending samples to reviewers, to storage thought leaders, and to high-profile technology journalists. To find these influencers in the consumer storage space, the public relations company employed social media listening to find out whom to contact.

By collecting data from tweets, articles, posts, reviews, video summaries, and tags, the public relations company provided the storage company with a list of influencers, setup interviews, sent out review copies, and schedule time with journalists at conferences to discuss and to try out the product for themselves. The influencers have the desired audience and can reach that audience on a regular basis at no cost to the storage company. The only marketing expense to the storage company is in hiring the public relations company that it used for the campaign.

Revealing the Audience

Scott Sims, CEO and co-found of Buzzlogix describes the process as follows:

“With social listening and analytics you can uncover your customer’s or audience’s pulse. You can identify topics, communication channels, entities, trends, author information, keywords, concepts, and store everything in a data warehouse that be analyzed with trends over a period of time. Feedback on products, services, competitors, industry trends, and other parameters can be looked at.”

Social media is the new word of mouth style of recommending products, services, and companies between users. However, not all social media sites carry the same user base nor do they have the same impact on a particular demographic.

As an example, a company that sells fashion accessories to women between the ages of 14 and 22 hired a social media listening company to gather sentiment, to gain insight into opinions, and to find out how better to engage those customers in conversations that will drive sales. The initial results were less than spectacular. The company decided to listen to five popular social media sites, but happened to leave out the one site whose use was the favorite of this particular demographic.

This example proves that social media listening isn’t perfect and it still requires some preliminary research by both parties. You have to look at the sites where the conversations occur. You can’t have social media listening success by simply focusing on your favorite three social media sites. You have to find out where the dialogs take place and then exploit the data they generate for you.

The consumer market has changed. Social listening isn’t only about getting closer to your customers, it’s about identifying how consumer preferences are changing. You have to carry on conversations with your customers and your potential customers. And you have to sense what the market wants.

If you’re clever about using social listening and studying what it tells you, you can uncover opportunities, avoid market risks, and use the data your customers generate to your advantage.

As Cheryl Lawson of SMTulsa puts it, “You have to think social. Bring your customers into the process, into the journey, and tell your story.”

Remember these key points about social listening and big data.

· Everyone has access to the same data.

· All data is not equally important.

· Expert analysis is the key to useful insights.

· You can’t change the data that already exists.

· Internet data lasts forever.

· Gather data from several sources for better analysis.

· Internet data is unstructured data.

Costs of Social Media Listening

How much does it cost to listen to social media? Some providers charge hundreds or even thousands of dollars per month for these services, which are usually based on a number of transactions per month, per hour, or per minute.

You might wonder why it costs so much to mine social media data. It’s there in the public domain, so why should it be expensive to look at it? It’s expensive because the computing power required for mining the data. Although high end computing has been commoditized to some degree, it’s still very expensive to use large computing clusters for these trivial appearing, but very non-trivial searches.

But how does an analytics service provider comb through all of the unstructured data to make any sense of it or make it structured? Some text mining service providers use Hadoop-based solutions, while others use MapReduce. Large volumes of data stored in billions of small records, such as those found in social media sites, seem impossible to sort through. However, great strides have been made in this area. For example, in 2008 MapReduce performed the first “petasort” by sorting a petabyte of 100-byte records. Using 4,000 computers, this sort took just over six hours to complete. In 2011, the same experiment using 8,000 computers sped to completion in 33 minutes. Also in 2011, those same 8,000 computers sorted ten petabytes of data in six-and-a-half hours. MapReduce is able to perform these big data sorts because it uses parallel, distributed computing.

Social listening has to do with gathering and gleaning data through analytics. Social media’s data is big, really big, to the tune of an estimated five exabytes per day. That’s really big data.

Five exabytes of data, it’s proposed, is the amount of data generated by human civilization from its dawn until 2003. Social media data is unstructured data. Analytics brings structure to the data however; it’s not all useful data. A lot of it is noise, but the parts that are usable are essential to your business’s success. Remember that you have two ears and one mouth for a reason: for listening twice as much as you speak.