Big Data and the Role of the Data Scientist

The latest adventurers are into analysing Big Data to get invaluable nuggets of insights. They scour the vast tracts of data, wrangle with it, clean it and analyse it to arrive at insights that a small data set can never reveal. Genomics, physics, Internet marketing and financial data are some of the examples where high volume, high-speed data known as big data is available. Big data is also available now from a variety of sources. Companies now have access to voices of customers they never had before. If the problem earlier was how to get data, the new one is how to understand and analyse all the data that is available.

Big data is a treasure trove everyone wants to explore and use to his or her benefit.

One of the most obvious examples of great use of big data is retail giant Walmart.

According, as of July 2014 Walmart’s sales stand at $405 billion and the total number of customers that visit per week is 100,000,000. 42% of purchases made by families earning less than $40,000 annually are made at Walmart. The data that Walmart has to handle is indeed tremendous. In its attempt to reach customers effectively and help them make better and more purchases Walmart developed tools to mine it’s big data.

Big Genome was one such product. This product helps capture information regarding what customers are looking for. It also analyses data available on social networking sites, and in-house purchase data to suggest products customers can buy. It offers the incentive of a discount as well. This tool helps understand customer relationships and understanding their preferences.

Their ShoppyCat application scours social networks to suggest products for customers and their loved ones. These two applications in tandem help use the data flowing into Walmart to propel future purchases. Walmart has an outfit, Walmart Labs that keeps coming up with newer ways to make sense of the large tracts of data they receive.

Companies and organisations are therefore looking at folks who can make sense of the hidden patterns and use it to the company’s benefit.

As we saw in the case of Walmart big data is helping companies reach out to customers in a more meaningful manner rather than just using mass one size fits all consumer message. It is using customer usage patterns to arrive at targeted marketing messages for them.

Customers are now more vocal than ever about their product and usage experiences. Applications can now go through this data and a data scientist can help the company make major or minor changes to the product based on their feedback.

This content is for paid members only.

Join our membership for lifelong unlimited access to all our data science learning content and resources.