View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Data
January 20, 2017updated 31 Jan 2017 4:20pm

Big data: It may be the new oil but have public failures damaged adoption?

Big data failed to predict Trump and Brexit, so is it worth the hype?

By James Nunns

Data is the new oil, at least that’s what we are told, along with the idea that big data is the saviour, the lifeblood of the modern business.

Yes, data is increasingly important to businesses certainly when it comes to making data-driven decisions rather than gut feel ones. But the big buzzword suffered a number of blows to its credibility in 2016.

Last year saw the weaknesses of big data exposed for all to see. The failure to accurately predict the result of Brexit, or any election result, saw high profile articles damning the reliance upon data and questioning the credibility of it.

It can be argued that the big data predictions were flawed, taking into account inaccurate data sources that painted a view of one segment of society rather than the whole picture, it’s perfectly understandable that these results would be wrong.

What these inaccurate results highlighted is that big data analytics projects need to be done the right way. Put rubbish in and you’ll get rubbish out.

UVA Centre for Politics predicted an easy win for Hillary Clinton.

UVA Centre for Politics predicted an easy win for Hillary Clinton.

Events like these haven’t diminished the importance of the area, if anything they’ve highlighted the importance of good data analytics.

Take for example the latest figures from the Robert Walters Global Salary Survey, which show that salaries for professionals specialising in data analytics and business intelligence can be expected to grow at an average of 4% this year.

Content from our partners
How to turn the evidence hackers leave behind against them
Why food manufacturers must pursue greater visibility and agility
How to define an empowered chief data officer

Partly that increasing salary is due to a well-documented skills gap, but it’s also to do with the importance of these roles in businesses.

James Murray, Associate Director at Robert Walters, said: “Data analytics is seen as an industry agnostic skillset and as a result data analytics professionals have a wide range of job options. This, combined with an increase in investment in data analytics, has driven permanent salaries and contract rates up.”

Further data points show that there is a significant increase in the number of big data initiatives in production.

The recent Big Data Executive Survey from NewVantage Partners, found that big data initiatives in production jumped from 48.2% in 2014 to 62.5% in 2015, with the percentage of firms reporting an expected investment in big data of greater than $50m expected to grow from 5.4% to 26.8%.

Matt Jones, Analytics Strategist at Tessella, said: “The results of this survey are encouraging, not only because data project numbers have grown, but because it demonstrates the growing recognition that data projects must be driven by identifying the insights required to drive the business, and not just seen as a pure technology issue.

“To add to this report, this growing success comes from a much better appreciation of what data means. Many early big data projects came from companies trying to emulate Google and Facebook. Most companies looking to harness data to improve their operations need to take a completely different approach – many now realise data projects can’t be cut and pasted, they need to be designed to meet their specific business challenges.”

What appears to be happening is that organisations across the board are realising that big data can be applied to their problems.

That’s not to say that there aren’t problems with actually successfully implementing a big data project, but it’s clear that it’s become vitally important to running a modern business.

A recent report released by MHR Analytics found that 88% of UK C-level executives agree that data analytics and BI will be crucial to the future of their business, citing factors such as driving growth 42%, helping drive cost reduction and efficiency savings 39% and the need to improve customer intelligence 36% as key factors as to why they are introducing data analytics into their organisation.

Challenges exist, the same report found that almost one in five businesses don’t have a chief analytics officer or a chief data officer, while 21% of the C-suite execs say that a lack of big data expertise is a challenge at board level.

These are clearly big problems that need to be solved, particularly so when you consider that 76% of organisations are planning a big data or data analytics project in the next 12 months.

If all the pieces aren’t in place then you can’t complain if the final puzzle picture doesn’t look like the cover of the box.

That is one of the reasons why the vendors, who are to some extent driving forward the industry and adoption, spent much of 2016 looking at improving the implementation of big data tools, see Cloudera as one.

While it’s clear that there are still many that will face problems with implementing big data projects, with very public failures demonstrating this, these failures aren’t too damaging and serve a greater purpose of highlighting the necessity of approaching it in the right way.

Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.
THANK YOU