Just one year ago big data analytics was still not that well understood, with few businesses actively pursuing an implementation.
Today big data – and big data science – is everywhere.
It lay at the heart of Obama’s re-election campaign with Dan Wagner, Obama’s chief analytics officer leading a ‘cave’ of data scientists to predict voter behaviour more accurately than any Presidential nominee before him and Nate Silver outperforming all the subjective based opinions from traditional political analysts.
Big data has also been behind some surprising innovations in the health sector. Google, for example, has become better than the Centre for Disease Control at predicting flu outbreaks, just by analysing what its users are searching for. A spike in searches for cold medicines, for example, usually prefigures a generalised flu outbreak .
Behind such innovations was the data scientist – charged with gathering whatever data they can and then analysing it to find meaningful patterns and insights.
And it’s not just in sport and politics that data scientists are making waves, they are more present than ever before in the business world as organisations look to realise competitive advantage.
Big data and business competition
The data scientist is a necessity for the modern business. Regardless of the industry, the business world is far more competitive than it has been in the past. In this environment and especially in harsh economic times, every small advantage matters. If businesses do not analyse their data to the fullest extent then they are not making the most of key competitive assets and will suffer as a result. Skilled data scientists are therefore crucial if businesses don’t want to be left behind.
At EMC we are seeing greater demand for data scientists than ever before as well as a proliferation of universities offering degrees in the subject, but there is still work to be done.
The impact of data scientists on the business
Why is this skill-set so important? It is not hyperbole to say that data scientists have the power to transform a business. This transformation is based on honing existing business models and ensuring optimal competitive edge, two vital contributors to business success.
The business networking site, LinkedIn, provides a good example of this. By elevating the role of the data scientist, LinkedIn has consistently provided its user base with innovative ways to network and build their resumés and professional reputations. This approach has guaranteed LinkedIn a competitive advantage, allowing the company to offer users professional tools before any other company in the social space, tools that are based squarely around the information its user base generates.
Short term solutions to the data science skills shortage
Businesses in EMEA – and the rest of the world for that matter – are currently suffering from a skills shortage when it comes to data scientists. In fact, according to Ventana Research, around half of organisations across the globe do not have the right skills in place to use predictive analytics tools properly . Business will need to make up this shortfall.
In the short term, outsourcing will provide a solution. Data scientists will be able to earn a very good living as consultants, providing their expertise to several businesses at any one time. For the longer-term, though, businesses are also looking to academia to plug the skills gap. For some time now pharmaceutical companies have been tempting academics from their colleges and universities with very generous salaries. Businesses across all sectors are now following suit and looking to scientists (usually life scientists and mathematicians) who are used to dealing with vast data sets in their studies and research.
When it comes to big data analytics, it is much easier to train a data scientist on the specifics of an industry than it is to train an industry business intelligence officer in the specifics of data science.
In-house data science skills
While relying on data scientists from consultancies will be important over the next few years, some businesses have already found that their requirements are too large and need some form of in-house skill to support them.
A number of large enterprises in the marketing, healthcare/life science, media and online commerce sectors have already employed in-house data scientists. We, at EMC, have recruited more than 25 data scientists with the remit to help our customers better understand and exploit the value of their data.
It is not all i doom and gloom however as many employees have entered the business environment across the years with the very skill sets associated with data scientists, but have never been called upon to use them. This "dormant capacity" can be re-invigorated through a skills refreshment development programme such as those offered by companies such as EMC.
The current skills gap and the inability to unearth replicable skills within your organisation however, means that we are still a long way from having full data science teams within most businesses. Currently, in-house data scientists will more often than not provide oversight to analysts from third party consultancies.
Long term solutions to the data science skills shortage
Eventually the high salaries commanded by experienced data scientists will set market forces to work, and we can expect to see the skills shortage addressed.
To assist this process, however, the industry must ensure that there are adequate training programmes in place, as well as supporting public sector initiatives to promote the uptake of advanced mathematics at schools and universities. Creating this pool of data science talent will be vitally important for the long term competitive health of businesses in EMEA, and it is something businesses from all sectors should be working together to achieve.
