View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Data
November 28, 2014

Demystifying Big Data – without diminishing the magic

Julien Sauvage, director of big data product marketing at Talend, on how to overcome the skills barrier hindering Big Data uptake.

By Cbr Rolling Blog

With Big Data analytics impacting almost every aspect of our lives, from our health to our bank balance, data scientists are taking on Harry Potter-like status. Held in awe for their wizardry and algorithms, they command high fees and big respect. So how can mere mortals learn how to benefit from Big Data themselves?

It’s a serious problem. In a recent Talend survey 39% of those organisations polled cited lack of skills or time as the main barrier to successfully managing and exploiting the potential opportunities of Big Data. And, as the whole issue of Big Data has evolved, the technology required to navigate it has become increasingly complex. Consequently, the pool of engineers capable of using it or even understanding it is shrinking.

Part of the problem is the smoke and mirrors surrounding Big Data. It’s seen as a complex and intricate technology that only the most talented IT department can fathom. The survey found that in 28% of cases, the initial push towards data-driven projects originated from this quarter, whereas only 13% said that the board or senior management team were the drivers.

This technical focus just makes the skills gap worse. Most IT departments don’t have the overview, resources or the power and influence to look at the strategic picture. Over half of those polled (55%) thought that to realise Big Data’s full potential, there was a need to create multi-disciplinary teams including IT, but also business heads, analysts and industry experts to build an overarching vision.

But in some ways there’s an onus on software vendors too to focus on simplifying the technology rather than making it even more opaque. Businesses need technologies that enable non experts to use Big Data – that would allow a regular data integration developer – someone who has been working with ETL for a few years, for example, to create and deploy Big Data concepts.
Hopefully it’s only a matter of time. Already Big Data ‘sandboxes’ or remote desktops are delivering preconfigured virtual environments designed to get Big Data projects off the ground quickly. They provide an ideal way for businesses to experiment and in turn remove the perception of risk.

Other products, such as the newly-released Talend 5.6 platform, simplify the most advanced new technologies to enable otherwise leading-edge tasks. These include the ingestion of the rapidly changing data involved in financial transactions and enable ever faster analysis for real-time decision-making.

When multi-discipline teams become involved with the strategic side of Big Data and are able to achieve their Big Data goals via self-service, the IT department is free to concentrate on the skills side – and these new solutions give them the confidence to do so and master the basics without the need for the hocus pocus of the technical experts.

Content from our partners
Unlocking growth through hybrid cloud: 5 key takeaways
How businesses can safeguard themselves on the cyber frontline
How hackers’ tactics are evolving in an increasingly complex landscape

In other words they give businesses the confidence to develop their own commercial deployments without being deterred by risk, cost or complexity. While there will always be the need for the deep-dive data scientists at some stage, the combination of a sound strategy, teamwork and these new technologies can often work a magic of their own.


Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.