View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Data
September 25, 2015

Big Data explosion sees Spark outgrow Hadoop

News: Improved performance one of the reasons why it is growing so quickly.

By James Nunns

Spark is starting to outgrow Hadoop, with many deploying it as a standalone solution.

The engine for Big Data processing is being deployed standalone by 48%, with 40% YARN within Hadoop and just 11% Apache Mesos.

Spark has now become the most active open source project in Big Data, with more than 600 contributors in the past 12 months, compared to 315 in the previous 12 months.

Not only is Spark growing extremely quickly, it is also being used for an increasingly diverse set of applications. Growth is particularly being seen in streaming, with 56% more users than in 2014.

According to research from Databricks, the technology is also breaking down technology barriers between data scientists and engineers.

It is becoming more common for the two areas to work together, as 41% of users identified themselves as data engineers, while 22% identified as data scientists.

The key driving forces behind Spark adoption are performance (91%), ease of programming (77%), ease of deployment (71%), advanced analytics capabilities (64%) and real-time streaming capabilities (52%).

Content from our partners
How to turn the evidence hackers leave behind against them
Why food manufacturers must pursue greater visibility and agility
How to define an empowered chief data officer

The survey questioned 1,400 from the Spark community.

 

Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.
THANK YOU