Spark is starting to outgrow Hadoop, with many deploying it as a standalone solution.
The engine for Big Data processing is being deployed standalone by 48%, with 40% YARN within Hadoop and just 11% Apache Mesos.
Spark has now become the most active open source project in Big Data, with more than 600 contributors in the past 12 months, compared to 315 in the previous 12 months.
Not only is Spark growing extremely quickly, it is also being used for an increasingly diverse set of applications. Growth is particularly being seen in streaming, with 56% more users than in 2014.
According to research from Databricks, the technology is also breaking down technology barriers between data scientists and engineers.
It is becoming more common for the two areas to work together, as 41% of users identified themselves as data engineers, while 22% identified as data scientists.
The key driving forces behind Spark adoption are performance (91%), ease of programming (77%), ease of deployment (71%), advanced analytics capabilities (64%) and real-time streaming capabilities (52%).
Content from our partners
The survey questioned 1,400 from the Spark community.
