View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Data
December 9, 2015

Hortonworks clears a path to big data with Spark analytics

News: Company aims to improve scale of Spark.

By James Nunns

The in-memory analytic capabilities of Spark are being used to advance the Hortonworks Data Platform.

The latest version of Apache Spark, 1.5.2, will appear on the HDP platform and include support for Spark SQL and Spark Streaming. The idea behind this is to help provide a clear path to using Spark, Hortonworks said.

Benefits of the inclusion include being able to deploy Spark-based applications alongside Hadoop workloads, something that will help increase the scale of Spark.

Hortonworks has identified three main areas of focus for Spark, which includes; data science acceleration, seamless data access and innovation at the core.

Data science acceleration aims to improve productivity by enhancing Apache Zeppelin and by contributing Spark algorithms and packing to ease the development of solutions.

The data access element is being addressed by improved integration between Spark and YARN, HDFS, Hive, HBase and ORC. Further optimisation is being done via the Data Source API.

In addition to the Spark integration, Hortonworks has also created a collaboration destination for developers, DevOps, customers and partners to get answers to questions, to collaborate and to share code examples from GitHub, called the Hortonworks Community Connection.

Content from our partners
How businesses can safeguard themselves on the cyber frontline
How hackers’ tactics are evolving in an increasingly complex landscape
Green for go: Transforming trade in the UK

Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.
THANK YOU