View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Data
March 29, 2016updated 04 Sep 2016 11:12pm

ODPi reins in Hadoop complexity with runtime specifications and standards

News: Backed by Hortonworks, IBM & SAS the aim is to make sure that applications work across multiple Hadoop distributions.

By James Nunns

The ODPi has published its first runtime specification in a move to standardise Hadoop applications.

Formerly known as the Open Data Platform initiative, the group was set up in 2015 as a way to try and make sure that applications would work across multiple Apache Hadoop distributions.

The collaborative Linux Foundation project has the backing of big name companies such as Hortonworks, IBM, Pivotal and SAS but has been snubbed by big leading Hadoop players MapR and Cloudera.

Accompanied by a test suite to simplify and speed up the development of data-drive applications, the runtime specification aims to tackle the problem of too much diversity and complexity in the Hadoop ecosystem, something which the group says is impeding adoption.

The runtime specification is built on Apache Hadoop version 2.7 and features HDFS, YARN and MapReduce components.

The ODPi test framework and self-certification uses Apache Big Top for packaging, testing, and configuration. The reason for using Big Top is so that it is closely aligned with the Apache Software Foundation.

Additional technical features include the runtime compliance tests being directly linked to the ODPi’s runtime specification; this is to assist with compliance.

Content from our partners
How businesses can safeguard themselves on the cyber frontline
How hackers’ tactics are evolving in an increasingly complex landscape
Green for go: Transforming trade in the UK

Rob Thomas, VP, product development, IBM Analytics said: "Big Data is the key to enterprises welcoming the cognitive era and there’s a need across the board for advancements in the Hadoop ecosystem to ensure companies can get the most out of their deployments in the most efficient ways possible.

"With the ODPi Runtime Specification, developers can write their application once and run it across a variety of distributions – ensuring more efficient applications that can generate the insights necessary for business change."

The features play to the ODPis objectives to standardise Hadoop to make it easier to use and to increase adoption of the technology.

It is unclear whether the initiatives standards will receive high levels of adoption but its backing by some big vendors will likely help.

Despite receiving the backing of the likes of IBM, SAS and Hortonworks, one of the leading companies in Hadoop, Cloudera, has no intention of joining.

Mike Olson, Chairman and Chief Strategy Officer at Cloudera, told CBR last year: "We had some early concern that this would be a major distraction in the market, but it’s certainly not playing out that way. Our position at the time of its launch was correct. In practice we haven’t seen ODP membership or non membership make much of a difference."

The ODPi has said that its Operations Specification will be available later this year. It will cover Apache Ambari, the Apache Software Foundation project that is designed for provisioning, managing, and monitoring Hadoop clusters.

John Mertic, senior manager of ODPi said: "ODPi complements the work done in the Apache projects by filling a gap in the big data community in bringing together all members of the Hadoop ecosystem."

Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.
THANK YOU