In 2005 a new technology appeared on the market that promised to help businesses to deal with extremely large data sets.
The technology known as Hadoop has been dogged by persistent concerns regarding its complexity and whether it can be successfully applied to everyday business use cases.
One company that has set about proving the value of the technology, in addition to the open source ethos, is Hortonworks. The company has helped to pave the way for Hadoop adoption and has set about disproving open source at scale critics by achieving a number of significant firsts.
Along with becoming the first Hadoop company to go public in 2014 it also became the first company to hit $100m revenue in four years.
Hortonworks made the most of the early adopters of its technology to help skyrocket it to the top of the Hadoop market and it’s now moving towards a second wave.
Rob Bearden, CEO, Hortonworks, told CBR: “All of the things we have done over the last five years, and really going back two – three years ago, those early adopters were really kicking in and that’s where we attained all these firsts came from.
“We brought on many hundreds of customers to do those proofs of concept, do those department level apps, prove those thesis out and they’re always very successful, that’s the great news.
“What that has done is to accelerate the adoption across the enterprise of those firsts that were happening three plus years ago, and what it’s done is to make it easier for those followers.”
Due to the open source community, those at the bleeding edge have been able to provide the followers with use cases that guide them where they need to go, essentially providing a road map for success when using Hadoop.
The knock on affect, according to the CEO, is that the early adopters are starting to roll out with wholesale adoption across the enterprise and so are the new “mid-majority adopters.”
“It’s been de-risked, we understand exactly what players to enable how to deploy, which users to leverage what data sets in what ways and exactly the ways to get value quickly,” Bearden said.
“That’s what fuelled the market in the last couple of years and now you are seeing that bigger early and mid-majority customer set begin just now roll out in wholesale across the enterprise. That’s what we are seeing happening from a market functioning and a market adoption and a market maturity, we’re starting to hit leverage points and points of inflection because of these leverage points, which is a cool thing and exciting thing.”
The company can attribute some of its success to helping create a new model of data analytics that’s changed the way businesses think about the technologies they use. No longer do businesses have to rely upon large proprietary data platforms that are often expensive and constraining, as the CEO put it: “they’ve been held hostage with what they can and can’t do with their data at extraordinarily high price points.”
With a wave of new data sets available in the form of mobile, social, web log data and high velocity data from the Internet of Things, businesses have sought out new ways to engage with customers by using this data.
Bearden said: “They can understand what their customers are doing, what their patterns are, where there customers will see value quicker, and they can have those insights they have in these data sets.
“In the old traditional relational EDW/analytic platforms just architecturally and financially they can’t bring that data under management architecturally or financially in a way that’s pragmatic. Hadoop is really the only platform that can do that and what they don’t want to do is repeat the position they got themselves in to with the proprietary platforms they’ve immersed their data architecture.”
Bearden see’s the new wave of data sets as a way for businesses to re-architect their entire data model, the strategies and data architects into a one that is data centric and allows them to evolve their business model and product offerings for both customers and trading partners.
“Hadoop has become the centre of the universe for them to do that. That’s what we built the company on for the last five and half years, around that thesis on that model and the thesis being that they are really going to build the next 20 years of architecture on a next generation platform,. They don’t really want to be locked in to a proprietary environment anywhere in the stack, they want that to be open source and that’s why we built that model the way we have and its paid of very significantly for us,” said Bearden.
Although Hortonworks can boast a significant amount of customers in the Fortune 500, open source companies haven’t always been seen as being able to properly service enterprise businesses. Essentially, their ability to scale has been questioned.
Bearden agreed that in a generic open source model there can be constraints when it comes to being able to provide the same level of enterprise predictability, reliability, scalabilities “all the ilities,” but that the founding principles of Hortonworks have helped it to overcome that.
“Evolve the core architecture of Hadoop from batch to being able to drive batch interactive in real time applications simultaneously over a central data architecture versus just a batch, we deliver that in a very big time way three years ago with Yarn.
“Second are we were going to insure that our HDP platform was built on the enterprise level services that exceeded the traditional data platforms in terms of functionality and in terms of our subscription offering value add services.”
Although there’s been a lot of success with Hadoop, there’s also been a lot of failures and getting value back from the technology hasn’t always been an easy path.
Part of the problem has always been the complexity of the technology and a lack of skills available to help manage and maintain it, but there’s also been difficulties in changing the mind-sets of those that have been working with limited proprietary technology for the past 20 years.
Bearden said that in the end it is incumbently the responsibility of Hortonworks to ensure that there is a knowledge transfer to customers when it comes to getting the maximum value and the most efficiency, but likened the situation to the ERP trend in the 1990’s.
“What the ERP did for the enterprise in the early 90s, and through, was just the radical transformation in which they could have complete visibility and pipeline and velocity of how they go from the order to cash cycle all the way through an ERP planning and financial process end to end,” said Bearden.
“It took a different way of approaching things versus the traditional very siloed way in which enterprises managed inventory, took orders, recorded transactions for their financial partners. But when doing that they created exponential value inside the enterprise.”
Big, open hypocrites? Open source big data analytics brings out the Beauty and the Beast in tech
Bearden referenced a statistic that of Fortune 500 companies that were early adopters of ERP, the top 12% were early adopters and they outperformed their peers by over 50% in the subsequent years, “the scoreboard doesn’t lie on that, because they were using the technology to get value. So it’s incumbent on us to do that knowledge transfer.”
The early adopters of Hadoop and Hortonworks will certainly be hoping for equally impressive performances.
