1. Growth rate on rise to $2.2bn
Over the next five years, the Hadoop-MapReduce market is expected to hit $2.2bn, a compound annual growth rate (GAGR) of 58%, according to MarketAnalysis.com.
The research firm put the growth down to the increasing amounts of data that has to be stored and analysed, as well as the cost-prohibitive pricing models imposed by the main vendors.
Peter Walker, Country Manager, UKI, at Information Builders, said: "A key driver of this growth is the increasing volume of both structured and unstructured enterprise data stemming from a variety of sources and formats. That’s why we recently announced we’re working with numerous partner integrations including: Cloudera, Hortonworks, HP Vertica, MapR, and Pivotal."
It follows research from IDC earlier this year, which predicted that worldwide revenues for the Hadoop-MapReduce market would rise to $812.8m by 2016, up from $77m in 2011.
2. Consolidation
Hadoop is relatively young in the technology maturity lifecycle, with only 10 years passing since the seminal MapReduce white papers were published by Google, according to MapR CEO and Cofounder, John Schroeder.
"Hadoop is in the innovation phase so vendors mistakenly adopting ‘Red Hat for Hadoop’ strategies are already exiting the market, most notably Intel and soon EMC Pivotal," he said.
Content from our partners
"In 2015, we’ll see the continued evolution of a new, more nuanced model of OSS to combine deep innovation with community development."
3. Hadoop vs traditional databases, MySql and data warehouses
Monte Zweben, co-founder and CEO of Splice Machine, says there will be "strong demand" for Hadoop to become more real-time and transactional, as it becomes a viable alternative for traditional database vendors like Oracle MySQL.
He said: "Hadoop will continue to expand its toolset to provide viable alternatives for traditional RDBMSs, MPP databases, ETL tools, and data warehouses."
Mike Hoskins, CTO at Actian, added: "As type, source and volumes of data continue to evolve, so will the interest in a variety of analytic approaches – from relational, graph, predictive to operational – to meet the changing needs of businesses."
"Enterprises will need modern analytics infrastructures that can scale and ingest any type and size of data from any source, transport it into Hadoop in a way that any analytics engine can read it, and perform a variety of advanced analytics techniques to identify meaningful insights in the necessary amount of time to make an impact on the business.
4. Costs will drive down adoption rate
Chris Boorman, CMO at automation firm Automic, told CBR that Hadoop is too costly and time intensive for many to justify its adoption.
"Although technologies such as Hadoop make real-time processing and visibility financially viable, their eventual uptake will be restricted because of their need to be integrated with existing enterprise tools in order to provide real business insight and value," he said.
"Additionally, businesses looking to implement Hadoop may need to hire several Hadoop specialists to write custom software and knit systems together. These specialists are heavily in demand and therefore expensive. Also the custom scripts they write have a tendency to load additional technical debt onto the codebase."
The prediction follows research by IDC, which expects Hadoop to be in use in 45% of European organisations by the end of 2015.
5. Cloudera and MarketLogic will become market leaders
Jon Wrennall, CTO of Fujitsu UK & Ireland, told CBR: "In 2015 we’ll continue to see the likes of Cloudera and MarkLogic build on these foundations and target a converged data platform aiming to take on more traditional transactional data from RDMS vendors."
Richard Noble, technical director at Tangent Snowball, a digital agency firm, also told CBR: "Based on the support for Hadoop (SAP, Amazon, Microsoft, SAS all offer a Hadoop based solution), and thanks to the open source nature of the project, I don’t think we’ll have a major competitor anytime soon."
"The market leader in this sphere will be the group that provides the easiest interface to big data processing, via whatever tools are built to work with and on top of Hadoop."
6. Apache Spark vs Hadoop
Alys Woodard, research director of European Big Data & Analytics at IDC, told CBR: "Alternatives to Hadoop will emerge; we are seeing a rise in use of Apache Spark for use cases where timeliness is important."
However, Dr. Mohammed Haji, Solutions Architect at Couchbase, said: "There are a number of tools in the market but either they come with a very large price tag or are not able or powerful enough to handle the large colossal data that hadoop can command and cater."
"We have seen a number of leading blue chip venders integrate their tools with Hadoop. Where in the past they claimed the fame for BigData now they competing to say their tools is fully integrated with Hadoop."
7. Skills shortage will disappear
The shortage of Hadoop skills will quickly disappear as enterprises turn to their existing application development teams to implement projects such as filling data lakes and developing MapReduce jobs using Java, according to Forrester Research.
Mike Gualtieri, an analyst of application development at Forrester, said: "For those familiar with business intelligence, SQL on Hadoop will open the door to familiar access to their data. And while experienced technology management professionals take the bull by the horns, the open source community and commercial vendors are building better tools to make using Hadoop easier to use."
He added: "CIOs won’t have to hire high-priced Hadoop consultants to get projects done. Hadoop projects will get done faster because the enterprise’s very own application developers and operations professionals know the data, the integration points, the applications, and the business challenges.
"Additional skills for more complicated applications, such as predictive analytics running inside Hadoop, can be built when needed over time."
8. Ease-of-use
Matt Davies, Head of Marketing, EMEA at Splunk, said: "As analytics on Hadoop become easier to use and more mainstream, anyone within an organisation will be able to take advantage of software to gain business insight from Hadoop in real-time, opening an organisation’s data to an entirely new audience. Key to this will be self-service analytics capabilities and the ability for anyone to ask the questions of big data regardless of their job role and technical know-how."
He added: "We are increasingly starting to see the rise of PAYG pricing plans with providers such as AWS, meaning the initial investment in terms of software, infrastructure and skills can be minimised. Companies will therefore have the freedom to experiment with Hadoop through ‘Big Data as a Service’ to demonstrate ROI and evaluate the possible return of a bigger investment. "
9. Larger scale deployments for 2015
Matt Aslett, research director of data platforms and analytics at 451 Research, said: "While we see tremendous interest in Hadoop, early adoption has been driven by isolated, tactical projects serving single applications often driven by developers and shadow IT or innovation labs."
"That has already begun to change among the very early adopters, with those tactical projects moving from POC into production and senior IT executive green-lighting larger-scale deployments driven by strategic imperatives to expand data storage, processing and analytics in the name of becoming more data-driven. We expect this trend to expand in 2015 as the early and late majority begins to get in on the act."
10. SQL to become Hadoop’s killer app for 2015.
SQL, the data querying language tool used by application developers, will become one of the most prolific use cases in the Hadoop ecosystem, according to Forrester Research.
Forrester’s Mike Gualtieri said: "Fast and ANSI-compliant SQL on Hadoop creates immediate opportunities for Hadoop to become a useful data platform for enterprises. AD&D professionals already know SQL and many existing technologies can already interface to it. Certainly Hadoop has many more use cases than SQL, but many enterprises will start with it because it is the low-hanging fruit."
Actian’s CTO Mike Hoskins added: "To transform Hadoop into a high-performance, fully-functional analytics platform, business analysts need easy access to the data that lives in Hadoop via SQL. As more organisations realise this need, we will see some vendors lose momentum in the industry as loosely-integrated or immature SQL-on-Hadoop offerings cave to those that can offer the performance, scale, maturity and end-to-end capability organisations need to finally leverage the long-awaited promise of big data with Hadoop."
