The IT industry is used to supplying more for less. Moore’s law tells us this. More and faster processing is accepted norm. And in the storage market there have been two constants. The first is about demand which says demand for digital storage capacity rises and the rate of demand accelerates. The second is that the price per unit of storage, byte, kilobyte, megabyte, gigabyte, terabyte always falls.
However we can reached the point where these norms are being challenged.
The trajectory is the same, demand will still rise and prices will continue to fall, but there are changes coming which will shift the tectonic plates of digital storage and of IT economics in general.
IT economics, an emerging discipline, being led by people like Dave Merrill, Hitachi Data Systems Chief Economist does not react to short term stimulus such as the recent gyrations in the world’s stock markets.
In the medium term whether in an upturn or a downturn the scale of new IT demand is pushing us to whole new IT architectures and cost and value models.
This new IT demand can only be met with a radical shake that will require an 85% cut in the cost of delivering IT.
IT economics is in some ways timeless and ‘not supposed to react to gyrations of stock markets, monetary policy’ and the like says Mr Merrill. It is not immune from the macro-economic environment but even traditional demand for IT will continue to grow because traditional demand drivers don’t change and new drivers point to exponential growth, he says.
Compliance and regulation is not as elastic as markets. If we’re talking to the telcos, hospitals and banks, demand is somewhat immune from the immediate impact of any downturn. They still have to keep records for 100 years. Those requirements don’t change and we saw this six or seven years ago during the crisis when behaviour changed but not demand, says Mr Merrill.
But today’s real opportunities are being driven by unprecedented demand which is itself going to be based on the new digital infrastructure. It has the catch all name of Internet of Things but that covers a multitude of advances and innovations.
A Critical Juncture
The real opportunities are now to be found in big data, IOT, and M2M. Today is when we build and deploy IOT infrastructures. These will generate IT and storage demand at a significantly larger scale than those of the traditional enterprise.
For IT, these new emerging platforms are about keeping pace with unprecedented data volumes.
"This is the rub of the new IOT economics. We can’t afford to run big data, IOT or M2M projects on traditional infrastructure," says Mr Merrill.
"These storage infrastructures that we build for IOT need to be roughly 15% per cent of the total of the cost of traditional infrastructure. So we’re talking about 85% of lower cost of ownership."
The vast data lakes being generated can’t be stored on RAID V or on commercial hypervisors. This is not traditional policy driven storage. It is totally new.
"There is a new kind of M2M data, log files, analytics, surveillance, geo informational tracking. As firms are looking to IOT and big data there is a rate of growth you can’t afford to run on traditional storage. We’re talking 200 petabyte big data farms. Applications that require dozens of petabytes, not single petabytes. In your IOT environment it will be ten times the capacity.
Mr Merrill says "Let’s think about IOT for a moment. We’re talking about sensor, wearable or M2M data which for the most part requires no back-up , no disaster recovery (DR) and no management service. The data is transient. Your business won’t fail if you lose this, you just regenerate the data. This is a great opening for cloud service providers. This is the radically different approach required built on new architectures delivered by new providers."
Data will be erasure coded, we will use highly dense 6 terabyte drives, KVM kernel, very highly densely packed infrastructure on commodity nodes.
The cost argument opens up a discussion on what is the cost of delivering IT to business today. Something that too few organisations have visibility of.
"Hitachi can offer a quick service to help. When they realise the cost of existing IT and then start to think about the cost of finding the value in big data it can cause a shock."
At the moment many organisations think of big data is a back office science project – fire up Hadoop, work on Splunk and begin. But it is not a standalone IT project. The question is what’s the value to the business? What will you learn about your customer from big data? Show me yourbusiness plan for big data and for IOT. There are few well thought through plans.
Where’s the money? (and other challenges)
What’s the business case for building IOT and big data farms," says Mr Merrill.
The squeeze at the moment is no-one yet knows the revenue streams that can be generated with all this data. How can it make money?
"Businesses are putting plenty of effort into it but they don’t know where it [revenue] will come from yet. So there is some IT guy coming in and asking for 5000 VMs and 5 petabytes of storage and they may not have a business model or a data scientist to mine it," he says.
They may build a Proof of Concept but then they realise that they can’t afford it at scale. And what you can’t afford is beyond the cost of acquisition. You can’t even afford the cost of power and cooling. It requires a radical review of IT.
"The business often doesn’t care about the IT side. They’re saying we can learn about the user data and sell it but this is an opportunity for IT to lead the business in finding true value of data and turning into a revenue stream for the business," Mr Merrill says.
How quickly can new IT happen?
Mr Merrill believes how fast the new IT can happen depends how fast the big data and IOT roll out will happen.
"We can put them into a sandbox in the cloud. We now have the basic economics to help them. The pieces are in place but businesses must ask themselves what is their rate of acceleration for doing IOT? The shortage is not in the data centre, in storage or in software. The shortage is in data scientists. Look at how firms such as Google, Amazon or Walmart are playing. The real rub will come from how we use the data. There was a splurge of data visualisation tools but we’re not yet pumping out graduates with qualifications to pull value from data.
In conclusion the key point is that Mr Merrill believes you can’t afford to run IOT and Big Data IT in current environments. The proof point is showing the business that they can run at an 85% lower cost.
The days of bad economics and bad planning are over.
He says, "The pull is we’ve got to understand the costs and recognise the value of data. We have to go in with eyes wide open on the investment case and on return of investment."
