Big data is a big deal. Like ‘cloud’ the phrase has taken on a life of its own quite outside what it actually entails, and gained a grinding inevitability to match. As Matt Quinn, CTO of TIBCO put it, every single app, hardware or company you use will soon be collecting lots of information.
But some believe there is a problem. "The majority of info that gets created today is consumed in the moment and then discarded," said Quinn. He believes that for big data to influence the minutiae of business something has to change. "The challenge with big data is that it has not gained relevance to your everyday life, to the operational decisions you are making on a daily basis."
The view is not universal. Tony Baer, principal analyst at Ovum said: "I have a huge amount of respect [for Quinn], but I think what he’s saying there is most businesses haven’t really got a handle on big data yet." Of course Quinn has a product to hawk, and this product is fast data – a phrase that Baer claims to have coined in 2011.
What is fast data? According to Quinn it’s "the ability to process big data in real time, which allows us to gain instant awareness of what is going on in this particular moment in time, and take appropriate action". In other words it is not the sort of data you leave an analyst to mull over for a couple of weeks, but data that is useful in the moment, giving you a very specific advantage.
The concept evolved on the basis of research conducted by Douglas Laney, who wrote about volume, velocity and variety, or what became known as the three Vs. Arguably fast data is already used by financial firms, trading stocks at the speed of light as described by Michael Lewis in his recent book "Flash Boys", but it has potential to move into more industries.
Quinn calls it both an "evolution" and "revolution" in big data, and that is not merely a marketing ploy. It could make airplanes more reactive to weather changes, or allow hospitals to monitor how its doctors and nurses are using soap dispensers. But it has its limits.
Content from our partners
"There are going to be industries that won’t necessarily need big data, and there will be industries that don’t really need fast data," Baer said. He gives life insurance as an example of an industry where instant processing is not much of a boon. Analytics firms want everybody to believe they need more data tools, but it really depends on the business. "You don’t want to adopt a ‘build it, they will come’, ‘field of dreams’ approach to this," Baer added.
But if fast data has been around for a couple of years, why is it only starting to flourish now? Baer estimated that vendor attention has only really been caught in the last year , perhaps a year and a half. "It’s not that we haven’t had this," he said. "We have had the ability but until now it required very extensive proprietary technology."
It is only now that processing the data has caught up with collecting it, a reminder that data is useless without analysis. As Quinn put it: "Many people realise that in order to achieve fast data there needs to be a new kind of infrastructure. One that is not beholden to the architecture of the past."
Now that everyone from TIBCO to Oracle are looking to capitalise on data, the trend represents a major opportunity for firms looking to improve their business. Baer believes next year could be the year fast data breaks. "I would say by next year you’re going to see adoption. [But] I don’t think you’re going to see 60% of businesses using it."
