The UK is in the vanguard of open data. Sirs Tim Berners-Lee and Nigel Shadbolt founded the Open Data Institute as a catalyst for "the evolution of open data culture to create economic, environmental, and social value".
In July 2014, Vince Cable, the-then Business Secretary announced that Companies House planned to make all of its digital data available free of charge, and in October it won international recognition for producing the first free companies’ accounts database in the world.
Companies House, along with Land Registry, Met Office and Ordnance Survey, has formed the Public Data Group that in its Summer 2014 Statement on Open Data estimates "the value of Open Data released by PDG at over £900m annually".
Finally, in June 2015, Companies House fulfilled Cable’s earlier promise by announcing that all public digital data held on the UK register of companies is indeed now freely accessible, on its new public beta search service.
The result is that a goldmine of more than 170 million digital records on companies including financial accounts company filings and director details is now freely available for businesses to sift through and search.
The work already done by Berners-Lee and Shadbolt and by the Government in supporting the Open Data Initiative is hugely positive, of course. Making public data sets freely available for analysis enables organisations to access and combine them with their own internal data to gain new commercial insights that would not have been achievable through viewing either data set in isolation.
Moreover, this capability potentially allows organisations to use information to drive innovation and generate enhanced economic, environmental, and social value. But ensuring data is more readily available is just one part of the battle, organisations also need the tools and technologies to extract value from it.
The key question is how do they do that? Last year, on the Forbes CIO Network website, Brian Ascher, partner at venture capital firm Venrock, noted that part of the technical solution was now in place. "In the last few years we’ve had a "Big Data" explosion and a host of new open source technologies like Hadoop, MapReduce, and Cassandra packaged by a set of new companies that help businesses manage and manipulate their ever expanding mountains of data".
However, Brian went on to observe that "for the most part, all of these business applications and more recent Big Data tools have left the burden of capturing real business insight, making decisions, and taking action on the business customers themselves".
Having identified a gap in our current technical capabilities, Brian then visualises a future: "A new breed of software company is emerging, however, that combines data science expertise with deep understanding of business problems. I call them Data Driven Solutions".
With the business landscape now fertile for open data, this is a timely moment for the emergence of this new brand of software solutions. After all, these data-driven applications are set up to ingest open data from a wide range of internal and external sources.
But they then go several steps further – extracting the entities of interest, establishing relationships between them; applying knowledge of what is important to the business community being served, and then analysing the emergent situation to present answers. Open data is effectively the raw material, data driven software the technology that makes sense of that data and turns it into business value and insight.
This knowledge, which is increasingly drawn from open data sources, can be presented to a human audience to make a decision, for example a deep analysis of an application for a business loan unearths a history of defaults that demand further consultation with applicants. In such cases, data-driven applications employ interactive data visualisation to help businesses rapidly understand situations informed by multiple data sources and drive fast time to insight.
We are already seeing many new commercial applications for this new breed of data driven software: data visualisation tools, or visual information management systems. Within professional services such as law, banking or insolvency for example, existing manual approaches to asset search are cumbersome and slow. In these cases, visualisation can cut time spent and costs incurred from the outset – but it’s far from the only use case.
So here is the new value chain: vast quantities of electronic information, much of it freely available as open data, plus data driven software, plus business process innovation. It’s a potent combination, yet it’s rapidly becoming a realistic proposition that I predict will continue to benefit technology-driven businesses of all types.
