Are we living in a world of hyperbole when statements such as “the data warehouse is dead” are thrown around? Or are we experiencing the rebirth of a modern data warehouse? – asks Simon Field, CTO UK, Snowflake?
Many have been left tired and scarred by clunky and capacity-constrained legacy data warehouses. Traditional, on premises systems have delayed access to data at a time when the world is becoming more data-driven and analytical speed is seen as a necessity to stay relevant and competitive. Beyond inefficient delays, legacy systems have also racked up significant operational and maintenance costs, resulting in many sleepless nights for IT and financial departments.
While this may seem like a forgone conclusion, advances in cloud technologies have revolutionised data warehousing, and given the data warehouse a much-needed facelift. At its very core, cloud computing is the catalyst in driving new innovations to equip businesses with faster, more intelligent insights.
The New Sharing Economy
One of the biggest trends emerging from cloud data warehousing, is the seamless experience in accessing and analysing internal and external data in real-time throughout the business ecosystem, as the notion of data sharing revolutionises the data economy. This removes the silos that often existed with legacy solutions.
Modern data sharing instantly levels the playing field for any organisation wanting to share data and consume shared data across its business units, with its customers and business partners, and to share and monetise governed data sets. Now, new business models and market opportunities can emerge which couldn’t previously be imagined.
Exemplifying this, UK charity Parkinson’s UK is capitalising on its cloud-built data warehouse to share relevant insights from its work with other research institutions in real-time. This is helping to accelerate the advancements and communication about the condition. Not only is this data being shared internally, but strategically shared with external partners such as pharmaceutical companies, universities, and other charities. Without the cloud powering this data sharing economy, tackling this condition simply wouldn’t happen fast enough.
Adding a Layer of Intelligence
While real-time data sharing is a major step forward for organisations, the cloud is also helping to tackle the understanding of vast amounts of data generated for businesses. According to research from IDC, data for organisations is growing at an alarming rate and is expected to reach an estimated 175 zettabytes worldwide by 2025, up from 33 zettabytes in 2018.
Artificial intelligence, or at least the more readily applicable machine learning models, are helping sift through vast volumes of data looking for important signals enabling organisations to optimise their businesses, thus removing the archaic approach of wading through data manually. Thankfully for most organisations, many of the largest cloud vendors such as Microsoft and AWS are already investing significantly in harnessing the power of machine learning tools, minimising the requirement for businesses to have deep expertise of the concept.
Much like the infrastructure of a cloud-built data warehouse, which is based on a consumption model rather than fixed charges, when dealing with large data sets, organisations can scale up processing and pull out the most interesting insights from data as and when required. By combining cloud and machine learning models, data scientists are shifting from spending 80% of their time gathering and cleaning data to create usable modelling features, to spending all of that time in productive analysis – a significant step forward in improving efficiencies.
Undoubtedly, there will always be an air of the ‘fear of the unknown’ when migrating to the new wave of cloud data warehouses. But the benefits far outweigh the apprehension. With cloud powering the rise of the data sharing economy and machine learning, data insights are being realised faster than previously imagined. The data warehouse is far from dead, instead, it’s a thriving solution which has been reborn again, thanks to the cloud.
Read this: Five Questions with… Snowflake Co-founder Benoit Dageville
