In a world where billions of interconnected devices communicate with countless cloud services on everything from digital transactions, to network log-ins, to room temperature, data has become a critical tool for generating ideas, problem-solving and above all, driving profit, writes James Hodge, Chief Technical Advisor, EMEA at Splunk.
James Hodge
The Economist declared data to be the world’s most valuable resource, and Forrester calls it “the new currency of business”.
The power of data relies on the ability of a business to combine disparate data to cover new insights, and leveraging analytics tools and artificial intelligence (AI) is crucial in this process. However, according to Splunk’s recent ‘State of Dark Data’ report, 55% of an organisation’s data is “dark” – unquantified and untapped. In a word, it is useless.
Many argue that we are entering a data revolution, where the success and failure of organisations will be based on how well they can utilise data and AI for business performance. Unlocking the benefits of dark data will involve strategic thinking, targeted technology investment, and comprehensive skills training.
Why Does Data Stay Dark?
Within the dark data of a business is records of everything the company needs to know about its market. It includes information on, transactions, applications, servers, networks, and mobile devices, all of which can be leveraged to understand factors that impact revenue such as customer preferences, purchasing trends and sector challenges.
In the majority of cases, organisations understand the value that data can have but do not invest the time and resources necessary for preparing it for use. Many other businesses simply do not understand the potential of data, and therefore do not see it as a business priority. Then there are those that are bogged down meeting day-to-day requirements rather than looking ahead for opportunities.
Our report found that 80% of business leaders believe their data is valuable, despite more than half of it being dark. The reasons for the difficulties identifying and recovering dark data are both technical and organisational. Companies are overwhelmed with data and short on the right talent and tools, while also facing internal challenges like company silos and unresponsive leadership.
Content from our partners
How Can We Turn On The Lights?
To tackle these barriers and start unlocking the benefits of dark data, businesses need to make data strategy a key organisational priority. When asked about the primary reasons behind the ongoing struggles to maximise data, 76% cited a lack of training for current employees, and 66% a lack of interest from company leaders.
This illustrates how data strategy, and the pursuit of dark data, has to be regarded as an essential business philosophy. As a starting point, the leaders of a business need to be educated on the value of data and the insights they can bring. From there, the business leader needs to take responsibility for driving advocacy across the organisation.
This starts with the introduction of training sessions to develop data skills – with the implementation of these programmes today, employees will have the rich-tapestry of skills to start generating positive business outcomes. They also need to review existing recruitment strategies. When hiring staff, businesses need to ensure that data skills are ingrained in the day-to-day responsibilities of the role.
What is the role of artificial intelligence?
The significance of skills and culture in driving data insights is critical, but humans alone are unable to derive insights at the scale and speeds required to allow a business to quickly adapt to industry trends. For this reason, businesses also need to be looking at the application of AI solutions.
AI technologies can be utilised when data is too varied and voluminous for humans to efficiently process. And a central value of AI is its ability to work with vast quantities of data at very high speed. Give AI more information to analyse and it can produce deeper, more accurate insights with greater business benefits. The value of data and AI therefore goes hand in hand – dark data could one day be an accelerant for even greater AI performance.
Between 61–67% of organisations see value in AI technologies for individual business use cases including operational efficiency, innovation, strategic decision making, recruitment, and customer experience. However, only 10-15% are currently deploying it for such functions. As industries become increasingly digital, businesses need to start readying their data systems for the AI era.
Why do businesses need to act now?
Businesses are currently in a conflicting position around the use of data. On the one hand, they recognise that dark data and artificial intelligence holds almost unlimited potential to transform business and society, and on the other, business and IT leaders have low confidence in their knowledge about AI and the data their company possesses.
However, having already witnessed a period of considerable digital transformation, we know how data is driving product development, supply chain, customer experience and overall business strategy to unprecedented degrees. The pace of business is much faster, and already real-time data and forward-looking predictive analytics are becoming common.
There is no question that further, and faster, transformation lies ahead, and organisations cannot afford to wait for this to take place. Only as we become more comfortable with the amount of data at our fingertips, and the things we can do with it, will we begin to uncover the greater potential of our dark data.
