The benefits of big data analytics are becoming increasingly well understood. In our Information Generation study, two thirds of business leaders say that valuable insights are driving their organisation to rethink how they do business, and big data analytics is cited as the number one trend driving the way their businesses operate internally. In fact, around 49% percent of businesses know they can get more valuable insights from information and data, but don’t know how. One specific challenge is infrastructure: in the Big Data League study, 42% of respondents stated that IT infrastructure limitations are actually holding them back.
The starting point for any analytics programme of scale is to consolidate data from operational silos into a single environment. After all, the greater the data set, the more valuable the results and insights will be. As businesses embark on their digital transformation journey, they need to consider how to consolidate data siloes to gain the full value analytics offers. This will help inform business decisions and evolve the business’ strategy in the future.
For anyone aiming to benefit from a data consolidation project, there are seven critical boxes that need to be ticked:
1. Ensure you have the flexibility to support multiple big data distributions
The big data analytics platforms your organisation uses today aren’t guaranteed to be the ones you’ll want to use tomorrow. Whether you’re dealing with Cloudera, Pivotal, IBM, Hortonworks or another player’s platforms, in the future, you may well have different requirements which may require a different platform.
As the key providers start to specialise, you may decide to deploy several platforms to capitalise on different types of analytics capability and expertise. Additionally when consolidating your infrastructure, ensure that you enable ‘multi-protocol’ support, giving the agility to switch between analytics platforms seamlessly, or use multiple applications at the same time.
2. Work to have a less cluttered data environment
With multiple data sources in a single environment, a business won’t need multiple copies of the same data. By having your data co-existing in two places, you create a range of challenges in terms of ensuring the data is always kept coherent between where it lives and where it is analysed, and you run risks of in-flight data corruption, out-of-date data being processed, and other errors emerging.
If you can work with a single copy of the data in a data lake environment, then it will make the operational management simpler, improve productivity and make compliance more straightforward, oh and provide better insights. By removing multiple copies of the data, your organisation will have less to store and much faster access to it.
3. Bring analytics to the data
If you move to a true ‘data lake’ environment, you also remove one of the most painful processes for analytics: moving large quantities of data. When businesses no longer need to move data sources in and out of their source applications into an analytics-friendly platform, they free up lots of resource for actually driving the programmes themselves.
Rather than bringing data to the analytics platform, by bringing analytics to the data environment, there is no need to export and import data, resulting in a much less complex process, a reduction in overheads and a much faster time to delivering business insights.
4. Build in security and compliance
With more organisations regularly headlining the news after becoming a victim of a cyber-attack, breach or leak – and with more stringent regulations being put in place for data protection – it’s more important than ever for businesses to have enterprise class features built into their data environments. Any form of data must be treated equally to how you would secure sensitive information.
The encryption of data should be subject to all regulatory and procedural processes and be an integral part of the overall security policy. This allows companies to have a trajectory of information and a formal, controlled authorisation process in place for use of data. Of course, organisations still need to operate ethically in their analytics programmes and ensure their customers are informed on how their data is likely to be used.
5. Extend your data pool from the core to the edge to the cloud
A lot of data centre projects start and end in your data centre. After all, it’s where the majority of your organisational data lives. However, as increasing amounts of ‘archive’ data are shipped off to the cloud, and as branch offices maintain a proportion in local data silos, it’s increasingly important to factor them into your consolidation plans if you want to get full value from analytics across the organisation.
Connecting multiple data environments means that businesses can gain insights from a single pool of data. After all, the ecommerce data held in your data centre won’t tell you anything about customer behaviour at a branch, and both will be vital concerns for future business strategy and change.
6. Expect large scale growth… and then expect it to be larger than you think
With digital technologies providing a more central part of the way we do business, and organisational data growing upwards of 40% a year, any projects you have need to have room to grow without the need for football fields of storage infrastructure. Worse still, with traditional legacy data architecture, businesses may require an army of full-time employees to manage it. For small environments, this may be fine, but given the growth of data, few enterprise environments will remain small for long.
7. Adapt to future data environments
Building an environment where you have multiple data silos in a standardised architecture enables businesses to adapt to any possible future environment – and the speed to adapt will be vital. By getting your ‘house in order’ and managing this process, it means than data can easily adapt to change quickly in the future. Those most agile will be those that grow and succeed against competition.
There’s no question the analytics opportunity is immense, and a vital consideration for businesses as they evolve. However, as we start to see more projects moving from proofs-of-concept to full deployments, longer term planning will be vital. And managing a process of data consolidation has to be top of that list.
