View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Data
March 20, 2015

Big Data doesn’t equal good data

Ed Wrazen of Trillium Software looks at data quality in Big Data.

By Cbr Rolling Blog

The appeal of Big Data is the opportunity to drive business improvement through better insights gleaned from a larger variety and volume of information than ever before. Big Data brings the potential for better customer insights, more detailed analytics and tighter risk and regulation management.

The ability to aggregate and analyze diverse datasets can results in revenue increases, more operational efficiency and a strong competitive edge over other industry players. According to Gartner, organisations integrating high value, diverse new information sources into a coherent information management infrastructure will outperform their industry peers financially by more then 20 percent.

However, whilst the potential of Big Data is alluring, myriad data sources raise concerns around quality and validity. A well-held belief amongst some business professionals is that, by virtue of sheer volume, quality errors in Big Data are less impactful. However, Gartner VP Ted Friedman recently fought back against this myopic view of Big Data: "In reality, although each individual flaw has a much smaller impact on the whole dataset than it did when there was less data, there are more flaws than before because there is more data."

Many CIOs are struggling to verify the large amounts of structured, unstructured and semi-structured data yielded from an increasingly diverse range of sources. Moreover, because much of today’s data is not validated at the point of creation, businesses are finding it difficult to implement a systematic data management approach that delivers data analysis, validation and assurance of Big Data before it’s utilised across crucial downstream applications such as CRM systems and business intelligence and analytics platforms.

Increasingly organisations are relying on third party data and information sources outside the business, such as social media streams, that are often unreliable, fragmented, and not automatically linked to existing internal data records. In the formative years of traditional analytics data quality was a costly afterthought, and it’s crucial that anyone establishing Big Data operations avoids repeating those mistakes.

To overcome these challenges organisations need to adopt a more thorough data management approach that enables seamless integration of data regardless of structure, source or application. Data science and predictive analytics applications can then query across unstructured data sources (where it was not possible before) and improve processing of previously inaccessible data. Business professionals can then more easily identify the right data to solve their specific business problems and align the right capabilities and skills to exploit the data in a way that positively affects their bottom line. For instance, from a CRM standpoint, matching data to individual customers can drive personalisation and provide an improved customer experience.

In order for Big Data initiatives to yield tangible business results, businesses need to manage the quality and validity of business information compiled in new storage systems such as Hadoop. To do otherwise risks producing misleading results. The old axiom "better data means better decisions" is just as important when dealing with Big Data, and if your data is to become a trusted information source and useful tool for employees, partners and customers, a robust Big Data governance framework is a must.

Content from our partners
Infosecurity Europe 2024: Rethink the power of infosecurity
Rethinking cloud: challenging assumptions, learning lessons
DTX Manchester welcomes leading tech talent from across the region and beyond

Organisations are beginning to wake up to this reality. A recent survey indicated that whilst only 11 per cent of IT leaders are currently looking into data quality, this percentage is expected to grow to 55 per cent in the next three years.

Big Data is a huge challenge for organisations but also an incredible opportunity to improve process efficiency, create more personalised customer experiences and drive innovation. Organisations that handle Big Data appropriately will undoubtebly be at the forefront of business innovation, leading the way in customer engagement and service quality.


By Ed Wrazen, VP Product Management: Big Data at Trillium Software


Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how Progressive Media Investments may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.