View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Cloud
September 15, 2015

YouGov utilises MongoDB for 70% storage reduction

Organisation uses MongoDB Enterprise Advanced.

By James Nunns

YouGov is implementing MongoDB to tackle its data in order to improve user experience.

YouGov, which engages in continuous conversations with millions of consumers in order to understand opinions, behaviours and preferences, has made MongoDB 3.0 its default non-relational database.

Previously reliant upon an internal database known as FastStore, the organisation migrated in order to gain a 70% reduction in storage.

The database it now uses serves Gryphon, a survey system built by YouGov. Jason Coombs, Executive Technical Director at YouGov, said: "All our surveys are stored in MongoDB, with each document capturing a user’s responses and activity for a questionnaire within an interview session.

"This raw data then serves our data products. We ETL (extract, transform and load) the data into our custom-developed column store for analytics, and export relevant surveys to our clients."

Once the data is ingested, the data is served to its products such as BrandIndex, which is a brand perception tool, or Pule, a consumer behaviour tracker.

The organisation captures on average 2-4GB of new data every hour, which can increase by 3x that amount at peak.

Content from our partners
How to turn the evidence hackers leave behind against them
Why food manufacturers must pursue greater visibility and agility
How to define an empowered chief data officer

Discussing the challenges of this deployment, Bryan Deeney, Senior Developer at YouGov, told CBR: "MongoDB’s dynamic schema provides great flexibility which both enables very rapid prototyping, and facilitates continuous iterative development of applications in production.

"However, this doesn’t relieve you of the responsibility of employing good schema design principles. As your application evolves over time, you need to regularly evaluate the changing demands placed on the database, and tweak your schema and indexing strategy accordingly.

Sometimes using new tools requires learning an equally new way of working. As someone previously accustomed to working with relational databases, I personally experienced this challenge when I first began working with MongoDB.

"To get all the benefits, it takes a real shift in philosophy and approach. You can’t simply replicate your previous ways of thinking about data. You have to envisage data in a different way. Be that learning to embrace denormalization where appropriate, or better understanding exactly what it means to have a flexible schema."

YouGov currently uses MongoDB Enterprise Advanced, which includes advanced software, support, certifications and licenses. The company’s Cloud Manager runs and automates the backup and deployment.

Joe Morrissey, VP of EMEA MongoDB said: "Seeing one of our earliest adopters weave MongoDB into a culture of constant innovation is gratifying. And it’s inspiring to see how that culture is delivering concrete benefits that propel the business forward. It’s a team we could all learn from."


Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.