YouGov is implementing MongoDB to tackle its data in order to improve user experience.
YouGov, which engages in continuous conversations with millions of consumers in order to understand opinions, behaviours and preferences, has made MongoDB 3.0 its default non-relational database.
Previously reliant upon an internal database known as FastStore, the organisation migrated in order to gain a 70% reduction in storage.
The database it now uses serves Gryphon, a survey system built by YouGov. Jason Coombs, Executive Technical Director at YouGov, said: "All our surveys are stored in MongoDB, with each document capturing a user’s responses and activity for a questionnaire within an interview session.
"This raw data then serves our data products. We ETL (extract, transform and load) the data into our custom-developed column store for analytics, and export relevant surveys to our clients."
Once the data is ingested, the data is served to its products such as BrandIndex, which is a brand perception tool, or Pule, a consumer behaviour tracker.
The organisation captures on average 2-4GB of new data every hour, which can increase by 3x that amount at peak.
Discussing the challenges of this deployment, Bryan Deeney, Senior Developer at YouGov, told CBR: "MongoDB’s dynamic schema provides great flexibility which both enables very rapid prototyping, and facilitates continuous iterative development of applications in production.
"However, this doesn’t relieve you of the responsibility of employing good schema design principles. As your application evolves over time, you need to regularly evaluate the changing demands placed on the database, and tweak your schema and indexing strategy accordingly.
Sometimes using new tools requires learning an equally new way of working. As someone previously accustomed to working with relational databases, I personally experienced this challenge when I first began working with MongoDB.
"To get all the benefits, it takes a real shift in philosophy and approach. You can’t simply replicate your previous ways of thinking about data. You have to envisage data in a different way. Be that learning to embrace denormalization where appropriate, or better understanding exactly what it means to have a flexible schema."
YouGov currently uses MongoDB Enterprise Advanced, which includes advanced software, support, certifications and licenses. The company’s Cloud Manager runs and automates the backup and deployment.
Joe Morrissey, VP of EMEA MongoDB said: "Seeing one of our earliest adopters weave MongoDB into a culture of constant innovation is gratifying. And it’s inspiring to see how that culture is delivering concrete benefits that propel the business forward. It’s a team we could all learn from."