View all newsletters
Receive our newsletter – data, insights and analysis delivered to you
  1. Technology
  2. Cloud
August 17, 2017

From Redshift to Snowflake: How Deliveroo overcame bottlenecks to deliver data success

With data being essential to its business, Deliveroo needed a solution to overcome bottlenecks it was dealing with.

By

Deliveroo, one of the most popular food delivery companies, has been going through something of a transformation.

The organisation, which is now introducing new technology that it says will “revolutionise food preparation and delivery in UK restaurants,” decided earlier in the year that it would migrate its data warehouse.

Henry Crawford, head of business intelligence at Deliveroo.

Previously sitting in Redshift, Deliveroo’s data has now been moved to Snowflake’s cloud data warehouse.

The company, who’s infrastructure sits on AWS, had decided to move due to a number of problems that had begun to arise with its use of Redshift, partly caused by the company’s rapid growth, it grew by 650% last year.

Henry Crawford, head of business intelligence at Deliveroo, told CBR: “We had collectively experience with it in previous organisations before Deliveroo and had similar problems. The concurrency problem for example, for a company aggressively growing, to have a known and obvious bottleneck in getting info out to the company, it’s the definition of not being scalable, and it wasn’t for a lack of trying.

“When we grew to such an extent the demands on data warehouse grew faster, growing employees and more complex queries and questions coming in as the business learns and gets faster,” all contributed to a need to change the technology that was being used.

The food delivery company is the definition of a data driven business. It uses vast amounts of data improve its services and to operate on a daily basis.

Content from our partners
Harnessing the power of low code and no code development
Signs your accounting software is no longer fit for your growing business
Incumbent banks must transform at speed, or miss the benefits of open banking
Read more: The data don’t lie: Using machine learning to fight insurance fraud

Earlier in the year at the AWS Summit in London the company revealed that it deals with 17 terabytes of compressed data, 1.8bn historical events, has 450+ active Looker users, and deals with 7000 warehouse queries per hour, and the daily count is said to be much higher.

Crawford said that around 90% of employees have access to the data and use tools such as Looker regularly, meaning that every single user will be using data to improve performance. “Using logistics is core to how we operate and crucial to making sure the customers are happy because we have to make sure the food is delivered in a timely manner.”

Far from being a one off change, Deliveroo is constantly assessing the technology it uses. The introduction of technology created by US food company Maple, which Deliveroo acquired in May this year, is the latest example.

The Deliveroo Toolkit, which is due to be introduced in Deliveroo Editions kitchens this year, and later to its 25,000 restaurant partners in 2018, is said to feature chefs with preparation time information on an item level basis, so that preparation and cooking speed can be optimised.

The toolkit also includes chefs and managers with a real-time kitchen overview dashboard so that bottlenecks can be identified and acted upon. There is also functionality to help restaurants more effectively manage their purchasing operations by predicting weekly order quantities.

Crawford told CBR that it constantly assess the technology that it’s using, “because you get new requirements all the time around SLAs of data, the quality of data, the quantity to be able to query. So it’s only ever going to grow and you’ve got to understand the data you are bringing in. Not just bringing it all in but bringing it in so it’s easy for people to access, aggregate, analyse, and make decisions on.”

Read more: Best hybrid cloud vendors: Microsoft vs IBM vs VMware vs AWS

The head of business intelligence said the end goal is having “data on the phone…we want to have something actionable at the end, something that makes it so you don’t have to manage it, makes it scalable, and makes it easier to manage it.”

The Deliveroo Editions project, mentioned above, is one use case that has been enabled by the move to Snowflake.

The technology change makes it easier for the company to use its data quickly, rather than battling through bottlenecks. 

Deliveroo Editions is where the company launches restaurants in areas where demand isn’t being met, such as in Camberwell.

“The challenge and interesting part is finding an area, or part of the country, that is missing, for example, a sushi restaurant, and we will partner with a chain and help it to expand into new areas,” said Crawford.

The decision making is data driven, based off transaction data over the last four years and what customers have been buying or wanting to buy. The company then brings in velocity on road data so that the restaurant can be set up in a good place.

“Snowflake helped bring in many new data sources that we think would be useful, customer profile, rest types and build models,” Crawford said.

The decision making for the technology change sat with the data engineering and analytics team, and due to the slow running nature of the Redshift technology, it was an easy decision to make.

The harder part of the decision making process was to forecast how much would be used, a compute per use basis.

Fortunately, the company also has “highly accurate models” that are good for short range forecasting, so that it can predict growth rate, but out to two years it becomes much harder. Decisions have to be made with the finance department, marketing and assumptions on how users will use the data and what customer volume will be in a year’s time.

“Intrinsically it’s hard, a lot of algorithms get run,” said Crawford.

Deliveroo is the perfect example of a technology migration caused by rapid expansion, the processes were in place to make it a smooth transition, and because the company had the right structure in place it could carry on business as usual.

Websites in our network
NEWSLETTER Sign up Tick the boxes of the newsletters you would like to receive. Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
I consent to New Statesman Media Group collecting my details provided via this form in accordance with the Privacy Policy
SUBSCRIBED

THANK YOU