Finding the right database to use is more difficult than ever due to the numerous offerings available in the market. No longer do businesses have a choice between two or three but dozens.
One business that had a database decision to make is OpenX, a company that creates programmatic advertising to help deliver optimal value to buyers and sellers. The company helps companies such as Forbes, EA, and Samsung.
CBR spoke to Matt Davis, Lead Site Reliability Engineer, OpenX about its decision to deploy the Riak NoSQL open source database which is created by Basho.
JN: Why did you choose the Riak NoSQL database?
Davis: “OpenX chose Riak-KV for several reasons, eight separate use-cases worth of reasons in fact. Not only is there the obvious need for a highly available key/value database reliable enough to serve as a front-end delivery data-store, but we also use it heavily in rugged multi-datacenter data pipeline systems that provide both metadata and serving statistics back into our ad-tech stack.
“Everything from traffic and ad quality to audience enrichment and publisher configurations is kept in Riak. We also employ a Riak-S2 storage cloud available to all our globally distributed datacenters.”
JN: What are the benefits that open source provides to your business?
Davis: “OpenX started as an open source project, and though our current business has evolved, we embrace open source software as part of our core infrastructure. In addition to Riak-KV, we have a heavy investment in such platforms as Apache Hadoop, HBase, Kafka and Spark.
“Our internal software is built with Erlang, Python and Java. We configure with SaltStack integrated with Git. There are containers, there is CentOS. We use these things not only because they work well, but also because we take an active part in those communities.
“Many OpenX Engineers – software and SRE alike – contribute to these FOSS projects, and many of us have given talks at conferences and written blog posts about them. Our people benefit from collaboration with the community, and our highly regarded programmatic platform benefits from the flexibility and customization of the open source software we leverage.”
JN: How does the technology influence the way you work?
Davis: “Well, really it’s in part of everything I do. Work, art, life, everything. From answering these questions to interacting with my distributed team to writing blog posts on the train and chatting with my family during the work day. Even the way I’m always “on-screen” the entire day at work directly influences my off hours.
“Instead of composing music on the computer, I build hardware and electro-acoustic instruments instead, play free improvisations in the studio instead of staring even more hours into screens.
“Technology pervades our lives, for better or for worse, and I try to find a balance to love all aspects of it. It makes me more agile in life, and for a socially awkward person like me, has actually helped me enrich human relationships in and out of work. I do not own a smart watch though.”
JN: What database were you using before Riak? Why did you change?
Davis: “Riak has covered a few use cases for us that may not have ever existed in any other database. The Riak-S2 storage system is one that stands out. In some cases early on, we migrated off Cassandra because of Riak’s superior reliability.
“Constant availability is essential in ad serving, and once we saw Riak in action against network partitions and node failures in addition to gaining more and more operational expertise, it started to become a no-brainer for other K/V use-cases.
“Yes there have been the occasional NoSQL newbie missteps, so we have also invested in education programs for teaching developers how – and where – it is best used.”
JN: What other databases did you evaluate before making your choice?
Davis: “In line with the question above, at some point we gained critical mass with Riak-KV and it has become our K/V system of choice, especially when considering the need to scale as linearly as possible.
“We have very deep experience running very large clusters, and developers have come up with some very creative ways to employ Riak in their designs as an alternative to other NoSQL databases.
“There are some instances where Riak was not the right choice, and that is fine too. The important thing is that you understand your use-case and choose a technology that meets your organization’s criteria (like participating in a strong community) and expertise in both ops and dev.”
JN: Did any particular challenges arise during deployment?
Davis: “There are not many challenges when deploying Riak itself, especially when you have as homogenous an environment as possible with a robust configuration management system.
“The real work is setting up instrumentation and visualization to help you recognize how your use-case operates within the database, what rhythms are created by the data, and what way early design choices and configuration settings morph and change as you scale.
“It is truly setting an ocean of data free to oscillate against the storm of indeterminacy that defines the ad-tech world, and Riak has proved to be a trusted foundation for OpenX to ride those waves.”
