View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Data
September 4, 2018updated 05 Sep 2018 1:13pm

ACID Breakthrough for data Artisans

"Data-intensive, real time applications such as fraud detection, machine learning, and real time trade pricing can now be upgraded to the streaming era"

By CBR Staff Writer

Berlin-headquartered data Artisans, founded by the creators of data stream processing engine Apache Flink, this morning rolled out a new patent-pending tech that makes fast, serialisable ACID transactions possible directly on streaming data; an industry first.

The “streaming ledger” (no, no blockchain is involved) can be used to power applications that require real-time decisions on streaming data, for example in financial services. It was revealed today at the Flink Forward Berlin conference.

In the context of transaction processing, the acronym ACID refers to the four key properties of a transaction: atomicity, consistency, isolation, and durability.

This means that A) all changes to data are performed as if they are a single operation; C) data is in a consistent state when a transaction starts and ends; I) the intermediate state of a transaction is invisible to other transactions, so transactions that run concurrently appear to be serialised and D) after a transaction successfully completes, changes to data persist and are not undone, even in the event of a system failure.

See also: Open Source Platform Aims to Democratise “Machine Learning Zoo”

The ledger builds on Apache Flink and provides the ability to perform serialisable transactions from multiple streams across shared tables and multiple rows of each table; “the data streaming equivalent of multi-row transactions on a key/value store or even across multiple key/value stores.”

The global streaming analytics market is projected to reach $47.75 billion by 2025, growing at 34.98 percent from 2017 to 2025, according to MarketInsightsReports, and data Artisans’ tool allows users to implement stream processing for a completely new class of applications that previously relied on relational database management systems.

“Data-intensive, real time applications such as fraud detection, machine learning, and real time trade pricing can now be upgraded to the streaming era effortlessly”, the company said in a blog.

Content from our partners
Scan and deliver
GenAI cybersecurity: "A super-human analyst, with a brain the size of a planet."
Cloud, AI, and cyber security – highlights from DTX Manchester

data Artisans: “The Crown Discipline”

data artisans

Stephan Ewen, co-founder and CTO at data Artisans said: “Guaranteeing serializable ACID transactions is the crown discipline of data management. It is a very hard problem – something that even some large established databases fail to provide. We are very proud to have come up with a way to solve this problem for real time data streams, and make it fast and easy to use”.

The company described the streaming ledger as “transcending a long-standing limitation of existing stream processing technology”. (Previous research has shown that nearly half FS IT decision makers would sacrifice consistency for time to market.)

data Artisans, which has offices in San Francisco, said it has released the programming API, including a serial runtime for local development, as a open-source project on GitHub.

“A Testament to the Power of Apache Flink”

data Artisans co-founder and CEO Kostas Tzoumas told Computer Business Review: “Until now, users of stream processing technology could only build applications that update a single row at a time with strong consistency (aka correctness) guarantees… users of stream processing can now build generalised applications with ACID guarantees without sacrificing the scalability, availability, and performance characteristics that data stream processing with Apache Flink offers.”

“We also see this as a testament to the power of Apache Flink and its unique capabilities to offer the building blocks for such an advanced technology.”

Want to read more? data Artisans has put out a whitepaper here.



Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how Progressive Media Investments may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.