Apache Pulsar, a major project by the open source Apache Foundation, today emerged from a two-year incubation period as one of the foundation’s “top level” projects.
Apache Pulsar is a so-called “pub-sub” or publish/subscribe messaging system developed then open-sourced by Yahoo! in Sept. 2016.
It comes with a REST Admin API for provisioning, administration, tools and monitoring and can be deployed on bare metal or Kubernetes.
Pub/sub is a form of asynchronous service-to-service communication used in serverless and microservices architectures. It can be used to enable event-driven architectures (a framework to orchestrate behaviour around events and their responses) or to decouple applications in order to increase performance, reliability and scalability.
See also: Apache Software Foundation: myNewt Flourishing, Check; Cloud Migration, Check; Audit Passed; Check; Five-Year Plan? Check…
Apache Pulsar was described by the Foundation as: “Highly scalable, low latency messaging platform running on commodity hardware. It provides simple pub-sub and queue semantics over topics, lightweight compute framework, automatic cursor management for subscribers, and cross-datacenter replication.”
Pulsar has run in production at Yahoo scale for over three years, with millions of messages per second across millions of topics and its original lead developer, Matteo Merli, the co-founder of Streamlio and recently named VP of Apache Pulsar said.
He added: “The fact that Apache Pulsar has gone from incubator project to top-level in two short years is a testament to the community growth around the project… it has become instrumental in a broad range of modern data-driven applications.”
One Click Retail (owned by the UK’s Ascential) is one of the early adopters of the tool. The company’s Jowanza Joseph said: “Because of Pulsar’s unique combination of messaging and stream processing, we’ve been able to replace multiple systems with one solution that works seamlessly in our Kubernetes environment.”
He added: “Pulsar functions has allowed us to dramatically simplify our stream processing pipeline and to reduce the cost associated with production grade stream processing systems. Seeing Pulsar become a top-level Apache project is a great milestone that validates our confidence in the current and future innovations of Pulsar and the Pulsar community.”
