View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Software
April 27, 2020updated 26 Jul 2022 10:22am

Apache Kafka 2.5: Latest Release Brings Host of Welcome Updates

Plenty of welcome new features and bug fixes...

By CBR Staff Writer

Apache Kafka — arguably one of the open source software world’s most significant projects from an enterprise perspective — has had a sweeping overhaul, with the release of version 2.5.

Apache Kafka 2.5 comes with seven notable new features and a 65 significant improvements. It also features 92 bug fixes.

Kafka is a publish-subscribe (“pub-sub”) messaging system that that lets users build distributed applications. It is widely used, including by AirBnB, Goldman Sachs, Netflix, and Twitter, to build real-time streaming data pipelines and applications that adapt to these data streams.)

Apache Kafka 2.5: What’s New?

New features rolled out this Month — resulting from the “Kafka Improvement Proposal” (KIP) process — include the following, among other new metrics and security features…

1) KAFKA-6049] – This new feature lets users add a co-group in the DSL (Domain Specific Language). This makes it easier for users to allow multiple streams to aggregate together to form a single larger object (e.g. a shopping website may have a cart stream, a wish list stream, etc.

2) [KAFKA-6144] – This new feature aims to improve availability when clusters are scaled up, by allowing Kafka to serve interactive queries from in-sync standbys. (Previously, interactive queries against state stores would fail when there is a rebalance in progress.)

3) [KAFKA-7251] – This new feature adds support for TLS 1.3 — an encryption protocol that improves security and speed.

4) [KAFKA-8843] – This new features adds support for TLS encryption between Kafka and configuration management software Apache Zookeeper.

Content from our partners
How to turn the evidence hackers leave behind against them
Why food manufacturers must pursue greater visibility and agility
How to define an empowered chief data officer

As contributor Confluent notes: “When deploying a secure Kafka cluster, it’s critical to use TLS to encrypt communication in transit. Apache Kafka 2.4 already ships with ZooKeeper 3.5, which adds TLS support between the broker and ZooKeeper. However, configuration information has to be passed via system properties as -D command line options on the Java invocation of the broker or CLI tool (e.g., zookeeper-security-migration), which is not secure. KIP-515 introduces the necessary changes to enable the use of secure configuration values for using TLS with ZooKeeper.

“ZooKeeper 3.5.7 supports both mutual TLS authentication via its ssl.clientAuth=required configuration value and TLS encryption without client certificate authentication via ssl.clientAuth=none.”

5) [KAFKA-9352] – This fixes unbalanced assignment of topic-partition to tasks, i.e. to balance Kafka load across all so-called mirrormaker instances (workers), this feature helps to evenly assign all topic-partition to the tasks.

6) [KAFKA-9445] – This feature allows users to fetch a key from a single partition rather than iterating over all the stores on an instance

Other welcome changes include KIP-447, which simplifies the API for applications that read from and write to Kafka transactionally.

Previously, this use case typically required separate producer instances for each input partition, but now there is no special requirement.

Confluent has provided a tidy overview of some of the other key updates in this video. Enjoy!

See also: Vulnerabilities in the Core: Key Lessons from a Major Open Source Census

Topics in this article: , , , ,
Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.
THANK YOU