Operations | Monitoring | ITSM | DevOps | Cloud

Kafka

Apache Kafka 3.8.0 Released

The Apache Kafka community is pleased to announce the release of Apache Kafka 3.8.0. As a release manager for this version, I want to personally thank all users and the team of 200+ contributors for the release of Apache Kafka 3.8.0. This was truly a community effort to bring new features and improvements to an open source streaming platform used by thousands of organizations worldwide.

Navigating Kafka: Challenges, Solutions, And The Future Of Real-Time Data Streaming

3). The complexities of real-time data streaming and auto-scaling in the cloud are no joke. But three are solutions to make it more simple and efficient. In this Data (R)evolution episode, Matan Mizrahi and Filip Yonov join us to discuss how Kafka effectively handles and optimizes data streams, the pursuit of standardized APIs, and the vital role of AI in optimizing complex systems. They also share how AI and open-source tools are transforming the landscape. Tune in to hear about the evolution of data streaming and the transformative impact of AI and cloud-native solutions.

Hot and cold data with Apache Kafka, Tiered Storage, and Iceberg

Utilizing the true potential of data streaming is key to business success. In this Data (R)evolution episode, we're joined by Josep Prat and Filip Yonov to dive into the transformative features of Apache Kafka and its evolving role in data architecture. They discuss the critical importance of collaboration and feedback in enhancing Kafka's capabilities, the future of "lake house" technology, exciting updates from the Open Source Program Office (OSPO), and the importance of Kafka's readiness to support evolving data formats—making it a backbone for modern data ecosystems.

Key Capabilities for Successful Kafka Management

Kafka is a powerful event streaming technology that is relatively easy to set up but can become extremely complicated to scale, especially without significant maintenance tasks. Any Kafka manager requires a robust Kafka management tool to efficiently operate, monitor, and maintain a Kafka cluster, especially in production environments. The following list comprises the most needed capabilities in a tool for a Kafka Manager.

The Top 8 Kafka Monitoring Tools

Apache Kafka has risen as a pivotal element in modern distributed systems, transforming data processing, storage, and distribution across diverse applications. Kafka, developed by Kafka, is an open-source distributed event streaming platform. It is designed to efficiently manage high volumes of real-time data, acting as a distributed messaging system.

The Challenges of Partition Rebalancing in Kafka Brokers and Effective Monitoring Strategies

Apache Kafka has become an essential component in data streaming and processing architectures due to its high throughput and scalability. However, as organizations scale up their Kafka usage, they often encounter challenges such as partition rebalancing across different brokers. This imbalance can lead to significant issues, including overloaded partitions that jam traffic, affecting performance and reliability.

Investigating Mysterious Kafka Broker I/O When Using Confluent Tiered Storage

Earlier this year, we upgraded from Confluent Platform 7.0.10 to 7.6.0. While the upgrade went smoothly, there was one thing that was different from previous upgrades: due to changes in the metadata format for Confluent’s Tiered Storage feature, all of our tiered storage metadata files had to be converted to a newer format.

Fixing Kafka Streams Uneven Tasks Distribution at Logz.io

At Logz.io we provide an observability platform with the ability to ship logs, metrics, and traces and then interact with them using our app. LogMetrics is an integral part of our observability offering, which bridges the gap between logs and metrics. It provides the seamless conversion of one type of signal to another. It empowers our customers to gain critical insights faster while also reducing their monitoring bill.