At Datadog, we operate 40+ Kafka and ZooKeeper clusters that process trillions of datapoints across multiple infrastructure platforms, data centers, and regions every day. Over the course of operating and scaling these clusters to support increasingly diverse and demanding workloads, we’ve learned a lot about Kafka—and what happens when its default behavior doesn’t align with expectations.

It all started with monolith architecture; business logic, user interfaces, and data layers were stored in one big program. As tightly coupled applications, a simple update to the program meant recompiling the entire application and redistributing the program to all users. That led to the difficulty of maintaining consistent program versions and distribution across all clients in order to ensure stability and alignment. This made the monolith approach inefficient and cumbersome.

Today, Sysdig is proud to announce our integration with the AWS Security Hub. AWS Security Hub consolidates alerts and findings from multiple AWS services including, Amazon GuardDuty, Amazon Inspector, as well as from AWS Partner Network (APN) security solutions, which Sysdig is already a part of. This single pane of glass gives you a comprehensive view of high-priority security alerts and compliance status across AWS accounts.

Another outstanding Kubernetes release, this time focused on making the CustomResource a first class citizen in your cluster, allowing for better extensibility and maintainability. But wait, there is much more! Here is the full list of what’s new in Kubernetes 1.15.