Operations | Monitoring | ITSM | DevOps | Cloud

If you’re using Apache Kafka for data streaming, then you’re likely to want to take data from one topic and write an amended version of that data to another topic, for instance: for filtering (give me the items that have been delivered), enrichment (add some more information to messages, from another data source) or anomaly detection (show me potential signs of trouble). The premier solution in this space is the Apache Kafka Streams library.

Let me start with something most Kafka contributors think but rarely say out loud: nobody understands all of Kafka. I'm not a core committer and have only contributed a few times, but those contributions I have made have been in part thanks to using coding assistants. There are some issues with this approach though, the Apache Kafka project is huge. It's split into many parts: the core, the server, the client libraries, the streams engine, the storage layer, the consensus code, and more.

Aiven for ClickHouse 26.3 is now available in Early Availability. This upstream Long-Term Support release makes full-text search generally available, enables asynchronous inserts by default, introduces materialized common table expressions, and brings a wide range of JSON and query-performance improvements. The upstream 26.3 release includes 27 new features and 40 performance optimizations.

TL;DR Agent projects turn into infrastructure projects fast. Managed agent platforms handle memory, integrations, scheduling, and scaling so builders can focus on the agent itself.

Occasionally, I get asked, "is pgvector enough for Enterprise AI Solutions or should I migrate to a dedicated vector database?" The honest answer is No Create EXTENSION vector; ALONE is not going to solve your solution for enterprise AI. That being said, I don't think you're asking the right question. The better question is, "does a dedicated vector database make sense for my workload or would pgvector suffice?".

Helsinki, Finland — Aiven has acquired Flow AI, a company building infrastructure for production-grade analytical AI agents. The integration of Flow AI technology will accelerate Aiven's product roadmap and make it easier for customers to securely and scalably run production AI applications and agents next to their data.

TL;DR OpenSearch 3.6 makes agentic search production-ready, with the AI-powered Launchpad provisioning full search apps in minutes and faster default vector search, and it's the first LTS release, bringing 18+ months of guaranteed support, SBOMs, and an upstream-first commitment (every fix goes back to the main project) so teams get fast-moving open source and a stable, supported platform at once.

Kafka is brilliant at moving events around, but sooner or later someone wants to actually query those events, perhaps aggregations, dashboards, or ad-hoc analytics over billions of rows. That is where ClickHouse comes in. It's the option for when stream processing is more than you need, but warehouse query latency is more than you'll tolerate.

I spend a good amount of my time deploying Aiven services for demos and examples. Traditionally the tools I reach for are: If I’m writing a program, I may also look to the Aiven API, perhaps using curl at the command line or in a shell script, or perhaps with direct HTTP requests in a Python program. The API is how the console and the CLI tool talk to Aiven, but I generally find that too low level to be comfortable, and I always have to look up how to pass in the Aiven user token.

You're building with Claude or Cursor, and you need to know what's actually happening on your Kafka cluster. Your AI assistant knows Apache Kafka in the abstract, but not your topics, your retention, or that a consumer group has been slipping since this morning. So you leave the editor and go digging through logs, a CLI, and a few dashboards, correlating by hand to answer questions like: The Aiven MCP (EA) turns each of those into a sentence you type where you already work.