Operations | Monitoring | ITSM | DevOps | Cloud

A few months ago, Darragh Curran, CTO at Fin (formerly Intercom) set a public goal to double engineering productivity and nearly tripled it instead. They did so by pulling a few levers: AI writing code at scale, building an AI-driven PR review system, leveraging observability as a trust mechanism, and with leadership becoming more hands-on through the transition.

It's been almost exactly one year since we issued our AI mandate here at Honeycomb, and we've been doing some reflection. When we issued our mandate, it's not like we hadn't been using AI. We were the first in the industry to bake a feature powered by AI into our product, way back in May of 2024. Many of us had been experimenting and using these tools in our spare time. But we believe that software is the killer app for AI.

When our company pushed everyone to start using AI tools, I thought about what it would mean for my team. As a remote company, we are already challenged by the lack of organic human connection. Every connection is planned and takes effort, and now, AI adds another layer. People now spend part of their day collaborating with a tool rather than with a person, which can take away from the time we spend learning from each other.

AI model drift is when an AI system's performance and accuracy degrades over time because the data, user behavior, or business environment has changed since the model was trained or evaluated. Even if latency, uptime, and infrastructure metrics remain healthy, model quality can quietly decline, leading to less accurate predictions, inconsistent responses, and reduced user trust.

BubbleUp has always been the fastest way to figure out what a group of outliers have in common. Draw a box around a band of slow traces, a cluster of errors, or any set of events you're interested in, and BubbleUp compares that selection to the baseline across every dimension you've sent us. It's how Honeycomb users find the "unknown unknowns" that dashboards can’t show you.

Last week, we sat down with the authors of Observability Engineering for a live AMA. We ended up getting so many questions (pre-submitted and live) that we couldn't get through them all. Charity, Liz, George, and Austin kindly stuck around afterward to answer more, ranging from low-hanging observability fruits and telemetry to AI and what software engineers can do that Claude can't. Missed the live session? Watch it on demand now.

A few months ago, I noticed something happening. I would spend all day working with LLMs—prompting them, reviewing their work, and correcting them—and when I wasn’t working on my own code, I was reviewing LLM-generated code. By the end of the day, I was exhausted. This was a very unusual thing for me: I’ve been a software developer at startups for 30 years, and while sometimes I might have gotten stressed out, I had never been exhausted by the actual act of writing code.

For the third consecutive year, Honeycomb has been recognized for its Ability to Execute and Completeness of Vision, and we believe for its strong vision around fast, flexible, high-cardinality querying that helps engineers understand not just that something broke, but why. The software development lifecycle has collapsed. The neat sequence of plan, build, test, and ship that teams have relied on for 20 years is now happening in a single afternoon. AI writes a large share of the code.

Roughly a year ago, I left Honeycomb’s SRE team to join the newly formed Tenant team, which works on our Private Cloud offering. This team held some significant challenges on its roadmap if it wanted to demonstrate that the offering was possible, would be worth the cost, and could be done without representing a heavy tax on the rest of the organization.

A year ago, I wrote “It’s the End of Observability (and I Feel Fine).” The upshot of that post was that AI was about to fundamentally change the way we approach systems design and operation in the future. In the grand tradition, I’d like to revisit my claims from then and see how my predictions panned out.