You probably know that Honeycomb is the most flexible observability tool around. Its powerful high-cardinality search makes working with real raw data quick and easy. But as you may have learned through hard experience, fetching those dots can still be quite a challenge.

In this blog miniseries, I’d like to talk about how to think about doing data analysis “the Honeycomb way.” Welcome to part 1, where I cover what a heatmap is—and how using them can really level up your ability to understand what’s going on with distributed software. Heatmaps are a vital tool for software owners: if you’re going to look at a lot of data, then you need to be able to summarize it without losing detail.

When we released derived columns last year, we already knew they were a powerful way to manipulate and explore data in Honeycomb, but we didn’t realize just how many different ways folks could use them. We use them all the time to improve our perspective when looking at data as we use Honeycomb internally, so we decided to share. So, in this series, Honeycombers share their favorite derived column use cases and explain how to achieve them.

Our new RubyGems.org public dataset is now available — use it to analyze global download traffic of all gems hosted on RubyGems!

Honeycomb strives to be a fast, efficient tool; our storage back-end satisfies the median customer query in 250ms (and the P90 in 1.3 seconds). Still, every system has its limits, and customers with large datasets know that querying over a long time range, grouping by high-cardinality columns, building complex derived columns, and throwing a quantile or heat map into the mix can lead to some pretty slow queries. If this sounds familiar: good news!

When we release something new, whether it’s a new SDK or Beeline or a new feature in the UI, we’ll often set a Honeycomb Trigger to keep track of its use. Sometimes we don’t necessarily have a customer we know is immediately going to use the feature in question, and we’re interested in when it happens.

If you’ve been paying attention, you know that although collecting and reviewing metrics and logs is a core part of running a stable and successful service, access to raw events and the ability to search and pivot on any dimension of your production environment, no matter how high-cardinality, is what will help your team debug and troubleshoot new problems and outages more quickly.

We’ve seen folks do amazing things using our Honeycomb Beelines–getting their apps instrumented in next-to-no time, expanding their observability, growing their understanding of what is happening in their code in production. Now, prepare to do more with improved tracing support!

We recently shipped an update to navigating inside Honeycomb. Thus far, results have been positive – thank you! We wanted to give you a behind the scenes look at why and how this work came along. It’s easy to develop a certain blindness after working with your own code day-in day-out. Staring at something over a long period of time wears down the jagged edges that once used to protrude and now appear as normal.