In this blog miniseries, I’m talking about how to think about doing data analysis, the Honeycomb way. In Part I, I talked about how heatmaps help us understand how data analysis works. In Part II, I’d like to broaden the perspective to include the subject of actually analyzing data.

Organizations that handle logging at scale eventually run into the same problem: too many events are being generated, and logging components can’t keep up. Even with persistent queues and other mitigating features enabled, there’s simply not enough of a buffer between log generators and log ingesters to handle the volume of log lines coming in.

Application Performance Monitoring, aka APM, is one of the most common methods used by engineers today to measure the availability, response times and behavior of applications and services. There are a variety of APM solutions in the market but if you’re familiar with the ELK Stack or are a Logz.io user, this article describes using a relatively new open source-based solution — Elastic APM.

Honeycomb has always been about flexibility, power, and speed — and about working with your data in a way that other vendors say is impossible. But now Honeycomb is also about being easier than ever to orient yourself and begin getting value out of your data right away.

Logging is an important topic in software development, especially if you need to analyze bugs and other unexpected events in your production environment. Implementing your logging often seems easy. But as you probably experienced yourself, logging is far more complex than it might seem. That’s why you can find lots of articles about it here on the blog.

Ever since humankind developed the ability to write, much of our progress has been made thanks to recording and using data. In ages long past, notes were made on the production and gathering of resources, the exact number of available soldiers and other important personnel, and were compiled and stored by hand. Because of this documentation method, important information was also prone to being misplaced, lost, or even mishandled.

I hosted a webinar where I covered why logging is important, how to choose a logging provider. And then shared our experience of setting up logging on Kubernetes containers, the Kubernetes logging framework and the logging best practices we’ve implemented internally and supported our customers who run Kubernetes in production.