(Field notes from O’Reilly’s Velocity 2019 Show, San Jose.) It was steamy hot in San Jose during O’Reilly’s Velocity show and the normally frigid AC temps in the expo hall were welcomed by all attendees, escaping the 104 degree temps. It got so bad, Charity Majors labeled it Satan Jose and the nearby Marriott hotel experienced a power outage for almost two full days, leaving guests hot under more than just their collars.

Observability-driven development requires both rich query capabilities and sufficient instrumentation in order to capture the nuances of developers’ intention and useful dimensions of cardinality. When our systems are running in containers, we need an equivalent to our local debugging tools that is as easy to use as Printf and as powerful as gdb.

This year was my third in a row attending (and now speaking at!) Monitorama. Because the organizers do a great job of turning introverts into extroverts for three days straight, it’s always a fun and exhausting time—but one of my favorite parts is how much folks continue talking about and sharing the content, days or weeks after it’s over. So, to continue the drumbeat, here were some of my highlights from this year.

Tracing is one of the key tools that Honeycomb offers to make sense of data. Over the last few weeks, we’ve made a number of improvements to our tracing interface — and, put together, those changes let you think about traces in a whole new way! Tracing makes it easier to understand control flow within a distributed system. We render traces with waterfall diagrams, which capture the execution history of individual requests.

Access to observability is becoming critical to organizations shipping software, running modern infrastructures in production, and to understanding how users are experiencing their service. To achieve success in delivering a complex service, it’s no longer optional to instrument for real visibility and ease of troubleshooting, to optimize alerting to enable a focused response, to do what is needed to drive toward real understanding and ownership of the code we deliver.

Everyone wants to be more efficient — to spend less time on the tedious things, and more time on the things that move the needle. As much as possible, if you can automate those tedious things, you should. With Honeycomb, we enable you to understand how your application behaves in production through the ability to iteratively ask questions of the system instrumentation data, no matter how granular. Honeycomb triggers enable you to be notified when specific things happen in your system.

Our latest product update features an intuitive home (landing) page that orients users with a quick, real-time view into what’s happening right now in your production systems. Home displays commonly used queries and breakdowns, and provides a jumping-off point to explore data about what’s happening in production.

Last week, Rachel published a guide describing the advantages of dynamic sampling. In it, we discussed varying sample rates to achieve a target collection rate overall, and having different sample rates for distinct kinds of keys. We also teased the idea of combining the two techniques to preserve the most important events and traces for debugging without drowning them out in a sea of noise.