This is a guest post by Ryan Ordway, DevOps Engineer at Oregon State University. At Oregon State University Libraries & Press (OSULP) we have been using Honeycomb for about 18 months. We were in the beginnings of automating our infrastructure and needed an APM solution that we could scale with. New Relic was becoming too expensive, and we couldn’t afford to monitor our whole infrastructure and trace all of our applications anymore. Thus began our Observability journey.

Here’s what happens when you obtain full-stack observability into your Cisco environment.

In part one, we introduced the duality of observability, controllability. As a reminder, observability is the ability to infer the internal state of a "machine” from externally exposed signals. Controllability is the ability to control input to direct the internal state to the desired outcome. So observability is a loop problem. And we need to stop treating it as the end state of our challenge in delivering performant, quality experiences to our users and customers.

A few months ago, Honeycomb released our SLO — Service Level Objective — feature to the world. We’ve written before about how to use it and some of the use scenarios. Today, I’d like to say a little more about how the feature has evolved, and what we did in the process of creating it. (Some of these notes are based on my talk, “Pitfalls in Measuring SLOs;” you can find the slides to that talk here, or view the video on our Honeycomb Talks page).

The duality of observability is controllability. Observability is the ability to infer the internal state of a "machine” from externally exposed signals. Controllability is the ability to control input to direct the internal state to the desired outcome. We need both in today's cloud native world. Quite often we find that observability is presented as the desired end state. Yet, in modern computing environments, this isn’t really true.

In today’s era of microservices, containers, and containerized applications, software architecture is more complex. Kubernetes is king in this environment, orchestrating an army of Docker containers in more distributed environments.

We’re really big fans of OpenTelemetry at Honeycomb. As we’ve blogged about before, OpenTelemetry is the next phase of the OpenTracing and OpenCensus projects. Instead of working on separate but similar efforts, those two projects have merged to create OpenTelemetry. This is wonderful for the larger community as it gives people a clear way to instrument their code for metrics and traces that isn’t specific to any tool or vendor. OpenTelemetry is a CNCF sandbox project.

With the majority of the workforce working remotely due to COVID-19, DevOps teams are still focused on delivering reliable, performant services. In these challenging times, ensuring that infrastructure and applications are available at their highest level is even more imperative—and worthy of recognition. We are all in this together, and in the spirit of supporting each other, we are excited to announce our Observability All-Star program.

Over the last 10 years, the landscape that we manage, maintain, and control as operators and developers has changed dramatically. We’ve shifted from monolith to microservices, from bare metal to VMs to containers to function-based computing — and it’s changed how we need to approach monitoring and observability.

When it comes to complex application integrations, RED monitoring provides a sensible and necessary common element to see how our systems are performing and to alert us to behavior which is detrimental to your customers and your business goals. So, what is RED? RED stands for rate, errors, duration and is an offshoot of the Google Golden Signals.