Restructuring How We Think About Alerts
Back in Alerts Are Fundamentally Messy, I made the point that the events we monitor are often fuzzy and uncertain. To make a distinction between what is valid or invalid as an event, context is needed, and since context doesn’t tend to exist within a metric, humans go around and validate alerts to add it. As such, humans are part of the alerting loop, and alerts can be framed as devices used to redirect our attention. In this post, I want to drive this concept a bit further.