Low MTTR is the much-desired nirvana-state in IT Operations. One of the most painful parts of the incident management lifecycle, which prevents the achievement of this nirvana, is triage: the time it takes first incident responders to determine the next action when facing a barrage of IT incidents. Why?

To understand the impact that stovepipes have on incident response, one need look no further than the 9/11 terrorist attacks that occurred in the United States. The CIA, DoD, and FBI all knew about the Al Qaeda terror threats before the planes hit the World Trade Center, but the 9/11 Commission found that a lack of data and intelligence sharing among the agencies limited each agency’s understanding of the looming terrorist threat; thereby, limiting their incident response.

SRE advocates addressing problems blamelessly. When something goes wrong, don’t try to determine who is at fault. Instead, look for systemic causes. Adopting this approach has many benefits, from the practical to the cultural. Your system will become more resilient as you learn from each failure. Your team will also feel safer when they don’t fear blame, leading to more initiative and innovation. Learning everything you can from incidents is a challenge.

An on-call schedule tells you and everyone in the team who will be the first responder when an issue happens in production. The on-call team member is responsible for investigating the issue, either fixing the issue herself or adding other people who can help fix it. Having an on-call schedule is important for building reliable systems because making someone responsible for production issues makes sure that they're not ignored.

Adding alerts across your monitoring tools is taking a proactive approach to reliability. But if there are too many alerts, then it can become counterproductive because team members will start ignoring alerts or remove the alerting altogether. Which is why you need a systematic approach to adding alerts and dealing with them.

Love 'em or hate 'em, mobile push notifications can be very useful. They are not as intrusive as a phone call and have better information formats and control than text messages. Which is why it can be very frustrating to not get push notifications for your favorite product because it doesn't have a mobile app. In this post, we will see how to get mobile push notifications from any service, even if they don't have a mobile app.

We’re excited to announce a new set of updates and enhancements to the PagerDuty platform! These updates are designed to help organizations accelerate cloud migration, provide premium levels of customer service, streamline collaboration and communication, and deliver a seamless customer experience in the moments that matter most.

Picture yourself flying first class. You board the plane first, you get champagne, and you feel as though you’re the most important. Why not treat your APIs the same way? In this talk, FireHydrant CEO and Co-Founder, Robert Ross (a.k.a @bobbytables) shares why putting your APIs first can be a game-changer for your business and how this mindset shaped the way FireHydrant was built.

In one of our recent webinars we discussed a substantial challenge IT Ops teams face in today’s complex IT environments: defining and clearly communicating incident/operational roles and processes, in an effort to create a well-coordinated incident management lifecycle. This lifecycle is essential for restoring service as quickly as possible when disruptions occur. Following are the highlights of that discussion, also recently published in an ApmDigest article.

The DevOps model unites development and IT operations to create a powerful organizational culture to achieve business goals more efficiently. Formerly siloed teams can now collaborate continuously to build more robust products, with increased confidence, and achieve business goals faster. The model has the power to transform operations, but there are barriers to DevOps that must be overcome first.