Site reliability engineering is a multifaceted movement that combines many practices, mentalities, and cultural values. It looks holistically at how an organization can become more resilient, operating on every level from server hardware to team morale. At each level, SRE is applied to improve the reliability of relevant systems. With such wide-reaching impact, it can be helpful to take time to reevaluate how to improve the reliability of a system.

This post outlines how to use Zabbix and iLert with multiple on-call teams, where each team is responsible for a set of host groups in Zabbix, and therefore, will only receive alerts for the services it is responsible for. But first, let’s start with the basic needs when being on-call.

With complex architectures, gaining visibility into systems is becoming more difficult. Additionally, with the move to remote work, it’s more important than ever before to adapt to new modes of work such as asynchronous collaboration. So how do we adjust to these changing times? In a CIO panel hosted by Lightspeed Venture Partners, industry experts came together to discuss these questions. Below are key insights from their conversation.

This is the first in a series of posts we’ll be publishing on trends we’re seeing in the retail industry and how IT organizations tasked with deploying and maintaining flawless digital customer experiences can take advantage of PagerDuty to ensure always-on reliability. It’s been a tough year for retail.

Fiserv, the Fortune 500 payments and financial technology provider, needed to streamline and automate its IT incident management process to detect and fix issues earlier and more quickly. The incident management workflow was complex, primarily because mergers and acquisitions over the years had made Fiserv’s IT environment very heterogeneous. “The challenges we were facing were enormous,” IT Director Chris Kreps says.

DevOpsDays Chicago 2020 was held on September 1, online. It was the first time the conference was held virtually due to the coronavirus pandemic. I was excited to attend for a couple of reasons. First, DevOpsDays Chicago is one of the better known and respected DevOpsDays held in the US. I’d never been able to attend it before, so it was great to get the opportunity. Also, I’d been missing the DevOpsDays community.

SLOs are key pillars in organizations’ reliability journeys. But, once you’ve set your SLOs, you need to know what to do with them. If they’re only metrics that you’re paged for once in a blue moon, they’ll become obsolete. To make sure your SLOs stay relevant, determine error budgets and policies for your teams. In this blog, we’ll look at the basics of error budgeting, how to set corresponding policies, and how to operationalize SLOs for the long term.

Last week I went camping with some friends. One of them did the shopping for all of us, so I sent him my share using a payment app. It took me less than 2 minutes to complete the transaction. A few years ago, a similar transaction would have me going to the bank to complete the task, or at a minimum, calling a bank teller and having him do it. Try to imagine a bank asking its customers to do any of these things today. It would probably lose all its customers in no time.

As you implement SRE practices and culture at your organization, you’ll realize everyone has a part to play. From engineers setting SLOs, to management upholding the virtue of blamelessness, to marketing teams conducting retrospectives on email campaigns, there’s no part of an organization that doesn’t benefit from the SRE mentality.

Datadog is an awesome tool for aggregating and visualizing the metrics that matter to you. Recently, Datadog launched a new Incident Management feature, which allows you to coordinate the activities around a problem that affected your service. In this example, I’ll walk through using Relay to roll back a Kubernetes deployment that caused a service impact, and show how the Datadog Incident timeline can keep everyone working on the incident in sync.