Nowadays, a working digital infrastructure is the lifeblood of almost any organization. The impact of a major IT incident can go far beyond the IT department, affecting a company’s revenue or incur costs in other areas of the business caused by service disruption. Therefore, in addition to the technical response to a major incident from the IT department, business stakeholders need to be involved as well, so they can prepare the business response.
One of the first things incident managers do when they get an alert page from Zenduty is to check the “Context” tab of the incident. Incident context is extremely critical to get a first responder’s view of what happened and what could possibly have caused it. Context tells you what happened before an incident. In the case of 40–50% of all incidents, Zenduty’s incident context can tell you within 5–10 seconds, what could be the cause of an incident.
DevOps pipelines enable teams to implement continuous software development processes, often by using automation and collaboration tooling. The overall goal is to quickly release software products, updates, and fixes. To ensure a DevOps pipeline works well, teams add management and monitoring tooling to the pipeline. This includes incident alert management, which supports the team’s efforts in monitoring the security of various software and environment components.
The ongoing pandemic and resulting economic downturn have led to dramatically changing market conditions. As a consequence, technology teams have become increasingly concerned with the need to minimize their financial risk and reduce costs to mitigate the effects of abruptly pivoting to a fully remote working environment. For some, there has been a struggle to maintain business continuity—i.e., keeping the physical components of the business running when everyone is working from home.
Efficient team communication requires the proper set of tools and processes, ensuring that the right persons receive timely messages. This way, recipients are well-informed of a critical issue, while having time to address the incident. Unfortunately, a large helicopter services company relied on time-wasting procedures to communicate with stakeholders, resulting in delayed incident response and resolution.
With many companies utilizing remote-working during the COVID-19 pandemic, effective communication is more important than ever. Unfortunately, being in the middle of responding to a global pandemic will not prevent your organization from suffering from other business disruptions. Likely disruptions you may face are: Cyber/ phishing attacks – these attacks can cripple your regular communication methods such as email, or may exploit ineffective communications to extract illegal payments.
Recently, one of our customers, a 20-member NOC team of a large B2C company, had set up Zabbix to monitor a network of over 1000+ servers, routers, and switches. The NOC team wanted to set up alerting, on-call scheduling, and an escalation matrix whenever a critical network component encountered any downtime. The NOC team used Slack as the primary communication channel and Zoom for real-time communication. For NOC teams like these running a very large operation, setting up alerting can be very tricky.