This talk was presented at Failover Conf on April 21, 2020.

Every disaster is a concatenation of smaller failures. How can we design software and processes to accept that we live in an imperfect world? Explore the concepts of resiliency, harm reduction, over-engineering, and planning for failure with real examples.

Risk Reduction is trying to make sure bad things happen as rarely as possible. It's anti-lock brakes and vaccinations and irons that turn off by themselves and all sorts of things that we think of as safety modifications in our life. We are trying to build lives where bad things happen less often.

Harm Mitigation is what we do so that when bad things do happen, they are less catastrophic. Building fire sprinklers and seatbelts and needle exchanges are all about making the consequences of something bad less terrible.

This talk is focused on understanding where we can prevent problems and where we can just make them less bad, and what kinds of tools we can use to make every disaster a disappointing fizzle.

Audiences will leave with a clearer understanding of risk and harm, and a set of tools than can be used to minimize future problems.

I'm going to talk about why we need to understand both avoiding problems and making them less catastrophic, and what kinds of tools are appropriate to each.

I think that developers need to be thinking about failure states more than we currently do. We talk about avoiding them, or testing them away, but we don't talk about how to make even failure a better experience.