Writing a postmortem: an interview exercise I like (2017)

Summary

First, some exciting news that’s relevant as context: starting in November, I’ll be joining Mapbox in their Washington, DC office. One thing that made me excited about joining Mapbox was how much I enjoyed their interview process. No part of it felt like it required extra “preparation” and it was clear that some significant thought had been put into its design. In my opinion, this is a pretty useful signal about an engineering organization. If we accept that finding great engineers (by whatever definition of “great” you subscribe to) is one of the largest contributors to the success of any company, it’s possible that a company with an interview process that appears disorganized or nonsensical is otherwise well-functioning, but it’s probably at least somewhat less likely. One aspect of the Mapbox interview process that I particularly liked was their request that I write a blameless postmortem as a take-home exercise. If the term is unfamiliar, postmortems are a tool designed to help facilitate a culture of building institutional knowledge and learning from the past. Whenever something goes wrong – an outage, bug in production, failure to meet an SLA, etc – anybody involved in the situation can call for a postmortem. The postmortem takes the form of a shared document where everybody can contribute their account of the incident to help identify its ultimate causes and propose changes to prevent it from happening again. The “blameless” aspect is crucial: a good postmortem avoids conclusions like “Dan wrote a bug and it brought down our service” and instead says “Dan wrote a bug and it brought down the service: we need to improve our testing and deployment processes to make sure that they catch this category of bugs in the future.” For many mistakes that initially look like they ought to be blamed on an individual, it’s possible to identify a deficiency in a process as the root cause. You can read more about blameless postmortems in the Google SRE book. I thought this ex...