Just Write a Test for It

Summary

This is a short appreciation post about Rust continuously guiding me towards doing The Right Thing™. Google Summer of Code 2025 is just around the corner, which means that a bunch of new contributors started sending pull requests to various Rust projects. One of the most popular projects seems to be bors, a from-scratch implementation of a merge queue bot that we aim to use to manage pull requests in the main Rust compiler repository. In the past couple of weeks, I have reviewed and merged tens of PRs in bors. Sadly, soon after I figured out that some of them broke the (staging) deployment of the bot. This was quite annoying, because I was deliberately trying to design the bot, tests and CI so that something like this shouldn’t happen. In practice, we would probably detect this issue in the staging environment, so it hopefully wouldn’t reach production, but it’s still a situation that I would like to avoid as much as possible. After investigating for a bit, I realized that the issue was in an SQL migration that looked something like this: ALTER TABLE foo ADD COLUMN bar NOT NULL; It looked innocent enough at a first glance, until I realized that adding a NOT NULL column to an already populated table without providing some DEFAULT value for the existing rows is not a good idea. This wasn’t caught by the existing test suite (even though it runs almost 200 end-to-end tests), because it always starts from an empty database, applies all migrations and only then runs the test code. I fixed the bug, but I didn’t want to stop there. My programming experience has taught me to (almost) always try to figure out how can I make sure that a specific bug (or ideally a whole class of bugs) won’t ever happen again after we first fix it. In this case, it seemed a bit tricky at first though, as the problem was in the structure of arbitrary SQL statements. I started by adding a warning to the bors development guide, urging people not to write migrations like this. While this is better t...