The Fifth Kind of Optimisation

Summary

A little while back I wrote about what I considered to be the four main kinds of optimisation: Use a better algorithm. Use a better data-structure. Use a lower-level system. Accept a less precise solution. In the same way that there are 10 “fifth Beatles”, I could have added many other kinds of optimisation, but I wanted to focus on what my experience has shown me to be the main contenders. In particular, I got a bit of pushback for “accept a less precise solution” but I think that the last 18 months of LLM advances have conclusively shown how many situations there are where this kind of optimisation is applicable. However, none of this excuses the fact that I missed off a technique that, even when I wrote that post, I use frequently: Use parallelisation. This post is not just a mea culpa, but attempts to give some retrospective justification for my stupidity. When I migrated this website to a Rust-based markdown system a couple of years ago, my software at first processed one page at a time. Irritatingly regularly, I would save a page, which automatically rebuilds the site, switch to my browser, press reload, and find that… nothing had changed. I’d managed to do all this before the site had rebuilt in the background, confusing the life out of me! A quick measurement showed that rebuilding the site took around 0.6s to run in “quick” mode. Unfortunately, profiling didn’t show any stand-out hot spots that I could try optimising. Fortunately, I had a trick up my sleeve: I made my website builder process pages in parallel using multi-threading. The “quick” build time more than halved (to below 0.3s) and suddenly I could no longer refresh my browser before the site had rebuilt. Multi-threading benefits “deploy” mode – the output of which you’re currently reading – even more, making it more than 3x faster. Using multiple threads to parallelise my website builder wasn’t about improving performance for the sake of it: the better performance I gained from parallelisation inc...