Why performance optimization is hard work

Summary

Why performance optimization is hard workApril 29, 2025 RedditI’m not talking about skill, knowledge, or convincing a world focused on radical acceleration that optimization is necessary. Performance optimization is hard because it’s fundamentally a brute-force task, and there’s nothing you can do about it.This post is a bit of a rant on my frustrations with code optimization. I’ll also try to give actionable advice, which I hope enchants your experience.Certain optimizations can only work together, while others lead to pessimizations when combined. To be an expert means to know what optimization avenues exist; to be a master means to know which ones to choose.I have a post on integer formatting in the works, covering a very particular algorithm design – and I still haven’t finished it because there’s like five different choices to make, I have no idea how they impact each other, and I need to analyze 25 variants to claim which one’s the best in conscience. Several of my projects are similarly stuck because I don’t have the willpower to implement a dozen combinations.Pruning “obviously” suboptimal approaches is all but a heuristic. I like to think I’m more in tune with an x86-64 CPU than most people, and it still manages to surprise me from time to time. Dumb algorithms can become more applicable due to vectorization, smart code can fail due to branch misprediction or store-to-load forwarding gone wrong.Optimization takes a lot of trial and error. I dislike the “intuition doesn’t work, profile your code” mantra because it seemingly says profiling is a viable replacement for theoretical calculations, which it isn’t. But I can’t argue that profiling is avoidable. I often joke that perf report is my go-to disassembler.Worse yet, you can’t trust “obviously” good code either. In a previous post, I optimized a single linear pass by replacing it with a superlinear sort. This is by no means a unique experience: just yesterday, I saw someone optimize best-of-class Barrett re...