The Uselessness of "Fast" and "Slow" in Programming

Summary

Every field of human activity has its unique characteristics, and programming is no exception. One of the unique aspects of software is how it spans such a large number of orders of magnitude. A software engineer may be slicing and dicing nanoseconds, or they may be trying to accelerate a computation that will run across thousands of cores for a month… and they may even be doing both at the same time! A single core in a nanosecond may cover 4 cycles. A thousand cores in a month covers about 2,600,000,000,000,000,000 cycles. Rounding a touch, that’s a range of about 19 orders of magnitude. A large supercomputer cluster, or if you choose to count GPU cores differently, may stretch even another couple of orders of magnitude. This is not an every day experience for most programmers, but even an 8-core 4GHz system covers 32,000,000,000 cycles in a second. Again rounding a bit that’s 10 orders of magnitude between “my code runs in a couple of cycles” and “my code takes all my CPU resources for a second”. I can not think of very many other disciplines that not only span that number of orders of magnitude, but are doing engineering across the entire range. Cosmology may care about quantum mechanics in order to try to determine the behavior of things like neutron stars, but there’s a vast swathe in the middle they don’t cover. Other ideas may leap to your mind, but even 10 orders of magnitude turns out to be really quite a bit! Thinking about things of one size in one moment, then something a billion times bigger or smaller, and caring about both of them and potentially also a range of magnitudes in between, is not common. And as such, our human brains are not very good at dealing with this. We do not have English terminology that can account for systems that span this range. Is something that takes 500 cycles “fast?” 50,000? 5 million? 50 million? To a human, all but perhaps that last one are equally “instant”. Then again, try to do them a billion times each and the differe...