Memory is slow, Disk is fast – Part 2

Summary

TL;DR Sourcing data directly from disk IS faster than caching in memory. I brought receipts. Because hardware got wider but not faster, the old methods don't get you there. You need new tools to use what is scaling and avoid what isn't. Introduction In part 1 I showed how some computer performance factors are scaling exponentially while others have been stagnant for decades. I then asserted, without proof, that sourcing data from disk can be faster than from memory. What follows is the proof. Computer Science dogma says that unused memory should be used to cache things from the filesystem because the disk is slow and memory is fast. Given that disk bandwidth is growing exponentially and memory access latency has stagnated this isn't always true anymore. Experimental set up We need data and something straight forward to do with the data. I used my free will or the illusion thereof to create a benchmark I cleverly call "counting 10s". I write some pseudo random integers between 0 and 20 to a buffer and then count how many of the integers are 10. I want to make sure we are doing all the counting in a single thread to simulate an Amdahl's Law situation. So how fast can we expect this to run? The upper limit would be the memory bandwidth. My testing rig is a server with an old AMD EPYC 7551P 32-Core Processor on a Supermicro H11SSL-i and 96GB of DDR4 2133 MHz and a couple of 1.92TB Samsung PM983a PCIe 3.0 SSDs I pieced together from EBay parts. Given the way this server is configured, the upper limit for memory bandwidth can be calculated as 3 channels * 2133MT/s * 8B/T / 4 numa domains = ~13GB/s for a single thread. It's kind of an odd system but that just makes it more fun to optimize for! The disks are rated at 3.1GB/s read BW each for an upper limit of 6.2GB/s. I made a raid0 volume with 4KB stripe size, formatted the the raid as ext4 with no journaling, and made sure it fully finished initializing the metadata before running the tests. sudo mdadm --create /dev/md0 -...