Use One Big Server (2022)

Summary

A lot of ink is spent on the "monoliths vs. microservices" debate, but the real issue behind this debate is about whether distributed system architecture is worth the developer time and cost overheads. By thinking about the real operational considerations of our systems, we can get some insight into whether we actually need distributed systems for most things. We have all gotten so familiar with virtualization and abstractions between our software and the servers that run it. These days, "serverless" computing is all the rage, and even "bare metal" is a class of virtual machine. However, every piece of software runs on a server. Since we now live in a world of virtualization, most of these servers are a lot bigger and a lot cheaper than we actually think. Meet Your Server This is a picture of a server used by Microsoft Azure with AMD CPUs. Starting from the left, the big metal fixture on the left (with the copper tubes) is a heatsink, and the metal boxes that the copper tubes are attached to are heat exchangers on each CPU. The CPUs are AMD's third generation server CPU, each of which has the following specifications: 64 cores 128 threads ~2-2.5 GHz clock Cores capable of 4-6 instructions per clock cycle 256 MB of L3 cache In total, this server has 128 cores with 256 simultaneous threads. With all of the cores working together, this server is capable of 4 TFLOPs of peak double precision computing performance. This server would sit at the top of the top500 supercomputer list in early 2000. It would take until 2007 for this server to leave the top500 list. Each CPU core is substantially more powerful than a single core from 10 years ago, and boasts a much wider computation pipeline. Above and below each CPU is the memory: 16 slots of DDR4-3200 RAM per socket. The largest capacity "cost effective" DIMMs today are 64 GB. Populated cost-efficiently, this server can hold 1 TB of memory. Populated with specialized high-capacity DIMMs (which are generally slower than the sm...