Serving 200M requests per day with a CGI-bin

Summary

In the early 2000s, we used to write a lot of CGI programs. This was the primary way to make websites dynamic at the time. These CGI programs were usually written in Perl, but sometimes in C or other languages to increase performance. The CGI mechanism is conceptually simple but powerful. When the web server receives an incoming request for a CGI script (e.g. /~jakegold/cgi-bin/guestbook.cgi), it: Sets up environment variables containing request metadata (HTTP headers, query parameters, request method, etc.) Spawns a new process to execute the CGI program Passes the request body (if any) to the program via stdin Captures the program’s stdout as the HTTP response Sends any error output from stderr to the error log The CGI program reads the environment variables to understand the request, processes it, and writes an HTTP response to stdout, starting with headers. One of the nice features of this is that the CGI program exits after handling a single request, so all of its file descriptors and memory are automatically freed by the operating system. This made the terrible code of the time run quite reliably. The developer experience was excellent as well. Deploying a new version of your CGI program was just a matter of copying it to the cgi-bin/ directory on your web server. Hug of death# Typical web servers of this time had 1-2 CPUs and 1-4 GB of memory. Most web servers ran Apache, which would fork an httpd process for every connection, each of which took a significant amount of memory. This would limit the maximum concurrency to fewer than 100 connections in most cases. This made it incredibly easy to Slashdot a website just by linking to it from a popular site. Modern servers# These days, we have servers with 384 CPU threads. Even a small VM can have 16 CPUs. The CPUs and memory are much faster as well. Most importantly, CGI programs, because they run as separate processes, are excellent at taking advantage of many CPUs! This got me curious about how fast CGI program...