Too Many Open Files

Summary

Too Many Open Files Jun 4, 2025 Recently I’ve been working on a pretty big rust project and to my surprise I couldn’t get tests to work properly. Running cargo test would start running all the tests in the repo and after a couple of milliseconds every single test would start to fail because of an error that I’m not very familiar with Io(Os { code: 24, kind: Other, message: "Too many open files" }) Fortunately, the error is pretty explicit and straightfoward so I was able to understand what was going on in a reasonable time. I’ve started digging a bit and learned some stuff along the way. Ever wondered how your programs juggle multiple tasks - reading files, sending data over the network, or even just displaying text on your screen - all at once? File descriptors are what make this all possible (in Unix systems). At its core, a file descriptor (often abbreviated as fd) is simply a positive integer used by the operating system kernel to identify an open file. In Unix, "everything is a file." Contrary to what the word says, a file descriptor doesn’t just refer to regular files on your disk. It can represent: Regular files: The documents, images, and code files you interact with daily. Directories: Yes, even directories are treated like files to some extent, allowing programs to list their contents. Pipes: Used for inter-process communication, allowing one program’s output to become another’s input. Sockets: The endpoints for network communication, whether it’s talking to a web server or another application on your local machine. Devices: Hardware devices like your keyboard, mouse, and printer are also accessed via file descriptors. When a program wants to interact with any of these resources, it first asks the kernel to "open" it. If successful, the kernel returns a file descriptor, which the program then uses for all subsequent operations (reading, writing, closing, etc.). By convention, every Unix process starts with at least three standard file descriptors automatic...