Io_uring, kTLS and Rust for zero syscall HTTPS server

Summary

Around the turn of the century we started to get a bigger need for high capacity web servers. For example there was the C10k problem paper. At the time, the kinds of things done to reduce work done per request was pre-forking the web server. This means a request could be handled without an expensive process creation. Because yes, creating a new process for every request used to be something perfectly normal. Things did get better. People learned how to create threads, making things more light weight. Then they switched to using poll()/select(), in order to not just spare the process/thread creation, but the whole context switch. I remember a comment on Kuro5hin from anakata, the creator of both The Pirate Bay and the web server that powered it, along the lines of “I am select() of borg, resistance is futile”, mocking someone for not understanding how to write a scalable web server. But select()/poll() also doesn’t scale. If you have ten thousand connections, that’s an array of ten thousand integers that need to be sent to the kernel for every single iteration of your request handling loop. Enter epoll (kqueue on other operating systems, but I’m focusing on Linux here). Now that’s better. The main loop is now: set_up_epoll() while True: new, read, write = epoll() epoll_add_connections(new) for con in read: process(con.read()) if con.read_all_we_need: epoll_remove_read_op(con) for con in write: con.write_buffer() if con.buffer_empty: epoll_remove_write_op(con) All the syscalls are pretty cheap. epoll() only deals in deltas, and it doesn’t have to be re-told the thousands of active connections. But they’re not without cost. Once we’ve gotten this far, the cost of a syscall is actually a significant part of the total remaining cost. We’re here going to ignore improvements like sendfile() and splice(), and instead jump to… io_uring Instead of performing a syscall for everything we want to do, commanding the kernel to do this or that, io_uring lets us just keep writing or...