The Scalar Select Anti-Pattern

Summary

Scalar Select Anti-Pattern May 14, 2025 I’ve written a number of stateful services starting with an event loop at the core: async for event in events_incoming: await process(event) I had to refactor this loop later, every time. For an example, see the direct cause for the article, this TigerBeetle PR. Let me write the refactoring down, so that I get it right from the get go next time! Let’s say I am implementing an LSP server for some programming language. There’s going to be three main sources of events: file modifications from user typing code in or git operations, requests from the client (“give me completions!”), output from compilation jobs running in the background with error messages and such. The “obvious” event loop setup for this case looks like this: events_incoming: Stream[Event] = merge( events_file_system, events_compiler, events_lsp, ) async for event in events_incoming: await process(event) Here, merge is an operator that takes a number of event streams, and merges them into one. This is a loop { select! { ... } } written with higher-order functions. Crucially, event streams are external to the process and are driven by the outside IO. You don’t really know or have control over when the user is typing! And process(event) takes time. This means that when we’ve finished processing the current event, there might be several events “ready”, already sitting in the address space of our process. Our “scalar select” will pick an arbitrary one, and that might create a lot of overhead. Here are some specific optimizations you can apply if you don’t ignore the fact that multiple events are available at the same time after the delay induced by processing the previous event. Prioritization The most obvious one, we can pick the order in which to process events. For the LSP example, if you have a code completion request, and a file modification request, you want to process file modification first. The rule-of-thumb priority is writes over reads over accepts (of new ...