We didn't rewrite our feed handler in Rust

Summary

We recently faced an interesting decision: what language should we use to rewrite our market data feed handler? Despite Rust's growing popularity in fintech and our own successful use of it in other systems, we chose C++. This post explores the technical reasoning behind that choice, diving into specific patterns where Rust's strict ownership model created friction for our use case. Our real-time market data architecture processes 14 million messages per second with sub-100 microsecond latency requirements. At its heart sits the feed handler, a colocated application that parses proprietary feeds from exchanges and normalizes them to our DBN format. The existing implementation had accumulated technical debt over time. It was overly generic, supporting workflows we didn't actually use. The architecture wasn't modular enough, which made optimizations invasive and difficult. Most critically, it had a complex concurrency model that led to a lot of context switching, and an abundance of shared state resulted in lock contention where spikes in one area could cascade across the application. Given these issues inherent in the architecture, we decided a rewrite was the best path forward. Our new implementation needed to: Keep things simple with minimal shared state and basic parallelism while avoiding premature optimizationHave a narrow focus and outsource as many tasks as possible to other services (like calculating OHLCV bars)Be fast enough to handle our 14M messages/second throughput with predictable sub-100μs latencies A language that compiled to native code was non-negotiable. We weren't starting from zero with Rust. At Databento, we've successfully deployed it across several critical systems: The reference implementation of our DBN encoding, which also handles serialization to CSV and JSONOur real-time market data gateways (the other colocated service that clients connect to)Our official Rust client library, a top 10 most downloaded crate in the finance categoryPython b...