HorizonDB, a geocoding engine in Rust that replaces Elasticsearch

Summary

At Radar, performance is a feature. Our platform processes over 1 billion API calls per day from hundreds of millions of devices worldwide. We provide geolocation infrastructure and solutions, including APIs for:Geocoding: Forward geocoding, reverse geocoding, and IP geocoding APIs with global coverage.Search: Address autocomplete, address validation, and places search APIs.Routing: Distance, matrix, route optimization, route matching, and directions APIs.Geolocation compliance: Detect current jurisdiction, distance to border, regulatory exclusion zones, and more.But as our products and data scale, so do our engineering challenges.To support this growth, we developed HorizonDB, a geospatial database written in Rust that consolidates multiple location services into a single, highly performant binary. With HorizonDB, we are able to power all of the above use cases with excellent operational footprint:Handle 1,000 QPS per core. Maintain a forward geocoding median latency of 50ms.Maintain a reverse geocoding median latency of <1ms.Scale linearly on commodity hardware.Why we replaced Mongo and ElasticsearchBefore HorizonDB, we split geocoding across Elasticsearch and microservices for forward geocoding, and MongoDB for reverse. Operating and scaling this stack was costly: Elasticsearch frequently fanned queries to all shards and required service-orchestrated batch updates, while MongoDB lacked true batch ingestion, required overprovisioning, and had no reliable bulk rollback for bad data.The architectureOur goals for this service included:‍Efficiency: The service can run on commodity machines, has predictable autoscaling, and is the single source of truth of all our geo entities.‍Operations: Data assets can be built and processed multiple times a day, changes can be deployed and rolled back trivially, and should be simple to operate.‍Developer experience: Developers should be able to run the service locally, and changes can be written and tested easily.With these goals i...