Cutting down Rust compile times from 30 to 2 minutes with one thousand crates

Summary

Rust is fast at runtime — but not so much at compile time. That’s hardly news to anyone who's worked on a serious Rust codebase. There's a whole genre of blog posts dedicated to shaving seconds off cargo build.At Feldera, we let users write SQL to define tables and views. Under the hood, we compile that SQL into Rust code — which is then compiled with rustc to a single binary that incrementally maintains all views as new data streams into tables.We’ve already pulled a lot of tricks in the past to speed up compilation: type erasure, aggressive code deduplication, limiting codegen lines. And that got us quite far. However, recently we started on-boarding a new, large enterprise client with fairly complicated SQL. They wrote many, very large programs with Feldera. For example, one of them was 8562 lines of SQL code that eventually is translated to ~100k lines of Rust code by the Feldera SQL-to-Rust compiler.To be clear, this isn’t some massive monolith we’re compiling. We’re talking about ~100k lines of generated Rust. That’s peanuts compared to something like the Linux kernel — 40 million lines (which manages to compile in a few minutes).And yet… this one program was taking around 25 minutes to compile on my machine. Worse, on our customer's setup it took about 45 minutes. And this was after we already switched the code that is generated to using dynamic dispatch and pretty much eliminated all monomorphization.Here's the log from the Feldera manager:[manager] SQL compilation success: pipeline 0196268e-7f98-7de3-b728-0ee339e449fa (program version: 2) (took 101.94s) [manager] Rust compilation success: pipeline 0196268e-7f98-7de3-b728-0ee339e449fa (program version: 2) (took 1617.77s; source checksum: cbffcb959174; integrity checksum: 709a17251475)Almost all the time is spent compiling Rust. The SQL-to-Rust translation takes about 1m40s. Even worse, the Rust build is doing the equivalent of a release build in cargo, so it happens from scratch every time (except for cargo ...