Reinventing how .NET builds and ships (again)

Summary

After I wrote my last post on how .NET builds and ships, I was cautiously optimistic that I wouldn’t be writing another one. Or at least not another one about how we build and ship. That problem was done and dusted. .NET had done it! We’d struck a balance between distributed repository development and the ability to quickly compose a product for shipping. Congratulations everyone, now the infrastructure teams could focus on other things. Security, cross-company standardization, support for building new product features. All the good stuff. …A year and a half later… We’re asking how much it will cost to build 3-4 major versions with a dozen .NET SDK bands between them each month. And keep their engineering systems up to date. And hey, there’s this late breaking fix we want to get into next week’s release, so can I check it in today and have the team validate tonight? It can’t be that hard, right? And I have this new cross-stack feature that I want to do some prototyping on…how can I build it? The answers were mostly frustrating: “It’ll cost a lot, and get worse over time.“ “I don’t think we have enough time for that fix, I can only guess how long the build will take, but it’s at least 36 hours before we can handoff to validation. Maybe more?“ “I’m sure we can keep that much infrastructure alive, but we’ll slowly drown under the cost of keeping it up to date.“ “How critical is it that you have a full stack to work with? It’ll take a while to set that up.“ These are not the answers we want to be giving. And so, we went back to the drawing board, looking for solutions. This blog post is about the Unified Build project: .NET’s effort to resolve many of these issues by moving product construction into a ‘virtual monolithic’ repository, consolidating the build into a series of ‘vertical builds’, while still enabling contributors to work outside the monolith. I’ll briefly tell the story of our product construction journey over the life of .NET. I’ll draw attention to the le...