Trusting builds with Bazel remote execution

Summary

The previous article on Bazel remote caching concluded that using just a remote cache for Bazel builds was suboptimal due to limitations in what can and cannot be cached for security reasons. The reason behind the restrictions was that it is impossible to safely reuse a cache across users. Or is it?In this article, we’ll see how leveraging remote execution in conjunction with a remote cache opens the door to safely sharing the cache across users. The reason is that remote execution provides a trusted execution environment for actions, and this opens the door to cross-user result sharing. Let’s see why and how.Remote execution basicsAs we saw in the article about action determinism, Bazel’s fundamental unit of execution is the action. Consequently, a remote execution system is going to concern itself with efficiently running individual actions, not builds, and caching the results of those. This distinction is critical because there are systems out there that work differently, such as Microsoft’s CloudBuild, Buildbuddy’s Remote Bazel, or even the shiny and new Bonanza.When we configure remote execution via the --remote_executor flag, Bazel enables the remote action execution strategy by default for all actions, just as if we had done --strategy=remote. But this is only a default and users can mix-and-match remote and local strategies by leveraging the various --strategy* selection flags or by specifying execution requirements in individual actions.A remote execution system is complicated as it is typically implemented by many services:Multiple frontends. These are responsible for accepting user requests and tracking results. These might include implement a second-level CAS to fan out traffic to clients.A scheduler. This is responsible for enqueuing action requests and distributing them to workers. Whether the scheduler uses a pull or push model to distribute work is implementation dependent.Multiple workers. These are responsible for action execution and are organized...