Running our Docker registry on-prem with Harbor

Summary

As of early 2025, we’re deploying all of our applications with Kamal using Docker as our containerization platform. The container registry that holds our app images is one of the most integral pieces of our deployment pipeline. Like many organizations, we’d been using external container registries for years. Our ecosystem was tightly coupled to both Dockerhub and Amazon’s Elastic Container Registry. However, as part of our cloud exit and kamalization journey, several issues started emerging: Cost: Not only does the paid license for Dockerhub produce a considerable invoice — pulling and pushing our images over the internet dozens of times a day caused us to hit the contracted bandwidth limit with our datacenter provider Deft repeatedly. We tried working around this by running pull-through caches, but this still locked us to Dockerhub. Performance: Migrating HEY to Kamal and expanding the deployment to another continent caused deploy time penalties — up to 45 seconds on uncached pulls per host. This was exacerbated once our largest application Basecamp 4 was moved to Kamal — suddenly deployments took minutes longer simply because of push/pull speeds out of our control. Security and Governance: We all hope to never leak credentials in our images, and yet it still happens — the scale ranging from easily mitigated to catastrophic. We wanted to eliminate that threat surface once and for all by keeping our artifacts where they belong — with us. Independence: Despite being on a paid account, we fell into the crunch of API limitations for arbitrary reasons a couple of times. In addition, we’d been keeping all of the images used in our Chef CI/CD infrastructure still on AWS. Our criteria for the solution to pick were fairly simple: reliable, performant, easy to set up, open-source. We evaluated running the default distribution implementation as our registry, but quickly set our eyes on Harbor. Harbor provided us with a more expandable and rich feature set right out of the box...