Publish a Python Wheel to GCP Artifact Registry with Poetry

Summary

Recently, I’ve been building a project in Python that doesn’t have a Docker image as output. Instead, I need a runnable file. Why? Because I need to talk to the machine directly and not to Docker. Because I need to talk to the GPU drivers directly and not to another abstraction layer. So, I needed to create a runnable file in Python. I needed to create a wheel file. All this with Poetry and integrate it into my CI/CD pipeline. Let’s break down the steps. Building the Project With Poetry My first solution was, when an VM is created in my Compute Instance Group, I clone the git project, install Poetry, sync the project and run it. Just like in my laptop. But! I need a token to clone the project. I never liked this solution. So, I’ve been thinking around for an alternative. I wanted a solution near to the Docker workflow: build locally, push to a registry, and download the latest version at the startup. I searched a way to do the same with a python project. here come the wheel files. By default, I build my project with poetry sync, because I don’t need a runnable file as I mainly use Docker images. But Poetry allows me to configure the install command a little bit further. Let’s start configuring that command to build a wheel file. Here is an extract of the pyproject.toml file: [tool.poetry] # ... packages = [ { include = "main_dir" }, ] It indicates to Poetry to include the main_dir directory when building a package, when building a wheel. Now the poetry build command does not require the no-root parameter anymore. And the wheel file will be available in the dist folder. Create a Python Registry to Store My Wheel File The project is hosted in GCP. And GCP has an artifact registry that allows me to specify which type of files I want to save there: Docker images, Python Wheel files, Java JAR files… Once the artifact registry is created, I need to configure two things: Poetry to push to the adequate place, and the VM to pull from the correct place. Let’s start in the Poe...