Hypervisor as a Library

Summary

Before we dive into the topic, let me introduce you my new friend catsay, a simple Go program which eats stdin and speaks like a cat: Cute! ... but it's not what I want to talk about. What makes this screenshot very exciting is, it's a Linux lightweight virtual machine running on Starina operating system! That said, this post is not about how hard it is to write a hypervisor (see my previous post for that). In fact, it's not that hard. The hardest part is to design how you interact with the hypervisor. In other words, designing the hypervisor API. Starina needs an attractive integration with Linux. In this post, I'd share a design pattern: hypervisor as a library. How do we run Linux apps in Linux today? First, if you were to write a Rust application which uses catsay, how do you integrate? In Rust, you would use std::process::Command to run catsay: Command::new("/bin/catsay") .stdin(stdin) .spawn() If you want to pass an environment variable, just add a single line: Command::new("/bin/catsay") .stdin(stdin) .env("CATSAY_MODE", "dog") .spawn() Nice. If you're interested in its output, add another parameter .stdout: let child = Command::new("/bin/catsay") .stdin(stdin) .env("CATSAY_MODE", "dog") .stdout(Stdio::piped()) .spawn() It's boringly obvious, right? However, on another OS like Starina, it's not obvious how to provide a Linux environment, or Linux compatibility. Linux compatibility Providing Linux compatibility is a challenging task, and there are some ways to achieve it. A popular way is to run Linux binaries as kinda in native: hook system calls and emulate them. Windows Subsystem for Linux (WSL 1) and FreeBSD Linuxulator are examples. In Starina, I took a different approach: run the real Linux kernel in a lightweight virtual machine. It might sound extreme, but it's already proven to be practical by WSL2. It has a huge drawback in the cloud computing environment where hardware-assisted virtualization is not available (or is slow) in cheap VM-based instances...