Implementing a RISC-V Hypervisor

Summary

To implement a seamless Linux integration into Starina, I decided to go with a Linux lightweight VM approach similar to WSL2. This means I need to implement a hypervisor that can run Linux. I had implemented an Intel VT-x based hypervisor before, but this time I wanted to try something different: RISC-V H-extension based hypervisor! This post is a diary of my journey of writing a RISC-V hypervisor incrementally. RISC-V H-extension RISC-V H-extension introduces new CPU modes and some more CSRs (so-called control registers) to implement hardware-assisted virtualization. Its design is similar to Intel VT-x in the sense that both host and guest modes have their own kernel mode and user mode. This design makes it easy to run host OS along with guests, that is, guests behave as normal host processes (e.g. QEMU and Firecracker). How can I test a hypervisor on macOS? Unlike Linux KVM-based hypervisors (more specifically, virtual machine monitors), Starina is a new operating system and has been tested on QEMU. In this case, you typically need to use nested virtualization where the hardware-assisted virtualization is emulated by the host OS. That's how I did it for Intel VT-x. Here's great news: QEMU itself can emulate RISC-V H-extension! You just need to add -cpu rv64,h=true to the QEMU command line. I presume this is thanks to RISC-V's simplicity and designers' foresight (and of course QEMU developers' effort!). Having a software emulation in QEMU is a key enabler when you're writing a new operating system from scratch because you can attach GDB to QEMU to debug the OS. Step 1: Entering the guest The first thing to do is to enter the guest state. In RISC-V, guest kernel mode is called VS-mode. In RISC-V, you just fill a few CSRs. Specifically, hstatus.SPV should be set to 1 before sret instruction: The kernel panicked with an interesting error name: instruction guest-page fault. Yes, CPU has entered the guest mode! Step 2: First ecall The next step is to run something in th...