Debugging Rustler on Illumos

Summary

Welcome to SYSTEM•ILLUMINATION! This is the first illumination I have written and the one that prompted me to start this space. This first session tackles several topics as you join me on the journey I took to debug Rustler misbehaving on OmniOS. I'm a beginner with illumos. This page serves a twofold purpose: to help me document and clarify my learnings as I delve into the illumos/Solaris world. And to shine a bit of light into a system that is fairly obscure and hard to get good info on. However, because I'm also learning it, I may be wrong on many things. Use the information in these pages as a starting point for your own investigations. From Linux to illumos Recently I decided I wanted to delve deeper into the illumos ecosystem; I had briefly used it a couple of years ago for a small server I was running. Two weeks ago, I had to decide on and build the final infrastructure for Katarineko a personal project I'm about to release. I had the choice between using my trusted Linux or giving in to the voice in my head that said USE ILLUMOS!. I did give in, maybe not the sanest choice, but it is my choice. Katarineko is built with Elixir, a language that runs on the Erlang virtual machine. For reasons I will not get into here, I also have a couple of NIFs written in Rust. NIFs are the main way you can call into compiled languages from Erlang/Elixir. The Rustler project is a very straightforward way of integrating NIFs written in Rust, it handles compilation lifecycle, exposing all the right structures and functions, and serialization between Erlang and Rust, including support for Serde. We'll get into how this works a bit further down. Migrating was quite simple. I used pkgsrc to get Elixir packages, set up some zones, and I was up and running within a day. At first, everything appeared to be running properly: there were no errors, and the page was running smoothly. I shared the beta version with a friend that was testing and he tried to use the Spaced-Repetition system...