Part 1: A Deep Dive into Rust and C Memory Interoperability

Summary

“Memory oppresses me.” - Severian, The Book of the New Sun Interviewer: “What happens if you allocate memory with C’s malloc and try to free it with Rust’s dealloc, if you get a pointer to the memory from C?” Me: “If we do it via FFI then there’s a possibility the program may continue working (because the underlying structs share the same memory layout? right? …right?)” Now if you have any experience working with memory management, you know that this is a dangerous answer. But I didn’t know it at the time. I was just trying to get through the interview. But I realized at that moment that I had been treating memory allocators like black boxes. I knew the rules - never mix allocators - but I didn’t truly understand why. So here’s my attempt at de-mystifying memory management, starting with the fundamentals and building a testing laboratory to explore what happens when different memory worlds collide. Prerequisites# To get the most from this article, you should be familiar with: Basic Rust and C programming Pointers and memory management concepts Command line tools (bash, gcc, cargo) Basic understanding of stack vs heap Don’t worry if you’re not an expert, I’m not one either - but I’ll explain concepts as best I can! Table of Contents# The Interview Question That Started Everything Why Memory Allocators Don’t Mix Memory Fundamentals: Building Our Mental Model Building a Memory Testing Laboratory First Experiments: Surprising Results Key Takeaways and What’s Next The Interview Question That Started Everything# It was Friday afternoon when I had an interview for an amazing startup which focuses on building very high performance systems. The interview experience was intense while being highly rewarding. We touched upon topics async runtimes, memory management, rust FFI etc. The intention wasn’t to test my language specific knowledge but being able to reason about how these systems work at a level closer to the machine. It caught me a little offguard. It’s not something I ...