Why did dlclose not unload the library? (2023)

Summary

Recently, at work, we were debugging an issue where dlclose was not unloading the library . You might wonder why that even matters - if your library contains any global symbols, then their state will persist across a dlopen, dlclose sequence. In other words, your library isn’t starting from a clean state every time. In the case we were debugging, this issue manifested in the following fashion: We had two libraries libA and libB and libA dynamically depends on libB. When you dlopen libA, it also implicitly loads libB since it is a dependency. But when you dlclose libA, you may expect libA to be unloaded but also libB (since it was brought in purely due to dlopening libA). Instead, what was strangely happening was that libA was being unloaded from the process address space but not libB. The next time you load libA & it talks to libB, libA is starting from a clean slate but libB is not. This caused an initialization function in libB to fail since it was already initialized. To give you a little bit more context, libA is a Rust library. libB is a C++ library. We’re dlopen’ing libA from another C++ codebase progA. What made this issue even more mysterious was that when we turned on logging through an environment variable in libA, the issue stopped happening. Ah, a Heisenbug! Skip down to The dlclose unmap condition if you just want to know about dlclose. But first, the story! A debugging tale It all started with some strange symptoms - a Rust construction routine was running multiple times in libA. But it was wrapped in a lazy_static, so it should be only initialized once! The suspicion was immediately on the library somehow being loaded multiple times. Though only one copy of the symbols should be present due to the way dlopen works, the various failure cases around visibility (including modes like RTLD_LOCAL) were always a bit confusing to me. We first started by confirming that the addresses of the static in both calls are the same. Normally, if you had two copies of ...