The iPhone 15 Pro's Depth Maps

Summary

Since 2017, Apple have supported depth maps in the images its iPhones capture either via LiDAR scanners, 3D time-of-flight scanner-less LIDAR or structured-light 3D scanning. These depth maps, along with other imagery, are stored in High Efficiency Image File Format (HEIF) container files. These files can contain multiple images and vast amounts of metadata. This format was originally designed between 2013 and 2015 and Apple adopted its HIEC variant in 2017. Since then, HEIC files are the default storage container format for images captured on the iPhone. But with that said, it is possible to use the JPEG format instead if things like depth maps and HDR are not of interest. Finn Jaeger, who is the head of VFX at Replayboys, a film production firm in Hamburg, Germany, posted a screenshot a few weeks ago showing how multiple depth maps were being produced by his iPhone. He announced he was working on a project called HEIC Shenanigans. This project contains scripts to separate out images and their metadata from HEIC containers as well as convert them into EXR Files. As of this writing, the project contains 374 lines of Python. In this post, I'll walk through Finn's codebase on an example image from an iPhone 15 Pro. My Workstation I'm using a 5.7 GHz AMD Ryzen 9 9950X CPU. It has 16 cores and 32 threads and 1.2 MB of L1, 16 MB of L2 and 64 MB of L3 cache. It has a liquid cooler attached and is housed in a spacious, full-sized Cooler Master HAF 700 computer case. The system has 96 GB of DDR5 RAM clocked at 4,800 MT/s and a 5th-generation, Crucial T700 4 TB NVMe M.2 SSD which can read at speeds up to 12,400 MB/s. There is a heatsink on the SSD to help keep its temperature down. This is my system's C drive. The system is powered by a 1,200-watt, fully modular Corsair Power Supply and is sat on an ASRock X870E Nova 90 Motherboard. I'm running Ubuntu 24 LTS via Microsoft's Ubuntu for Windows on Windows 11 Pro. In case you're wondering why I don't run a Linux-based desktop a...