Apple's Cubify Anything: Scaling Indoor 3D Object Detection

Summary

CA-1M and Cubify Anything This repository includes the public implementation of Cubify Transformer and the associated CA-1M dataset. Paper Apple Cubify Anything: Scaling Indoor 3D Object Detection Justin Lazarow, David Griffiths, Gefen Kohavi, Francisco Crespo, Afshin Dehghan CVPR 2025 Repository Overview This repository includes: Links to the underlying data and annotations of the CA-1M dataset. Links to released models of the Cubify Transformer (CuTR) model from the Cubify Anything paper. Basic readers and inference code to run CuTR on the provided data. Basic support for using images captured from own device using the NeRF Capture app. Installation We recommend Python 3.10 and a recent 2.x build of PyTorch. We include a requirements.txt which should encapsulate all necessary dependencies. Please make sure you have torch installed first, e.g.,: pip install torch torchvision Then, within the root of the repository: pip install -r requirements.txt pip install -e . CA-1M versus ARKitScenes? This work is related to ARKitScenes. We generally share the same underlying captures. Some notable differences in CA-1M: Each scene has been exhaustively annotated with class-agnostic 3D boxes. We release these in the laser scanner's coordinate frame. For each frame in each capture, we include "per-frame" 3D box ground-truth which was produced using the rendering process outlined in the Cubify Anything paper. These annotations are, therefore, independent of any pose. Some other nice things: We release the GT poses (registered to laser scanner) for every frame in each capture. We release the GT depth (rendered from laser scanner) at 512 x 384 for every frame in each capture. Each frame has been already oriented into an upright position. NOTE: CA-1M will only include captures which were successfully registered to the laser scanner. Therefore not every capture including in ARKitScenes will be present in CA-1M. Downloading and using the CA-1M data Data License All data is released und...