Tiny worlds: A minimal implementation of DeepMind's Genie world model

Summary

TinyWorlds is a minimal autoregressive world model built on Google Deepmind's Genie Architecture. World models can't use action-less internet video to scale like VEO3. Deepmind's Genie solves this by inferring the actions between frames using no prior action data. TinyWorlds is meant to help people understand the clever autoregressive, unsupervised method Deepmind likely used to achieve scalable world models. Table of Contents Getting Started # Installation git clone https://github.com/AlmondGod/tinyworlds.git cd tinyworlds pip install -r requirements.txt export WANDB_API_KEY= < YOUR_WANDB_API_KEY > export PYTHONPATH= " /workspace/tinyworlds: $PYTHONPATH " # Training # 1. download data from huggingface python scripts/download_assets.py datasets --pattern " zelda_frames.h5 " # 2. run training python scripts/full_train.py --config configs/training.yaml -- --dataset=ZELDA # Inference # 1. pull pretrained sonic checkpoints from huggingface python scripts/download_assets.py models --suite-name sonic # 2. run inference python scripts/run_inference.py --config configs/inference.yaml -- use_latest_checkpoints=true dataset=SONIC Overview Why World Models? How do we bend reality to our will? we generate reality itself… A world model is simply a function mapping the current state of the environment to the next state of the environment. To predict the next state accurately, the function must compress all information in the world into a set of laws. So the world model captures all the inherent structure and emergent phenomena of the world. In fact, all of deep learning, and all of intelligence, is trying to compress the universe into a model. A model that can predict important aspects of the next state of the universe, by learning heuristics about how it operates. And the universe can also be thought of as a world model. It is a constantly running mapping from state to state, following a set of laws but with many layers of emergent behavior over these laws. So far, we've seen wo...