Apple Research unearthed forgotten AI technique and using it to generate images

Summary

Today, most generative image models basically fall into two main categories: diffusion models, like Stable Diffusion, or autoregressive models, like OpenAI’s GPT-4o. But Apple just released two papers that show how there might be room for a third, forgotten technique: Normalizing Flows. And with a dash of Transformers on top, they might be more capable than previously thought. First things first: What are Normalizing Flows? Normalizing Flows (NFs) are a type of AI model that works by learning how to mathematically transform real-world data (like images) into structured noise, and then reverse that process to generate new samples. The big advantage is that they can calculate the exact likelihood of each image they generate, a property that diffusion models can’t do. This makes flows especially appealing for tasks where understanding the probability of an outcome really matters. But there’s a reason most people haven’t heard much about them lately: Early flow-based models produced images that looked blurry or lacked the detail and diversity offered by diffusion and transformer-based systems. Study #1: TarFlow In the paper “Normalizing Flows are Capable Generative Models”, Apple introduces a new model called TarFlow, short for Transformer AutoRegressive Flow. At its core, TarFlow replaces the old, handcrafted layers used in previous flow models with Transformer blocks. Basically, it splits images into small patches, and generates them in blocks, with each block predicted based on all the ones that came before. That’s what’s called autoregressive, which is the same underlying method that OpenAI currently uses for image generation. Images of various resolutions generated by TarFlow models. From left to right, top to bottom: 256×256 images on AFHQ, 128×128 and 64×64 images on ImageNet. Source: Normalizing Flows are Capable Generative Models The key difference is that while OpenAI generates discrete tokens, treating images like long sequences of text-like symbols, Apple’s ...