How Can AI Researchers Save Energy? By Going Backward

Summary

This loss is a fundamental aspect of how computers operate. For example, when a computer adds two numbers together, it returns a single number for the total: 2 + 2 = 4. There’s a loss of information as you go from two numbers to one. You could have added 2 and 2, or you could have combined 1 and 3. The missing information makes the calculation irreversible. Computers that process information this way — and almost all of them do — are always going to lose some information as heat, no matter what. Landauer wondered if a machine could get around this limitation by simply never deleting data. Such a device would need to keep a record of every operation, every pair of numbers added at every step. These records would rapidly fill its memory, making such a computer unusable in practice, despite the energy savings. Landauer soon moved on, convinced that reversible computing was a dead end. A decade later, he learned that he had been mistaken. Hitting Reverse Charles Bennett, a younger colleague of Landauer’s at IBM, argued in 1973 that there was another option. Instead of saving every single scrap of information, you could run each calculation forward, store the result you care about, then run the calculation backward. Bennett’s idea, which he called uncomputation, is a little like if Hansel and Gretel picked up their trail of breadcrumbs on the way back home: The pair are guaranteed not to get lost, and they don’t waste any breadcrumbs. Uncomputation means you are left with only the data you want, and you never lose track of it. Because none of the initial information is deleted, you never lose energy to heat. Unfortunately, uncomputation also takes twice as long as an ordinary computation, which makes it impractical. Still, Bennett continued to improve on his idea. In 1989, he showed that you can uncompute in much less time by using slightly more memory. Researchers began to tinker with the details, finding ways to shave off further memory and time. But computers don’t lo...