IBM's Dmitry Krotov wants to crack the 'physics' of memory

Summary

Dmitry “Dima” Krotov was among the first to congratulate AI pioneer, John Hopfield, on his Nobel Prize in Physics last fall. “John, wow!” he texted Hopfield on the morning the award became public. “Just WOW!!”As Hopfield’s close collaborator, Krotov has helped explain to the world following the announcement how Hopfield’s single-layer digital neural network led to the “deep” networks in use today. At Princeton, the two researchers invented something called dense associative memory, which lifted the memory storage limits of those early Hopfield networks, opening them to practical applications.Now a researcher at IBM, Krotov is carrying on Hopfield’s ideas by building computational models to improve artificial intelligence, and to even understand the underpinnings of intelligence itself.IBM's Dmitry Krotov studies computation in biological and digital neural networks using the mathematical tools of physics.Associative memory may never displace transformers as the backbone of generative AI, but it could provide ideas for making AI more transparent and comprehensible to us humans. And Krotov’s parallel work, using associative memory to model biological computation, could help explain how our brains manage to squeeze so much information into such a small space.It may not be immediately clear what this all has to do with physics, the science concerned with the nature and evolution of physical matter. But if you have the time, Krotov will elaborate at length, something he now does regularly following talks he gives beyond IBM’s Cambridge lab where he works.“Computation is a physical process,” he said recently. “We can study the flow of bits just as we study the flow of atoms.”Rolling down the “energy” landscapeTraditional software runs on hard-coded instructions for processing data. The nodes and weights underpinning today’s LLM chatbots, by contrast, are designed to learn from raw data, with no explicit instruction, allowing them to take on more complex, open-ended proble...