Horses: AI progress is steady. Human equivalence is sudden

Summary

Horses So after all these hours talking about AI, in these last five minutes I am going to talk about: horses. Engines, steam engines, were invented in 1700. And what followed was 200 years of steady improvement, with engines getting 20% better a decade. For the first 120 years of that steady improvement, horses didn't notice at all. Then, between 1930 and 1950, 90% of the horses in the US disappeared. Progress in engines was steady. Equivalence to horses was sudden. But enough about horses. Let's talk about chess! Folks started tracking computer chess in 1985. And for the next 40 years, computer chess would improve by 50 Elo per year. That meant in 2000, a human grandmaster could expect to win 90% of their games against a computer. But ten years later, the same human grandmaster would lose 90% of their games against a computer. Progress in chess was steady. Equivalence to humans was sudden. Enough about chess! Let's talk about AI. Capital expenditure on AI has been pretty steady. Right now we're - globally - spending the equivalent of 2% of US GDP on AI datacenters each year. That number seems to have steadily been doubling over the past few years. And it seems - according to the deals signed - likely to carry on doubling for the next few years. But from my perspective, from equivalence to me, it hasn't been steady at all. I was one of the first researchers hired at Anthropic. This pink line, back in 2024, was a large part of my job. Answer technical questions for new hires. Back then, me and other old-timers were answering about 4,000 new-hire questions a month. Then in December, Claude finally got good enough to answer some of those questions for us. In December, it was some of those questions. Six months later, 80% of the questions I'd been being asked had disappeared. Claude, meanwhile, was now answering 30,000 questions a month; eight times as many questions as me & mine ever did. Now. Answering those questions was only part of my job. But while it took horses...