The AI coding trap

Summary

If you ever watch someone “coding”, you might see them spending far more time staring into space than typing on their keyboard. No, they (probably) aren’t slacking off. Software development is fundamentally a practice of problem-solving, and so, as with solving a tricky crossword, most of the work is done in your head. In the software development lifecycle, coding is the letters filled into the crossword, only a small amount of effort compared to all the head scratching and scribbled notes. The real work usually happens alongside coding, as the developer learns the domain, narrows down requirements, maps out relevant abstractions, considers side effects, tests features incrementally, and finally squashes bugs that survived this rigorous process. It looks something like this: But with AI-driven coding, things play out very differently. “Code first, ask questions later” AI coding agents such as Claude Code are making it astonishingly fast to write code in isolation. But most software lives within complex systems, and since LLMs can't yet hold the full context of an application in memory at once, human review, testing, and integration needs will remain. And that is a lot harder when the code has been written without the human thinking about it. As a result, for complex software, much of the time will be spent on post hoc understanding of what code the AI has written. This is the root of the difference between marketing copy that boasts of the paradigm shifting speed of writing code with AI (often framed as “10X faster”), and the marginal productivity gains in delivering working software seen in the wild (usually closer to 10%). An even more dispiriting upshot of this is that, as developers, we spend an ever greater proportion of our time merely fixing up the output of these wondrous babbling machines. While the LLMs get to blast through all the fun, easy work at lightning speed, we are then left with all the thankless tasks: testing to ensure existing functionality isn...