Why AI Isn't Ready to Be a Real Coder

Summary

Artificial intelligence (AI) has transformed the coding sphere, with AI coding tools completing source code, correcting syntax errors, creating inline documentation, and understanding and answering questions about a codebase. As the technology advances beyond automating programming tasks, the idea of full autonomy looms large. Is AI ready to be a real coder?A new paper says not yet—and maps out exactly why. Researchers from Cornell University, MIT CSAIL, Stanford University, and UC Berkeley highlight key challenges that today’s AI models face and outline promising research directions to tackle them. They presented their work at the 2025 International Conference on Machine Learning.The study offers a clear-eyed reality check amid all the hype. “At some level, the technology is powerful and useful already, and it has gotten to the point where programming without these tools just feels primitive,” says Armando Solar-Lezama, a co-author of the paper and an associate director at MIT CSAIL, where he leads the computer-aided programming group. He argues, however, that AI-powered software development has yet to reach “the point where you can really collaborate with these tools the way you can with a human programmer.”Challenges With AI Coding ToolsAccording to the study, AI still struggles with several crucial facets of coding: sweeping scopes involving huge codebases, the extended context lengths of millions of lines of code, higher levels of logical complexity, and long-horizon or long-term planning about the structure and design of code to maintain code quality.Koushik Sen, a professor of computer science at UC Berkeley and also a co-author of the paper, cites fixing a memory safety bug as an example. (Such bugs can cause crashes, corrupt data, and open security vulnerabilities.) Software engineers might approach debugging by first determining where the error originates, “which might be far away from where it’s crashing, especially in a large codebase,” Sen explains. The...