A software developer sits down to a task she has done a hundred times, in a codebase she knows well. Today she is allowed to use AI. Asked to predict how much it will speed her up, she says about twenty-four percent. She works, the AI at her side, and finishes. Asked how it went, she is sure she was faster, maybe twenty percent.
Then they check the screen recording. She was not twenty percent faster. She was nineteen percent slower.1
What was the study?
This is the central finding of a 2025 randomised controlled trial by a research group called METR. They took experienced open-source developers, gave them real tasks in repositories they knew, and let them use the best AI coding tools available. The developers expected a twenty-four percent speed-up. Even after finishing, they believed they had gained twenty percent. In fact, on average, the work took them nineteen percent longer with AI than without it.2 Outside experts had predicted a large speed-up too. Everyone was wrong, and wrong in the same direction.
Why the gap between feeling and reality?
There are two stopwatches running on every piece of work you do with AI. One is in your head, and it reports how the work felt. The other is on the wall, and it records what actually happened. The promise of AI is told entirely by the first stopwatch.
The screen recordings explained the slowdown. Developers spent less time actually coding and searching, which felt like progress, but more time prompting the model, waiting for it, and reviewing and correcting what it produced.3 The work felt smoother and easier, and smoother and easier is exactly what fools the internal stopwatch. Effort fell while elapsed time rose.
Does this mean AI is useless for coding?
No, and the honesty cuts both ways. The result is specific to expert developers working in mature codebases they already understand. That is the hardest case for AI to help, because the human already holds the context the model lacks, and verifying the model’s output costs more than the help is worth. Other rigorous studies show real gains for less experienced workers and for well-defined tasks.4 METR itself has flagged that the slowdown estimate carries a wide confidence interval and is being studied further.5
What is the lesson for everyone, not just coders?
Trust the wall clock, not the feeling. You are, when it comes to your own time, an unreliable narrator, and AI is unusually good at telling you a flattering story. Before you conclude that a tool is saving you time, check whether it actually is. The feeling of speed is not evidence of speed.
This is why the rest of the method starts with measurement. You cannot reclaim time you cannot see, and the time you think you saved is often time you spent supervising a machine that produced something you then had to redo. The cure is not to abandon AI. It is to stop grading it on how it feels.
Frequently asked questions
Did a study find AI makes developers slower?
Yes. The 2025 METR randomised trial found that experienced open-source developers took 19 percent longer to complete tasks with AI than without it, in codebases they knew well, even though they predicted and perceived a speed-up.
Why did developers think AI sped them up when it slowed them down?
Because the work felt smoother and easier. Screen recordings showed less time coding and searching but more time prompting, waiting, and reviewing AI output, which felt like progress while adding to the clock.
Does this mean AI never helps coding?
No. The result is specific to expert developers in mature, familiar codebases. Other studies show clear gains for less experienced workers and well-defined tasks. The lesson is to measure real time rather than trust the feeling.
About the author
Tom Goodwin is the author of Don’t Work Harder, a book about taking the time AI gives back as time rather than more work. He is a co-founder of GAMEPLAN and writes on productivity, technology, and the economics of the working week.
Footnotes
Footnotes
-
METR (Joel Becker, Nate Rush, Elizabeth Barnes, David Rein), “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity,” arXiv:2507.09089 (July 2025). ↩
-
METR (2025): developers predicted a 24 percent speed-up and estimated a 20 percent speed-up afterward, against a measured 19 percent slowdown. ↩
-
METR (2025): screen recordings showed less time coding and searching, but more time prompting, waiting, and reviewing AI output. ↩
-
For example, Brynjolfsson, Li, and Raymond, “Generative AI at Work” (2023), found a 14 percent gain among support agents, concentrated among the least experienced. ↩
-
METR, “We are Changing our Developer Productivity Experiment Design” (February 2026): the estimate has a wide confidence interval and the result is specific to expert developers in mature codebases. ↩