Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
The workflow of the software engineer has been transformed in recent years through the flow of artificial intelligence coding tools Indicator The GitHub Copilot, which promises to enhance productivity by writing code lines automatically, installing errors, and test changes. Tools are run by the artificial intelligence models from Openai, Google DeepMind, and Hone -and XAI that they have Their performance increased quickly On a set of software engineering tests in recent years.
However, a New study It is published on Thursday by the Non -profit artificial intelligence group to the extent in which the tools of artificial intelligence coded today are enhanced by the productive developers of experienced developers.
Metr conducted a random experience governed by this study by employing 16 experienced source developers and making them complete 246 real tasks on large code warehouses in which they contribute regularly. The researchers assigned nearly half of these tasks randomly as “AI-Lebored”, giving developers permission to use modern artificial intelligence coding tools such as Cursor Pro, while the other half of the tasks prevents the use of artificial intelligence tools.
Before completing the tasks allocated to them, developers expected that the use of artificial intelligence coding tools would reduce the time of completion by 24 %. This was not the case.
The researchers said: “It is surprising that we find that allowing the sponing organization actually increases the 19 % completion time – developers are slower when using artificial intelligence tools.”
It is worth noting that only 56 % of the study developers have experience in using the index, which is the main artificial intelligence tool provided in the study. While almost all developers (94 %) have experience in using some web -based LLMS in their coding function, this study was the first time that some indicator was used specifically. Researchers note that developers have been trained to use the index in preparation for study.
However, the results of Metr raises questions about the supposed global productivity gains promised by artificial intelligence coding tools in 2025. based on the study, developers should not assume that artificial intelligence coding tools – specifically what is known as “programmers” – will immediately accelerate their work.
Metr researchers refer to some possible reasons that make artificial intelligence slowing the developers instead of speeding them up: developers spend more time to push artificial intelligence and wait for response when using programmers in elegance instead of actually coding. Artificial intelligence also tends to struggle in the rules of large and complex software instructions, which this test used.
The authors of the study are keen not to extract any strong conclusions of these results, noting that they do not believe that artificial intelligence systems are currently failing to accelerate or most of the program developers. last Wide -scale studies It has shown that artificial intelligence coding tools accelerate the software engineer.
Authors also notice that the progress of artificial intelligence has been great in recent years and that they will not expect the same results even three months from now. Metr has also found that artificial intelligence coding tools have greatly improved their ability Kamel complex, long horizon tasks In recent years.
However, the research provides another reason to be skeptical of the promised gains of artificial intelligence coding tools. Other studies have shown that today’s artificial intelligence coding tools can Provide errors In some cases, Security gaps.