Scientists warn that artificial intelligence is wreaking havoc in the world of research

Scientific papers depend on readers trusting their information. That’s why it’s annoying that A New study Researchers associated with Cornell and UCLA found 146,900 artificial intelligence-generated fake citations in scientific papers hosted across four major research databases.

One of the major limitations of large language models like Gemini and ChatGPT is their tendency to produce information that seems plausible but is incorrect. A phenomenon known as hallucinations. If a researcher relies on a chatbot to formulate citations without verifying them, the model may generate completely fabricated references.

While scientific papers are often hidden from the public eye, the research they publish has a profound impact on our lives. Everything from Internet to Lithium-ion batteries It started as a research paper.

But when scientists submit papers suggesting AI hallucinations, it can erode confidence in the quality of the research.

Trivial science

The research team analyzed 111 million references from 2.5 million scientific papers. They searched for citations with titles that the team could not match to any publication. While some of these cases were simply misspellings, the team also found hallucinations.

Unscrupulous researchers have been faking citations long before chatbots existed, so the team also examined unparalleled citation rates in papers published before 2023, when chatbots were not yet ubiquitous.

“We found a sharp rise in non-existent references after widespread adoption of the MBA,” the authors wrote in the paper.

The team also found that bad citations were spread across many papers rather than concentrated in just a few. This suggests that the problem is widespread, with many researchers relying on AI-generated references without fully verifying them.

Warning signs

Osha Haley, a professor of management at Wichita State University, told CNET via email that she sees the spread of fake citations as a serious warning.

“Fake or AI-generated citations undermine trust in the scientific record that provides the foundation on which peer review and cumulative knowledge rest,” Healey said. “It is worrying that these doubts are now coming from within academia itself and from early-career researchers.”

The four databases where researchers found fake citations were arXiv, bioRxiv, SSRN, and PubMed Central. These organizations, known as scientific repositories, play a major role in the world of research.

Before publishing a paper in a scientific journal, authors often upload it to a scientific repository, which increases its visibility and allows the global scientific community to access it immediately. The new paper on hallucinogenic citations for artificial intelligence is currently hosted on arXiv.

Recently, arXiv has taken steps to stem the flow of false citations. Organization Announced Tuesday It will ban authors who present their works with hallucinogenic citations or with any sign of AI content that has not been carefully vetted.

“The body of science has become diluted. A lot of AI stuff is either wildly wrong or doesn’t make sense. It’s just noise,” said arXiv’s scientific director, Stein Sigurdsson. He said CNET’s Caitlin Shedrawi returns in February. “It makes it difficult to find what’s really going on, and can mislead people.”