Google’s new Scholar Labs search uses artificial intelligence to find relevant studies

Google has it Announce It’s testing a new AI-powered search tool, Scientists’ laboratorieswhich are designed to answer detailed research questions. But her presentation highlighted a larger question about finding “good” scientific studies. How much will scientists trust a tool that abandons typical ways of measuring a study’s popularity with the scientific establishment in favor of reading relationships between words to help highlight good research?

The new search tool uses artificial intelligence to identify key topics and relationships in a user’s query and is currently available to a limited group of logged in users. A demo video from Scholar Labs showed a question about brain-computer interfaces (BCIs). I have a PhD in BCIs, so I was excited to see what Scholar Labs came up with.

The first result was a review paper of BCI research published in 2024 in a journal called Applied sciences. Scholar Labs includes explanations for why the results match the query, so it notes that the paper discusses research into a non-invasive signal called EEG and surveys some of the leading algorithms in this field.

But I noticed that Scholar Labs lacks the common metrics filters used to separate “good” studies from “not so good” studies. One measure is the number of times a study has been cited by other studies since its publication, which loosely translates to the popularity of the paper. It is also related to time: a recently published study may have no citations or their number may number in the hundreds within a few months; A study from the 1990s may sell thousands. Another measure is the “impact factor” of a scientific journal. Journals that publish studies that are widely cited have a higher impact factor and thus have a reputation for being more rigorous or meaningful to the scientific community. Applied sciences Self-reports a Impact factor 2.5. natureFor comparison, its impact factor says it is 48.5.

The original Google Scholar has an option to sort studies by “relevance” and lists the number of citations for each result. Google spokeswoman Lisa Ogwueke said the goal of the new Scholar Labs is to search for “the most useful research for user research.” Edge It does this by ranking research papers in the same way that researchers themselves do, Google says, by “weighting the full text of each document, where it was published, who wrote it, as well as how often and recently it has been cited in other scientific literature.”

However, the new Scholar Labs will not sort or select results based on a paper’s number of citations or a journal’s impact factor, Ugwueke said. Edge.

“Impact factors and citation counts depend on the research field of the papers, and it may be difficult for most users to guess appropriate values ​​in the context of specific research questions,” Ugwueke wrote. “Restriction by impact factor or number of citations may often lead to missing key research papers – in particular, papers in interdisciplinary areas/adjacent fields/journals or recently published articles,” Ugwueke added.

Metrics like the number of citations and impact factor are “very rough assessments of the quality of a paper,” Matthew Schrag, an assistant professor of neuroscience at Vanderbilt University Medical Center, said in an interview with Newsweek. Edgeagreeing with Google’s statement. He said they “talk more about the social context of the paper” rather than its quality, although “those two things are hopefully interconnected.”

Schrag, who conducts research on Alzheimer’s disease, is one of several investigative scientists who have reported on Alzheimer’s disease. Questionable data in published scientific studies. The efforts of data investigators like Schrag, and the close attention of the scientific community in general, have led to studies being withdrawn from prestigious journals due to… Processed imagesCorrections issued by Nobel Prize winnersand Federal investigations to false data

However, it is difficult to do so no Use the number of citations or reputation of the journal to casually examine the study, especially when entering a new field. Tufts University rehabilitation sciences professor James Smoliga, a frequent user of the original Google Scholar, finds himself believing that highly cited research is more trustworthy. “I’m as guilty of this as everyone else,” he said. Edge. He does so despite his presence expose Methods used in a study with thousands of citations. “And I know that’s not the case, but I still fall into this trap, so what else am I going to do?”

I replicated the Scholar Labs pilot inquiry into BCI research for stroke patients in PubMed, a leading repository of biomedical and health research managed by the National Library of Medicine of the US National Institutes of Health. Unlike Scholar Labs, PubMed relies heavily on filters and associated terms orsand andQ. I narrowed my results to review only clinical research articles, that is, only those conducted in humans, from the past five years. I excluded preprints, which are studies that are published directly into a paper repository such as arXiv or bioRxiv without undergoing review by other scientists. Two of the six findings focused exclusively on EEG as the primary type of non-interventional BCI used to help stroke patients.

Users will be able to request “recent” research papers in their query and specify a time period in their request, Ugwueke added, and Scholar Labs uses “full text papers” to find results that match a user’s query.

Google calls Scholar Labs “a new direction for us” and says it plans to incorporate user feedback in the future. It has a waiting list for access.

Schrage believes that AI-powered research, like the new Scholar Labs research, has a place in the scientific ecosystem. He added that it could, in theory, cast a wider net to surface papers that were slipping through the cracks, or add additional context about the paper’s popularity across social media platforms. He said the studies need comprehensive evaluation, which artificial intelligence may be able to address. “You have to have an idea of ​​what standards exist in the field in terms of accuracy and whether the study meets that,” he added.

Ultimately, scientists have a responsibility to identify impactful science, Schrag said. It requires reading and engaging with the scientific literature “to be the final arbiters and not let algorithms be the final arbiters of what we consider to be high quality.”