I am a professional fact checker. Artificial intelligence is wrong more often than you think


Nearly half Americans say so Use artificial intelligence To find information and generate ideas. It’s not hard to see why. With the development of social media slope— and Google to a glorified landing page for Reddit threads and content farms — most of us are hungry for something reliable. In addition, so are chatbots cooperatingIsn’t it? The first time I interacted with one of these sites, I asked if he knew it was a huge drain on resources. Half an hour later, I had a new vegan cream cheese recipe.

I’ve never tried the recipe. Instead, I found a man-made one that my LLM may have discarded. This is how these models work, of course. They repackage collective knowledge into something that feels tailor-made for you. This may be fine for dairy alternatives (unless you’re a vegan blogger). But on the order of the world, and fact— The focus of my role as WIRED’s fact-checker — the stakes are dramatically higher.

Over the past year or so, more and more people have looked at me with great pity. A fact-checker at a magazine certainly won’t be long in this AI-upgraded world. Call me foolish, but I’m not that worried. I have concluded that very little of humanity’s collective knowledge lives on the Internet. And according to my research, AI is more wrong than people might think.

Obviously Tom Wolfe Fact checkers thought, according to the writer Colin Dickeyas “a cabal of women and mediocre editors all cooperating in weakening and weakening the great writer’s prose.” By definition, it’s not bad (although my boss and many of my colleagues are men). What can I say? It’s our mission, Unlike artificial intelligenceto be annoying.

WIRED’s fact-checking department is old school: careful line-by-line annotations, primary sources whenever possible, and extensive ethical and legal review. We question basic assumptions, look for new or conflicting information, contact people and talk to them – and confirm. It’s fast-paced peer review, working as best it can at the same pace as the news itself.

As far as I can tell, artificial intelligence has not come to this process yet. What he’s here for is “after-the-fact” fact-checking, that is, Snopes-style analysis of the reality of something after the fact. In the United Kingdom, an initiative called The whole truth It has built its own artificial intelligence tools to help thwart the spread of misinformation. These tools, used in more than 40 countries, process massive amounts of data, from social media posts to podcast transcripts, and then pinpoint specific claims that humans can investigate further. “You definitely need a human,” says Mark Frankel, head of public affairs at Full Fact.

The reason for this is simple: AI still gets things wrong. As a fact-checker, I’d like to be able to tell you exactly how many times. But it’s not that easy. Since 2018, nearly 17,000 research papers have been published Published on arXiv In the MBA, many of them focused specifically on the issue of its reliability. However, it is worth trying to identify a working figure.

In any article presented to WIRED’s fact-checking desk, there is usually a fair amount of “topic”: statistics, news events, quotes, anything that helps put the topic in context. Fact checkers tend to Google this basic and practical information Overviews of scary artificial intelligenceIt constitutes my main interaction with artificial intelligence. In my professional opinion, it is unusable, and wrong, in about a third of cases.

That may be a generous assessment, though. March 2025 study from Tu Center for Digital Journalism It found that more than 60 percent of responses from AI-powered search engines were inaccurate. A study conducted by the BBC highlights the error of chatbots Closer to 45 percentthe number I see cited most often. Since the percentages are so far apart, let me put this more clearly: AI might be wrong in about half of the cases.

Leave a Reply

Your email address will not be published. Required fields are marked *