AI-powered disinformation hordes are coming for democracy

“We are moving into a new phase of information warfare on social media platforms, where technological developments have made the classic bot approach obsolete,” says Jonas Kunst, professor of communications at BI Norwegian Business School and one of the report’s co-authors.

For experts who have spent years tracking and combating disinformation campaigns, this paper represents a terrifying future.

“What if AI wasn’t just hallucinating information, but thousands of AI chatbots were working together to give the appearance of popular support where there was none? That’s the future this paper imagines — Russian troll farms on steroids,” says Nina Yankovic, a former disinformation czar in the Biden administration who is now CEO of the American Sunlight Project.

The researchers say it’s unclear whether this tactic has actually been used, because current systems in place to track and identify coordinated inauthentic behavior are unable to detect it.

“Because of their elusive properties that mimic humans, they are very difficult to detect and assess their existence,” says Kunst. “We lack access to most (social media) platforms because the platforms are becoming increasingly restricted, so it’s difficult to get insight there. Technically, it’s definitely possible. We’re sure it’s being tested.”

Kunst added that these systems are likely to remain subject to some human oversight while they are developed, and he expects that while they may not have a massive impact on the 2026 US midterm elections in November, they will very likely be deployed to disrupt the 2028 presidential election.

Accounts that are indistinguishable from humans on social media platforms are just one problem. The ability to map social networks at scale will allow those coordinating disinformation campaigns to target customers in specific communities, ensuring the greatest impact, researchers say.

“Using these capabilities, swarms can localize themselves for maximum impact and tailor messages to the beliefs and cultural cues of each community, enabling more precise targeting than that experienced with previous botnets,” the researchers wrote.

These systems can essentially improve themselves, using responses to their posts as comments to improve their logic in order to better communicate the message. “With enough signals, they can run millions of microA/B tests, propagate winning variants at machine speed, and iterate much faster than humans,” the researchers wrote.

In order to combat the threat posed by AI swarms, the researchers propose creating an “AI Impact Observatory,” which would consist of people from academic groups and NGOs working to “consolidate evidence, improve situational awareness, and enable faster collective response rather than imposing top-down reputational penalties.”

One group that was not included were executives from the social media platforms themselves, primarily because the researchers believe their companies incentivize engagement more than anything else, and thus they have little incentive to identify these crowds.

“Suppose AI swarms become so frequent that you can’t trust anyone, and people leave the platform,” says Kunst. “Of course, it threatens the model. If they just increase engagement, it’s better for the platform not to disclose that, because it seems like there’s more engagement, seeing more ads, that would be positive for the valuation of a particular company.”

In addition to the lack of action on the part of platforms, experts believe there is little incentive for governments to participate. “The current geopolitical landscape may not be conducive to ‘observatories’ that essentially monitor online discussions,” Oleinik says, something Yankovic agrees with: “The scariest thing about this future is that there is very little political will to address the damage caused by AI, which means[AI swarms]may soon become a reality.”