OpenAI and Humanity’s Signature Message to Prevent AI-Developed Bioweapons

CEOs for Several major AI companies are urging members of Congress to adopt new laws that would make it harder for bad actors to develop bioweapons using their technology.

Demis Hassabis of Google DeepMind, Sam Altman of OpenAI, Dario Amodei of Anthropic, and Mustafa Soliman of Microsoft AI were among the signatories. Public speech Advocate for laws requiring companies selling synthetic DNA and RNA to screen customers and orders to prevent misuse of genetic material.

The letter, organized by the nonpartisan Institute for Progress and the right-leaning Foundation for American Innovation, acknowledges that given the pace of AI development, “there is a real possibility that the cognitive barriers that have historically prevented bad actors from obtaining bioweapons will erode.”

Scientist Arthur Kornberg was the first to successfully synthesize DNA in the 1950s. Now this process has become automated, with dozens of companies around the world using commercial synthesizers to “print” and sell customized genetic sequences for use in scientific research, drug development, and diagnostics. Many providers sell only to qualified researchers, biotechnology companies, and educational institutions, but not all are clients of veterinarians or the gene sequences they order.

In 2017, Canadian researchers Triggered the alarm When they used $100,000 worth of mail-order DNA to recreate the extinct horsepox virus. Critics said the same methodology could be used to create smallpox, a closely related deadly virus. Gene synthesis has become cheaper since then.

Combined with advances in artificial intelligence, it is now possible to design dangerous new toxins and pathogens using large language models, although some biological training will likely still be needed to make an effective virus from scratch. Although bioterrorist attacks have been rare, they have the potential to cause mass casualties, public panic, and economic losses. One major concern is that AI-engineered pathogens could intentionally or inadvertently trigger a global pandemic.

“AI tools enable the user to determine very quickly where to go for sequences of requests that would not be subject to screening,” says David Relman, a microbiologist and biosecurity expert at Stanford University, who signed the letter. “If asked appropriately, they can also tell you how to change the nature of your request, so that even those doing the screening may be less able to detect what you are trying to do.”

Other signatories include scientists, national security experts and executives from gene manufacturing companies Twist Bioscience and Ansa Biotechnologies. These companies are members of the International Gene Synthesis Consortium, which was formed in 2009 to implement voluntary screening practices. Many companies already use software to screen orders for “sequences of concern” that could contribute to an organism’s toxicity or ability to cause disease.

“If you have technology that can make DNA, you have to make sure you use it responsibly, and part of that is making sure you understand what you’re making and who you’re making it for,” says James Deggans, vice president of policy and biosecurity at Twist Bioscience. The company has supported implementation of the Official Rules for years.

Federalism Guidelines Introduced during the Biden administration would require scientists and companies that receive federal funding to request synthetic gene sequences from providers that screen purchases. A Bipartisan bill Introduced earlier this year in the Senate, it would require all genetic synthesis providers operating in the United States to screen orders and customers for bad actors or dangerous pathogens.

But screening tools aren’t perfect. Last year, researchers from Microsoft published a He studies It shows that AI protein design tools were able to generate potentially dangerous genetic sequences, which bypassed companies’ screening programs. The models suggested new protein sequences with structures similar to those known to be dangerous.

Jeff Ralston, former president of Y Combinator and a partner at the Safe AI Fund, believes that AI labs with biological models should screen users themselves.

“It should be very difficult, if not impossible, to ask a model to help you do something imminently dangerous,” says Ralston, who also signed the letter.

Relman agrees that regulations on screening procedures are only part of the solution. “Because the test may fail in some cases, we must have other control points,” he says. “This is where AI companies will have to step up.”