How did the government decide that OpenAI’s frontier model was safe to release?

OpenAI is releasing its latest advanced LLM software, Sol, for wide public access. Sol is at least on par with Anthropic’s Fable, a model whose abilities (or ownership) so stressed the White House that it was briefly banned from public access.

So how did these models get approved for release? Short answer: No one is quite sure.

“Honestly, I don’t have a clear view of these precise processes, so yeah, I don’t feel like I have enough information to say whether it’s adequate or not,” Meena Narayanan, a senior research analyst at Georgetown’s Center for Security and Emerging Technology, told TechCrunch. “Anthropic said they have been in talks with the government, have developed a classifier to detect jailbreak attempts, and have implemented defensive gap strategies to prevent future jailbreaks, but the exact shape of this dialogue between the government, Anthropic, and OpenAI is unclear.”

Dean W. Paul, a former Trump policy advisor who now works at OpenAI, books “No one knows what the requirements are to get a license,” he wrote in his newsletter last month.

Andy Konwinski, a computer scientist who co-founded Databricks, Perplexity, and the Laude Institute, said he’s never talked to anyone who understands the process, even staff at frontier labs. “It’s an existential problem,” he tells TechCrunch. “Whether it’s safety or not, it’s about who has the power to make the decisions — who’s gatekeeping and deciding permissions?”

Eighteen months into the Trump administration, there is still little clarity about how to move forward, despite — or as some critics claim — because of the industry figures who set policy. Last month after Weeks of infightingan executive order laying out a roadmap for evaluating border models has been published, but the details haven’t been filled in yet, other than what won’t be there. “There will be no FDA for AI,” said Sriram Krishnan, a former Andreessen Horowitz partner who served as a senior AI advisor in the White House until last month. He said Financial Times.

Notably, there is still no agreement on what types of models require government auditing, or which agency or agencies should conduct these evaluations. For now, the Commerce Department’s Center for AI and Innovation Standards appears to be taking the lead, but the executive order directs six Cabinet agencies to determine the final process by early August. What has emerged in the meantime is, at best, a private matter.

OpenAI CEO Sam Altman He said CNBC reported that the process included conversations with officials such as Commerce Secretary Howard Lutnick, Treasury Secretary Scott Besent, and US National Cyber ​​Director Sean Cairncross, but it’s not clear which experts tested the models or how they did so. OpenAI declined to share details of the government process with TechCrunch, but pointed to the results of several external evaluations conducted by organizations such as UK AISI, SecureBio and Irregular in the latest model. Safety card.

As with Anthropic’s Fable release, OpenAI previewed the model for the government and selected users before wider release, but we don’t know who all those users are or how they were selected. In late June Blog post“We do not believe this type of government access process should become the default in the long term,” the company said, saying it would work with the government to develop a different path forward.

However, the background to those conversations includes Altman an offer Up to 5% of OpenAI shares in exchange for management’s so-called “Trump accounts” and OpenAI chief Greg Brockman’s role as president. Largest publicly known donor Trump’s midterm political process. It is difficult for outside observers to separate these activities from the government’s watered-down approach to regulating Seoul.

On the other hand, Anthropic’s Fable was briefly withdrawn from wider access when the US government banned its use by foreign citizens, partly due to real concerns about users jailbreaking the form to access hacking capabilities and partly due to personality clashes between Anthropic and the Trump administration. The threat of an export ban may have prompted OpenAI to be more cooperative with (unknown) government requests.

From an industry perspective, a hands-off approach to regulation may be nice, but an approach that relies on personal relationships with management officials creates uncertainty and poor incentives.

Konwinski told TechCrunch that he’s concerned that the real experts in this technology — “safety researchers, alignment researchers, interpretability researchers, but also people who work in the data, people from all over the group” — aren’t playing enough of a role in the model release process.

Konwinski He argues “Open commons” is the best way to truly balance safety and innovation. He points to models such as the US Food and Drug Administration, the National Institutes of Health, or national laboratories, which bring together researchers, government officials and private companies to reach consensus on safety issues.

Some of this is due to the incentives of capitalism that have motivated AI researchers for more than a decade, and which played out in the courtroom during Elon Musk’s lawsuit against OpenAI’s corporate structure. Paul points out that the nature of the AI ​​business requires companies to recover a significant portion of their training costs soon after their models are launched and outperform the competition.

“Even if their intentions are good, there are very clear legal obligations and fiduciary responsibility built into operating procedures,” Konwinski points out.

Ball, in His positionHe argued that the way forward would depend on third-party audit organizations, licensed by the government, that would evaluate frontier labs’ approach to safety. Konwinski is also optimistic about new institutional forms such as focused research organizations that can help more disinterested experts from academia and the nonprofit world access and evaluate pioneering models.

For now, the secrecy surrounding AI development will not go away, but it will also create political challenges for an industry that Americans aspire to. A look growing with suspicion. “There is no sense that responsible people are driving these changes forward,” Ramzi Arbaci Dosso, a computer science professor at the University of Wisconsin-Madison, said last week at the Open Borders Conference.

At the same event, David Siegel, a computer scientist who founded Two Sigma, one of the most successful quantitative hedge funds, asked the audience to “imagine a situation, which I think would be very bad, where a small number of companies control the technology; the government, in its secret laboratories, evaluates whether the technology is suitable for use; and the general public and the scientific community don’t have access to any of this stuff.”

It seems we don’t need to imagine it.