Satya Nadella has issued a shocking warning to companies using artificial intelligence

Of all the discussions about the potential downsides of AI, there is one concern that is causing the most concern among AI enthusiasts in Silicon Valley. Their fear is that giant AI labs selling proprietary models somehow behave like Trojans.

The concern is that as startups and enterprises use AI models from labs like OpenAI and Anthropic, the labs will gain increased access to those companies’ most sensitive business information. Model makers can then use that knowledge for themselves, and potentially become competitors to their customers. Those who issue such warnings range from Venture capitalists like Jason Calacanis to Palantir CEO Alex Karp.

And now for a surprise Blog post Published on Monday, Microsoft CEO Satya Nadella has joined the crowd. Nadella warns that AI users (“buyers,” as he calls them) are paying twice. They are intentionally spending to use AI tokens, but they are also, unconsciously, handing over valuable data in the process.

“You’re basically paying for intelligence twice, once with money, and again with something more valuable: the proprietary knowledge you have to uncover to make that intelligence useful. And the better you want the model to perform, the more of that knowledge you have to feed it!” he writes.

What’s even more alarming, he adds, is that companies are literally teaching models about the nuances of their business.

“Models learn from the ‘exhaust’, from the instructions people write, from the tools agents use, and especially the corrections people make when the model is wrong. All correction is distilled into institutional knowledge,” he writes.

This is “the kind of knowledge that no competitor could ever buy,” and yet companies deliver it.

If AI companies can freely exploit the Internet to train their models, Nadella says, it’s only fair that companies study — or “scrape” — those models in return. “Distillation” is the practice of using the output of one’s own model to learn how it works and training a new, often cheaper, model based on those insights. In February, Anthropic accused Chinese open source models of Send millions of claims to Claude As a way to improve their own models, they urged the US government to crack down on export controls.

Nadella’s point is that model makers can’t have it both ways. It is hypocritical for them to freely train on the world’s data while preventing others from doing the same with their models.

“While there is a need for great innovation to come from model providers with fair use rights to train models on public data, I find it ironic that the status quo would then turn around and impose restrictive conditions on distillation,” the Microsoft CEO wrote.

Nadella is particularly concerned when model makers reserve “the right to learn from customer usage and interaction data.”

Nadella’s solution is something the CEO of a giant cloud provider would suggest. It wants companies to “retain ownership” of their data including claims, comments, etc. So he urges her to build her own “private learning environments” in the cloud (where her data is likely already stored anyway, which might mean Microsoft’s cloud, Azure). He also wants companies to build what he calls “orchestration layers” — essentially, a way to easily switch between AI models from different providers rather than being locked into one provider. Tools such as AI “gates” that allow companies to do precisely this are becoming increasingly popular.

While Nadella never uses the phrase “open source” as a way to retain ownership, this is an obvious subtext. However, there is another subtext.

Large enterprises, many of which still have some of their own data centers in addition to using the cloud, are already moving to open source models installed on their own premises (“on-premises,” in industry parlance). Edit Levin, founder and CEO of Solo.io — which makes networking and security software that helps organizations manage AI systems — says she sees exactly this shift happening with her customers. After experimenting with proprietary model makers, they began asking themselves: “Can I take an open source model and run it in-house? It will do roughly 90% of what the big model does. And it will cost much less,” she told TechCrunch. “They understand it, and they can control it.”

Solo.io was selected last year as the technology powering… The Linux Foundation’s Agentgateway project. Her company counts companies such as T-Mobile, ADP and SAP among clients. She sees companies increasingly installing open source models in-house and sees it as the next big wave in the use of AI in enterprises.

She’s not alone. Both Vercel — known as a website building and hosting platform, which recently added AI model switchers — and OpenRouter, a company that helps developers route requests through different AI models — are seeing a significant increase in traffic to Open source models. In fact, open forms account for 29% of all directed traffic Through the Vercel portal last month.

With the CEO of Microsoft, a company that has invested in both OpenAI and Anthropic, now publicly urging companies to be wary of using proprietary models, we’re betting this trend continues to grow. “In consuming intelligence,” Nadella writes, “you create intelligence. What you create must be your own.”