Why early GPU financiers are turning to inference chips in $400M deal

General Compute, an AI inference cloud startup, has secured a $400 million loan from Upper90, a technology investment firm. This may be the first deal to feature inference-specific chips as collateral, chips designed to run already-trained AI models quickly and efficiently, rather than the more expensive chips used to build the models in the first place.

The funding is the latest sign that markets are responding to concerns about the prices of AI tools and tokens by shifting to infrastructure that runs open source models at a lower cost than the latest LLMs from frontier labs.

Founded by CEO Finn Poklosky, General Compute It raised $15 million A seed round in May to build a new inference cloud around silicon from SambaNova, an Intel-backed chip maker. (Neoclouds are designed specifically for AI workloads, unlike the general-purpose infrastructure offered by traditional scaling tools like AWS or Azure.)

The company’s SN50 chips are designed for inference. They’re energy efficient and don’t require expensive water cooling systems, which means they can be deployed more quickly than GPUs across a wider range of data centers. General Compute says the new chips will provide 16 times faster inference than GPU-based clouds.

The challenge is to get a lot of these chips, especially when you’re a brand new company.

Upper90 co-founder and CEO Billy Libby, a former quantitative trader at Goldman Sachs, had a clue to this: In 2021, his company financed GPU purchases by Crusoe, the energy-focused data center startup, which is believed to have been the first loan-to-value for advanced chips.

Traditional lenders avoided such deals at the time due to the risks and uncertainties surrounding GPU depreciation. But as CoreWeave turned chip-backed loans into a business model and then became the basis for a massive IPO, this type of financing became popular.

“When we funded Nvidia GPUs as the first group to do this, the market was inefficient,” Libby told TechCrunch. “We can really put together something as early entrants, and get some kind of compensation for the risk.”

Now that GPUs are relatively well understood Maybe overbuyUpper90 is turning to companies like General Compute to ride the next wave of the AI ​​boom. “We think open source models will be important, and we went and looked for a player last year that was inferred,” Libby said. “Not everyone needs a supercomputer, but they do need inference and artificial intelligence.”

This hypothesis has become stronger, as companies that provide access to open models, such as OpenRouter and Fireworks, have raised new rounds with huge valuations. New models like Kimi’s K3, which was introduced as recently as this week, have proven to compete with the latest releases from Anthropic and OpenAI in coding standards. New chipmakers such as Groq and Cerebras have attracted interest from acquirers and public markets alike.

General Compute’s ability to access chips outside the Nvidia ecosystem is important for the same reason. TensorWave, another AI infrastructure company, is making a similar bet on a partnership with AMD. As more alternatives to Nvidia emerge, compute providers not locked into Nvidia’s deals may have an advantage in providing cost-effective inference.

“There are a bunch of chips that are starting to expand that have amazing (total cost of ownership), or can run much faster than Nvidia, but there aren’t very many buyers for them,” Poklowski said. “By teaming up with Upper90, it’s not just ‘a cool startup got some money to buy some computers.’” Like this is the first sign of capital organizing itself and breaking up Nvidia’s monopoly dominance.