Microsoft announces a powerful new AI inference chip

Microsoft announced the launch of its latest chipset, the Maia 200, which the company describes as a silicon backbone designed to scale AI inference.

The 200 who follow the company Maya 100 released in 2023The company said it is technically equipped to run powerful AI models at faster speeds and more efficiently. Maia comes with over 100 billion transistors, delivering over 10 petaflops at 4-bit resolution and nearly 5 petaflops at 8-bit performance – a significant increase over its predecessor.

Inference refers to the computing process of running a model, in contrast to the computing required to train it. As AI companies mature, inference costs have become an increasingly important part of overall operating costs, leading to renewed interest in process improvement methods.

Microsoft hopes the Maia 200 will be part of this improvement, making AI work with less interruption and less power usage. “In practice, a single Maia 200 node can effortlessly run today’s largest models, with plenty of room for even larger models in the future,” the company said.

Microsoft’s new chip is also part of a growing trend of tech giants turning to self-designed chips as a way to reduce their dependence on NVIDIA, which High-end graphics processing units It is becoming increasingly important to the success of AI companies. Google, for example, has its own thermal polyurethane (TPU) processing units, which are not yet sold as chips. The computing power can be accessed through its cloud. Then there’s Amazon Trainium, the e-commerce giant’s AI accelerator chip, which It just launched its latest release,Trainium3, in December. In each case, TPUs can be used to offload some compute that would otherwise be allocated to NVIDIA GPUs, reducing the overall hardware cost.

With Maia, Microsoft puts itself in a position to compete with those alternatives. In its press release on Monday, the company noted that the Maia offers three times higher FP4 performance than third-generation Amazon Trainium chips, and higher FP8 performance than Google’s seventh-generation TPU.

Microsoft says Maia is already hard at work feeding the company’s AI models through its super intelligence team. It also supports the operations of Copilot, its chatbot. As of Monday, the company said it had invited a variety of parties — including developers, academics and frontier AI labs — to use the Maia 200 software development kit in their workloads.