OpenAI unveils its first custom chip made by Broadcom

On Wednesday, OpenAI unveil The first custom-designed inference processor, designed and manufactured in collaboration with Broadcom. The new processor, named Jalapeño, is specifically designed to meet the unique needs of OpenAI’s inference systems. The company said OpenAI’s artificial intelligence models helped develop the chip.

While the chip is still being tested, OpenAI says early results show better performance per watt than current modern alternatives.

It was a partnership Officially announced in OctoberBut so were plans for OpenAI chips It has been rumored for a long time As a way to reduce the company’s reliance on Nvidia GPUs. Google and Amazon Both have built custom chips to serve a similar purpose, often called “AI accelerators” — silicon specifically designed to speed up machine learning workloads.

OpenAI President Greg Brockman explained the company’s approach to developing chips On her insider podcastshortly after the Broadcom partnership was announced.

“We have a deep understanding of the workload,” Brockman said in the episode. “We were really looking at specific workloads that are underserved, (and asking) how can we build something that’s going to be able to accelerate what’s possible?”

Jalapeño is specifically designed for inference, which is the process of running pre-built AI models in response to user commands. In the announcement, OpenAI emphasized the low cost of running the chip when running coding models in real time. More performance-intensive tasks like pre-training will likely still rely on Nvidia hardware, but even small reductions in inference costs could do a lot to improve the company’s bottom line.

Improving this inference system may be a critical factor in the economics of AI moving forward, and it will likely happen at every level of the stack. OpenAI is already building agent products like Codex and the models that run them, as well as the data centers to run those models. Moving to custom-designed chips allows the company to move forward with this process, the company explained in its announcement.

“OpenAI doesn’t just develop parametric models or build products on top of them, it designs the infrastructure underneath them: the chip architecture, cores, memory systems, networking, scheduling, deployment systems, and product experience,” the company wrote. “Because OpenAI works across the stack, each layer can be optimized around the same goal: making its models faster, more reliable, and more affordable for users.”