OpenAI and Broadcom Reveal Jalapeño Custom AI Inference Chip to Challenge Nvidia
OpenAI has unveiled benchmark results for its new custom AI inference chip, Jalapeño, co-developed with Broadcom. Tested on SemiAnalysis's InferenceX platform, the chip demonstrated 1.5 to 1.9 times higher energy efficiency and significantly lower latency compared to Nvidia's GB200 and GB300 systems. OpenAI's entry into custom silicon marks a significant shift in the AI hardware market, potentially reducing its reliance on Nvidia's dominant GPUs. By co-designing hardware and software, OpenAI can optimize inference costs and speed, which are critical for scaling agentic AI workloads. Jalapeño specifically targets bottlenecks in the prefill and communication phases of inference by minimizing data movement and keeping model states like KV cache local. OpenAI plans to deploy the chip on a small scale by the end of 2025, with wider deployment scheduled for 2027.
## BACKGROUND
AI inference is the process where a trained model runs to generate responses or perform tasks, which differs from the resource-heavy training phase. Nvidia currently dominates this space with systems like the GB300 NVL72, which integrates Blackwell Ultra GPUs and Grace CPUs. SemiAnalysis's InferenceX is an open-source benchmark platform used to evaluate the real-world performance of inference chips and frameworks.