AMD and Cerebras Partner on Low-Latency AI Inference Architecture
AMD has partnered with Cerebras to develop a low-latency AI inference solution that combines AMD's Helios rack-scale systems with Cerebras's Wafer-Scale Engine. This hybrid architecture aims to deliver a 5x increase in token throughput per watt. By separating the prefill and decoding stages of LLM inference, this partnership addresses the critical memory bandwidth bottleneck in AI workloads. It offers a highly efficient alternative to competing solutions from NVIDIA and Groq. In this setup, AMD's Helios racks handle compute-heavy prompt processing and large context windows, while Cerebras's Wafer-Scale Engine, featuring integrated SRAM, handles the memory-bound token generation phase.
## BACKGROUND
Large Language Model (LLM) inference is split into two phases: prefill, which processes the input prompt in parallel, and decoding, which generates output tokens sequentially. Because prefill is compute-bound and decoding is memory-bandwidth bound, running them on the same hardware can cause resource contention. Prefill-decode disaggregation separates these workloads onto specialized hardware to optimize latency and throughput.