Samsung Details zHBM Architecture to Boost AI Response Speeds Tenfold

Samsung presented its zHBM (3D-stacked memory) roadmap, aiming to increase AI accelerator response speeds from 100 to 1,000 tokens per second per user. By vertically bonding HBM memory directly onto AI accelerators, the design targets up to 8x the speed of HBM5 and 3x the energy efficiency. As agentic AI applications demand unprecedented bandwidth for real-time inference, traditional memory bottlenecks limit processing speeds. Direct vertical stacking minimizes data transfer distances, reinforcing Samsung's leadership position in next-generation high-performance memory. Samsung also plans to sample its zNAND-O 3D storage solution starting in 2028. Additionally, experts from OpenAI and Intel noted that while Compute Express Link (CXL) helps scale cold data storage affordably, it cannot replace HBM for high-speed GPU data transfers.