SK hynix and SanDisk Unveil High Bandwidth Flash (HBF) Standard for AI Inference
SK hynix and SanDisk have introduced the High Bandwidth Flash (HBF) standard, a new NAND-based memory technology targeting up to 3TB/s bandwidth. This standard is designed specifically to alleviate memory bottlenecks during AI inference workloads. By offering up to 8 to 16 times the capacity of High Bandwidth Memory (HBM) at a similar cost, HBF could democratize large-scale AI inference by allowing massive models to run efficiently without relying on expensive HBM. HBF leverages SanDisk's CMOS directly Bonded to Array (CBA) technology and delivers performance within 2.2% of unlimited-capacity HBM. It aims to bridge the gap between high-speed HBM and high-capacity storage solutions.
## BACKGROUND
AI inference, especially for Large Language Models (LLMs), is highly dependent on memory bandwidth because model weights must be constantly loaded into the processor. While High Bandwidth Memory (HBM) offers the necessary speed, it is extremely expensive and limited in capacity. Traditional NAND flash memory has high capacity but lacks the bandwidth required for real-time AI processing.