Asianometry Explores High Bandwidth Flash for AI Workloads
Popular tech educational channel Asianometry examined High Bandwidth Flash (HBF), an emerging NAND-based memory architecture pioneered by SanDisk to address memory constraints in AI hardware. HBF connects multiple 3D NAND flash arrays in parallel to dramatically increase bandwidth while retaining high capacity. As Large Language Models (LLMs) continue to expand in scale, traditional High Bandwidth Memory (HBM) suffers from capacity limits and high manufacturing costs. HBF promises 8x to 16x the memory capacity of HBM at comparable bandwidths, which could allow single GPUs to run massive AI models far more efficiently. HBF leverages SanDisk's CMOS directly Bonded to Array (CBA) technology to enable access to parallel NAND arrays, reaching up to 4TB of capacity on GPU accelerators. However, because it relies on flash memory instead of DRAM, system architects must account for distinct latency profiles and finite write endurance limitations.
## BACKGROUND
AI inference heavily relies on fast memory bandwidth to stream billions of model parameters to computing cores, creating a bottleneck known as the memory wall. High Bandwidth Memory (HBM) stacks DRAM chips vertically to deliver high speeds, but its physical density limits make scaling capacity very expensive. Flash memory (NAND) offers high density and lower cost per gigabyte, but historically suffered from much slower read and write throughput compared to DRAM.