NVIDIA Announces Nemotron Bolt 3.5 74B A9B Model
NVIDIA is releasing the Nemotron Bolt 3.5 74B A9B, a 74-billion parameter mixture-of-experts model featuring 9 active parameters that fits into 24GB of memory. Additionally, the Nemotron 3.5 Bolt E4B 4GB is scheduled for release on October 23rd. This model offers a high-performance open-weights option optimized for local deployment by balancing high processing speed with strong intelligence within a modest memory footprint. It allows users to run advanced AI capabilities on standard consumer hardware. The model uses advanced 2-bit quantization and quantization-aware training techniques to drastically reduce memory usage while retaining quality. It is designed to be faster than typical dense models while maintaining higher intelligence than similarly sized MoE models.
## BACKGROUND
A mixture-of-experts model divides an AI network into multiple specialized expert models to scale up capacity without increasing active compute costs. Quantization reduces the precision of model weights to save memory, and quantization-aware training helps mitigate the accuracy loss that normally occurs during this process.