NVIDIA Releases Nemotron-3.5-Bolt-74B VLM Optimized for Local 24GB Hardware
NVIDIA has released Nemotron-3.5-Bolt-74B-A9B-24GB, a 74-billion parameter vision-language model featuring a Mamba-2-Transformer hybrid LatentMoE architecture. It utilizes native 2-bit quantization with quantization-aware training to fit comfortably within 24GB VRAM hardware while supporting text, image, and video inputs. This release bridges the gap between massive cloud-scale multimodal models and consumer hardware by delivering a model that runs locally on standard 24GB GPUs. It achieves an optimal balance of speed and intelligence, outperforming dense models in speed and mixture-of-experts models in accuracy per GB. The model features 74 billion total parameters with 9 billion active parameters, a context length of up to 256K tokens, and is available in GGUF format. It supports reasoning mode via chat-template controls and employs a speculative decoding drafter for enhanced generation performance.
## BACKGROUND
Mamba-2-Transformer hybrid models combine traditional attention mechanisms with state space models to achieve linear scaling and high efficiency. Quantization-aware training incorporates precision reduction directly into the training process, allowing models to retain higher accuracy when converted to low-bit formats like 2-bit.