NVIDIA Releases Nemotron-3.5-Lightning-30B Model on Hugging Face
NVIDIA has released the NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 model on Hugging Face. This model is optimized for fast inference, featuring 30 billion total parameters with only 3 billion active parameters. This release represents a significant advancement in sparse Mixture-of-Experts (MoE) models, allowing the local LLM community to run a highly capable model with minimal computational overhead. It provides a balance of high capacity and low resource requirements for consumer hardware. The model utilizes the BF16 (bfloat16) data format to accelerate inference performance while maintaining numerical stability. With only 3 billion active parameters (A3B) per token, it significantly reduces the memory bandwidth and compute power required during generation.
## BACKGROUND
NVIDIA's Nemotron is a family of open-source AI models designed for reasoning, coding, and agentic applications. The Mixture-of-Experts (MoE) architecture scales model capacity by routing inputs to specific "expert" sub-networks, meaning only a fraction of the total parameters (active parameters) are computed for any given input. BF16 is a 16-bit floating-point format that offers a wider dynamic range than standard FP16, making it ideal for deep learning.