Mistral Releases Mistral Large 4, a 1-Trillion Parameter MoE Model
Mistral AI has released Mistral Large 4, its largest natively multimodal Mixture-of-Experts model featuring approximately 1 trillion total parameters and 49 billion active parameters per token. GGUF quantizations have already appeared on Hugging Face alongside merged support in llama.cpp. This release represents a significant milestone for the open-weights community, delivering a frontier-class generalist model with 1 million context length, advanced reasoning capabilities, and state-of-the-art visual grounding. It allows developers and researchers to run highly capable multimodal systems locally or via API. The model features a granular Mixture-of-Experts architecture with 128 routed experts (4 active per token) and 1 shared expert, trained on 3,800 NVIDIA Grace Blackwell GPUs. It supports a 1M context window, natively handles variable-sized images, offers configurable reasoning effort, and is distributed under the Mistral Ecosystem Open-Weight License.
## BACKGROUND
Mixture-of-Experts (MoE) is a neural network architecture where a large total number of parameters is broken into smaller sub-networks called experts, but only a small fraction of them are activated for each processed token to maintain fast computation speeds. GGUF is a popular file format designed by the llama.cpp team to store quantized language models for efficient inference on consumer and enterprise hardware.