Microsoft Completes Pre-Training of SOTA MAI-Base-2 Model
Microsoft has completed the pre-training of its new SOTA foundation model, MAI-Base-2, which outperforms all tested open-weight models. The model boasts a total of 2.8 trillion parameters with 80 billion active parameters, trained across 6,000 NVIDIA GB200 GPUs in 38 days. This milestone highlights Microsoft's expanding in-house AI capabilities, achieving high pre-training efficiency that is reported to be 4 to 12 times more efficient than comparable models. It demonstrates significant progress in model-infrastructure co-design and hardware optimization using advanced Blackwell architecture. During its 38-day training cycle, MAI-Base-2 maintained an average floating-point operations utilization rate of 21% and an effective throughput of 89%. Microsoft attributes this success to scientific scaling, model-infrastructure co-design, and data scaling, and has already initiated preparations for MAI-Base-3.
## BACKGROUND
Large language models are often built using Mixture-of-Experts (MoE) architectures, which activate only a subset of parameters for any given token to improve computational efficiency. The NVIDIA Blackwell microarchitecture and GB200 systems represent advanced data center hardware designed to accelerate heavy AI workloads and massive model training.