Musk Announces 2.5-Trillion Parameter Grok 4.8 Built on Custom C++ Stack
Elon Musk announced that xAI's Grok 4.8, a massive 2.5-trillion-parameter AI model, is finishing its initial training phase this week using a brand-new custom C++ software stack. Following this pre-training stage, the model will transition into reinforcement learning (RL) for fine-tuning. Reaching 2.5 trillion parameters makes Grok 4.8 one of the largest AI models to date, and moving from traditional Python frameworks like PyTorch to a custom C++ stack could set a new benchmark for computational efficiency at scale. The announcement coincided with minor delays for Grok 4.7, which xAI held back after discovering that overly strict RL length penalties caused the model to abandon complex reasoning tasks prematurely and skip self-verification.
## BACKGROUND
Training frontier LLMs typically relies on Python frameworks like PyTorch, but low-level implementations in C++ can dramatically cut execution overhead and optimize GPU usage. Furthermore, reinforcement learning is applied after initial pre-training to align model outputs, often applying length penalties to keep responses concise.