llama.cpp Adds CPU Support for Maple 20B-A1B Ternary MoE Architecture
Pull Request #27000 in llama.cpp by AlexGabbia introduces CPU support for the Maple 20B-A1B architecture. This new model architecture combines a 20-billion parameter capacity with ternary quantization and a Mixture-of-Experts design. Combining ternary quantization with sparse Mixture-of-Experts (MoE) routing enables consumer hardware to run larger LLMs with significantly lower memory bandwidth and VRAM demands. This approach makes running capable 20B-class models on standard CPUs and low-VRAM machines much more feasible. The Maple 20B-A1B architecture has 20 billion total parameters but only activates 1 billion active parameters per token during inference. The initial PR implementation specifically targets optimized CPU execution inside the ggml/llama.cpp engine.
## BACKGROUND
Mixture-of-Experts (MoE) is a neural network architecture that routes input tokens to specific sub-networks (experts), allowing high total parameter capacity without spending compute on every parameter per token. Ternary neural networks quantize weights into just three values (-1, 0, 1), which significantly reduces memory footprint and substitutes heavy multiplication operations with lightweight additions.