Developer Builds Custom llama.cpp Branch for MoE Expert Expansion
A developer created an experimental custom branch of llama.cpp designed to support expert expansion in Mixture of Experts (MoE) models. Built with assistance from GLM 5.3 Flash, the implementation was successfully tested on Apple's Metal framework and is now seeking community feedback across other hardware platforms. Enhancing MoE support in llama.cpp allows local LLM users to run high-parameter specialized models more efficiently on consumer hardware. Community-driven experimental branches help accelerate feature adoption and optimize inference speeds across diverse hardware backends. The developer noted that the custom branch performs noticeably better on Apple Metal compared to their previous DS4 implementation. However, the project remains in an early stage and requires extensive testing on non-Apple hardware such as NVIDIA CUDA or AMD ROCm GPUs, as well as on different MoE model architectures.
## BACKGROUND
Mixture of Experts (MoE) is an architecture that routes inputs to specialized sub-networks called experts via a gating network, allowing models to increase parameter count without proportionally increasing computational cost during inference. llama.cpp is a widely used open-source framework written in C/C++ that enables efficient local inference of large language models across multiple hardware platforms.