llama.cpp Release b11026 Adds Tensor Skipping Fix for Qwen MoE Models
llama.cpp released build b11026, introducing a fix to skip `gate_up_exps` tensors when `TENSOR_SKIP` is set in Qwen MoE model configurations. This adjustment is specifically required for architectures like Qwen 3.5 MoE when Multi-Token Prediction (MTP) tensors are fused but not loaded during inference. This update prevents tensor loading failures and unexpected memory overhead when executing modern Qwen Mixture-of-Experts (MoE) variants locally. It improves overall compatibility as open-source LLM architectures adopt advanced features like Multi-Token Prediction. The change, implemented via PR #29014, ensures `gate_up_exps` tensors are bypassed when `TENSOR_SKIP` is flag-enabled. Pre-built release artifacts were generated for macOS, Linux, Windows, and Android across various hardware backends including CPU, CUDA 12/13, Vulkan, ROCm, and SYCL.
## BACKGROUND
llama.cpp is a widely used C/C++ inference framework designed to run Large Language Models efficiently on consumer-grade CPU and GPU hardware. Mixture-of-Experts (MoE) models route tokens through specialized sub-networks called experts, while Multi-Token Prediction (MTP) is a technique that enables models to predict multiple tokens per forward pass to accelerate inference speed.