llama.cpp b10604 Adds Support and Optimizations for DeepSeek-V4
The llama.cpp release b10604 introduces initial support and performance optimizations for the DeepSeek-V4 (DSV4) model architecture. Key updates include tensor splitting, shared expert delayed allreduce, and model saving capabilities for DSV4. This release enables local deployment and efficient inference of the latest DeepSeek-V4 models on consumer and edge hardware. By optimizing tensor splitting and communication overhead, it lowers the barrier for running advanced, long-context models locally. Technical changes include setting a coarser granularity for head splits, implementing a delayed allreduce mechanism for shared experts, and allowing dflash to return on specific devices. Additionally, the release fixes dspark and adds model saving support for DSV4.
## BACKGROUND
llama.cpp is a popular open-source LLM inference engine designed for high performance on a variety of hardware, especially using CPU and consumer GPUs. DeepSeek-V4 is a state-of-the-art model architecture featuring a hybrid attention mechanism designed for highly efficient long-context processing.