llama.cpp v0.3.0 Released with GLM-4.5-Air and DeepSeek 4 Support
llama.cpp v0.3.0 has been released, introducing support for the dots3-note multimodal model with a new DSA-ISWA KV cache and Multi-Token Prediction (MTP) for GLM-4.5-Air. It also includes tensor-split and multi-sequence rollback fixes for DeepSeek 4, alongside upgrading the underlying ggml engine to v0.22.0. This release enhances local LLM inference capabilities by integrating support for cutting-edge models like GLM-4.5-Air and DeepSeek 4. Upgrading to ggml v0.22.0 improves performance and hardware utilization, particularly for Apple Silicon users via parallel Metal kernel compilation. The update introduces a new DSA-ISWA KV cache type for dots3-note and enables tensor-split mode via the `-sm tensor` flag for DeepSeek 4. Additionally, the underlying ggml v0.22.0 upgrade brings meta-backend tensor splitting and a non-in-place `ggml_clamp` operation.
## BACKGROUND
llama.cpp is a popular open-source C/C++ library designed for efficient local inference of LLMs, utilizing the ggml tensor library to optimize performance on consumer hardware. KV (Key-Value) caching is a technique used to speed up LLM generation by storing past attention states, while Multi-Token Prediction (MTP) allows models to predict multiple future tokens at once to accelerate decoding.