llama.cpp Release b11039 Improves Sliding Window Attention and Model-Saver Compatibility
llama.cpp released build b11039, updating its model loader and saver logic to correctly read and write Sliding Window Attention (SWA) patterns across 15 supported architectures. The update introduces llama_model_base::load_swa_pattern() and serializes Multi-Head Latent Attention (MLA) parameters for SWA layers. This release guarantees bit-exact roundtrip model saving and loading for architectures using SWA and MLA, such as Gemma 3, Cohere 2, and OLMo 2. By ensuring per-layer SWA patterns are not silently ignored or corrupted during conversion, it prevents subtle inference degradation in GGUF model files. The updated model saver writes sliding_window_pattern explicitly as an array of flags per layer rather than collapsing it to a scalar period. It also serializes MLA key/value lengths and KV LoRA ranks required for models like dots3note, enabling bit-exact roundtrip test passes across 15 architectures.
## BACKGROUND
Sliding Window Attention (SWA) limits self-attention computation to a fixed context window around each token, reducing the quadratic computational cost of traditional Transformers to linear complexity. Multi-Head Latent Attention (MLA) uses low-rank projections to compress Key-Value (KV) cache memory footprints, allowing LLMs to handle longer contexts more efficiently.