llama.cpp b10514 Released with Support for IBM Granite SWA Models
The llama.cpp project has released build b10514, which adds conversion and inference support for IBM's GraniteSWAForCausalLM and GraniteMoeSWAForCausalLM model architectures. This release allows developers to run IBM's Granite models featuring Sliding Window Attention (SWA) and Mixture of Experts (MoE) locally with high efficiency. It expands the ecosystem of open-source models compatible with llama.cpp, making advanced business-focused LLMs more accessible. The update introduces conversion infrastructure for the `rope_pattern` array, supports per-layer RoPE (Rotary Position Embedding) determination, and handles MoE parameters as optional during conversion. It also fixes sliding window pattern logic and naming conventions for feed-forward network (FFN) gate inputs.
## BACKGROUND
IBM Granite is a family of open AI models designed for business applications. Sliding Window Attention (SWA) is a mechanism that restricts attention to a fixed-size window of neighboring tokens, reducing the computational complexity of Transformers from quadratic to linear. Mixture of Experts (MoE) is an architecture that activates only a subset of the model's parameters for each input, improving efficiency.