ggerganov/llama.cpp Released b11868 with Qualcomm Hexagon Optimization
Release b11868 of llama.cpp introduces a KV-head team split optimization for multicore HMX flash attention on Qualcomm Hexagon hardware. This update merges previous flags into a single GGML_HEXAGON_FA_SPLIT parameter to ensure stranded cores participate when KV heads are fewer than available cores. This optimization improves hardware utilization and inference efficiency for Large Language Models running on edge devices powered by Qualcomm Snapdragon and Hexagon NPUs. It helps edge AI practitioners extract better performance from mobile and embedded hardware architectures. The new GGML_HEXAGON_FA_SPLIT flag accepts values where 0 disables the feature for token-splitting only, and 1 enables head-split combined with team-split. When the number of attention heads divides evenly across cores, the system defaults to a plain head-split strategy.
## BACKGROUND
Llama.cpp is an open-source software library that runs Large Language Models locally on various hardware platforms by utilizing the GGML tensor library. Qualcomm Hexagon is a family of digital signal processors and neural processing units designed to deliver high-performance, low-power AI computations on mobile and edge devices.