ggerganov/llama.cpp released b11574 with Hexagon DSP optimizations
Llama.cpp release b11574 introduces performance optimizations for Qualcomm Hexagon DSP tensor copying and concatenation operations. This update focuses on improving short-row handling and reducing DMA latency for edge devices. These optimizations significantly improve on-device AI inference performance and efficiency on Qualcomm hardware. This benefits mobile and edge deployment ecosystems where power consumption and processing speed are critical constraints. The update implements a threaded DMA and HVX gather-row helper, unrolls gather-row loops, and adds a fastpath for f16 data types. It also resolves random hangs in mdev setups by making vtcm_acquire sync state atomic.
## BACKGROUND
Qualcomm Hexagon is a family of digital signal processors (DSP) and neural processing units (NPU) designed to deliver high-performance, low-power processing for mobile and edge applications. Direct Memory Access (DMA) and Hexagon Vector Extensions (HVX) allow hardware subsystems to transfer data and execute vector operations independently and efficiently without overloading the main CPU.