llama.cpp b11338 Released with Qualcomm Hexagon DMA Optimizations
llama.cpp release b11338 introduces shared strided Direct Memory Access (DMA) copy optimizations for CPY and CONCAT tensor operations on Qualcomm Hexagon DSPs. This release enables any-dimension CONCAT operations over DMA alongside stability fixes and additional safeguards for unsupported execution conditions. These optimizations improve memory transfer efficiency and hardware acceleration when running large language models on Snapdragon mobile and edge platforms featuring Hexagon NPUs/DSPs. Offloading tensor copy and concatenation operations to DMA reduces CPU overhead and enhances overall inference latency on Snapdragon devices. The update removes the broken `CONCAT_DMA_MIN_ROW` logic under 64-bit DMA mode and adds missing `dma_queue_flush()` calls to ensure proper data flushing. It also ensures that extended buffer tensors mapped to hardware memory are correctly accessible via DMA, developed in collaboration with Qualcomm engineers.
## BACKGROUND
llama.cpp is a popular open-source LLM inference engine designed for efficient model execution across diverse hardware backends. Qualcomm Hexagon is a specialized digital signal processor (DSP) and Neural Processing Unit (NPU) architecture embedded in Snapdragon processors to accelerate machine learning workloads. Direct Memory Access (DMA) allows hardware subsystems to transfer memory independently of the main CPU, which speeds up memory-heavy tensor manipulation operations.