llama.cpp Release b11560 Adds Qualcomm Hexagon IM2COL_3D Support
Llama.cpp build b11560 introduces IM2COL_3D support for Qualcomm Hexagon hardware alongside performance optimizations. The update rewrites the im2col operation to utilize a canonical DMA double-buffering pipeline and eliminates inefficient floating-point conversions. This release improves hardware-accelerated large language model inference on mobile and edge devices powered by Qualcomm Snapdragon chips. It enables better efficiency and processing speed for developers running local AI workloads on mobile architectures. The update includes direct memory access (DMA) paths for IM2COL_3D, moves kernel parameters and vector tight-coupled memory (VTCM) layout precomputation into the host, and supports non-contiguous views.
## BACKGROUND
llama.cpp is an open-source software library that performs inference on large language models like Meta's Llama using the GGML tensor library. Qualcomm Hexagon is a family of digital signal processors (DSPs) and neural processing units integrated into Snapdragon mobile processors to handle heavy computational tasks with low power consumption.