ggerganov/llama.cpp released b11875 with Hexagon DSP Optimizations
Llama.cpp release b11875 introduces generic dimension collapse and reshape optimizations tailored for Qualcomm Hexagon unary, binary, and matrix multiplication operations. It also includes support for the --compare option within the profile.py utility script. These hardware-specific enhancements improve the execution efficiency of large language models on edge devices powered by Qualcomm Snapdragon processors. This update benefits mobile and embedded developers aiming to achieve better local AI performance. The update refines the collapse logic to perform splits when the number of rows is small, and handles all dimension collapsing at the host level while removing outdated unary helpers.
## BACKGROUND
Llama.cpp is an open-source library that enables efficient inference for large language models like Llama using C/C++. Qualcomm Hexagon is a specialized digital signal processor architecture integrated into Snapdragon systems-on-chip to accelerate complex mathematical and AI workloads.