llama.cpp Release b11717 Adds Hexagon DSP Support for Q1_0 and Q2_0
llama.cpp release b11717 introduces support for Hadamard MUL_MAT and GET_ROW operations for flat and tiled Q1_0 and Q2_0 quantization formats on Qualcomm Hexagon architectures. This update also reduces packet counts in the q8_1 quantizer and vec_dot_q1 to optimize performance. This release enhances on-device AI performance and efficiency on mobile devices and edge hardware powered by Snapdragon processors. By optimizing extreme quantization formats for specialized hardware accelerators, it allows larger language models to run smoothly with lower power consumption. The updates specifically target Qualcomm Hexagon DSP and NPU hardware implementations, adding flat and tiled GET_ROWS for Q1_0 and Q2_0. Contributions to this release include work by developers from Qualcomm to improve low-level vector dot products and packet counts.
## BACKGROUND
Qualcomm Hexagon is a family of digital signal processors (DSP) and neural processing units (NPU) designed for low-power, high-performance edge AI and mobile computing. Q1_0 and Q2_0 are extremely compact model quantization formats in llama.cpp, where Q1_0 uses roughly 1.125 bits per weight for sign-only quantization in extreme resource-constrained environments.