llama.cpp b11005 Fixes Qualcomm Hexagon Backend Graph Splitting Bug
The b11005 release of llama.cpp updates its Hexagon hardware acceleration backend to accept zeroed Rotary Position Embedding (RoPE) probes during tensor placement checks. This fix prevents the inference engine from rejecting the operator and falling back to CPU execution during model setup. By eliminating unnecessary CPU fallbacks, inference execution graphs remain unified on the Qualcomm Hexagon DSP, improving execution efficiency on mobile and edge devices. For architecture models like Gemma, this fix reduces execution graph splits from 5 down to 2 at every full-attention layer. The issue stemmed from Hexagon's `supports_op` check rejecting RoPE probes where `n_dims == 0` or `freq_base == 0`. By supporting these zeroed probe requests, `rope_freqs` computations remain on the hardware accelerator instead of placing positional frequency operations back on the host CPU.
## BACKGROUND
llama.cpp is an open-source C/C++ LLM inference engine optimized for running language models across various hardware backends, including Qualcomm Hexagon DSPs. Rotary Position Embedding (RoPE) is a technique used in modern Transformer models to inject positional awareness into tokens. When a hardware backend rejects an operator, llama.cpp splits the computational graph, forcing CPU fallbacks that incur significant data transfer overhead.