llama.cpp Release b11703 Adds Hexagon HVX Vectorization for Softplus
llama.cpp release b11703 introduces Hexagon HVX vectorization for the softplus activation function to replace scalar loops with high-performance vector operations. This update builds on existing backend primitives without introducing new numerical approximations. This optimization improves execution efficiency on hardware equipped with Qualcomm Hexagon processors, such as mobile and edge devices utilizing Snapdragon chips. It contributes to faster and more efficient local large language model inference on diverse hardware architectures. The implementation replaces scalar logf(1.0f + expf(x)) loops with a shared hvx_softplus_f32_aa() helper. It preserves the scalar reference's x > 20 guard via a select instruction to prevent numerical saturation issues for large inputs.
## BACKGROUND
Qualcomm Hexagon Vector eXtensions (HVX) is a high-performance SIMD coprocessor integrated into Hexagon digital signal processors to accelerate parallel workloads like computer vision and machine learning. Meanwhile, llama.cpp is an open-source library that enables efficient CPU and hardware-accelerated inference for large language models.