llama.cpp Release b11500 Fixes Memory Alignment in Fused Bias-Add Operations
Open-source LLM inference framework llama.cpp released version b11500, fixing memory alignment handling in Hexagon matrix multiplication routines. The patch ensures the system does not incorrectly assume aligned memory access when bias addition is fused with matrix multiplication. This fix prevents potential memory corruption or crashes on hardware supporting Hexagon DSP/NPU acceleration, such as Qualcomm Snapdragon chips. It maintains reliable local AI inference execution when running optimized fused kernels on mobile and edge devices. The update specifically targets `hex-mmadd` routines (pull request #30133) by removing the requirement for aligned read and write access during fused bias-add operations. Binaries for this automated release were updated across supported platforms, including macOS, Linux, Windows, Android, and Snapdragon environments.
## BACKGROUND
llama.cpp is a popular lightweight library designed for executing LLMs on local hardware across diverse compute backends, including Qualcomm Hexagon NPUs. Fused operations combine multiple steps, such as matrix multiplication and bias addition, into a single execution pass to reduce memory overhead. However, certain hardware instructions require careful handling of memory addresses if data is not aligned to specific byte boundaries.