llama.cpp Release b11811 Adds KleidiAI SVE Kernels
Release b11811 of llama.cpp introduces new Q8_0 SVE dotprod GEMV and SVE i8mm GEMM kernels via the KleidiAI library. These additions improve performance for artificial intelligence workloads running on supported Arm architectures. This update significantly optimizes large language model inference speeds on Arm CPUs by leveraging specialized hardware acceleration instructions. It directly benefits users running local AI models on mobile devices, servers, and edge hardware powered by Arm processors. The update incorporates KleidiAI micro-kernels specifically tuned to exploit Arm CPU capabilities for Q8_0 quantized matrix operations. This includes optimized routines for both General Matrix-Vector (GEMV) and General Matrix-Matrix (GEMM) multiplications.
## BACKGROUND
KleidiAI is an open-source library that provides optimized performance-critical routines, or micro-kernels, tailored for Arm CPUs to accelerate artificial intelligence workloads. Scalable Vector Extension (SVE) is an instruction set architecture extension for Arm processors designed for high-performance vector processing.