llama.cpp Release b10902 Adds OpenCL Binary Kernel Support for A8 Q4_0
llama.cpp build release b10902 introduces OpenCL binary kernel support for A8 Q4_0 matrix multiplication. The release also packages updated pre-compiled binaries across macOS, Linux, Windows, Android, and backends like CUDA, Vulkan, ROCm, and OpenCL. Providing dedicated OpenCL binary kernels for 4-bit quantization enhances execution efficiency on non-NVIDIA hardware, such as Qualcomm Adreno and integrated GPUs. This continues llama.cpp's goal of making local LLM inference faster and more accessible across diverse, resource-constrained devices. This release incorporates PR #28268, targeting matrix multiplication with 8-bit activations and 4-bit weights (A8 Q4_0) under OpenCL. Binary packages cover CUDA 12/13, ROCm 10.0, OpenVINO, SYCL, Vulkan, and Windows OpenCL Adreno, though KleidiAI support on macOS Apple Silicon remains disabled.
## BACKGROUND
llama.cpp is a popular open-source C/C++ library designed for efficient inference of Large Language Models (LLMs) on consumer-grade hardware. Quantization methods like Q4_0 reduce memory consumption by storing model parameters using 4-bit precision instead of 16-bit floating-point numbers. OpenCL is a cross-platform framework that allows software to run parallel computations across heterogeneous hardware, including mobile and integrated GPUs.