llama.cpp Release b12389 Improves OpenCL Backend and Fixes Bugs
llama.cpp version b12389 has been released with multiple bug fixes and enhancements for the OpenCL backend. The update specifically addresses out-of-bounds memory access, kernel dispatch errors, and optimization issues across various operations. These fixes ensure greater stability and reliability for users running large language model inference using hardware accelerators supported by OpenCL. It improves overall compatibility and performance consistency for supported GPUs and devices. The update resolves out-of-bounds read issues in q4_K matrix-vector operations, fixes kernel dispatch and out-of-bounds writes for f16 swiglu_oai, and corrects row broadcasting and tensor striding for copy operations. Additionally, it handles batched group normalization and prevents double rounding for bf16 sigmoid.
## BACKGROUND
llama.cpp is a popular open-source software library designed to perform efficient inference on various large language models using the GGML tensor library. The OpenCL backend allows these models to run on a wide variety of hardware accelerators, including mobile GPUs and desktop graphics cards that support the OpenCL standard.