ggerganov/llama.cpp Released b11558 with OpenCL Bug Fixes
llama.cpp release b11558 introduces several bug fixes for the OpenCL backend, including handling image limits for Q4_K dense bin kernels, correcting broadcasting for Q4_0, and preventing incomplete subgroups in rms_norm. These fixes improve the stability and performance of running large language models locally on hardware accelerated by OpenCL, preventing crashes or incorrect calculations when dealing with quantized weights and specific tensor operations. The update specifically adds a fallback mechanism when Q4_K weight images exceed device limits, refines conditions for dense bin kernels, and fixes broadcast issues for Q4_0 tensors.
## BACKGROUND
llama.cpp is a popular open-source software library used for running inference on large language models locally across various hardware backends. OpenCL is an open framework for writing programs that execute across heterogeneous platforms, enabling hardware acceleration on devices like GPUs.