llama.cpp Release b11741 Adds OpenCL Cooperative-K dp4a GEMM Support
Llama.cpp release b11741 introduces cooperative-k dp4a GEMM support in OpenCL for q4_0, q4_k, and q6_k quantized formats. This update also fixes alignment issues to improve compatibility and performance for supported hardware. This improvement enhances LLM inference speed and efficiency for users running models via OpenCL backend implementations. It ensures better hardware utilization across devices that rely on OpenCL acceleration for quantized tensor operations. The update focuses on optimizing general matrix multiplication (GEMM) specifically for 4-bit and 6-bit K-quantized and standard formats using dp4a vector instructions in OpenCL. Proper alignment adjustments were implemented to ensure stability during these low-precision mathematical operations.
## BACKGROUND
Quantization techniques like q4_0, q4_k, and q6_k reduce the memory footprint of large language models by compressing weights to lower bit-widths, making them runnable on consumer hardware. GEMM (General Matrix Multiply) is a fundamental computational building block in neural networks, while instructions like dp4a accelerate dot-product operations on compatible hardware architectures.