llama.cpp Release b12031 Adds OpenCL Support for IQ3 Formats
Release b12031 of llama.cpp introduces initial OpenCL support for iq3 quantization formats, featuring dedicated optimizations for Adreno GPUs. This update improves Large Language Model inference performance on mobile devices and hardware powered by Adreno graphics, expanding hardware accessibility for low-bit quantization models. The implementation adds iq3_s and iq3_xxs AOS/SOA matrix vector operations, tiled matrix multiplication, noshuffle GEMM/GEMV, and dp4a instructions tailored for Qualcomm Adreno GPUs.
## BACKGROUND
Quantization reduces the memory footprint and accelerates inference of AI models by compressing weights to lower bit-widths, while OpenCL provides an open standard for cross-platform parallel programming on heterogeneous hardware.