llama.cpp b10687 Optimizes OpenCL for Adreno GPUs
The llama.cpp b10687 release introduces OpenCL optimizations that improve LLM prefill performance on specific Qualcomm Adreno GPU generations. Specifically, it defaults to a faster matrix multiplication path for Adreno X2E and bypasses a slow tiled path on Adreno A7X. This update delivers a 25% prefill speedup on Adreno X2-90 and a 9% speedup on Adreno 740, making local mobile AI inference faster on supported Qualcomm hardware. It highlights the ongoing effort to optimize LLM execution on edge devices by tailoring kernels to specific GPU architectures. The optimization targets the prefill phase (where batch size N >= 16) and leaves the decode phase untouched. It resolves compiler register spilling issues on the Adreno A7X by routing batched F32xF32 operations to a per-row F32 kernel instead of the tiled path.
## BACKGROUND
LLM inference is split into two phases: the prefill phase, which processes the input prompt in parallel, and the decode phase, which generates output tokens sequentially. General Matrix Multiplication (GEMM) is the fundamental mathematical operation driving these computations, and optimizing GEMM kernels for specific hardware architectures is crucial for efficient local execution.