ggerganov/llama.cpp Released b11702 with OpenCL MxFP4 MoE Support
Llama.cpp release b11702 introduces a new MxFP4 MoE binary kernel optimized via OpenCL specifically for Qualcomm Adreno X1 hardware devices. This update was contributed by Qualcomm engineers to enhance edge AI performance on supported platforms. This addition enables efficient execution of low-precision Mixture-of-Experts models on consumer-grade Qualcomm hardware, expanding the reach of edge AI. It bridges advanced model quantization techniques with widespread mobile and PC graphics architectures. The update specifically adds the binary kernel named gemm_moe_mxfp4_q8_1_dp4a_ila_ldck for X1 devices and enables dp4a clc support for Snapdragon X1E. Co-authored by Li He from Qualcomm, it targets hardware-accelerated low-bit matrix multiplication.
## BACKGROUND
MxFP4 refers to a 4-bit microscaling floating-point data format designed for efficient AI workloads through block quantization and shared scaling factors. Mixture of Experts (MoE) is a neural network architecture that activates only a subset of parameters per input to scale model capacity efficiently, though it requires specialized compression techniques due to high memory overhead.