ggerganov/llama.cpp Released b11985 with Apple Metal Optimizations
Llama.cpp release b11985 introduces major performance optimizations for Apple Metal, featuring a faster MXFP4 matrix-vector multiplication kernel and enhanced threadgroup utilization for unary, binary, and GLU operations. These updates eliminate threadgroup memory reliance for MXFP4 decoding and scale workloads across multiple threadgroups for improved hardware efficiency. These optimizations deliver faster local large language model inference speeds for users running models on Apple Silicon hardware via the Metal backend. Such continuous performance refinements help maintain llama.cpp's role as a core engine for local AI tools. The new MXFP4 kernel decodes fp4 values using an fp16 bit-trick instead of a threadgroup lookup table and loads packed nibbles as aligned 32-bit words. Additionally, contiguous unary and binary operations now launch with up to 256 threads per threadgroup, and GLU rows are split across threadgroups rather than running on a single one.
## BACKGROUND
Llama.cpp is an open-source software library designed to run inference on large language models efficiently using local hardware resources like CPUs and GPUs. Apple Metal is Apple's proprietary graphics and compute API that allows developers to harness the full power of Apple Silicon GPUs for machine learning tasks.