llama.cpp Release b10172 Fixes WebGPU and Buffer Offset Bugs
The llama.cpp project has released build b10172, which introduces bug fixes for the ggml-webgpu backend. Key updates include resolving alignment issues for buffer offsets larger than 4GB, correcting view offsets, and fixing binding alias issues to support all architectures. These fixes improve the stability and compatibility of running large language models locally in web browsers using WebGPU, especially when handling large models that exceed 4GB memory limits. It ensures smoother cross-platform execution of LLMs on diverse hardware architectures. The release addresses edge cases in `set_rows`, adds an overlap GLU variant, and temporarily skips certain failing models like DeepSeek-32 to clean up the tracing infrastructure. Additionally, the macOS Apple Silicon build with KleidiAI enabled has been disabled in this release.
## BACKGROUND
llama.cpp is a popular open-source port of Meta's LLaMA model in C/C++ designed for efficient local LLM inference. WebGPU is a modern web standard that allows web applications to perform high-performance GPU computations directly in the browser, and ggml-webgpu serves as the backend enabling llama.cpp to leverage this technology. KleidiAI is an open-source library developed by Arm to optimize AI workloads on Arm CPUs.