llama.cpp Release b10721 Fixes WebGPU Backend Crashes
The llama.cpp project has released version b10721, which includes a bug fix to prevent crashes in the WebGPU backend. Specifically, it resolves an issue in the ggml_backend_tensor_get() implementation when the tensor offset is not a multiple of four. This patch improves the stability of running large language models in web browsers and environments utilizing WebGPU for hardware acceleration. It ensures smoother inference performance and prevents unexpected application crashes during tensor operations. The fix addresses alignment constraints in WebGPU memory operations by handling non-four-multiple offsets in the ggml_backend_tensor_get() function. The release also provides pre-built binaries for various platforms, though KleidiAI support for macOS Apple Silicon is currently disabled.
## BACKGROUND
llama.cpp is an open-source library designed for efficient LLM inference, built on top of the ggml tensor library. WebGPU is a modern web standard and API that allows web applications to access a system's GPU for high-performance graphics and machine learning tasks. Proper memory alignment, such as offsets being multiples of four, is often a strict requirement for GPU APIs to perform efficient data transfers.