ggerganov/llama.cpp Released Version b11575 with Metal Tensor Bug Fix
Release b11575 of llama.cpp introduces a specific bug fix for the Metal tensor kernel_mul_mm error, which previously affected large tensors on Apple Silicon hardware. This update addresses the multiplication issue in the Metal backend shader files. This point release is important for users running large language models locally on Apple Mac devices using GPU acceleration. Ensuring stability with large tensors prevents crashes and computational errors during inference sessions. The fix specifically modifies the ggml-metal kernel implementation for matrix multiplication (kernel_mul_mm) to handle large tensors correctly. Code contributions for this patch were assisted by Codex and reviewed by community developers.
## BACKGROUND
llama.cpp is an open-source library that enables efficient local inference for large language models using the GGML tensor library. The Metal backend provides GPU acceleration for these tensor operations specifically on Apple devices utilizing Apple Silicon and the Metal API.