llama.cpp Release b10188 Fixes Metal Memory Leak on macOS
The llama.cpp project has released build b10188, which addresses a memory leak issue in the Metal backend. This leak occurred when a model was loaded and then freed without executing any GPU operations. This fix improves memory management stability for macOS and iOS users running local LLMs, preventing system-wide wired memory accumulation during model initialization or teardown cycles. It ensures that applications using llama.cpp as a backend do not waste system resources when models are loaded but not immediately run. The update resolves the issue by running dummy work only when residency sets are used and wrapping the relevant function in conditional compilation macros. Additionally, the release introduces a regression test that measures system-wide wired memory to prevent future occurrences.
## BACKGROUND
llama.cpp is a popular open-source C/C++ library designed for efficient local inference of large language models, serving as the backend for tools like Ollama and LM Studio. On Apple hardware, it utilizes Apple's Metal API to accelerate computations on the GPU. Metal residency sets are used to manage resource allocation and residency in GPU memory, but improper handling can lead to memory remaining "wired" or locked in system RAM.