llama.cpp b10694 Released with RPC Fix for Older macOS Versions
llama.cpp has released version b10694, which features a bug fix for the Remote Procedure Call (RPC) backend on macOS versions that do not support Remote Direct Memory Access (RDMA). This release addresses issue #27815 to restore compatibility for older macOS systems. This update ensures that users running older macOS environments can continue to utilize llama.cpp's distributed inference capabilities without encountering RPC failures. It maintains backward compatibility for Mac clusters that rely on network-based offloading for large language models. The release includes pre-built binaries for various platforms, though the macOS Apple Silicon build with KleidiAI enabled remains disabled. The fix specifically targets the RPC backend, which distributes model weights and KV cache across local and remote devices.
## BACKGROUND
llama.cpp is a popular open-source project that enables efficient LLM inference on consumer hardware. Its RPC backend allows users to split model computations across multiple machines, such as connecting multiple Macs over a network. Recent macOS updates introduced RDMA (Remote Direct Memory Access) over Thunderbolt to speed up these connections, but older macOS versions lacking this feature experienced issues with llama.cpp's RPC implementation.