llama.cpp Release b10989 Enables HIP AllReduce Support for AMD ROCm GPUs
Open-source LLM inference framework llama.cpp released patch build b10989, enabling HIP AllReduce support for AMD ROCm GPUs. The release also includes updated pre-built binaries across Linux, Windows, macOS, and Android platforms. Enabling AllReduce under HIP improves multi-GPU communication and execution performance on AMD graphics hardware running the ROCm platform. This strengthens open-source LLM scaling capabilities on non-NVIDIA GPU hardware. The update re-enables llama.cpp's internal multi-GPU AllReduce implementation on HIP by utilizing available host memory functions like `cudaHostAllocPortable` and `cudaHostAllocMapped`. Binaries provided in this release cover CUDA 12/13, ROCm, Vulkan, SYCL, OpenVINO, and CPU targets across x86 and ARM architectures.
## BACKGROUND
llama.cpp is a high-performance open-source C/C++ library designed for local Large Language Model (LLM) inference across diverse hardware backends. AMD's ROCm ecosystem relies on HIP (Heterogeneous-computing Interface for Portability), a C++ API designed to write code portable between AMD and NVIDIA GPUs. AllReduce is a collective communication operation in distributed computing that aggregates data from all participating GPUs and redistributes the result back to each GPU.