ggerganov/llama.cpp Released b12449
Release b12449 of llama.cpp introduces support for allowing up to llama_max_devices() configurations for both the --tensor-split and --fit-target arguments. This update improves multi-device configuration flexibility for advanced hardware setups. This improvement benefits power users and engineers running very large language models across complex multi-GPU or multi-device hardware configurations. It removes previous restrictions on device count mapping, enabling smoother scaling. The release includes precompiled binaries for a wide variety of platforms, including macOS, Linux, Windows, and Android, with specific support for backend accelerators like CUDA, ROCm, Vulkan, and OpenVINO.
## BACKGROUND
llama.cpp is a widely used open-source software library that enables efficient inference of large language models on local hardware with minimal setup. The tensor-split feature allows users to distribute model layers proportionally across multiple hardware accelerators to bypass single-device VRAM limitations.