llama.cpp Release b12367 Adds SM103 SASS Support for CUDA
The llama.cpp project has released build b12367, which introduces SM103 SASS support for non-native CUDA builds along with standard multi-platform binaries. This update includes typical compilation targets and packages across various operating systems and hardware backends. This release ensures better compatibility and hardware enablement for newer NVIDIA GPU architectures in non-native CUDA environments. It helps developers and users running large language models smoothly transition to newer hardware instruction sets. The specific pull request #30099 contributed by NVIDIA adds SM103 SASS instructions specifically for non-native builds. Alongside this, the release provides precompiled binaries for macOS, Linux, Android, and Windows supporting diverse accelerators like Vulkan, OpenVINO, and ROCm.
## BACKGROUND
llama.cpp is a popular open-source software library that enables efficient inference of large language models on consumer hardware, particularly emphasizing low resource usage. SASS refers to NVIDIA's low-level assembly language used inside its GPUs, which target specific compute capabilities like SM103.