ggerganov/llama.cpp released b12444 with CDNA HIP/CUDA updates
llama.cpp release b12444 introduces a HIP/CUDA update that enables the GDN mma kernel on the CDNA architecture. This update brings specific hardware optimization improvements for supported accelerator cards. This release improves execution efficiency and performance for running large language models locally on specific hardware architectures. It expands hardware compatibility and optimization for users utilizing AMD CDNA accelerators through HIP. The pull request #31327 specifically targets the integration of the GDN mma kernel within the HIP/CUDA layer for CDNA architecture devices. The release packages binaries across various environments, including multiple CUDA and ROCm versions.
## BACKGROUND
llama.cpp is an open-source software project that enables efficient local inference of large language models with minimal dependencies across diverse hardware. HIP is a C++ runtime API and kernel language that allows developers to create portable applications for AMD and NVIDIA GPUs.