llama.cpp Releases Build b10990 with DMA Memory Copy Optimization
llama.cpp has released build b10990, introducing a minor optimization (#28906) in hex-cpy to utilize Direct Memory Access (DMA) when source and destination memory buffers are contiguous. Offloading memory copy operations to DMA reduces CPU involvement during data movement, improving overall data transfer efficiency during LLM inference. The patch specifically targets contiguous memory regions in hex-cpy routines. The release provides pre-built binaries for macOS, Linux, Windows, iOS, and Android across CPU, CUDA 12/13, Vulkan, ROCm, and SYCL backends.
## BACKGROUND
llama.cpp is an open-source library that enables efficient, local inference of Large Language Models across various hardware architectures. Direct Memory Access (DMA) is a computer architecture feature that allows hardware subsystems to read/write system memory independently of the central processing unit (CPU).