llama.cpp Release b11913 Adds Hexagon VTCM Configuration
llama.cpp release b11913 introduces Hexagon VTCM size configuration overrides to improve hardware support for Snapdragon platforms. This update adds the GGML_HEXAGON_VTCM_SIZE capability, allowing developers to configure vector tight-coupled memory parameters. This improvement enhances the efficiency of running large language models on mobile and edge devices utilizing Qualcomm Snapdragon hardware. It optimizes memory management on Hexagon NPUs, leading to better performance in on-device AI inference. The update introduces the GGML_HEXAGON_VTCM_SIZE parameter as an override option that accepts standard memory suffixes like K, M, and G. It also removes the older NTHREADS cap in favor of streamlined memory management.
## BACKGROUND
llama.cpp is a popular tensor library and inference engine built on GGML that enables running large language models on consumer hardware. Qualcomm Hexagon is a digital signal processor (DSP) and neural processing unit (NPU) architecture featuring Vector Tightly Coupled Memory (VTCM) for high-speed local data sharing.