ggerganov/llama.cpp Released b11759 with Flash Attention for ggml-zdnn
llama.cpp release b11759 introduces the flash attention operation for the ggml-zdnn backend. It also adds a scratch buffer pool to improve memory management for IBM architecture support. This update improves inference efficiency and performance on specialized IBM hardware environments. It helps broaden the hardware acceleration ecosystem supported by the library. The implementation introduces flash attention (fattn) and transitions away from scratch slots in favour of a dedicated scratch buffer pool. The contribution was authored by Aaron Teo from IBM.
## BACKGROUND
llama.cpp is an open-source library that runs inference for large language models locally using the GGML tensor library. FlashAttention is an algorithm designed to speed up the transformer attention mechanism by reducing memory read and write operations between different levels of hardware memory.