llama.cpp Release b10795 Adds SYCL Kernel Fusion Optimizations
llama.cpp release b10795 introduces SYCL backend optimizations that fuse RMS_NORM+MUL+ADD and ADD+ADD operation chains. This update allows multiple sequential tensor operations to execute in a single GPU kernel call under the GGML_SYCL_ENABLE_FUSION flag. Kernel fusion reduces memory bandwidth overhead and GPU kernel launch latency, accelerating LLM inference on hardware supported by SYCL, such as Intel GPUs. It improves execution efficiency for residual connections and normalization layers, which are heavily used in Transformer models. The fused ADD+ADD operation supports various data types including f32, f16, bf16, i32, and i16, alongside broadcast and non-contiguous tensors. If an unsupported matrix or precision combination is encountered, execution gracefully falls back to launching two separate add() kernels.
## BACKGROUND
SYCL is an open, cross-platform C++ programming standard enabling code execution across heterogeneous hardware such as GPUs and accelerators. Kernel fusion is an optimization technique that merges consecutive math operations into a single GPU execution step to minimize round-trips to global VRAM. RMSNorm and residual addition are key operational building blocks frequently computed across Transformer layers in modern large language models.