llama.cpp Release b10455 Adds SYCL Support for AdamW and SGD Optimizers
The llama.cpp project has released build b10455, which introduces SYCL backend support for the AdamW (OPT_STEP_ADAMW) and SGD (OPT_STEP_SGD) optimization steps. This update allows these specific optimization operations to run on hardware compatible with the SYCL framework. Adding SYCL support for these optimization steps enables developers to perform model training and fine-tuning tasks more efficiently across a broader range of heterogeneous hardware, such as Intel GPUs. This expands llama.cpp's capabilities beyond inference into hardware-accelerated optimization workloads. The release provides pre-built binaries for multiple platforms, including Windows and Ubuntu with SYCL support (FP16 and FP32). However, certain builds, such as macOS with KleidiAI and Ubuntu with ROCm 7.14, were disabled in this release due to ongoing integration issues.
## BACKGROUND
llama.cpp is a popular open-source project designed for efficient LLM inference across various hardware backends. SYCL is an open, cross-platform programming model by the Khronos Group that allows developers to write code for heterogeneous processors (like CPUs, GPUs, and FPGAs) using standard C++. AdamW and SGD (Stochastic Gradient Descent) are widely used optimization algorithms in machine learning for updating model weights during training.