llama.cpp b10580 Adds Support for dots3-note Multi-Modal Model
The latest release of llama.cpp (b10580) introduces support for the "dots3-note" vision and audio multi-modal model. This update includes implementation for model conversion and C++ inference support. This update allows users to run the dots3-note multi-modal model locally with efficient C++ inference. It expands llama.cpp's capabilities in handling complex, long-context audio and vision tasks. The integration involves updates to the GGUF tensor mapping (`gguf-py`) and the core C++ implementation for multi-modal tensor mapping. The release also provides pre-built binaries for various platforms including macOS, Windows (with CUDA 12/13), Linux, and Android.
## BACKGROUND
llama.cpp is a popular open-source project designed for efficient LLM inference in C/C++ on consumer hardware. The dots3-note model, developed by Dots Studio, is an open-weight Mixture-of-Experts (MoE) model featuring 280 billion total parameters, 16 billion active parameters, and a massive 512K token context window.