llama.cpp Release b10906 Fixes Speculative Decoding Bug After Image Inputs
llama.cpp released build b10906, which fixes a token position tracking bug in the server component during speculative decoding following an image input. The fix ensures that the actual token position (`pos0`) is passed to the draft model instead of the cumulative token count (`n_past`). This bug fix resolves potential generation errors and misalignment when serving multimodal LLMs with speculative decoding enabled. It affects all drafting models in the server component, improving inference accuracy after image processing. The update explicitly renames the draft parameter from `n_past` to `pos0` to correctly reflect token positions rather than counts. The fix applies universally to all speculative drafters—not just DFlash—when processing image inputs in `llama-server`.
## BACKGROUND
llama.cpp is a popular open-source C/C++ framework designed for running Large Language Models locally across diverse hardware architectures. Speculative decoding is an optimization technique where a smaller draft model generates candidate tokens for a larger target model to verify, accelerating overall text generation speed. When processing multimodal inputs like images, tracking the exact positional index of tokens is crucial for maintaining synchronization between draft and target models.