~/VISION LANGU/mmflash-faster-speculative-decoding-for-vision-language-models

MMFlash: Faster Speculative Decoding for Vision-Language Models

MMFlash introduces a novel speculative decoding framework that accelerates vision-language models by adaptively upweighting visually dependent tokens and employing sparse attention for long videos. Across various benchmarks, it achieves lossless decoding speedups of up to 3.58×. This framework addresses a major bottleneck in multimodal generation by significantly speeding up inference without sacrificing output quality. It enables more efficient deployment of advanced vision-language models on long videos and resource-constrained environments. The framework utilizes a vision-free diffusion drafter that reads target hidden states without re-encoding media, and it supports drop-in serving with stock SGLang and vLLM without requiring custom patches. Pre-trained drafts have been released for models like Qwen3.5 and Gemma 4.

## BACKGROUND

Speculative decoding is an inference optimization technique where a smaller draft model proposes candidate tokens that a larger target model verifies in parallel, cutting latency while preserving the original output distribution. Vision-language models (VLMs) extend large language models by jointly interpreting both images and text.

## REFERENCES

## KEYWORDS

#vision-language-models#speculative-decoding#ai-inference#multimodal-ai#machine-learning-research

$ subscribe --daily

MMFlash: Faster Speculative Decoding for Vision-Language Models | Daily News