MMFlash: Faster Speculative Decoding for Vision-Language Models

1National University of Singapore

NUS
The same question answered by Qwen3.5-9B with autoregressive decoding and with three speculative-decoding methods: a MathVista question, recorded in real time on one NVIDIA H200 with SGLang (greedy decoding).

Highlights

  • Diffusion drafting for VLMs: MMFlash brings block-diffusion drafting to vision-language models. The drafter proposes a block of 16 tokens in a single forward pass and the target verifies the block in one pass, so decoding stays lossless while running 3.0× faster on Qwen3.5-4B / 9B and up to 4.7× on Qwen3.8-27B (one H200, batch size 1, greedy).
  • Vision-free drafter: the drafter never re-encodes the image or video. It reads hidden states of the target, where the visual context is already fused, so its cost does not grow with the number of visual tokens.
  • Visual-dependence-aware training: a DFlash drafter's acceptance drops sharply on tokens that depend on the image, and most rejected blocks break there. MMFlash measures each training token's visual dependence on the target and focuses the drafter's loss on those tokens.
  • Long videos: a sparse draft context (attention sinks, text, strided visual tokens and a recent window) keeps drafting cheap on 40k-token video prompts; the drafter is trained and served with the same pattern.
  • Drop-in serving: the released drafts run with stock SGLang and vLLM (DFLASH), no patches needed (tested with SGLang 0.5.23 and vLLM 0.33.0).

Models

Each draft is paired with its target model in a speculative-decoding server. Every model card on Hugging Face has the serving command and per-benchmark results.
DraftTarget modelDraft sizeBlock size
Qwen3.5-4B-MMFlashQwen/Qwen3.5-4B537M16
Qwen3.5-9B-MMFlashQwen/Qwen3.5-9B1.05B16
Qwen3.8-27B-MMFlashQwen/Qwen3.8-27B1.73B16
Gemma4-E4B-it-MMFlashgoogle/gemma-4-E4B-it537M16

BibTeX

@article{feng2026mmflash,
  title={MMFlash: Faster Speculative Decoding for Vision-Language Models},
  author={Feng, Sicheng and Chen, Zigeng and Wang, Song and Fang, Gongfan and Wang, Xinchao},
  journal={arXiv preprint arXiv:2611.18482},
  year={2026},
}