// event · library

vLLM v0.31.0: DeepSeek-V4.1-Flash FlashMLA, fast-restart preload daemon, Model Runner V2 speculative decoding

library

vendor: vllm-projectpublished: 8d agoconfidence: 0.90id: b0e0722a-d22b-415a-8445-0ca06b469beb

vLLM 0.31.0 ships 717 commits with DeepSeek-V4.1-Flash FlashMLA + NVFP4 compressed KV as SM100 default, a new `vllm preload` daemon that keeps post-quantized weights resident across engine restarts (plus CRIU-based `vllm snapshot create/restore`), Model Runner V2 draft-model speculative decoding with custom logits processors, MoonEP balanced EP all2all, prefill context parallelism, new scheduling controls, and tighter security on per-request multimodal kwargs.

capability delta

An agent can now serve DeepSeek-V4.1-Flash and GLM-5.3-Flash with FlashMLA/NVFP4 KV, restart vLLM engines in seconds via a resident weight-cache daemon or CRIU snapshots, and run draft-model speculative decoding on Model Runner V2.

curator note

Major vLLM release; agent-relevant as a serving runtime powering local/self-hosted agents. Note several breaking changes: `tokenizer_mode="slow"` removed, online `quantization="fp8"` replaced by `fp8_per_tensor`, per-request `mm_processor_kwargs` gated behind `--trust-request-mm-kwargs`.

// install

pip install vllm==0.31.0

pypi · vllm @ 0.31.0

// example

bash

# Launch the weight-cache daemon so engine restarts are fast
vllm preload --model deepseek-ai/DeepSeek-V4.1-Flash --quantization fp8_per_tensor &

# Serve with Model Runner V2 + draft-model speculative decoding
vllm serve deepseek-ai/DeepSeek-V4.1-Flash \
  --speculative-model deepseek-ai/DeepSeek-V4.1-Flash-Draft \
  --all2all-backend moonep \
  --max-num-active-seqs 128

// links

vllminferencedeepseekspeculative-decodingflashmlanvfp4moeserving

source: github_release · https://github.com/vllm-project/vllm/releases/tag/v0.31.0