// event · library

vLLM 0.32.0: /v1/systemone structured decisions, generic HF parsers, NVFP4 KV cache, Model Runner V2 default

library

vendor: vllm-projectpublished: 1h agoconfidence: 0.90id: cb460077-4d1d-49b3-bc0b-d5d854827816

vLLM 0.32.0 ships a new POST /v1/systemone endpoint for structured choice/noul/score decisions, generic `hf` tool and reasoning parsers that read a checkpoint's response_template, NVFP4 KV cache across SM8x/SM12x/SM90/SM100, Model Runner V2 as default on ROCm (GPU n-gram speculator, Elastic EP, PP with prefill CP), faster startup via `vllm download-kernels`/`preload`, 43-85% more req/s with load-balanced API workers, plus new models (EmbeddingGemma 2, MiniCPM-V 4.7, Nemotron

capability delta

An agent can now serve structured choice/score decisions via /v1/systemone, use generic HF tool/reasoning parsers without model-specific code, and run NVFP4 KV cache plus Model Runner V2 speculative decoding for higher-throughput inference.

curator note

Major release (814 commits, 363 contributors). Highly agent-relevant: structured decision endpoint, generic tool/reasoning parsers, and significant throughput wins change how agents can be served.

// install

pip install vllm==0.32.0

pypi · vllm @ 0.32.0

// example

bash

pip install vllm==0.32.0

# Serve a model with the generic HF tool/reasoning parser and NVFP4 KV cache
vllm serve meta-llama/Llama-3.1-8B-Instruct \
  --tool-call-parser hf \
  --reasoning-parser hf \
  --kv-cache-dtype nvfp4 \
  --api-server-count 4

# Structured decision via the new /v1/systemone endpoint
curl http://localhost:8000/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"type":"choice","state":"The user wants to book a flight.","choices":["search_flights","book_flight","cancel"]}'

// links

vllminferencestructured-outputspeculative-decodingkv-cachenvfp4tool-callingopenai-compatible

source: github_release · https://github.com/vllm-project/vllm/releases/tag/v0.32.0