Traceary

Catalog / vLLM

0.11.1

9 months agoaddedfixedchangedOriginal notes

Highlights

This release includes 1456 commits from 449 contributors (184 new contributors)!

Key changes include:

  • PyTorch 2.9.0 + CUDA 12.9.1: Updated the default CUDA build to torch==2.9.0+cu129, enabling Inductor partitioning and landing multiple fixes in graph-partition rules and compile-cache integration.
  • Batch-invariant torch.compile: Generalized batch-invariant support across attention and MoE backends, with explicit support for DeepGEMM and FlashInfer on Hopper and Blackwell GPUs.
  • Robust async scheduling: Fixed several correctness and stability issues in async scheduling, especially when combined with chunked prefill, structured outputs, priority scheduling, MTP, and DeepEP / DCP. We expect --async-scheduling to be enabled by default in the next release.
  • Stronger scheduler + KV ecosystem: Improved test coverage in CI and made scheduler behavior more robust with KV connectors, prefix caching, and multi-node deployments.
  • Anthropic API Support: Added support for the /v1/messages endpoint, allowing users to interact with vllm serve using Anthropic-compatible clients.

Detailed release notes will be updated in the next few days.

What's Changed

  • [Bugfix] Improve GLM4 MoE Reasoning Parser's is_reasoning_end Condition (@frankwang28 #25355)
  • [Docs] Add Toronto Meetup (@mgoin #25773)
  • [CI] Add E2E Blackwell Quantized MoE Test (@mgoin #25723)
  • [V1] address post issues related to #20059 (part 1); cascade attention reenable by default (@fhl2000 #23046)
  • [CI] Fix FlashInfer AOT in release docker image (@mgoin #25730)
  • [spec decode] Consolidate speculative decode method name for MTP (@zixi-qi #25232)
  • Reduce the Cuda Graph memory footprint when running with DBO (@SageMoore #25779)
  • Kernel-override Determinism [1/n] (@bwasti #25603)
  • [Bugfix] Optimize CpuGpuBuffer initialization (@namanlalitnyu #25447)
  • [Spec decode] automatically disable mm for text-only draft models (@jmkuebler #25667)
  • [Core] Don't count preempted tokens in prefix cache hit rate (@zhuohan123 #25787)
  • Add option to restrict media domains (@russellb #25783)
  • Add flashinfer-build.sh and register precompiled cu128 wheel in Dockerfile (@mgoin #25782)
  • [Multimodal][Speculative Decoding]Eagle Eagle3 mm support, enablement on qwen2.5vl (@david6666666 #22872)
  • [Bugfix] Allow Only SDPA Backend for ViT on B200 for Qwen3-VL (@yewentao256 #25788)
  • [CI/Build] Consolidate model loader tests and requirements (@DarkLight1337 #25765)
  • [CI/Build] Add timing to Model Executor Test (@22quinn #25799)
  • [CI/Build] Reorganize root-level V1 tests (@DarkLight1337 #25767)
  • [Misc] Fix codeowners override for v1 sample and attention (@22quinn #25037)
  • [Misc] Update openai client example file for multimodal (@ywang96 #25795)
  • [Bugfix] Add missing image_size for phi4_multimodal (@Renovamen #25796)
  • [Bugfix] Merge MM embeddings by index instead of token IDs (@DarkLight1337 #16229)
  • Validate API tokens in constant time (@russellb #25781)
  • Add filtering for chat template kwargs (@russellb #25794)
  • Fix GPTQ model loading in Transformers backend (@hmellor #25770)
  • [Bugfix] Fix triton import precommit failure (@tlrmchlsmth #25803)
  • [Bugfix][WideEP] Apply TP Attn + EP MoE fix to other models (@tlrmchlsmth #24982)
  • [docs] Resolve transcriptions API TODO (@yyzxw #25446)
  • [env] default nixl side port conflicts with kv-event zmq port (@panpan0000 #25056)
  • [Core] Refactor self.model() to call a helper for subclassing. (@patrick-toulme #25084)
  • [torch.compile]: Add VLLM_DEBUG_DUMP_PATH environment variable (@ZJY0516 #25651)
  • [Bug]: Set LD_LIBRARY_PATH to include the 'standard' CUDA location (@smarterclayton #25766)
  • [Core] GC Debug callback (@Jialin #24829)
  • [Bugfix][NIXL] Fix Async Scheduler timeout issue (@NickLucche #25808)
  • [MM] Optimize memory profiling for scattered multimodal embeddings (@ywang96 #25810)
  • [Bugfix] Fix Qwen3-VL regression from #24982 (@ywang96 #25814)
  • [VLM] Update Qwen3-VL max_num_video_tokens calculation for configurable video profiling (@Isotr0py #25557)
  • Fix random dataset mismatched token length with config. (@weireweire #24937)
  • Update GLM-4.5 Doc transformers version (@zRzRzRzRzRzRzR #25830)
  • [Bugfix] fix Qwen3VLMoe load when pp > 1 (@JJJYmmm #25838)
  • Remove redundant cudagraph dispatcher warning (@mgoin #25841)
  • [Misc] fix tests failure by using current_platform (@kingsmad #25825)
  • [P/D] NIXL Updates (@robertgshaw2-redhat #25844)
  • Add Phi4FlashForCausalLM to _PREVIOUSLY_SUPPORTED_MODELS (@tdoublep #25832)
  • [XPU]Fix xpu spec decoding UTs, avoid using cuda graph (@jikunshang #25847)
  • [Bugfix] Fallback ViT attn backend to SDPA for blackwell (@ywang96 #25851)
  • [V0 Deprecation][Models] Remove all V0 condition for mm embeddings merge (@Isotr0py #25331)
  • [Misc] Remove more get_input_embeddings_v0 (@DarkLight1337 #25857)
  • update to latest deepgemm for dsv3.2 (@youkaichao #25871)
  • [Bugfix] Fix requirements paths in install instructions (@yingjun-mou #25827)
  • [Model][Bugfix] Fix issues in MiDashengLM implementation for quantized models (@zhoukezi #25854)
  • [torch.compile] serialize cudagraph_mode as its enum name instead of value (@ZJY0516 #25868)
  • [Cuda2CPU][P/D] Add cuda2cpu support in NixlConnector (@chenxi-yang #24690)
  • [Bugfix][Speculative Decoding] Fix Eagle3 quantization config issue (@rahul-tuli #25883)
  • [CI/Build] Include Transformers backend test in nightly transformers test (@Isotr0py #25885)
  • [Model] Remove MotifForCausalLM (@jeejeelee #25866)
  • [Bugfix] Use correct key "ignore" for config.json non-quantized layers (@leejnau #25706)
  • [BugFix][torch.compile] KV scale calculation issues with FP8 quantization (#21640) (@adabeyta #25513)
  • [Doc] Add documentation for vLLM continuous benchmarking and profiling (@namanlalitnyu #25819)
  • [Bugfix][ROCm] Fixing trying to import non-existent symbols from libnccl.so (@gshtras #25605)
  • [Kernel] Chunk-aligned mamba2 (@tdoublep #24683)
  • [Doc] Polish example for torchrun dp (@zhuohan123 #25899)
  • [NIXL] Increase default KV block eviction timeout on P (@NickLucche #25897)
  • [V0 Deprecation] Remove vllm.worker and update according imports (@aarnphm #25901)
  • Test Prompt Embeds/LoRA compatibility and Enable LoRA Support for OPT Models (@qthequartermasterman #25717)
  • [Bug] Fix Weight Loading for Block FP8 Cutlass SM90 (@yewentao256 #25909)
  • [Benchmark] Support benchmark throughput for external launcher DP (@zhuohan123 #25913)
  • MoveVllmConfig from config/__init__.py to config/vllm.py (@hmellor #25271)
  • [BugFix] Fix DP/EP hang (@LucasWilkinson #25906)
  • [BugFix] Pass config_format via try_get_generation_config (@acisseJZhong #25912)
  • [Model][Bugfix] Fix MiDashengLM audio encoder mask by removing incorrect logical_not (@zhoukezi #25925)
  • [Bugfix]: Clean up chunked prefill logging when using whisper (@simondanielsson #25075)
  • [New Model] DeepSeek-V3.2 (Rebased to Main) (@zyongye #25896)
  • [Doc] Add Cambricon MLU support (@a120092009 #25942)
  • Updated TRL integration docs (@sergiopaniego #25684)
  • [Bugfix][Model]fix ernie45 moe gate&bias dtype to float32 (@CSWYF3634076 #25936)
  • [Model] Move vision_feature_select_strategy into resolve_visual_encoder_outputs (@DarkLight1337 #25938)
  • [perf] Use CPU tensor to reduce GPU->CPU sync (@lhtin #25884)
  • [NIXL] Add support for MLA caches with different latent dim (@NickLucche #25902)
  • [CI] Move applicable tests to CPU (@rzabarazesh #24080)
  • [Fix] Improve CPU backend compatibility for RISC-V (@ihb2032 #25816)
  • [Kernel][Moe Configs] Add more tuned triton configs for ExpertsInt8 and FP8 (@Josephasafg #25858)
  • Add Hugging Face Inference Endpoints guide to Deployment docs (@sergiopaniego #25886)
  • [Bugfix][Model] Fix inference for Hunyuan dense models (@Anionex #25354)
  • [Bugfix] Fix accuracy issue of TRTLLM FP8 MOE and improve logging (@pavanimajety #25895)
  • [Bugfix] Token type and position embeddings fail to be applied to inputs_embeds (@DarkLight1337 #25922)
  • [bugfix][deepseek] fix flashmla kernel selection (@youkaichao #25956)
  • [Bug] Fix AttributeError: 'QKVParallelLinear' object has no attribute 'orig_dtype' (@yewentao256 #25958)
  • [Doc] Improve MM Pooling model documentation (@DarkLight1337 #25966)
  • [Docs] Add moe kernel features doc (@bnellnm #25297)
  • OffloadingConnector: Fix GPU block tracking bug (@orozery #25856)
  • [Llama4] [multimodal] Fix misplaced dtype cast of cos_sin_cache in Llama4VisionRotaryEmbedding (@cjackal #25889)
  • [Bench] Add DeepSeekV32 to MoE benchmark (@jeejeelee #25962)
  • [V1] [P/D] Add Support for KV Load Failure Recovery (@sdavidbd #19330)
  • Add explicit pooling classes for the Transformers backend (@hmellor #25322)
  • [Docs] Remove API Reference from search index (@hmellor #25949)
  • [gpt-oss] use vLLM instead of openai types for streaming (@qandrew #25186)
  • [Misc] Make EP kernels install script support uv (@LucasWilkinson #25785)
  • [Model] MTP fallback to eager for DeepSeek v32 (@luccafong #25982)
  • Update launch_bounds_utils.h for correct compile on Multiple Cuda Arch - PTXAS out of range Warning (@DrStone1971 #25843)
  • [Log] Optimize Log for FP8MOE (@yewentao256 #25709)
  • Fix INT8 quantization error on Blackwell GPUs (SM100+) (@certainly-param #25935)
  • [MM] Add text-only mode for Qwen3-VL (@ywang96 #26000)
  • [Bugfix] Fix __syncwarp on ROCM (@zhewenl #25996)
  • [BugFix] Fix default kv-cache-dtype default for DeepseekV3.2 (@LucasWilkinson #25988)
  • Update to Transformers v4.56.2 (@hmellor #24638)
  • [Misc]allow disable pynccl (@luccafong #25421)
  • [Doc] updating torch.compile doc link #25989)
  • [BugFix][MM] Fix Nonetype error when video is cache in qwen2.5-omni-thinker (@wwl2755 #26004)
  • [Misc] Factor out common _apply_feature_select_strategy (@DarkLight1337 #26003)
  • [CI] Only capture a single CUDA graph size in CI by default (@hmellor #25951)
  • [MISC] Fix misleading batch_size_capture_list when cuda_graph_sizes < 4 (@billishyahao #25829)
  • [Benchmark] Finish documented v0.11.0 deprecation of --endpoint-type (@natoscott #26007)
  • [Bugfix] Apply same sampling parameters for both n=1 and n>1 (@kmaehashi #26005)
  • [NVIDIA] Blackwell Family (@johnnynunez #24673)
  • Fix test_mamba_ssm_ssd.py due to missing _query_start_loc_to_chunk_indices_offsets (@hl475 #25995)
  • [CI] Tweaks to GPT-OSS Eval (Blackwell) for stability (@mgoin #26030)
  • [BugFix][DP/EP] Fix CUTLASS MLA hang under load (@LucasWilkinson #26026)
  • [ROCm][Build] Add support for AMD Ryzen AI MAX / AI 300 Series (@hyoon1 #25908)
  • [Bug] Fix Negative Cuda Memory Usage (@yewentao256 #25683)
  • [BugFix] ChunkedLocalAttention is currently not CG compatible (@LucasWilkinson #26034)
  • Support RL online quantization with torchao (@jerryzh168 #23014)
  • [ROCm][Bugfix] Add missing parameter to ROCm backend (@gshtras #26029)
  • [Misc] Make handling of SamplingParams clearer in n>1 case (@njhill #26032)
  • Run:ai model streamer add GCS package support (@pwschuurman #24909)
  • Update base image to 22.04 (jammy) (@huydhn #26065)
  • Change size of single CUDA graph for CI to 4 (@tdoublep #26089)
  • [FA/Chore] Bump vllm-flash-attention (@LucasWilkinson #25537)
  • [Model] Use merge_by_field_config for MM models (A-C) (@DarkLight1337 #26073)
  • [Model] Use merge_by_field_config for MM models (D-F) (@DarkLight1337 #26076)
  • [Platform][CI] Added OOT platform interface e2e test that running on Ascend NPU (@leo-pony #25470)
  • [Qwen][ROCm] Flash Attention Rotary Embeddings (@vllmellm #24642)
  • [CI] Add Blackwell DeepSeek FP8 FlashInfer MoE tests (@mgoin #26040)
  • [CI/Build] Replace vllm.entrypoints.openai.api_server entrypoint with vllm serve command (@DarkLight1337 #25967)
  • [BugFix] Fix FI accuracy issue when used for MLA prefill (@LucasWilkinson #26063)
  • [Small] Prevent bypassing media domain restriction via HTTP redirects (@huachenheli #26035)
  • [Deepseek v3.2] Support indexer prefill chunking (@heheda12345 #25999)
  • EAGLE 3: Fix preamble so that measured speedup over Eagle 1 becomes 32% instead of 5% on MTBench (@ekagra-ranjan #25916)
  • [Mamba][KVCacheManager] Simplify kv cache manage logic for mamba + MTP (@heheda12345 #25119)
  • [Perf] Fix and reapply move apply w8a8 block fp8 linear to class (@ElizaWszola #25696)
  • Fix MTP with deepep_low_latency (@MatthewBonanni #25904)
  • [Bugfix] Disable cascade attention with FlashInfer (@mgoin #26130)
  • [Log] Optimize DeepGEMM Missing Log (@yewentao256 #26106)
  • [Bug][Benchmark] Fix duplicate req in oversampling (@ekagra-ranjan #26140)
  • [Attention] Move Backend enum into registry (@MatthewBonanni #25893)
  • [CI/Build] Conditionally register cutlass_fp4_group_mm to fix building on Hopper (@mgoin #26138)
  • [DeepSeek] Improve performance of DS MLA cache kernel (@MatthewBonanni #26132)
  • [Bug]: Limit num_reqs in dummy_run when max_num_seqs is small (@benchislett #26144)
  • [gpt-oss] disable tool server initialization if no tool in request (@qandrew #25790)
  • [Build/CI] Revert back to Ubuntu 20.04, install python 3.12 with uv (@tlrmchlsmth #26103)
  • [ROCm] [VL] [Bugfix] Fix vit flash attn dispatcher logic for ROCm (@tjtanaa #26104)
  • [Bugfix] Fix import gemm_afp4wfp4 failure on AMD (@zhewenl #26068)
  • [Model] Use merge_by_field_config for MM models (G) (@DarkLight1337 #26117)
  • FusedMoE support for the Transformers backend (@hmellor #22650)
  • [BUG] Reorder model config creation (@ahao-anyscale #26124)
  • [Misc] Remove typing.List (@varun-sundar-rabindranath #26150)
  • [Input] Remove unused prompt field (@DarkLight1337 #26097)
  • [Perf] Optimize reshape_and_cache CUDA Kernel (@ZJY0516 #25955)
  • add(v1): RequestStatesStats to RequestOutput (@huijjj #24947)
  • [Model] Use merge_by_field_config for MM models (InternVL family) (@DarkLight1337 #26153)
  • [test utils] correct wrong typing (@yannicks1 #26159)
  • [CI] Fix distributed hybrid tests in CI (@tdoublep #26155)
  • [NIXL][Misc] Expose metrics from NIXL for logging to CLI (@NickLucche #25388)
  • [openai] Fix missing tool usage check (system message) (@levunet #24768)
  • [Multi Modal] Configurable MM Profiling (@wwl2755 #25631)
  • [Doc] Fixed shape description for fused_batched_moe.py (@Egor-Krivov #25668)
  • Quick fix for IMA with the Prefix Prefill kernel during graph capture (@SageMoore #25983)
  • [Renderer] Move Processor out of AsyncLLM (@KKSK-DON #24138)
  • Re-enable prefill of max model length (@yannicks1 #24446)
  • [backends][short_conv] CUDA graph piecewise edits (@paulpak58 #24215)
  • [Model] Supplement to PR 24862: Pass param prefix to LLMHead (@whx-sjtu #25805)
  • [CI/Build] do not enforce precompilation on tpu ci tests (@sixiang-google #25992)
  • [Model] Fixed stream generator for gpt-oss + spec-decoding (@astralord #26027)
  • [Renderer] Move Processor out of LLMEngine (@DarkLight1337 #26165)
  • Fix undefined symbol: cutlass_moe_mm_sm100 (@jasl #26098)
  • [BugFix][QWEN-VL]fix wrong apply_rotary_emb_torch selection introduced by #24642 (@xuechendi #26123)
  • Stop mergify from keeping stale PRs alive (@hmellor #26169)
  • Avoid division by zero in cache DS MLA kernel (@MatthewBonanni #26174)
  • Fix V1 engine serialization error with Ray distributed executor (@nrghosh #26148)
  • [Quantization/NVFP4] Speed up TRTLLM NVFP4 MOE weight loading and fix K/V scale loading for MLA Attn (@pavanimajety #25968)
  • [Perf] Remove hardcoded num_warps=1 (@chelsea0x3b #26183)
  • [Refactor] Optimize FP8 MOE Backend Choice and Log (@yewentao256 #26044)
  • [responsesAPI] add better error messaging for long prompts (@qandrew #25724)
  • [Bugfix] Relax tokenizer regex for mixtral to include 'tokenizer.model' (@BowenBao #25964)
  • [CI] Push multiarch manifests as nightly builds (@csahithi #25764)
  • [Misc] Add penalties sampling parameters to serve tool (@southfreebird #25974)
  • [BugFix] Fix de-functionalization pass for rotary_embedding (@angelayi #23953)
  • [CI] Fix Pre-commit Mypy Error (@yewentao256 #26181)
  • [GPTOSS][DP/EP][Marlin] Enable GPTOSS DP/EP using Marlin kernels (@varun-sundar-rabindranath #25488)
  • Fix issue of using only the part of video frame [Nemotron Nano] (@BloodAxe #26186)
  • [Bugfix] Fix qwen3 vl dummy data generation with overrides (@ywang96 #26193)
  • [BugFix] Use async Mistral Tokenizer in Chat Completions (@bbrowning #26134)
  • Add batch invariant kernel override for FlashInfer backend [2/n] (@bwasti #25769)
  • [cpu][perf] Accelerate unquantized-linear for AArch64 through oneDNN/ACL and weight prepack (@fadara01 #25948)
  • [V1] [Hybrid] Mamba2 Automatic Prefix Caching (@s3woz #25752)
  • Support expert parallel in Transformers backend (@hmellor #26162)
  • [Model] Support nested structures for TensorSchema (@DarkLight1337 #26212)
  • [Misc] Require merge_by_field_config argument (@DarkLight1337 #26214)
  • [Misc] Remove unused executor.apply_model (@DarkLight1337 #26215)
  • [CI Failure] fix_test_auto_prefix_cache_support (@hl475 #26053)
  • Revert "Add batch invariant kernel override for FlashInfer backend [2/n]" (@DarkLight1337 #26220)
  • Add Olmo 3 reasoning parser (@soldni #26054)
  • [Core] Enable decode of context length equal to max model length (@yannicks1 #26168)
  • [Bugfix] Fix _reqs_to_process leak on abort (@NickLucche #26012)
  • [Model] CLIP Embedding Support (@DarkLight1337 #26010)
  • Fix tensor device and dtype placement in Qwen2VL model (@yuafng #26219)
  • [V1] [Hybrid] Remove code to override default CUDA graph configuration (@tdoublep #26226)
  • [CPU] Refine batch reorder of CPU attention backend (@bigPYJ1151 #26096)
  • [Frontend] Cache chat template kwargs resolution (@Isotr0py #26227)
  • [Renderer] Clean up renderer code (@DarkLight1337 #26216)
  • [Model] Use merge_by_field_config for MM models (H-L) (@DarkLight1337 #26230)
  • [Easy] Add str repr for IterationStats (@22quinn #26232)
  • [Bugfix] Allow --skip-tokenizer-init with echo and return_token_ids (@DarkLight1337 #26238)
  • Add documentation for granite 4 tool calling (@maxdebayser #26175)
  • [Perf][Easy] Early stop in request_block_hasher (@Jialin #26112)
  • [Bugfix]: Assertion error when using FlashInfer backend (@simondanielsson #25933)
  • [Bugfix] Always apply MM processor even when no MM items are passed (@DarkLight1337 #26240)
  • [Bugfix][Hardware][RISC-V] Limit supported dtypes to float32 to avoid scheduler segfault (@ihb2032 #26228)
  • [Refactor][Kernel] support loading kernel from other place (@ILikeIneine #25823)
  • Convert formatting to use ruff instead of yapf + isort (@hmellor #26247)
  • Remove all references to yapf as it's no longer used (@hmellor #26251)
  • Remove all cases of fmt: on/off (@hmellor #26253)
  • fix(tests): Resolve late binding of loop variable in assert message lambda (@ihb2032 #26249)
  • Fix per file ruff ignores related to typing (@hmellor #26254)
  • Update ruff pre-commit hooks version (@hmellor #26255)
  • [CI] fix mamba kernel test (@ZJY0516 #26250)
  • [NVIDIA] flashinfer TRTLLM attention prefill token limit (@jasonlizhengjian #25998)
  • Fix per file ruff ignores related to simplification (@hmellor #26259)
  • [CI] Add Blackwell LM Eval Small Models test to nightly (@mgoin #26052)
  • [DOC] Update production-stack.md (@elieserr #26177)
  • [CI] Add comment about the single cudagraph capture size that is used (@tdoublep #26252)
  • [V1] [Hybrid] Some additional clean-up in Mamba2 prefix caching (@tdoublep #26222)
  • [Doc] Edited minor typo (@orangeng #26266)
  • [MISC] Add heheda12345 to CODEOWNERS of vllm/config/cache.py (@heheda12345 #26270)
  • [CI][gpt-oss] Enable python tool tests in CI (@wuhang2014 #24315)
  • Fix per file ruff ignores related to line length (@hmellor #26262)
  • Bump actions/stale from 10.0.0 to 10.1.0 (@dependabot[bot] #26272)
  • [Benchmarking] Add disable_shuffle option for dataset loading (@ymoslem #26258)
  • [Misc] Clean up unnecessary E501 ignore (@ywang96 #26274)
  • [Docs] Edit HF Inference Endpoints documentation (@ariG23498 #26275)
  • [Doc] add KAITO to integrations (@abhisheksheth28 #25521)
  • [Frontend] Consolidate tokenizer init code (@DarkLight1337 #26276)
  • [Model] Use merge_by_field_config for MM models (Llava family) (@DarkLight1337 #26280)
  • Support expert parallel load balancing in Transformers backend (@hmellor #26287)
  • [Bugfix] Fix mrope in Transformers Backend (@zucchini-nlp #26087)
  • Fix DotsOCR tensor type (@what-in-the-nim #26281)
  • [Model] EVS support for nano_nemotron_vl (@tomeras91 #26269)
  • [Attention] Remove unused reorder_batch method (@MatthewBonanni #24463)
  • [Tests] conftest: Extending VllmRunner and HfRunner to accept token_ids as input (@yannicks1 #26295)
  • [CI Bugfix] Make sure TRTLLM attention is available in test_blackwell_moe (@mgoin #26188)
  • Support llama3 eagle3 head with llama4 verifier (@rahul-tuli #25961)
  • [Misc] auto_tune: kill specific vllm process (@karan #26304)
  • [Bugfix][Spec Decode] Fix wrong valid_mask for padded speculation when chunked prefill occurs (@seven-mile #26231)
  • Add bias handling to CPUFusedMOE kernel (@cfRod #26289)
  • [Bugfix] Fix gemma3 with transformers backend (@zucchini-nlp #23178)
  • [Benchmark] Enable MM Embedding benchmarks (@DarkLight1337 #26310)
  • [Docs] Fix broken table in moe_kernel_features doc (@varun-sundar-rabindranath #26314)
  • [BugFix] Pad input buffers in _dummy_run (@varun-sundar-rabindranath #26209)
  • [Bugfix] Allow skipping MoE in NVFP4 (fix for MTP) (@benchislett #25987)
  • [ROCm] Split AITER unified attention into its own backend (@gshtras #25507)
  • [Perf] Add decode full-graph support to FlashInfer-MLA backend (@benchislett #26313)
  • [Misc] Define EP kernel arch list in Dockerfile (@simon-mo #25635)
  • [Docs][DBO] Add initial doc that describes the DBO implementation (@SageMoore #26024)
  • [Core] Simplify the Dp padding/should ubatch coordination logic (@SageMoore #25768)
  • [UX] Support nested dicts in hf_overrides (@mgoin #25727)
  • [BUG] Fix file parsing for load_format runai_streamer_sharded (@ahao-anyscale #26324)
  • [Model] Define merge_by_field_config MM interface (U-Z) (@ayushsatyam146 #26261)
  • [Deprecation] Deprecate LLM.set_tokenizer (@DarkLight1337 #26333)
  • [responsesAPI][bugfix] serialize harmony messages (@qandrew #26185)
  • [Model] Define merge_by_field_config MM interface (R-T) (@ayushsatyam146 #26260)
  • [BugFix] Update KV block hash type from BlockHash to ExternalBlockHash in kv_events_subscriber - #26264 (@atalhens #26265)
  • [V0 Deprecation] Remove VLLM_USE_V1 from docs and scripts (@DarkLight1337 #26336)
  • Optimize KV cache distribution for asymmetric pipeline parallelism (@gholmes829 #25164)
  • Add topk logits torch op for DS3.2. (@dcampora #25945)
  • Add TRL example notebook to RLHF docs (@sergiopaniego #26346)
  • [Docs] add docs for cuda graph v1 (@fhl2000 #24374)
  • [Model] Use merge_by_field_config for MM models (Ovis family) (@Isotr0py #26308)
  • [Feature][OCP MX] Support mxfp6 and mixed mxfp6-mxfp4 (@fxmarty-amd #21166)
  • [Model] Add support for ModernBertForTokenClassification (@antrec #26340)
  • [Misc] Move LRUCache into its own file (@DarkLight1337 #26342)
  • [V0 Deprecation] Remove VLLM_USE_V1 from tests (@DarkLight1337 #26341)
  • [Model] Lfm2Moe (@paulpak58 #26344)
  • [ci] Rename test_mxfp4_moe.py to test_ocp_mx_moe.py (@fxmarty-amd #26364)
  • [CI] Add Qwen3 MoE NVFP4 to Blackwell lm-eval (@mgoin #26316)
  • [deepseek] add EP8 FusedMOE config for H200 and B200 (@heheda12345 #26331)
  • [Bug] Fix Shape Validation for Fallback while Enabling E8M0 for DeepGEMM (@yewentao256 #26322)
  • [Bugfix] Add missing sink tensor into flash attn cascade attn implementation (@plliao #26325)
  • [Frontend] CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops (@morrison-turnansky #26113)
  • [V1] Logit processors for rejection sampler (@southfreebird #19482)
  • [Spec Decode] Enable efficient speculative decoding with FlashInfer-MLA (@benchislett #25984)
  • [TPU] update TPU benchmark threshold (@jcyang43 #25713)
  • Add more libraries to rlhf.md (@mgoin #26374)
  • [Bugfix] Fix MTP+FlashInfer crash when trtllm kernels are available but disabled (@benchislett #26361)
  • Revert #24446 and #26168 (@tdoublep #26332)
  • [Misc] Clean up cruft from previous FlashMLA sparse implementation (@LucasWilkinson #26125)
  • [torchao] safetensors integration (@liangel-02 #25969)
  • Add SwigluOAI implementation for CPUFusedMOE (@isharif168 #26347)
  • [Core] Simplify setting new_token_ids in CachedRequestData (@njhill #26388)
  • fix(v1/kv_cache): resolve async KV transfer bug in cascade attention (@ayushsatyam146 #23485)
  • Add gather_indexer_k_quant_cache kernel (@Barry-Delaney #25931)
  • [Bugfix] Incorrect MM data format in vllm bench throughput (@DarkLight1337 #26395)
  • fix[DP][v1]: Prevent hangs from mismatched worker configurations (@ayushsatyam146 #26218)
  • [TPU] Rename tpu_commons to tpu_inference (@utkarshsharma1 #26279)
  • [Feature] Enable E8M0 by Default on Hopper for DeepGEMM, 5% E2E throughput improvement (@yewentao256 #26197)
  • [Misc] add usedforsecurity=False in md5 hash call (@dtrifiro #26357)
  • [Model] Allow passing custom number of max tiles to Nano 2 VL (@BloodAxe #26403)
  • [Docs] Have mergify leave a comment with the docs preview link (@hmellor #26412)
  • [CI] Pooling models mteb test disable enforce_eager (@noooop #26408)
  • [Benchmarks] Add support for Qwen 3 VL MoE tuning (@lgeiger #26419)
  • Tidy vllm/config/__init__.py to only add classes and functions (@hmellor #26405)
  • [NIXL][non-cuda] Add install script for nixl with non-cuda ucx (@xuechendi #25959)
  • [Refactor] Refactor FP8 & INT8 Quant Folder inside w8a8 (@yewentao256 #25293)
  • [CI Failure] Fix pre-commit issue for install_nixl_from_source_ubuntu.py (@mgoin #26424)
  • [Bugfix] Fix vllm bench ... on CPU-only head nodes (@Aydin-ab #25283)
  • [Bug] Fix DeepGEMM Attention Test (@yewentao256 #26423)
  • [Benchmarks] Fix imports in FP8 tuning script (@lgeiger #26407)
  • [Bug] Fix Test in Batch Invariant (@yewentao256 #26128)
  • Remove Python 3.9 support ahead of PyTorch 2.9 in v0.11.1 (@hmellor #26416)
  • [Feature] Change cache.py with pydantic validation (@vrdn-23 #26390)
  • [Attention] Implement universal BACKEND_MAP (@MatthewBonanni #25900)
  • [Bugfix][Flashinfer] fix VLLM_USE_TRTLLM_ATTENTION issue for models with diff hyperparameters (@elvischenv #25924)
  • [BugFix] Fix failing test quantization/test_compressed_tensors.py::test_compressed_tensors_fp8_block_enabled (@morrison-turnansky #26436)
  • [Kernel] Centralize platform kernel import in current_platform.import_kernels (@NickLucche #26286)
  • [Models] Improve iteration over layers (@lgeiger #26425)
  • [Bugfix] Respect min_tokens in scheduler stop check (@elaineyz #26317)
  • [Kernels] Modular kernel refactor (@bnellnm #24812)
  • [Attention] Register FLASHMLA_SPARSE (@MatthewBonanni #26441)
  • Separate MLAAttention class from Attention (@therealnaveenkamal #25103)
  • [Misc] Redact ray runtime env before logging (@ruisearch42 #26302)
  • [Bugfix] Set the minimum python version for gpt-oss (@jeejeelee #26392)
  • [Minor] Change warning->warning_once in preprocess (@zhuohan123 #26455)
  • [Bugfix] Catch and log invalid token ids in detokenizer #2 (@njhill #26445)
  • [Bugfix] Incorrect another MM data format in vllm bench throughput (@huydhn #26462)
  • [Hardware][AMD] Enable FlexAttention backend on ROCm (@mawong-amd #26439)
  • [MM][Doc] Add documentation for configurable mm profiling (@wwl2755 #26200)
  • [Core][KVConnector] Propagate all tokens on resumed preemptions (@QierLi #24926)
  • [Hybrid]: Decouple Kernel Block Size from KV Page Size (@zhiyuan1i #24486)
  • [CI/Build] Fix model nightly tests (@DarkLight1337 #26466)
  • [Core] Relax the LoRA max rank (@jeejeelee #26461)
  • Update Dockerfile and install runai-model-streamer[gcs] package (@pwschuurman #26464)
  • Bump Flashinfer to v0.4.0 (@elvischenv #26326)
  • [Model] Gemma3: Fix GGUF loading and quantization (@lucianommartins #26189)
  • Enable RMSNorm substitution for Transformers backend (@hmellor #26353)
  • Add: Support for multiple hidden layers in Eagle3 (@rahul-tuli #26164)
  • [torchao] Add support for ModuleFqnToConfig using regex (@jerryzh168 #26001)
  • [Misc] Misc code simplifications (@njhill #26450)
  • [doc] add Volcengine as a compute sponsor (@youkaichao #26477)
  • [Feature] Use pydantic validation in lora.py and load.py configs (@simondanielsson #26413)
  • [Misc] Upgrade more code to Python 3.10 (@DarkLight1337 #26463)
  • [Bugfix] Fix SHM cache initialization (@DarkLight1337 #26427)
  • [Models][Qwen3VL] Optimise _validate_and_reshape_mm_tensor (@lgeiger #26426)
  • [Bugfix] Move current_platform import to avoid python import cache. (@iwzbi #16601)
  • [V0 deprecation] Remove QKVCrossParallelLinear implementation (@Isotr0py #26475)
  • [Feature] Use pydantic validation in parallel.py config (@simondanielsson #26417)
  • Revert #26113 "[Frontend] CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops" (@ZJY0516 #26472)
  • Upgrade Pydantic to v2.12.0 and remove hack for Python 3.13 (@hmellor #26481)
  • [Models][Qwen] Replace pad with cat for better performance (@lgeiger #26486)
  • [Attention][DCP] Support DCP with query length > 1 (MTP) with FA3 (@minosfuture #25049)
  • [Model] Apply shared experts overlap optimization to all models with shared experts (@bnellnm #26145)
  • [BUGFIX] Add cu_tokens_across_sp to DPMetadata (@SageMoore #26457)
  • [Bugfix] Enable padded FP4 quantization (@roikoren755 #25947)
  • [Bugfix] Disable moe inplace for torch >= 2.9 (@bnellnm #26497)
  • [Flashinfer][gpt-oss] Support FP8-qkv Flashinfer TRTLLM Sinks Attention (@elvischenv #25674)
  • [Core] Remove unused prev_sampled_token_ids_invalid_indices input batch field (@njhill #26514)
  • [UX] Add FlashInfer as default CUDA dependency (@mgoin #26443)
  • [Bugfix] Fix CUDA graph selection bug in FlashInfer at high concurrency (@benchislett #26499)
  • [Bug] Fix modular_kernel: ZeroDivisionError: integer division or modulo by zero (@yewentao256 #26528)
  • [CI] Fix Pre-commit Issue Cannot determine type of "rank" and "world_size" (@yewentao256 #26448)
  • Refactor MistralTokenizer (@juliendenize #26358)
  • [DP][ray] Support different VLLM_RAY_DP_PACK_STRATEGY (@ruisearch42 #23849)
  • [Core] Small simplification in GPUModelRunner._update_states() (@njhill #26508)
  • [Chore]: One pythonic tool parser test uses the wrong parser (@bbrowning #26515)
  • [Spec-Decode] Support piecewise cudagraphs for Eagle head (@LucasWilkinson #25109)
  • fix test_simple_inductor_graph_partition (@BoyuanFeng #26522)
  • [deepseek] kernel block size for UniformTypeKVCacheSpecs (@heheda12345 #26559)
  • [Metrics] Log multi-modal cache stats and fix reset (@DarkLight1337 #26285)
  • [GPT-OSS] Add support for arrays at tool message content (@luis5tb #25593)
  • Remove LoRA bias support (@ashwin-phadke #25807)
  • [CI] fix ruff format (@chaunceyjiang #26579)
  • [bugfix][DCP] fix block_size of hash in DCP prefix caching (@heheda12345 #26296)
  • [NIXL] Ignore abort on already-finished request (@markmc #25067)
  • [Bugfix] Convert untraceable GroupShape to list for AMD impl (@Lucaskabela #26535)
  • [BugFix] Fix noop elimination edge case (@andylolu2 #26394)
  • [CI] fix test_run_batch.py::test_completions - AssertionError (@chaunceyjiang #26578)
  • [BugFix][torch.compile] Fix fused_scaled_matmul_reduce_scatter signature for PyTorch 2.8 (@jasonlizhengjian #26038)
  • Added test_top_k_per_row to test-pipeline.yaml. (@dcampora #26569)
  • [Bugfix] Make DP padding optional in coordinate_batch_across_dp (@SageMoore #26375)
  • Silu v2 (@elvircrn #25074)
  • [Metrics] Add test for multi-modal cache stats logging (@markmc #26588)
  • [torch.compile] Make inductor partition rules respect splitting_ops #25691 (@baonudesifeizhai #25845)
  • [Bugfix] fixed top_logprobs: -1 does not appear to work as intended (@chaunceyjiang #26470)
  • [Model][Qwen3VL] Compute cu_seqlens on CPU to remove (@lgeiger #26496)
  • [Model] Add FlexOlmo model implementation (@2015aroras #24923)
  • [Transform] [Quantization] Add QuTLASS support to vLLM (@LopezCastroRoberto #24440)
  • Add Qwen3-Omni moe thinker (@wangxiongts #25550)
  • Update pre-commit hook versions (@hmellor #26591)
  • Update CUDA architecture list in build pipeline for 12.9.1 wheels (@wseaton #26592)
  • Fix some typing issues found by mypy==1.18.2 (@hmellor #26596)
  • [BUG] Qwen3-next MTP. Fix attn metadata build bug (@vadiklyutiy #26564)
  • [BugFix] Fix async scheduling + request preemption (@njhill #26385)
  • Cache the environment variable check for batch invariance (@bwasti #26510)
  • AOT Compilation for torch.compile (Bundled) (@zhxchen17 #24274)
  • [BugFix] Make penalties and bad_words work with async scheduling (@njhill #26467)
  • [Frontend] Improve the performance of is_reasoning_end (@chaunceyjiang #25735)
  • [CI/Build] Fix ppc64le CPU build and tests (@npanpaliya #22443)
  • [XPU] Upgrade NIXL to remove CUDA dependency (@zhenwei-intel #26570)
  • [MM] Move Qwen3Omni MRoPE impl to model file (@ywang96 #26608)
  • [Bugfix][Multi Modal] Fix incorrect Molmo image processing (@sangho-vision #26563)
  • [Refactor]: Use M-RoPE interface directly while defining model class instead of maintaining model specific M-RoPE implementation in mrope.py (@divyanshsinghvi #24172)
  • fix(nix): Allow local oneDNN path to fix vLLM CPU build failure (@ihb2032 #26401)
  • Add EAGLE-3 Speculative Decoding Support for Qwen3 MoE (@rahul-tuli #26485)
  • [CPU] fix the issue when the node is '-' cause json decode error. (@muzian666 #26562)
  • [Refactor]Reduce duplicate code in serving_chat (@chaunceyjiang #26627)
  • [compile] Add patched_fused_scaled_matmul_reduce_scatter (@angelayi #26604)
  • [Bugfix][Qwen3VL] fix deepstack in qwen3vl (@JJJYmmm #26626)
  • [Bugfix] Fix qwen-moe packed_modules_mapping (@jeejeelee #26634)
  • [Benchmark] Support Infinity API (@DarkLight1337 #26641)
  • CP: make correct_attn_out robust to 4‑D views and fix Triton arg binding (@hl475 #26509)
  • [compile] Fix inductor partition config (@angelayi #26645)
  • [EPLB] Support ernie4.5-moe (@HsChen-sys #22100)
  • Add @noooop to codeowner for pooling models (@noooop #26652)
  • [PERF] [Qwen3-next] Speed up gated RMSNorm (@vadiklyutiy #26207)
  • [MISC] Rename the torch profiler filename as instance_id+rank_id for merging the Profiler results of each Rank (@noooop #25867)
  • [Bugfix][CI/Build] Fix failing Mteb CI (@Isotr0py #26638)
  • [Bugfix][DCP] Set default CUDAGraphMode to PIECEWISE for DCP (@FENP #26574)
  • [TEST][BUG FIX] Fix DP GPU_ID issue (@xuechendi #26442)
  • Update Optional[x] -> x | None and Union[x, y] to x | y (@hmellor #26633)
  • [Feature] Add support for naver/splade-v3 (BERT-based sparse embedding model) (@gjgjos #26339)
  • [Models][Qwen3VL] Speedup fast_pos_embed_interpolate (@lgeiger #26647)
  • [easy] fix pre commit error on trunk (@hl475 #26665)
  • [CI/Build] Add tool to build vllm-tpu wheel (@mgoin #19165)
  • [Misc] cache result of disable_inplace (@bnellnm #26666)
  • [Bugfix][Core]Fix block table out-of-range issue in priority scheduling (@quanliu1991 #26661)
  • [FIX] Throwing an exception when the model does not support pool tasks (#25840) (@yyzxw #25855)
  • docs: wrong command in structured_outputs README (@yihong0618 #26677)
  • [Model] Fix Skywork R1V mlp (@jeejeelee #26673)
  • [Model] Add reasoning_parser and tool_parser for Ernie45 thinking (@CSWYF3634076 #25027)
  • Ignore large reformatting PRs in git blame (@hmellor #26690)
  • [Model][0/N] Improve all pooling task | clean up (@noooop #25817)
  • [ResponseAPI] Simplify input/output message serialization (@Jialin #26620)
  • [Bugfix] Fix out of bound index issue for Jina-embedding-v3 RoPE with cuda graph (@Isotr0py #26687)
  • [unrevert] Add batch invariant kernel override for FlashInfer backend [2/n] (@bwasti #26373)
  • [Hardware][CPU] Disable torch.compile for RISC-V to prevent APIError (@ihb2032 #26693)
  • [FEATURE]: Use pydantic validation in multimodal.py config (@andycandy #26629)
  • [UX] Speedup DeepGEMM warmup with heuristics (@mgoin #25619)
  • [P/D] [NixlConnector] kv load recovery integration (@wseaton #26171)
  • [Misc] Separate prompt logging to debug (@aitsvet #26713)
  • [CI/Build] upgrade compressed-tensors to 0.12.2 t

These notes run past the length kept in the archive. The rest is on the publisher’s page.