Code

Upstream merge 25 1 6#350

Code

Merged

gshtras merged 202 commits intomainROCm/vllm:mainfrom upstream_merge_25_1_6ROCm/vllm:upstream_merge_25_1_6Copy head branch name to clipboardJan 8, 2025

+31,168-16,735

Commits

Commits on Dec 16, 2024

• [misc] Layerwise profile updates (vllm-project#10242)

  Show description for efbce85
  

  varun-sundar-rabindranath

  and

  Varun Sundar Rabindranath

  authored
  efbce85

  Copy full SHA for efbce85

• [core] overhaul memory profiling and fix backward compatibility (vllm-project#10511)

  Show description for 551603f
  youkaichao

  authored
  551603f

  Copy full SHA for 551603f

• [Docs] hint to enable use of GPU performance counters in profiling tools for multi-node distributed serving (vllm-project#11235)

  Show description for 35ffa68
  

  bk-TurbaAI

  and

  mgoin

  authored
  35ffa68

  Copy full SHA for 35ffa68

• [ci][tests] add gh200 tests (vllm-project#11244)

  Show description for c301616
  youkaichao

  authored
  c301616

  Copy full SHA for c301616

Commits on Dec 17, 2024

• [torch.compile] fast inductor (vllm-project#11108)

  Show description for 88a412e
  

  youkaichao

  and

  tlrmchlsmth

  authored
  88a412e

  Copy full SHA for 88a412e

• fix gh200 tests on main (vllm-project#11246)

  Show description for 35bae11
  youkaichao

  authored
  35bae11

  Copy full SHA for 35bae11

• [CI] Add test case with JSON schema using references + use xgrammar by default with OpenAI parse (vllm-project#10935)

  Show description for 0064f69
  mgoin

  authored
  0064f69

  Copy full SHA for 0064f69

• [Frontend] Add OpenAI API support for input_audio (vllm-project#11027)

  Show description for 66d4b16
  

  kylehh

  and

  DarkLight1337

  authored
  66d4b16

  Copy full SHA for 66d4b16

• [V1][VLM] Proper memory profiling for image language models (vllm-project#11210)

  Show description for 59c9b6e
  

  ywang96

  and

  ywang96

  authored
  59c9b6e

  Copy full SHA for 59c9b6e

• [Platform] platform agnostic for EngineArgs initialization (vllm-project#11225)

  Show description for e88db68
  wangxiyuan

  authored
  e88db68

  Copy full SHA for e88db68

• [V1][Core] Use weakref.finalize instead of atexit (vllm-project#11242)

  Show description for 2bfdbf2
  tlrmchlsmth

  authored
  2bfdbf2

  Copy full SHA for 2bfdbf2

• [Misc] Kernel Benchmark for RMSNorm (vllm-project#11241)

  Show description for 02222a0
  

  ywang96

  and

  BBuf

  authored
  02222a0

  Copy full SHA for 02222a0

• [Misc] Allow passing logits_soft_cap for xformers backend (vllm-project#11252)

  Show description for f9ecbb1
  Isotr0py

  authored
  f9ecbb1

  Copy full SHA for f9ecbb1

• [Bugfix] Fix request cancellation without polling (vllm-project#11190)

  joerunde

  authored
  2d1b9ba

  Copy full SHA for 2d1b9ba

Commits on Dec 18, 2024

• [Bugfix] Set temperature=0.7 in test_guided_choice_chat (vllm-project#11264)

  mgoin

  authored
  c77eb8a

  Copy full SHA for c77eb8a

• [V1] Prefix caching for vision language models (vllm-project#11187)

  Show description for bf8717e
  comaniac

  authored
  bf8717e

  Copy full SHA for bf8717e

• [Bugfix] Restore support for larger block sizes (vllm-project#11259)

  Show description for 866fa45
  kzawora-intel

  authored
  866fa45

  Copy full SHA for 866fa45

• [Bugfix] Fix guided decoding with tokenizer mode mistral (vllm-project#11046)

  wallashss

  authored
  8b79f9e

  Copy full SHA for 8b79f9e

• [MISC][XPU]update ipex link for CI fix (vllm-project#11278)

  yma11

  authored
  f04e407

  Copy full SHA for f04e407

• [Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (vllm-project#10995)

  Show description for 60508ff
  

  5 peopleauthored
  60508ff

  Copy full SHA for 60508ff

• [Bugfix] Fix broken phi3-v mm_processor_kwargs tests (vllm-project#11263)

  Show description for 996aa70
  Isotr0py

  authored
  996aa70

  Copy full SHA for 996aa70

• [CI][Misc] Remove Github Action Release Workflow (vllm-project#11274)

  simon-mo

  authored
  362cff1

  Copy full SHA for 362cff1

• [FIX] update openai version (vllm-project#11287)

  Show description for f954fe0
  jikunshang

  authored
  f954fe0

  Copy full SHA for f954fe0

• [Bugfix] fix minicpmv test (vllm-project#11304)

  Show description for ca5f54a
  joerunde

  authored
  ca5f54a

  Copy full SHA for ca5f54a

• [V1] VLM - enable processor cache by default (vllm-project#11305)

  Show description for fdea8ec
  alexm-redhat

  authored
  fdea8ec

  Copy full SHA for fdea8ec

Commits on Dec 19, 2024

• [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (vllm-project#11311)

  Show description for 5a9da2e
  tlrmchlsmth

  authored
  5a9da2e

  Copy full SHA for 5a9da2e

• [Model] IBM Granite 3.1 (vllm-project#11307)

  Show description for 17ca964
  tjohnson31415

  authored
  17ca964

  Copy full SHA for 17ca964

• [CI] Expand test_guided_generate to test all backends (vllm-project#11313)

  Show description for a30482f
  mgoin

  authored
  a30482f

  Copy full SHA for a30482f

• [V1] Simplify prefix caching logic by removing num_evictable_computed_blocks (vllm-project#11310)

  heheda12345

  authored
  c6b0a7d

  Copy full SHA for c6b0a7d

• [VLM] Merged multimodal processor for Qwen2-Audio (vllm-project#11303)

  Show description for 6142ef0
  DarkLight1337

  authored
  6142ef0

  Copy full SHA for 6142ef0

• [Kernel] Refactor Cutlass c3x (vllm-project#10049)

  Show description for 8936316
  

  varun-sundar-rabindranath

  and

  Varun Sundar Rabindranath

  authored
  8936316

  Copy full SHA for 8936316

• [Misc] Optimize ray worker initialization time (vllm-project#11275)

  Show description for f26c4ae
  

  ruisearch42

  and

  comaniac

  authored
  f26c4ae

  Copy full SHA for f26c4ae

• [misc] benchmark_throughput : Add LoRA (vllm-project#11267)

  Show description for 9835673
  

  varun-sundar-rabindranath

  and

  Varun Sundar Rabindranath

  authored
  9835673

  Copy full SHA for 9835673

• [Feature] Add load generation config from model (vllm-project#11164)

  Show description for 5aef498
  

  liuyanyi

  and

  DarkLight1337

  authored
  5aef498

  Copy full SHA for 5aef498

• [Bugfix] Cleanup Pixtral HF code (vllm-project#11333)

  Show description for a0f7d53
  DarkLight1337

  authored
  a0f7d53

  Copy full SHA for a0f7d53

• [Model] Add JambaForSequenceClassification model (vllm-project#10860)

  Show description for 6c7f881
  

  3 peopleauthored
  6c7f881

  Copy full SHA for 6c7f881

• [V1] Fix multimodal profiling for Molmo (vllm-project#11325)

  Show description for 7379b3d
  

  ywang96

  and

  ywang96

  authored
  7379b3d

  Copy full SHA for 7379b3d

• [Model] Refactor Qwen2-VL to use merged multimodal processor (vllm-project#11258)

  Show description for e24113a
  

  Isotr0py

  and

  DarkLight1337

  authored
  e24113a

  Copy full SHA for e24113a

• [Misc] Clean up and consolidate LRUCache (vllm-project#11339)

  Show description for cdf22af
  DarkLight1337

  authored
  cdf22af

  Copy full SHA for cdf22af

• [Bugfix] Fix broken CPU compressed-tensors test (vllm-project#11338)

  Show description for 276738c
  Isotr0py

  authored
  276738c

  Copy full SHA for 276738c

• [Misc] Remove unused vllm/block.py (vllm-project#11336)

  Ghjk94522

  authored
  e461c26

  Copy full SHA for e461c26

• [CI] Adding CPU docker pipeline (vllm-project#11261)

  Show description for a985f7a
  

  zhouyuan

  and

  khluu

  authored
  a985f7a

  Copy full SHA for a985f7a

Commits on Dec 20, 2024

• [Bugfix][Hardware][POWERPC] Fix auto dtype failure in case of POWER10 (vllm-project#11331)

  Show description for 48edab8
  Akashcodes732

  authored
  48edab8

  Copy full SHA for 48edab8

• [ci][gh200] dockerfile clean up (vllm-project#11351)

  Show description for 7801f56
  

  3 peopleauthored
  7801f56

  Copy full SHA for 7801f56

• [Misc] Add tqdm progress bar during graph capture (vllm-project#11349)

  Show description for b880ffb
  mgoin

  authored
  b880ffb

  Copy full SHA for b880ffb

• [Bugfix] Fix spec decoding when seed is none in a batch (vllm-project#10863)

  Show description for 86c2d8f
  wallashss

  authored
  86c2d8f

  Copy full SHA for 86c2d8f

• [misc] add early error message for custom ops (vllm-project#11355)

  Show description for c954f21
  youkaichao

  authored
  c954f21

  Copy full SHA for c954f21

• [doc] backward compatibility for 0.6.4 (vllm-project#11359)

  Show description for 1ecc645
  youkaichao

  authored
  1ecc645

  Copy full SHA for 1ecc645

• [V1] Fix profiling for models with merged input processor (vllm-project#11370)

  Show description for 04139ad
  ywang96

  authored
  04139ad

  Copy full SHA for 04139ad

• [CI/Build] fix pre-compiled wheel install for exact tag (vllm-project#11373)

  Show description for 7c7aa37
  dtrifiro

  authored
  7c7aa37

  Copy full SHA for 7c7aa37

• [Core] Loading model from S3 using RunAI Model Streamer as optional loader (vllm-project#10192)

  Show description for 995f562
  omer-dayan

  authored
  995f562

  Copy full SHA for 995f562

• [Bugfix] Don't log OpenAI field aliases as ignored (vllm-project#11378)

  Show description for d573aea
  mgoin

  authored
  d573aea

  Copy full SHA for d573aea

• [doc] explain nccl requirements for rlhf (vllm-project#11381)

  Show description for 5d2248d
  youkaichao

  authored
  5d2248d

  Copy full SHA for 5d2248d

• Add ray[default] to wget to run distributed inference out of box (vllm-project#11265)

  Show description for 47a0b61
  Jeffwan

  authored
  47a0b61

  Copy full SHA for 47a0b61

Commits on Dec 21, 2024

• [V1][Bugfix] Skip hashing empty or None mm_data (vllm-project#11386)

  Show description for dd2b563
  WoosukKwon

  authored
  dd2b563

  Copy full SHA for dd2b563

• [Bugfix] update should_ignore_layer (vllm-project#11354)

  Show description for 51ff216
  horheynm

  authored
  51ff216

  Copy full SHA for 51ff216

• [V1] Make AsyncLLMEngine v1-v0 opaque (vllm-project#11383)

  Show description for 584f0ae
  rickyyx

  authored
  584f0ae

  Copy full SHA for 584f0ae

• [Bugfix] Fix issues for Pixtral-Large-Instruct-2411 (vllm-project#11393)

  Show description for c2d1b07
  

  ywang96

  and

  ywang96

  authored
  c2d1b07

  Copy full SHA for c2d1b07

Commits on Dec 22, 2024

• [CI] Fix flaky entrypoint tests (vllm-project#11403)

  Show description for 29c7489
  ywang96

  authored
  29c7489

  Copy full SHA for 29c7489

• [cd][release] add pypi index for every commit and nightly build (vllm-project#11404)

  Show description for 4a91397
  

  youkaichao

  and

  ywang96

  authored
  4a91397

  Copy full SHA for 4a91397

• [cd][release] fix race conditions (vllm-project#11407)

  Show description for 72d9c31
  youkaichao

  authored
  72d9c31

  Copy full SHA for 72d9c31

• [Bugfix] Fix fully sharded LoRAs with Mixtral (vllm-project#11390)

  Show description for f1d1bf6
  n1hility

  authored
  f1d1bf6

  Copy full SHA for f1d1bf6

• [CI] Unboock H100 Benchmark (vllm-project#11419)

  Show description for 048fc57
  simon-mo

  authored
  048fc57

  Copy full SHA for 048fc57

Commits on Dec 23, 2024

• [misc][perf] remove old code (vllm-project#11425)

  Show description for f30581c
  youkaichao

  authored
  f30581c

  Copy full SHA for f30581c

• mypy type checking for vllm/worker (vllm-project#11418)

  Show description for e51719a
  

  lucas-tucker

  and

  lucast2021

  authored
  e51719a

  Copy full SHA for e51719a

• [Bugfix] Fix CFGGuide and use outlines for grammars that can't convert to GBNF (vllm-project#11389)

  Show description for 5bfb30a
  mgoin

  authored
  5bfb30a

  Copy full SHA for 5bfb30a

• [Bugfix] torch nightly version in ROCm installation guide (vllm-project#11423)

  Show description for 2e72668
  terrytangyuan

  authored
  2e72668

  Copy full SHA for 2e72668

• [Misc] Add assertion and helpful message for marlin24 compressed models (vllm-project#11388)

  dsikka

  authored
  b866cdb

  Copy full SHA for b866cdb

• [Misc] add w8a8 asym models (vllm-project#11075)

  dsikka

  authored
  8cef6e0

  Copy full SHA for 8cef6e0

• [CI] Expand OpenAI test_chat.py guided decoding tests (vllm-project#11048)

  Show description for 63afbe9
  mgoin

  authored
  63afbe9

  Copy full SHA for 63afbe9

• [Bugfix] Add kv cache scales to gemma2.py (vllm-project#11269)

  mgoin

  authored
  60fb4f3

  Copy full SHA for 60fb4f3

• [Doc] Fix typo in the help message of '--guided-decoding-backend' (vllm-project#11440)

  yansh97

  authored
  94d545a

  Copy full SHA for 94d545a

• [Docs] Convert rST to MyST (Markdown) (vllm-project#11145)

  Show description for 32aa205
  rafvasq

  authored
  32aa205

  Copy full SHA for 32aa205

• [V1] TP Ray executor (vllm-project#11107)

  Show description for a491d6f
  ruisearch42

  authored
  a491d6f

  Copy full SHA for a491d6f

Commits on Dec 24, 2024

• [Misc]Suppress irrelevant exception stack trace information when CUDA… (vllm-project#11438)

  Show description for 4f074fb
  shiquan1988

  authored
  4f074fb

  Copy full SHA for 4f074fb

• [Frontend] Online Pooling API (vllm-project#11457)

  Show description for 9edca6b
  DarkLight1337

  authored
  9edca6b

  Copy full SHA for 9edca6b

• [Bugfix] Fix Qwen2-VL LoRA weight loading (vllm-project#11430)

  Show description for b1b1038
  jeejeelee

  authored
  b1b1038

  Copy full SHA for b1b1038

• [Bugfix][Hardware][CPU] Fix CPU input_positions creation for text-only inputs with mrope (vllm-project#11434)

  Show description for 7a5286c
  Isotr0py

  authored
  7a5286c

  Copy full SHA for 7a5286c

• [OpenVINO] Fixed installation conflicts (vllm-project#11458)

  Show description for 461cde2
  ilya-lavrenov

  authored
  461cde2

  Copy full SHA for 461cde2

• [attn][tiny fix] fix attn backend in MultiHeadAttention (vllm-project#11463)

  Show description for 5c79632
  MengqingCao

  authored
  5c79632

  Copy full SHA for 5c79632

• [Misc] Move weights mapper (vllm-project#11443)

  Show description for 196c34b
  jeejeelee

  authored
  196c34b

  Copy full SHA for 196c34b

• [Bugfix] Fix issues in CPU build Dockerfile. Fixes vllm-project#9182 (vllm-project#11435)

  Show description for 409475a
  terrytangyuan

  authored
  409475a

  Copy full SHA for 409475a

• [Model] Automatic conversion of classification and reward models (vllm-project#11469)

  Show description for 3f3e92e
  DarkLight1337

  authored
  3f3e92e

  Copy full SHA for 3f3e92e

Commits on Dec 25, 2024

• [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (vllm-project#11472)

  ruisearch42

  authored
  9832e55

  Copy full SHA for 9832e55

• [Misc] Update disaggregation benchmark scripts and test logs (vllm-project#11456)

  Show description for fc60166
  Jeffwan

  authored
  fc60166

  Copy full SHA for fc60166

• [Frontend] Enable decord to load video from base64 (vllm-project#11492)

  Show description for b689ada
  DarkLight1337

  authored
  b689ada

  Copy full SHA for b689ada

• [Doc] Improve GitHub links (vllm-project#11491)

  Show description for 6ad909f
  DarkLight1337

  authored
  6ad909f

  Copy full SHA for 6ad909f

Commits on Dec 26, 2024

• [Misc] Move some multimodal utils to modality-specific modules (vllm-project#11494)

  Show description for 51a624b
  DarkLight1337

  authored
  51a624b

  Copy full SHA for 51a624b

• Mypy checking for vllm/compilation (vllm-project#11496)

  Show description for dbeac95
  

  lucas-tucker

  and

  lucast2021

  authored
  dbeac95

  Copy full SHA for dbeac95

• [Misc][LoRA] Fix LoRA weight mapper (vllm-project#11495)

  Show description for aa25985
  jeejeelee

  authored
  aa25985

  Copy full SHA for aa25985

• [Doc] Add QVQ and QwQ to the list of supported models (vllm-project#11509)

  Show description for 7492a36
  

  ywang96

  and

  DarkLight1337

  authored
  7492a36

  Copy full SHA for 7492a36

• [V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (vllm-project#10681)

  Show description for dcb1a94
  

  sroy745

  and

  WoosukKwon

  authored
  dcb1a94

  Copy full SHA for dcb1a94

• [Model] Modify MolmoForCausalLM MLP (vllm-project#11510)

  Show description for f57ee56
  jeejeelee

  authored
  f57ee56

  Copy full SHA for f57ee56

• [Misc] Add placeholder module (vllm-project#11501)

  Show description for eec906d
  DarkLight1337

  authored
  eec906d

  Copy full SHA for eec906d

• [Doc] Add video example to openai client for multimodal (vllm-project#11521)

  Show description for b85a977
  

  Isotr0py

  and

  DarkLight1337

  authored
  b85a977

  Copy full SHA for b85a977

• [1/N] API Server (Remove Proxy) (vllm-project#11529)

  robertgshaw2-redhat

  authored
  720b10f

  Copy full SHA for 720b10f

• [Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (vllm-project#11523)

  Show description for 2072924
  

  3 peopleauthored
  2072924

  Copy full SHA for 2072924

• [2/N] API Server: Avoid ulimit footgun (vllm-project#11530)

  robertgshaw2-redhat

  authored
  55fb97f

  Copy full SHA for 55fb97f

Commits on Dec 27, 2024

• Deepseek v3 (vllm-project#11502)

  Show description for f49777b
  

  3 peopleauthored
  f49777b

  Copy full SHA for f49777b

• [Docs] Document Deepseek V3 support (vllm-project#11535)

  Show description for 82d24f7
  simon-mo

  authored
  82d24f7

  Copy full SHA for 82d24f7

• Update openai_compatible_server.md (vllm-project#11536)

  Show description for 0c0c201
  

  robertgshaw2-redhat

  and

  simon-mo

  authored
  0c0c201

  Copy full SHA for 0c0c201

• [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (vllm-project#11394)

  Show description for 371d04d
  WoosukKwon

  authored
  371d04d

  Copy full SHA for 371d04d

• [V1] Fix yapf (vllm-project#11538)

  Show description for 81b979f
  WoosukKwon

  authored
  81b979f

  Copy full SHA for 81b979f

• [CI] Fix broken CI (vllm-project#11543)

  robertgshaw2-redhat

  authored
  46d4359

  Copy full SHA for 46d4359

• [misc] fix typing (vllm-project#11540)

  Show description for eb881ed
  youkaichao

  authored
  eb881ed

  Copy full SHA for eb881ed

• [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (vllm-project#11534)

  robertgshaw2-redhat

  authored
  1b875a0

  Copy full SHA for 1b875a0

• [BugFix] Fix quantization for all other methods (vllm-project#11547)

  robertgshaw2-redhat

  authored
  2339d59

  Copy full SHA for 2339d59

• [Platform] Move model arch check to platform (vllm-project#11503)

  Show description for 6c6f7fe
  MengqingCao

  authored
  6c6f7fe

  Copy full SHA for 6c6f7fe

• Update deploying_with_k8s.md with AMD ROCm GPU example (vllm-project#11465)

  Show description for d003f3e
  

  AlexHe99

  and

  DarkLight1337

  authored
  d003f3e

  Copy full SHA for d003f3e

• [Bugfix] Fix TeleChat2ForCausalLM weights mapper (vllm-project#11546)

  Show description for 2c9b8ea
  jeejeelee

  authored
  2c9b8ea

  Copy full SHA for 2c9b8ea

• [Misc] Abstract the logic for reading and writing media content (vllm-project#11527)

  Show description for 7af553e
  DarkLight1337

  authored
  7af553e

  Copy full SHA for 7af553e

• [Doc] Add xgrammar in doc (vllm-project#11549)

  Show description for 5ce4627
  Chen-0210

  authored
  5ce4627

  Copy full SHA for 5ce4627

• [VLM] Support caching in merged multi-modal processor (vllm-project#11396)

  Show description for 1014180
  DarkLight1337

  authored
  1014180

  Copy full SHA for 1014180

• [MODEL] LoRA support for Jamba model (vllm-project#11209)

  Show description for 55509c2
  ErezSC42

  authored
  55509c2

  Copy full SHA for 55509c2

• [Misc]Add BNB quantization for MolmoForCausalLM (vllm-project#11551)

  Show description for 0240402
  jeejeelee

  authored
  0240402

  Copy full SHA for 0240402

• [Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix (vllm-project#11566)

  Show description for dde1fa1
  Isotr0py

  authored
  dde1fa1

  Copy full SHA for dde1fa1

• [Bugfix] Fix for ROCM compressed tensor support (vllm-project#11561)

  selalipop

  authored
  ac79799

  Copy full SHA for ac79799

Commits on Dec 28, 2024

• [Doc] Update mllama example based on official doc (vllm-project#11567)

  Show description for a607312
  heheda12345

  authored
  a607312

  Copy full SHA for a607312

• [V1] [4/N] API Server: ZMQ/MP Utilities (vllm-project#11541)

  robertgshaw2-redhat

  authored
  df04dff

  Copy full SHA for df04dff

• [Bugfix] Last token measurement fix (vllm-project#11376)

  Show description for b5cbe8e
  

  rajveerb

  and

  ywang96

  authored
  b5cbe8e

  Copy full SHA for b5cbe8e

• [Model] Support InternLM2 Reward models (vllm-project#11571)

  Show description for d34be24
  

  Isotr0py

  and

  DarkLight1337

  authored
  d34be24

  Copy full SHA for d34be24

• [Model] Remove hardcoded image tokens ids from Pixtral (vllm-project#11582)

  Show description for b7dcc00
  ywang96

  authored
  b7dcc00

  Copy full SHA for b7dcc00

• [Hardware][AMD]: Replace HIPCC version with more precise ROCm version (vllm-project#11515)

  Show description for 59d6bb4
  hj-wei

  authored
  59d6bb4

  Copy full SHA for 59d6bb4

• [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (vllm-project#11581)

  Show description for 42bb201
  WoosukKwon

  authored
  42bb201

  Copy full SHA for 42bb201

• [Doc] Minor documentation fixes (vllm-project#11580)

  Show description for d427e5c
  DarkLight1337

  authored
  d427e5c

  Copy full SHA for d427e5c

• [bugfix] interleaving sliding window for cohere2 model (vllm-project#11583)

  Show description for 328841d
  youkaichao

  authored
  328841d

  Copy full SHA for 328841d

• [V1] [5/N] API Server: unify Detokenizer and EngineCore input (vllm-project#11545)

  Show description for 4fb8e32
  robertgshaw2-redhat

  authored
  4fb8e32

  Copy full SHA for 4fb8e32

Commits on Dec 29, 2024

• [Doc] Convert list tables to MyST (vllm-project#11594)

  Show description for 32b4c63
  DarkLight1337

  authored
  32b4c63

  Copy full SHA for 32b4c63

• [v1][bugfix] fix cudagraph with inplace buffer assignment (vllm-project#11596)

  Show description for dba4d9d
  youkaichao

  authored
  dba4d9d

  Copy full SHA for dba4d9d

• [Misc] KV cache transfer connector registry (vllm-project#11481)

  Show description for faef77c
  KuntaiDu

  authored
  faef77c

  Copy full SHA for faef77c

• Remove print statement in DeepseekScalingRotaryEmbedding (vllm-project#11604)

  mgoin

  authored
  0aa38d1

  Copy full SHA for 0aa38d1

Commits on Dec 30, 2024

• [v1] fix compilation cache (vllm-project#11598)

  Show description for 3682e33
  youkaichao

  authored
  3682e33

  Copy full SHA for 3682e33

• [Docker] bump up neuron sdk v2.21 (vllm-project#11593)

  Show description for 628ec6c
  liangfu

  authored
  628ec6c

  Copy full SHA for 628ec6c

• [Build][Kernel] Update CUTLASS to v3.6.0 (vllm-project#11607)

  Show description for 970d6d0
  tlrmchlsmth

  authored
  970d6d0

  Copy full SHA for 970d6d0

• [CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (vllm-project#11618)

  Show description for 5dbf854
  bigPYJ1151

  authored
  5dbf854

  Copy full SHA for 5dbf854

• [platforms] enable platform plugins (vllm-project#11602)

  Show description for b12e87f
  youkaichao

  authored
  b12e87f

  Copy full SHA for b12e87f

• [VLM] Abstract out multi-modal data parsing in merged processor (vllm-project#11620)

  Show description for 8d9b672
  DarkLight1337

  authored
  8d9b672

  Copy full SHA for 8d9b672

• [V1] [6/N] API Server: Better Shutdown (vllm-project#11586)

  robertgshaw2-redhat

  authored
  5886aa4

  Copy full SHA for 5886aa4

• [Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel (vllm-project#11631)

  whyiug

  authored
  36e7670

  Copy full SHA for 36e7670

• [benchmark] Remove dependency for H100 benchmark step (vllm-project#11572)

  khluu

  authored
  ccb1aab

  Copy full SHA for ccb1aab

Commits on Dec 31, 2024

• [Model][LoRA]LoRA support added for MolmoForCausalLM (vllm-project#11439)

  Show description for a2a40bc
  

  3 peopleauthored
  a2a40bc

  Copy full SHA for a2a40bc

• [Bugfix] Fix OpenAI parallel sampling when using xgrammar (vllm-project#11637)

  Show description for 74fa1d1
  mgoin

  authored
  74fa1d1

  Copy full SHA for 74fa1d1

• [Misc][LoRA] Support Rank Stabilized LoRA (RSLoRA) (vllm-project#6909)

  Show description for 82c49d3
  

  JohnGiorgi

  and

  jeejeelee

  authored
  82c49d3

  Copy full SHA for 82c49d3

• [Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (vllm-project#11565)

  sakunkun

  authored
  2c57188

  Copy full SHA for 2c57188

• [V1] Simpify vision block hash for prefix caching by removing offset from hash (vllm-project#11646)

  heheda12345

  authored
  8c3230d

  Copy full SHA for 8c3230d

• [V1][VLM] V1 support for selected single-image models. (vllm-project#11632)

  Show description for e7c7c5e
  

  3 peopleauthored
  e7c7c5e

  Copy full SHA for e7c7c5e

Commits on Jan 1, 2025

• [Benchmark] Add benchmark script for CPU offloading (vllm-project#11533)

  Show description for 0c6f998
  

  ApostaC

  and

  KuntaiDu

  authored
  0c6f998

  Copy full SHA for 0c6f998

• [Bugfix][Refactor] Unify model management in frontend (vllm-project#11660)

  Show description for 4db72e5
  joerunde

  authored
  4db72e5

  Copy full SHA for 4db72e5

• [VLM] Add max-count checking in data parser for single image models (vllm-project#11661)

  Show description for 365801f
  

  DarkLight1337

  and

  ywang96

  authored
  365801f

  Copy full SHA for 365801f

• [Misc] Optimize Qwen2-VL LoRA test (vllm-project#11663)

  Show description for 11d8a09
  jeejeelee

  authored
  11d8a09

  Copy full SHA for 11d8a09

• [Misc] Replace space with - in the file names (vllm-project#11667)

  Show description for f962f42
  houseroad

  authored
  f962f42

  Copy full SHA for f962f42

• [Doc] Fix typo (vllm-project#11666)

  Show description for 6d70198
  serihiro

  authored
  6d70198

  Copy full SHA for 6d70198

• [V1] Implement Cascade Attention (vllm-project#11635)

  Show description for 7300144
  WoosukKwon

  authored
  7300144

  Copy full SHA for 7300144

• [VLM] Move supported limits and max tokens to merged multi-modal processor (vllm-project#11669)

  Show description for a115ac4
  

  DarkLight1337

  and

  Isotr0py

  authored
  a115ac4

  Copy full SHA for a115ac4

Commits on Jan 2, 2025

• [VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (vllm-project#11674)

  Show description for 23c1b10
  DarkLight1337

  authored
  23c1b10

  Copy full SHA for 23c1b10

• [mypy] Pass type checking in vllm/inputs (vllm-project#11680)

  Show description for b6087a6
  CloseChoice

  authored
  b6087a6

  Copy full SHA for b6087a6

• [VLM] Merged multi-modal processor for LLaVA-NeXT (vllm-project#11682)

  Show description for 8c38ee7
  DarkLight1337

  authored
  8c38ee7

  Copy full SHA for 8c38ee7

• According to vllm.EngineArgs, the name should be distributed_executor_backend (vllm-project#11689)

  chunyang-wen

  authored
  84c35c3

  Copy full SHA for 84c35c3

• [Bugfix] Free cross attention block table for preempted-for-recompute sequence group. (vllm-project#10013)

  Show description for 2f38518
  kathyyu-google

  authored
  2f38518

  Copy full SHA for 2f38518

• [V1][Minor] Optimize token_ids_cpu copy (vllm-project#11692)

  Show description for b55ed6e
  WoosukKwon

  authored
  b55ed6e

  Copy full SHA for b55ed6e

• [Bugfix] Change kv scaling factor by param json on nvidia gpu (vllm-project#11688)

  Show description for 187e329
  

  bjmsong

  and

  mdattack

  authored
  187e329

  Copy full SHA for 187e329

• Resolve race conditions in Marlin kernel (vllm-project#11493)

  Show description for 5dba257
  wchen61

  authored
  5dba257

  Copy full SHA for 5dba257

• [Misc] Minimum requirements for SageMaker compatibility (vllm-project#11576)

  nathan-az

  authored
  68d3780

  Copy full SHA for 68d3780

Commits on Jan 3, 2025

• Update default max_num_batch_tokens for chunked prefill (vllm-project#11694)

  SachinVarghese

  authored
  2f1e8e8

  Copy full SHA for 2f1e8e8

• [Bugfix] Check chain_speculative_sampling before calling it (vllm-project#11673)

  Show description for 07064cb
  houseroad

  authored
  07064cb

  Copy full SHA for 07064cb

• [perf-benchmark] Fix dependency for steps in benchmark pipeline (vllm-project#11710)

  khluu

  authored
  fd3a62a

  Copy full SHA for fd3a62a

• [Model] Whisper model implementation (vllm-project#11280)

  Show description for e1a5c2f
  

  aurickq

  and

  sfc-gh-aqiao

  authored
  e1a5c2f

  Copy full SHA for e1a5c2f

• [V1] Simplify Shutdown (vllm-project#11659)

  robertgshaw2-redhat

  authored
  80c751e

  Copy full SHA for 80c751e

• [Bugfix] Fix ColumnParallelLinearWithLoRA slice (vllm-project#11708)

  Show description for 61fed92
  zinccat

  authored
  61fed92

  Copy full SHA for 61fed92

• [V1] Improve TP>1 Error Handling + Stack Trace (vllm-project#11721)

  Show description for 1543914
  

  robertgshaw2-redhat

  and

  tlrmchlsmth

  authored
  1543914

  Copy full SHA for 1543914

• [Misc]Add BNB quantization for Qwen2VL (vllm-project#11719)

  Show description for a655eb3
  

  jeejeelee

  and

  Isotr0py

  authored
  a655eb3

  Copy full SHA for a655eb3

• Update requirements-tpu.txt to support python 3.9 and 3.11 (vllm-project#11695)

  Show description for bf0d97d
  mgoin

  authored
  bf0d97d

  Copy full SHA for bf0d97d

• [V1] Chore: cruft removal (vllm-project#11724)

  robertgshaw2-redhat

  authored
  ad0d567

  Copy full SHA for ad0d567

Commits on Jan 4, 2025

• [V1] log GPU blocks num for MultiprocExecutor (vllm-project#11656)

  WangErXiao

  authored
  e5d7ed0

  Copy full SHA for e5d7ed0

• Update tool_calling.md (vllm-project#11701)

  Bryce1010

  authored
  9c93636

  Copy full SHA for 9c93636

• Update bnb.md with example for OpenAI (vllm-project#11718)

  bet0x

  authored
  d1d4939

  Copy full SHA for d1d4939

• [V1] Add RayExecutor support for AsyncLLM (api server) (vllm-project#11712)

  jikunshang

  authored
  fbf2564

  Copy full SHA for fbf2564

• [V1] Add kv cache utils tests. (vllm-project#11513)

  Show description for d91457d
  xcnick

  authored
  d91457d

  Copy full SHA for d91457d

• [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (vllm-project#11233)

  Show description for 300acb8
  yanburman

  authored
  300acb8

  Copy full SHA for 300acb8

• [VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision (vllm-project#11717)

  Show description for eed11eb
  DarkLight1337

  authored
  eed11eb

  Copy full SHA for eed11eb

• [Bugfix] Fix precision error in LLaVA-NeXT (vllm-project#11735)

  Show description for ba214df
  DarkLight1337

  authored
  ba214df

  Copy full SHA for ba214df

• [Model] Remove unnecessary weight initialization logic (vllm-project#11736)

  Show description for 65c0892
  

  DarkLight1337

  and

  Isotr0py

  authored
  65c0892

  Copy full SHA for 65c0892

• [Bugfix][V1] Fix test_kv_cache_utils.py (vllm-project#11738)

  Show description for 4783143
  jeejeelee

  authored
  4783143

  Copy full SHA for 4783143

Commits on Jan 5, 2025

• [MISC] Replace c10::optional with std::optional (vllm-project#11730)

  Show description for 4068f4b
  houseroad

  authored
  4068f4b

  Copy full SHA for 4068f4b

• [distributed] remove pynccl's redundant stream (vllm-project#11744)

  cennn

  authored
  635b897

  Copy full SHA for 635b897

• fix: [doc] fix typo (vllm-project#11751)

  Show description for eba1717
  

  RuixiangMa

  and

  Lancer

  authored
  eba1717

  Copy full SHA for eba1717

• [Frontend] Improve StreamingResponse Exception Handling (vllm-project#11752)

  robertgshaw2-redhat

  authored
  33fc1e2

  Copy full SHA for 33fc1e2

Commits on Jan 6, 2025

• [distributed] remove pynccl's redundant change_state (vllm-project#11749)

  cennn

  authored
  9e764e7

  Copy full SHA for 9e764e7

• [Doc] [1/N] Reorganize Getting Started section (vllm-project#11645)

  Show description for 402d378
  DarkLight1337

  authored
  402d378

  Copy full SHA for 402d378

• [Bugfix] Remove block size constraint (vllm-project#11723)

  comaniac

  authored
  408e560

  Copy full SHA for 408e560

• [V1] Add BlockTable class (vllm-project#11693)

  Show description for 06bfb51
  WoosukKwon

  authored
  06bfb51

  Copy full SHA for 06bfb51

• [Misc] Fix typo for valid_tool_parses (vllm-project#11753)

  Show description for f8fcca1
  ruisearch42

  authored
  f8fcca1

  Copy full SHA for f8fcca1

• [V1] Refactor get_executor_cls (vllm-project#11754)

  ruisearch42

  authored
  022c5c6

  Copy full SHA for 022c5c6

• [mypy] Forward pass function type hints in lora (vllm-project#11740)

  Show description for 9c74971
  

  lucas-tucker

  and

  lucast2021

  authored
  9c74971

  Copy full SHA for 9c74971

• k8s-config: Update the secret to use stringData (vllm-project#11679)

  Show description for 2a622d7
  surajssd

  authored
  2a622d7

  Copy full SHA for 2a622d7

• [VLM] Separate out profiling-related logic (vllm-project#11746)

  Show description for 996357e
  DarkLight1337

  authored
  996357e

  Copy full SHA for 996357e

• [Doc][2/N] Reorganize Models and Usage sections (vllm-project#11755)

  Show description for ee77fdb
  DarkLight1337

  authored
  ee77fdb

  Copy full SHA for ee77fdb

• [Bugfix] Fix max image size for LLaVA-Onevision (vllm-project#11769)

  Show description for 9279b9f
  ywang96

  authored
  9279b9f

  Copy full SHA for 9279b9f

• [doc] explain how to add interleaving sliding window support (vllm-project#11771)

  Show description for 4ca5d40
  youkaichao

  authored
  4ca5d40

  Copy full SHA for 4ca5d40

• Merge remote-tracking branch 'upstream/main'

  gshtras

  committed
  4773c29

  Copy full SHA for 4773c29

• format

  gshtras

  committed
  267c1a1

  Copy full SHA for 267c1a1

Commits on Jan 8, 2025

• Merge branch 'main' into upstream_merge_25_1_6

  gshtras

  authored
  97067c0

  Copy full SHA for 97067c0