Skip to content

Commit

Permalink
feat(doc): be more explicit on which tool_choice options are supporte…
Browse files Browse the repository at this point in the history
…d in the doc
  • Loading branch information
gcalmettes committed Nov 9, 2024
1 parent 32a35fb commit 426e098
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion docs/source/serving/openai_compatible_server.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,9 +189,10 @@ The order of priorities is `command line > config file values > defaults`.
---

## Tool calling in the chat completion API
vLLM supports only named function calling in the chat completion API. The `tool_choice` option `required` is **not yet supported** but on the roadmap.
vLLM currently supports named function calling, as well as the `auto` and `none` options for the `tool_choice` field in the chat completion API. The `tool_choice` option `required` is **not yet supported** but on the roadmap.

It is the callers responsibility to prompt the model with the tool information, vLLM will not automatically manipulate the prompt.
Please see below for recommended configuration and chat templates to use when function calling is to be used with the different models.


### Named Function Calling
Expand Down

0 comments on commit 426e098

Please sign in to comment.