Skip to content

Commit

Permalink
Update tools max tokens to 100 (#33)
Browse files Browse the repository at this point in the history
* Use 50 max tokens for function calls

* Bump tools tokens to 100, needed for groq l3.1 8b
  • Loading branch information
juberti authored Jul 29, 2024
1 parent 358cd72 commit 776ea2d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion .github/workflows/bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
curl -f -X POST "https://ai-benchmarks.fly.dev/bench?mode=audio&max_tokens=20&num_requests=1"
# Run the benchmarks
declare -A max_tokens=(
["tools"]=50
["tools"]=100
)
default_max_tokens=20
regions=("sea" "iad" "cdg")
Expand Down

0 comments on commit 776ea2d

Please sign in to comment.