Skip to content

Commit

Permalink
Update llm-benchmark.md
Browse files Browse the repository at this point in the history
  • Loading branch information
vprelovac authored Dec 21, 2024
1 parent 99786cb commit 52f29ce
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/kagi/ai/llm-benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Reasoning models are optimized for multi-step reasoning and often produce better

| Model | Accuracy (%) | Tokens | Total Cost ($) | Median Latency (s) | Speed (tokens/sec) |
|-----------------------------|---------------|--------|----------------|------------------|-------------------|
| **Google** gemini-2.0-flash-thinking-exp-1219 | 51.61% | 52323 | 2.26607 | 4.67 | n/a |
| **Google** gemini-2.0-flash-thinking-exp-1219 | 51.61 | 52323 | 2.26607 | 4.67 | n/a |
| **Qwen** QWQ-32B | 50.00 | 45293 | 0.02835 | 15.46 | n/a |
| **OpenAI** o1-mini | 37.10 | 42965 | 0.53978 | 5.24 | n/a |

Expand Down

0 comments on commit 52f29ce

Please sign in to comment.