Skip to content

Commit

Permalink
Update currentMetrics.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@juberti
juberti authored Apr 11, 2024
1 parent 198cce8 commit 58bdf22
Showing 1 changed file with 42 additions and 41 deletions.
83 changes: 42 additions & 41 deletions src/data/currentMetrics.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,44 +2,45 @@
date: '04/10/2024'
---

Provider/Model | TTR | TTFT | TPS | Tok | Total
gpt-4-turbo | 0.39 | 0.40 | 39 | 79 | 2.40
gpt-4-0125-preview | 0.35 | 0.46 | 23 | 91 | 4.39
gpt-4-1106-preview | 0.38 | 0.46 | 26 | 64 | 2.85
fixie-westus.azure/gpt-4-1106-preview | 0.29 | 0.29 | 23 | 78 | 3.64
fixie-eastus2.azure/gpt-4-1106-preview | 0.64 | 0.64 | 12 | 79 | 7.33
gpt-3.5-turbo-0125 | 0.20 | 0.21 | 82 | 60 | 0.94
gpt-3.5-turbo-1106 | 0.27 | 0.29 | 106 | 47 | 0.72
fixie-westus.azure/gpt-3.5-turbo-1106 | 0.09 | 0.09 | 97 | 53 | 0.62
fixie-eastus2.azure/gpt-3.5-turbo | 0.15 | 0.15 | 72 | 96 | 1.46
claude-3-opus-20240229 | 0.99 | 0.99 | 20 | 77 | 4.71
claude-3-sonnet-20240229 | 0.37 | 0.37 | 45 | 74 | 1.98
claude-3-haiku-20240307 | 0.32 | 0.32 | 87 | 100 | 1.47
claude-2.1 | 0.39 | 0.39 | 19 | 52 | 3.12
claude-instant-1.2 | 0.45 | 0.45 | 64 | 100 | 2.00
command-r-plus | 0.10 | 0.19 | 33 | 78 | 2.54
command-r | 0.19 | 0.25 | 98 | 138 | 1.65
command-light | 0.18 | 0.22 | 50 | 101 | 2.23
gemini-pro | 0.45 | 0.63 | 214 | 100 | 1.09
gemini-1.5-pro-preview-0409 | 1.19 | 1.60 | 83 | 59 | 2.30
fixie-mistral.eastus2.azure | 2.51 | 2.51 | 999 | 65 | 2.51
api.fireworks.ai/mixtral-8x7b-instruct | 0.16 | 0.16 | 217 | 89 | 0.56
api.groq.com/mixtral-8x7b-32768 | 0.12 | 0.18 | 519 | 94 | 0.36
text.octoai.run/mixtral-8x7b-instruct | 0.34 | 0.37 | 64 | 88 | 1.73
api.perplexity.ai/sonar-medium-chat | 0.17 | 0.17 | 81 | 100 | 1.39
fixie-llama-2-70b.westus3.azure | 1.33 | 1.33 | 20 | 100 | 6.17
fixie-llama-2-70b.eastus2.azure | 2.11 | 2.11 | 24 | 100 | 6.16
api.fireworks.ai/llama-v2-70b-chat | 0.13 | 0.14 | 116 | 100 | 0.99
api.groq.com/llama2-70b-4096 | 0.07 | 0.27 | 289 | 101 | 0.62
text.octoai.run/llama-2-70b-chat-fp16 | 0.12 | 0.16 | 28 | 100 | 3.75
api.perplexity.ai/pplx-70b-chat | 0.16 | 0.16 | 54 | 100 | 1.98
togethercomputer/llama-2-70b-chat | 0.25 | 0.26 | 67 | 100 | 1.73
api.fireworks.ai/llama-v2-13b-chat | 0.21 | 0.23 | 144 | 100 | 0.91
text.octoai.run/llama-2-13b-chat-fp16 | 0.12 | 0.14 | 57 | 100 | 1.88
togethercomputer/llama-2-13b-chat | 0.28 | 0.29 | 45 | 100 | 2.47
api.fireworks.ai/llama-v2-7b-chat | 0.19 | 0.21 | 194 | 100 | 0.72
api.perplexity.ai/pplx-7b-chat | 0.14 | 0.14 | 127 | 100 | 0.92
togethercomputer/llama-2-7b-chat | 0.18 | 0.18 | 89 | 100 | 1.30
@cf/meta/llama-2-7b-chat-fp16 | 0.64 | 0.64 | 19 | 101 | 5.91
@cf/meta/llama-2-7b-chat-int8 | 0.46 | 0.46 | 35 | 101 | 3.32
Neets-7B | 0.46 | 0.47 | 82 | 100 | 1.67
| Provider/Model | TTR | TTFT | TPS | Tok | Total |
| :--------------------------------------- | :--- | :--- | :-- | :-- | :---- |
| gpt-4-turbo | 0.39 | 0.40 | 39 | 79 | 2.40 |
| gpt-4-0125-preview | 0.35 | 0.46 | 23 | 91 | 4.39 |
| gpt-4-1106-preview | 0.38 | 0.46 | 26 | 64 | 2.85 |
| fixie-westus.azure/gpt-4-1106-preview | 0.29 | 0.29 | 23 | 78 | 3.64 |
| fixie-eastus2.azure/gpt-4-1106-preview | 0.64 | 0.64 | 12 | 79 | 7.33 |
| gpt-3.5-turbo-0125 | 0.20 | 0.21 | 82 | 60 | 0.94 |
| gpt-3.5-turbo-1106 | 0.27 | 0.29 | 106 | 47 | 0.72 |
| fixie-westus.azure/gpt-3.5-turbo-1106 | 0.09 | 0.09 | 97 | 53 | 0.62 |
| fixie-eastus2.azure/gpt-3.5-turbo | 0.15 | 0.15 | 72 | 96 | 1.46 |
| claude-3-opus-20240229 | 0.99 | 0.99 | 20 | 77 | 4.71 |
| claude-3-sonnet-20240229 | 0.37 | 0.37 | 45 | 74 | 1.98 |
| claude-3-haiku-20240307 | 0.32 | 0.32 | 87 | 100 | 1.47 |
| claude-2.1 | 0.39 | 0.39 | 19 | 52 | 3.12 |
| claude-instant-1.2 | 0.45 | 0.45 | 64 | 100 | 2.00 |
| command-r-plus | 0.10 | 0.19 | 33 | 78 | 2.54 |
| command-r | 0.19 | 0.25 | 98 | 138 | 1.65 |
| command-light | 0.18 | 0.22 | 50 | 101 | 2.23 |
| gemini-pro | 0.45 | 0.63 | 214 | 100 | 1.09 |
| gemini-1.5-pro-preview-0409 | 1.19 | 1.60 | 83 | 59 | 2.30 |
| fixie-mistral.eastus2.azure | 2.51 | 2.51 | 999 | 65 | 2.51 |
| api.fireworks.ai/mixtral-8x7b-instruct | 0.16 | 0.16 | 217 | 89 | 0.56 |
| api.groq.com/mixtral-8x7b-32768 | 0.12 | 0.18 | 519 | 94 | 0.36 |
| text.octoai.run/mixtral-8x7b-instruct | 0.34 | 0.37 | 64 | 88 | 1.73 |
| api.perplexity.ai/sonar-medium-chat | 0.17 | 0.17 | 81 | 100 | 1.39 |
| fixie-llama-2-70b.westus3.azure | 1.33 | 1.33 | 20 | 100 | 6.17 |
| fixie-llama-2-70b.eastus2.azure | 2.11 | 2.11 | 24 | 100 | 6.16 |
| api.fireworks.ai/llama-v2-70b-chat | 0.13 | 0.14 | 116 | 100 | 0.99 |
| api.groq.com/llama2-70b-4096 | 0.07 | 0.27 | 289 | 101 | 0.62 |
| text.octoai.run/llama-2-70b-chat-fp16 | 0.12 | 0.16 | 28 | 100 | 3.75 |
| api.perplexity.ai/pplx-70b-chat | 0.16 | 0.16 | 54 | 100 | 1.98 |
| togethercomputer/llama-2-70b-chat | 0.25 | 0.26 | 67 | 100 | 1.73 |
| api.fireworks.ai/llama-v2-13b-chat | 0.21 | 0.23 | 144 | 100 | 0.91 |
| text.octoai.run/llama-2-13b-chat-fp16 | 0.12 | 0.14 | 57 | 100 | 1.88 |
| togethercomputer/llama-2-13b-chat | 0.28 | 0.29 | 45 | 100 | 2.47 |
| api.fireworks.ai/llama-v2-7b-chat | 0.19 | 0.21 | 194 | 100 | 0.72 |
| api.perplexity.ai/pplx-7b-chat | 0.14 | 0.14 | 127 | 100 | 0.92 |
| togethercomputer/llama-2-7b-chat | 0.18 | 0.18 | 89 | 100 | 1.30 |
| @cf/meta/llama-2-7b-chat-fp16 | 0.64 | 0.64 | 19 | 101 | 5.91 |
| @cf/meta/llama-2-7b-chat-int8 | 0.46 | 0.46 | 35 | 101 | 3.32 |
| Neets-7B | 0.46 | 0.47 | 82 | 100 | 1.67 |

0 comments on commit 58bdf22

Please sign in to comment.