Skip to content

Commit

Permalink
colons and default sorting
Browse files Browse the repository at this point in the history
changed arrows to colons. default grid sorting is now Total Time ASC
  • Loading branch information
benlower committed Apr 16, 2024
1 parent b9a2489 commit cc8dd01
Show file tree
Hide file tree
Showing 4 changed files with 21 additions and 18 deletions.
8 changes: 4 additions & 4 deletions website/src/components/BenchmarkDefinitions.astro
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ import { ModelDefinition, TTFTDefinition, TPSDefinition, TotalTimeDefinition } f
<div class="text-base">
<p id="definitions" class="text-xl font-bold mb-4">Definitions</p>
<ul class="list-none">
<li class="pb-2"><b>{ ModelDefinition.title } </b>{ ModelDefinition.definition }</li>
<li class="pb-2"><b>{ TTFTDefinition.title } </b>{ TTFTDefinition.definition }</li>
<li class="pb-2"><b>{ TPSDefinition.title } </b>{ TPSDefinition.definition }</li>
<li class="pb-2"><b>{ TotalTimeDefinition.title } </b>{ TotalTimeDefinition.definition }</li>
<li class="pb-2"><b>{ ModelDefinition.title }: </b>{ ModelDefinition.definition }</li>
<li class="pb-2"><b>{ TTFTDefinition.title }: </b>{ TTFTDefinition.definition }</li>
<li class="pb-2"><b>{ TPSDefinition.title }: </b>{ TPSDefinition.definition }</li>
<li class="pb-2"><b>{ TotalTimeDefinition.title }: </b>{ TotalTimeDefinition.definition }</li>
</ul>
</div>
</Section>
12 changes: 6 additions & 6 deletions website/src/components/Methodology.astro
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ import Section from './Section.astro';
<Section>
<div class="text-base">
<p id="methodology" class="text-xl font-bold mb-4">Methodology</p>
<p class="py-2"><b>Distributed Footprint </b>We run our tools daily in multiple data centers using <a class="underline hover:text-orange-600" href="https://fly.io/docs/reference/regions/" target="_blank">Fly.io</a>. Currently we run in cdg, iad, and sea.</p>
<p class="py-2"><b>Short Requests </b>Input requests are a short, single message (~20 tokens) and ask for a brief output response.</p>
<p class="py-2"><b>Max Output </b>Max output tokens is set to 100 to avoid distortion of TPS values from long outputs.</p>
<p class="py-2"><b>Warmup for Consistent Setup </b>A warmup connection is made to remove any connection setup latency.</p>
<p class="py-2"><b>TTFT Roundtrip </b>The TTFT clock starts when the HTTP request is made and stops when the first token result is received in the response stream.</p>
<p class="py-2"><b>Try 3, Keep 1 </b>For each provider, three separate inferences are done, and the best result is kept (to remove any outliers due to queuing etc).</p>
<p class="py-2"><b>Distributed Footprint: </b>We run our tools daily in multiple data centers using <a class="underline hover:text-orange-600" href="https://fly.io/docs/reference/regions/" target="_blank">Fly.io</a>. Currently we run in cdg, iad, and sea.</p>
<p class="py-2"><b>Short Requests: </b>Input requests are a short, single message (~20 tokens) and ask for a brief output response.</p>
<p class="py-2"><b>Max Output: </b>Max output tokens is set to 100 to avoid distortion of TPS values from long outputs.</p>
<p class="py-2"><b>Warmup for Consistent Setup: </b>A warmup connection is made to remove any connection setup latency.</p>
<p class="py-2"><b>TTFT Roundtrip: </b>The TTFT clock starts when the HTTP request is made and stops when the first token result is received in the response stream.</p>
<p class="py-2"><b>Try 3, Keep 1: </b>For each provider, three separate inferences are done, and the best result is kept (to remove any outliers due to queuing etc).</p>
</div>
</Section>

Expand Down
6 changes: 3 additions & 3 deletions website/src/components/Sources.astro
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ import Section from './Section.astro';
<Section>
<div class="text-base">
<p id="source" class="text-xl font-bold">Source</p>
<p class="py-2"><b>Raw Data </b>All data is in this public <a class="underline hover:text-orange-600" href="https://storage.googleapis.com/thefastest-data" target="_blank">GCS bucket.</a></p>
<p class="py-2"><b>Benchmarking Tools </b>The full test suite is available in the <a class="underline hover:text-orange-600" href="https://github.com/fixie-ai/ai-benchmarks" target="_blank">ai-benchmarks repo.</a></p>
<p class="py-2"><b>Website </b>Full source code for this site is on <a class="underline hover:text-orange-600" href="https://github.com/fixie-ai/fastest.ai" target="_blank">GitHub.</a></p>
<p class="py-2"><b>Raw Data: </b>All data is in this public <a class="underline hover:text-orange-600" href="https://storage.googleapis.com/thefastest-data" target="_blank">GCS bucket.</a></p>
<p class="py-2"><b>Benchmarking Tools: </b>The full test suite is available in the <a class="underline hover:text-orange-600" href="https://github.com/fixie-ai/ai-benchmarks" target="_blank">ai-benchmarks repo.</a></p>
<p class="py-2"><b>Website: </b>Full source code for this site is on <a class="underline hover:text-orange-600" href="https://github.com/fixie-ai/fastest.ai" target="_blank">GitHub.</a></p>
</div>
</Section>

Expand Down
13 changes: 8 additions & 5 deletions website/src/utils/DataGridDefinitions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,8 @@ const columnTTFT = {
headerName: TTFTDefinition.title,
headerTooltip: TTFTDefinition.definition,
headerClass: headerClass,
maxWidth: 90,
// minWidth: 80,
// maxWidth: 90,
valueFormatter: (p: ValueFormatterParam) => p.value.toFixed(2)
};

Expand All @@ -104,7 +105,8 @@ const columnTPS = {
headerName: TPSDefinition.title,
headerTooltip: TPSDefinition.definition,
headerClass: headerClass,
maxWidth: 90,
// minWidth: 90,
// maxWidth: 90,
valueFormatter: (p: ValueFormatterParam) => p.value.toFixed(2)
};

Expand All @@ -114,8 +116,8 @@ const columnNumTokens = {
headerName: TokensDefinition.title,
headerTooltip: TokensDefinition.definition,
headerClass: headerClass,
// minWidth: 100,
maxWidth: 100,
minWidth: 100,
// maxWidth: 100,
wrapHeaderText: true
};

Expand All @@ -126,9 +128,10 @@ const columnTotalTime = {
headerTooltip: TotalTimeDefinition.definition,
headerClass: headerClass,
// minWidth: 100,
maxWidth: 100,
// maxWidth: 100,
wrapHeaderText: true,
valueFormatter: (p: ValueFormatterParam) => p.value.toFixed(2),
sort: 'asc',
};

export const gridOptions = {
Expand Down

0 comments on commit cc8dd01

Please sign in to comment.