added the benchmarking resuls on cuda

premAI-io · Dec 7, 2023 · 5552dfc · 5552dfc
1 parent 5f8dd03
commit 5552dfc
Showing 1 changed file with 1 addition and 0 deletions.
diff --git a/docs/llama2.md.template b/docs/llama2.md.template
@@ -17,6 +17,7 @@
 | tinygrad                     |      -       | 20.32 ± 0.06  |      -        |      -        |
 | onnx                         |      -       | 54.16 ± 3.15  |      -        |      -        |
 | transformers (pytorch)       | 0.44 ± 0.44  | 0.44 ± 0.44   |      -        |      -        |
+| exllamav2                    |      -        |      -       | 163.41 ± 5.58 | 120.17 ± 0.73 |
 
 *(Data updated: `<LAST_UPDATE>`)