Update README.md
Browse files
README.md
CHANGED
|
@@ -155,9 +155,8 @@ Below are DeciCoder's pass@1 on MultiPL HumanEval scores
|
|
| 155 |
| HF Inference Endpoints | 1,364.2 | 9.03 | 3,244.4 | 8.8 |
|
| 156 |
| Infery LLM | 3,889.3 | 3.075 | 11,676.8 | 1.729 |
|
| 157 |
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
>- Throughput (tokens/sec) - Measured with optimal batchsize per hardware - A10 on BS 128, A100 on BS 512
|
| 161 |
|
| 162 |
## Documentation
|
| 163 |
|
|
|
|
| 155 |
| HF Inference Endpoints | 1,364.2 | 9.03 | 3,244.4 | 8.8 |
|
| 156 |
| Infery LLM | 3,889.3 | 3.075 | 11,676.8 | 1.729 |
|
| 157 |
|
| 158 |
+
- Latency - Total generation time of batch size 1 (prefill+generate)
|
| 159 |
+
- Throughput (tokens/sec) - Measured with optimal batchsize per hardware - A10 on BS 128, A100 on BS 512
|
|
|
|
| 160 |
|
| 161 |
## Documentation
|
| 162 |
|