Update
Browse files
README.md
CHANGED
|
@@ -50,9 +50,9 @@ We report our results on in-house dataset, representing LLM queries inside our c
|
|
| 50 |
|
| 51 |
* Using 2x H100 80GB HBM in tensor parallel regime.
|
| 52 |
* temperature equals 1.
|
| 53 |
-
* speculative num steps parameter equals
|
| 54 |
* speculative Eagle topk parameter equals 1.
|
| 55 |
-
* speculative num draft tokens parameter equals
|
| 56 |
|
| 57 |
| bs | tps w/o Eagle | tps w Eagle | Eagle acc len | Speedup |
|
| 58 |
|----|---------------|-------------|---------------|----------|
|
|
|
|
| 50 |
|
| 51 |
* Using 2x H100 80GB HBM in tensor parallel regime.
|
| 52 |
* temperature equals 1.
|
| 53 |
+
* speculative num steps parameter equals 3.
|
| 54 |
* speculative Eagle topk parameter equals 1.
|
| 55 |
+
* speculative num draft tokens parameter equals 4.
|
| 56 |
|
| 57 |
| bs | tps w/o Eagle | tps w Eagle | Eagle acc len | Speedup |
|
| 58 |
|----|---------------|-------------|---------------|----------|
|