README: add full_acc@0 row + cond_acc@0 note (complete per-position table)
Browse files
README.md
CHANGED
|
@@ -37,17 +37,24 @@ replacement in the same vLLM serving path.
|
|
| 37 |
|
| 38 |
Per-position draft accuracy on a fixed held-out val split, measured during
|
| 39 |
training (teacher-forced argmax match; this is a *training-time* metric and is
|
| 40 |
-
not directly comparable to runtime accept-length).
|
| 41 |
-
|
|
|
|
|
|
|
|
|
|
| 42 |
|
| 43 |
| metric | init | final | rel. |
|
| 44 |
| --- | --- | --- | --- |
|
| 45 |
| val_loss | 7.69 | 5.62 | -27% |
|
|
|
|
| 46 |
| full_acc@1 | 0.253 | 0.367 | +45% |
|
| 47 |
| full_acc@2 | 0.085 | 0.184 | +116% |
|
| 48 |
| cond_acc@1 | 0.412 | 0.559 | +36% |
|
| 49 |
| cond_acc@2 | 0.336 | 0.499 | +49% |
|
| 50 |
|
|
|
|
|
|
|
|
|
|
| 51 |
A runtime `accept_length` benchmark (vLLM 0.20, `num_speculative_tokens=3`) is
|
| 52 |
pending and will be added once measured.
|
| 53 |
|
|
|
|
| 37 |
|
| 38 |
Per-position draft accuracy on a fixed held-out val split, measured during
|
| 39 |
training (teacher-forced argmax match; this is a *training-time* metric and is
|
| 40 |
+
not directly comparable to runtime accept-length). `acc@i` is the accuracy at
|
| 41 |
+
TTT position `i` (`i = 0, 1, 2`). `full_acc@i` requires positions `0..i` all
|
| 42 |
+
correct; `cond_acc@i` is conditioned on `0..i-1` being correct. Continual FT
|
| 43 |
+
lifts the weaker later positions most, while the already-strong position 0
|
| 44 |
+
holds:
|
| 45 |
|
| 46 |
| metric | init | final | rel. |
|
| 47 |
| --- | --- | --- | --- |
|
| 48 |
| val_loss | 7.69 | 5.62 | -27% |
|
| 49 |
+
| full_acc@0 | 0.614 | 0.656 | +7% |
|
| 50 |
| full_acc@1 | 0.253 | 0.367 | +45% |
|
| 51 |
| full_acc@2 | 0.085 | 0.184 | +116% |
|
| 52 |
| cond_acc@1 | 0.412 | 0.559 | +36% |
|
| 53 |
| cond_acc@2 | 0.336 | 0.499 | +49% |
|
| 54 |
|
| 55 |
+
(`cond_acc@0` equals `full_acc@0` by definition — position 0 has no prior to
|
| 56 |
+
condition on — so it is the same 0.614 -> 0.656.)
|
| 57 |
+
|
| 58 |
A runtime `accept_length` benchmark (vLLM 0.20, `num_speculative_tokens=3`) is
|
| 59 |
pending and will be added once measured.
|
| 60 |
|