# RESULTS — EAGLE-3 draft for North-Mini-Code-1.0 (cohere2_moe)

## 1. Best trained draft (champion)
| metric | value |
|--------|-------|
| exp | **exp7_bigdata_ep10** |
| tau (Σ held-out eval acc, 7 positions) | **4.25** (acc0=0.71) |
| config | LlamaForCausalLMEagle3, 1 layer, hidden 2048, target_hidden 2048, FFN 12288, draft_vocab 32000, aux layers [1,23,45] |
| data | ~8.3k code-instruction samples (magicoder-evol-instruct), 10 epochs, lr 1e-4 |
| draft size | 366 MB (bf16) |

**Lever ladder findings (offline held-out tau):**
- baseline FFN8192/4ep = 3.82 → lr3e-4 = 2.00 (too high ✗) → FFN12288 ≈ +0 (noise) →
  10ep = 4.02 → 20ep = 4.10 (epochs saturate) → **+more data (8.3k) = 4.25 (winner)**.
- **More data beat more epochs.** Next lever = self-distillation on the model's own generations.

## 2. Real serving in vLLM (North-Mini-Code + our EAGLE draft)
Brought up on vLLM **main** (0.23.1rc1.dev, nightly) per Cohere guidance, on an A100-80 (driver 580/CUDA13),
with `cohere_melody` + `--tool-call-parser/--reasoning-parser cohere_command4`. Required two fixes:
- `--hf-overrides '{"first_k_dense_replace":1}'` — transformers 5.12 drops this field, so vLLM
  mis-built dense layer-0 as MoE (weight KeyError). Override restores it.
- **Patched `vllm/model_executor/models/cohere2_moe.py`** to implement the EAGLE3 interface
  (`SupportsEagle3`: capture aux hidden states from layers [1,23,45], `set/get_aux_hidden_state_layers`).
  vLLM's cohere2_moe shipped without it → "Model does not support EAGLE3 interface".

## 3. vibetest quality (14 qa scenarios, OpenAI endpoint)
| harness set | base (no spec) | + EAGLE |
|-------------|----------------|---------|
| GPQA-physics G1–G8 (PhD MCQ) | 4/8 | 3/8 |
| Security/exploit X1–X6 | 6/6 | 6/6 |
| **total** | **10/14** | **9/14** |

EAGLE-3 is distribution-lossless in theory; the ±1 difference is bf16 spec-decode greedy-path
divergence on borderline PhD-physics ties (off-domain for a code model), not a real regression.
Security set is identical 6/6.

## 4. EAGLE speed in vLLM (the honest result)
| | base | + EAGLE |
|--|------|---------|
| gen throughput | **~118 tok/s** | **~38 tok/s** |
| mean acceptance length | — | **1.28** |
| per-position accept | — | 0.22, 0.03, 0.001, 0, 0 |

**EAGLE was net SLOWER here.** With acceptance length only ~1.28, the draft+verify overhead
(5 spec tokens) is not repaid → ~3× slowdown. EAGLE only wins when acceptance >~2.

**Why acceptance is low (≠ training acc0 0.71):** train↔serve **hidden-state mismatch**. The draft
was trained OFFLINE on hidden states dumped from HuggingFace transformers; vLLM feeds it its own
internal aux hidden states (different fused add-norm / residual representation). Plus domain shift
(code-trained draft vs physics/security eval prompts). The offline metric (tau) is real for the
offline setup but does not transfer to vLLM's serving representation.

**Fix / next step:** ONLINE EAGLE-3 training in vLLM/SpecForge (Speculators), where the draft learns
on the exact serving-time hidden states — this is the officially recommended path and is what closes
the acceptance gap. Alternatively, align the offline aux-hidden-state capture to vLLM's representation.

## 5. Deliverable
- Champion draft (exp7, tau=4.25) ready to push: `alexdragannikolich/North-Mini-Code-1.0-EAGLE3` (needs HF token).
- Full harness reproducible: gen_hidden_states.py (offline), SpecForge train_eagle3 (offline),
  vLLM main serve + cohere2_moe EAGLE3 patch, vibetest qa.