Competitive push — measured
Built: 2026-07-22T12:21:35.589705+00:00 Law pin: D1D38A
Where we stand after this push
Product hybrid chat (no ref peeking)
| Metric | Value |
|---|---|
| sacreBLEU | 40.64 |
| BLEU-4 | 76.2 |
| BLEU-1 | 90.72 |
| chrF | 74.46 |
| Routes | densify=2800 neural=400 |
| Gap to chat mid bar 45 | 4.36 |
WMT14 de→en news
| System | sacreBLEU |
|---|---|
| opus-mt-de-en_b8 | 33.79 |
| nllb-600M_b8 | 33.49 |
| oracle_ensemble | 37.61 |
| Best | 37.61 (oracle_ensemble) |
| Gap to mid 40 | 2.39 |
| Gap to stretch 48 | 10.39 |
Accurate competitive read
| Arena | Status |
|---|---|
| A Catalog / classical / FSOT | Winning unique |
| B1 Chat open-set product hybrid | Climbing |
| B2 News mid-parity (≥40) | Short by 2.39 sacre |
| B2 News stretch SOTA (≥48) | Short by 10.39 sacre |
Left to beat commercial MT on news
- Finetune student on WMT train (largest expected gain toward +6)
- Optional larger NLLB / more pairs
- FLORES multi-pair public bar when unlocked
- Keep hybrid router as product default
See also: docs/COMPETITIVE_ROADMAP.md