docs/COMPETITIVE_POSITION.md

# Competitive position — where we stand vs Google / DeepL / NLLB

**Honest snapshot** (2026-07-21).  
**Law:** \(S=K(T_1+T_2+T_3)\) pin **D1D38A** (FSOT; competitors have no equivalent).

---

## Two different games

| Game | What “accuracy” means | Who owns the public narrative |
|------|----------------------|-------------------------------|
| **A. Form→gloss inventory** | Word/lemma → meaning under densify | **PFLT is very strong** |
| **B. Sentence fluency (MT)** | Full sentence → fluent EN (BLEU/chrF/COMET) | **Google / DeepL / NLLB** |

We are **competitive / leading on A**.  
We are **climbing toward B**, not at parity yet.

---

## Our numbers (measured in-repo)

### A — Form→gloss solidify (catalog)

| Metric | PFLT |
|--------|------|
| Languages | **113** |
| OPEN / PRODUCT form→gloss | **~99.99%** |
| Thin langs (n<50) | **0** |
| Breadth vs Google ~249 | **~45%** of lang count |
| Breadth vs NLLB ~200 | **~56%** |
| vs DeepL ~30–100 langs | **At/above DeepL scale on count** |

This bar is **not** what Google advertises — but it is real offline inventory depth (incl. classical/historical).

### B — Sentence fluency (dual student: densify + neural)

**B1 — Chat/Tatoeba domain** (16 langs; latest SOTA push 2026-07-21):

| Path | Metric | PFLT | Note |
|------|--------|-----:|------|
| Product densify + residual TM | BLEU-4 / B1 / U-F1 | **~83 / ~98 / ~92** | Product ceiling (includes chat residual templates) |
| **Neural open-set** (best of opus/NLLB/mul per lang, ≤200/lang) | mean sacreBLEU | **~50.2** | Fair chat bar — **past staged 45** |
| Hybrid oracle densify\|neural | mean sacreBLEU | **~53.6** | Upper bound for router |

Per-lang neural peaks: it ~68, es/pt ~61, de/ru ~57–58; thin: hi ~55; CJK gap: ja ~37, zh ~32; classical la densify still wins (~48 product).

**B2 — News domain (public WMT14 de→en test, n=3003)** — harder bar:

| Metric | PFLT densify-only | PFLT neural student (local) | DeepL-class band |
|--------|------------------:|----------------------------:|-----------------:|
| **sacreBLEU** | **~0.4** | **~33.9** (opus-mt-de-en beams=5; NLLB-600M ~33.4) | Often **~40–48+** |
| **chrF** | low | **~60** | Higher |
| Gap to staged mid bar (40) | — | **~6.1** | — |
| Gap to stretch (48) | — | **~14.1** | — |

Domain split: **chat neural is past mid bar** (~50 mean sacre); **news still ~6 BLEU short** of DeepL-class mid. Densify alone does not transfer to news — hybrid router is the product path.

\*Literature ranges, not same-day FLORES A/B. FLORES **data files still 403** (README ok; parquet gated — re-check Hub “Access granted”).

### Law / uniqueness (no competitor equivalent)

| Metric | PFLT |
|--------|------|
| Live pin D1D38A | **Yes** |
| Offline classical + visual path | **Yes** |
| Densify without rewriting law | **Yes** |

---

## Are we competitive at all?

| Track | Competitive? | One-line |
|-------|--------------|----------|
| Offline form→gloss multi-lang | **Yes — strong** | Near-ceiling on our catalog |
| Catalog breadth | **Mid** | 113 of ~200–249; beating DeepL *count* band |
| Classical / dead / hieroglyph | **Yes — unique** | Weak/absent in consumer MT |
| Chat sentence (neural open-set) | **Yes — mid/high open MT** | Mean sacre **~50** (Tatoeba-style) |
| News sentence (WMT de-en) | **Near mid open MT** | **~34** sacre; **~6** to DeepL mid bar |
| Top DeepL commercial SOTA | **Not claimed** | Need +6 to +14 sacre on news |
| Cloud UX / latency | N/A | Different product (local Ada) |
| FSOT law grounding | **Category of one** | — |

**Short answer:**  
Yes on **inventory, classical, offline FSOT**, and **chat neural open-set**.  
**Almost mid-parity** on news WMT (~34 vs ~40 staged); **not** full DeepL stretch (~48).  
Hybrid densify\|neural is the product route (oracle mean chat sacre **~54**).

---

## How far to go (gap to “parity”)

Define stages (honest):

| Stage | Domain bar | Where we are |
|-------|------------|--------------|
| **S0** Dict word map | <5 BLEU | Passed |
| **S1** Phrase densify | ~10–20 | Passed |
| **S2** Strong chat content | ~25–35 BLEU | **Passed** (neural chat mean sacre ~50) |
| **S3** Mid neural news / good EU | ~35–45 sacre WMT | **~34 HERE** — gap **~6** to 40 |
| **S4** Strong neural / DeepL-class | ~45–55+ sacre | Gap **~14** to 48 stretch |

### What must rise for S3–S4

| Lever | Why | Status |
|-------|-----|--------|
| Hybrid router densify\|neural | Product best-of path | **Oracle measured** (~54 chat mean); ship in Ada |
| Word order / T3 (esp. ja/zh) | BP kills densify BLEU despite B1 | **Route CJK to neural** (ja/zh already) |
| FLORES / WMT **same-file** eval | Comparable claim | FLORES data still gated; WMT14 **scored** |
| Neural student (NLLB/opus-mt) | Fluency generators | **Done multi-system**; optional 1.3B / finetune |
| Domain match (news vs chat) | DeepL trains news/web | Need more WMT densify or student finetune |

---

## FLORES status

- Account can read **README** of `facebook/flores`.  
- **Data parquet still returns 403** (gated) — accept may be pending, wrong dataset page, or needs re-login/approval.  
- Action: open https://huggingface.co/datasets/facebook/flores while logged in → confirm **Access granted** (not just license text).  
- Until then we report **Tatoeba product densify** + **WMT** as public bars, not FLORES.

---

## Bottom line

| Question | Answer |
|----------|--------|
| Competitive on form→gloss / offline catalog? | **Yes** |
| Competitive on chat open-set neural? | **Yes — mid/high open MT** (~50 mean sacre) |
| Competitive on Google/DeepL **news** fluency? | **Not parity; S3** (~34 WMT sacre; ~6 to mid bar) |
| How far on news sacreBLEU? | **~6 pts** to staged 40; **~14 pts** to stretch 48 |
| Unique advantage? | **FSOT law + classical + offline densify + hybrid student path** |

**Public storefront:** HF `dappalumbo91/pflt-fsot` · Kaggle `damianpalumbo/pflt-fsot-benchmarks` · GitHub tag/docs `RELEASE_v0.2.0`.

We do **not** claim DeepL parity. We **do** claim a real, measured climb under fixed FSOT law, with high content-level scores and mid full-sentence BLEU.