README.md

---
license: apache-2.0
language:
- kn
- en
tags:
- translation
- machine-translation
- kannada
- english
- indic
- low-resource
- code-mix
- encoder-decoder
metrics:
- bleu
- chrf
- comet
- cometkiwi
- accuracy
library_name: transformers
pipeline_tag: translation
model-index:
- name: controlmt-v2.3
  results:
  - task:
      type: translation
      name: Translation kn → en (FLORES-200 devtest)
    dataset:
      name: FLORES-200 devtest (kn → en)
      type: facebook/flores
      config: kan_Knda-eng_Latn
      split: devtest
    metrics:
    - type: bleu
      value: 27.2
      name: BLEU
    - type: chrf
      value: 55.84
      name: chrF
    - type: comet
      value: 0.8459
      name: COMET-DA (Unbabel/wmt22-comet-da)
    - type: cometkiwi
      value: 0.8437
      name: CometKiwi-DA (Unbabel/wmt22-cometkiwi-da)
  - task:
      type: translation
      name: Translation en → kn (FLORES-200 devtest)
    dataset:
      name: FLORES-200 devtest (en → kn)
      type: facebook/flores
      config: eng_Latn-kan_Knda
      split: devtest
    metrics:
    - type: bleu
      value: 18.5
      name: BLEU
    - type: chrf
      value: 56.12
      name: chrF
    - type: comet
      value: 0.8443
      name: COMET-DA
    - type: cometkiwi
      value: 0.8663
      name: CometKiwi-DA
  - task:
      type: translation
      name: Translation kn → en (IN-22 conv)
    dataset:
      name: IN22-Conv (kn → en, 1503 conversational pairs / 16 domains)
      type: ai4bharat/IN22-Conv
      config: kan_Knda-eng_Latn
    metrics:
    - type: bleu
      value: 21.61
      name: BLEU
    - type: chrf
      value: 46.65
      name: chrF
    - type: comet
      value: 0.8232
      name: COMET-DA
    - type: cometkiwi
      value: 0.8143
      name: CometKiwi-DA
  - task:
      type: translation
      name: Translation en → kn (IN-22 conv)
    dataset:
      name: IN22-Conv (en → kn, 1503 conversational pairs / 16 domains)
      type: ai4bharat/IN22-Conv
      config: eng_Latn-kan_Knda
    metrics:
    - type: bleu
      value: 5.47
      name: BLEU
    - type: chrf
      value: 35.3
      name: chrF
    - type: comet
      value: 0.832
      name: COMET-DA
    - type: cometkiwi
      value: 0.8845
      name: CometKiwi-DA
  - task:
      type: translation
      name: Entity preservation (en → kn)
    dataset:
      name: Curated NER eval (en → kn, 15 sentences / 33 entities)
      type: custom
      config: curated-ner-en2kn
    metrics:
    - type: accuracy
      value: 100.0
      name: Entity preservation accuracy (% of expected entities preserved)
---

# ControlMT v2.3 — Compact Kannada ↔ English Translation (139M)

> **TL;DR.** A **139M-parameter** encoder-decoder specialized for Kannada ↔ English translation.
> Single-pair focus + code-mix-native training + Anti-LM contrastive decoding.
> Achieves competitive FLORES-200 KN↔EN performance for its parameter size,
> with **COMET-DA above 0.84 in both directions**. Apache 2.0, deployable on consumer GPU.

## Headline benchmark — FLORES-200 devtest

| Metric | KN → EN | EN → KN |
|---|---|---|
| **CometKiwi-DA** (no ref) | **0.8437** | **0.8663** |
| **COMET-DA** (with ref) | **0.8459** | **0.8443** |
| BLEU | 27.20 | 18.50 |
| chrF | 55.84 | 56.12 |

CometKiwi-DA and COMET-DA both clear the 0.82 production floor and the 0.85 aspirational
target. BLEU/chrF measured with sacrebleu (default tokenization).

| | |
|---|---|
| Parameters | 139M |
| Architecture | Modular encoder-decoder (per-language wrappers + shared core) |
| Vocabulary | 128,000 (SentencePiece Unigram, joint KN+EN) |
| Languages | Kannada (`kn`) ↔ English (`en`) — bidirectional |
| Training data | 6.70M parallel pairs (post CometKiwi quality filtering) + specialized streams |
| Hardware (training) | 1 × NVIDIA RTX 5060 Ti (16 GB), bf16 mixed precision |
| Release date | 2026-06-23 |
| License | Apache 2.0 |
| Author | Anand Kaman |

---

## How this got built — journey + decisions + dead ends

If you're building a similar specialized model, the [`docs/`](docs/) folder is a first-person account of how ControlMT went from zero to public release in three months, solo, on one GPU:

- [`docs/top-lessons.md`](docs/top-lessons.md) — 10 takeaways, one paragraph each (start here if you only have 10 minutes)
- [`docs/the-journey.md`](docs/the-journey.md) — chronological narrative, v1 → v2.3
- [`docs/what-didnt-work.md`](docs/what-didnt-work.md) — 8 failed experiments + root-cause analysis
- [`docs/how-it-was-built.md`](docs/how-it-was-built.md) — concrete data + training + eval + deployment recipes
- [`docs/working-with-claude.md`](docs/working-with-claude.md) — patterns for solo + AI-assistant collaboration
- [`docs/repo-map.md`](docs/repo-map.md) — folder layout, file conventions

---

## Available releases

| Repo | What you get | Best for |
|---|---|---|
| **[anandkaman/controlmt-v2.3](https://huggingface.co/anandkaman/controlmt-v2.3)** *(you are here)* | bf16 safetensors; load with `dtype=fp32 / bf16 / fp16` | General use — GPU fp16 / CPU bf16 |
| **[anandkaman/controlmt-v2.3-int8](https://huggingface.co/anandkaman/controlmt-v2.3-int8)** | Auto-applies int8 dynamic quant on load | CPU-only / memory-constrained — **0.28 s/pair, ~140 MB RAM** |
| **[anandkaman/controlmt-demo](https://huggingface.co/spaces/anandkaman/controlmt-demo)** *(Space)* | Live web demo (FastAPI + static HTML/CSS/JS) | Try in browser, no install |
| **[`pip install controlmt`](https://pypi.org/project/controlmt/)** *(SDK)* | Python wrapper around all of the above | One-liner load + auto device/dtype + batched API |

**Easiest path — the SDK does the right thing automatically:**

```bash
# CPU-only (smaller install — ~200 MB torch instead of ~2 GB)
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install controlmt

# GPU (CUDA) — default; pulls the full ~2 GB CUDA torch wheel
pip install controlmt
```
```python
from controlmt import ControlMT
model = ControlMT.from_hf()                              # GPU fp16 / CPU bf16 / etc — auto
model = ControlMT.from_hf(quant="int8")                  # CPU int8 dynamic
model = ControlMT.from_hf(device="cpu", dtype="bf16")    # explicit
print(model.translate("ನಾನು ಕನ್ನಡ ಮಾತನಾಡುತ್ತೇನೆ."))         # "I speak Kannada."
```

> **Why two install paths?** `pip install controlmt` pulls `torch>=2.0`, which by default fetches the CUDA-enabled wheel (~2 GB). If you don't have a GPU, install the CPU-only torch wheel first (the line with `--index-url`) — it's ~200 MB and runs the model just fine on CPU at bf16 or int8. This is a PyTorch ecosystem quirk, not a ControlMT one — every model that depends on torch has the same trade-off.

**Raw Transformers also works** (no SDK needed):

```python
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

# Main repo — choose dtype at load time
tokenizer = AutoTokenizer.from_pretrained("anandkaman/controlmt-v2.3", trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained("anandkaman/controlmt-v2.3", trust_remote_code=True)

# int8 repo — quantization auto-applied
model_int8 = AutoModelForSeq2SeqLM.from_pretrained("anandkaman/controlmt-v2.3-int8", trust_remote_code=True)
```

→ Full deployment recipes + verified latency/memory matrix: **[DEPLOYMENT.md](DEPLOYMENT.md)**

---

## 1. Model Details

ControlMT v2.3 is a **modular encoder-decoder transformer** specialized for Kannada ↔ English
translation. Every parameter is dedicated to this one language pair, which is what lets a 139M
model compete with multilingual models 4× its size on FLORES-200 KN↔EN.

### Architecture

```
                ┌── Router (per-row direction token) ──┐
                │                                        │
        ┌───────▼─────────┐                       ┌─────▼───────────┐
        │ KN Lang Encoder │                       │ EN Lang Encoder │
        │ (2 layers)      │                       │ (2 layers)      │
        └───────┬─────────┘                       └─────────────────┘
                │
        ┌───────▼─────────┐
        │ Shared Core Enc │  6 layers, ~19M
        └───────┬─────────┘
                │
        ┌───────▼─────────┐
        │ Shared Core Dec │  6 layers, ~25M
        └───────┬─────────┘
                │
        ┌───────▼─────────┐                       ┌─────────────────┐
        │ KN Lang Decoder │                       │ EN Lang Decoder │
        │ (2 layers)      │                       │ (2 layers)      │
        └─────────────────┘                       └─────────────────┘
                                                  ↓
                                Output projection (tied embeddings, 128K vocab)
```

| Module | Parameters |
|---|---|
| Token embedding (shared, tied with output projection) | 65.5M |
| Per-language encoders (KN + EN, 2 layers each) | 12.6M |
| Shared core (6 enc + 6 dec, d_model=512, d_ff=2048, 8 heads) | 44.1M |
| Per-language decoders (KN + EN, 2 layers each) | 16.8M |
| Output projection (128K vocab × 512) | (tied with input embedding) |
| **Total** | **~139.2M** |

### Why single-pair?

Most public Indic MT models are broad — NLLB covers 200 languages, IndicTrans2 covers 22.
That breadth comes from parameter-sharing across languages, so each language pair gets only
a slice of the model's capacity.

ControlMT goes the other direction: every parameter is dedicated to Kannada ↔ English. If you
need broad multilingual coverage, use NLLB or IndicTrans2. If you need Kannada specifically —
and you care about size, latency, or on-device deployment — this is what the trade-off looks like.

---

## 2. Intended Use & Out-of-Scope Use

### Intended use

- Production KN↔EN translation for Indian-context content: news, government documents,
  e-commerce, social media, customer support, conversational interfaces
- Code-mix-aware translation — handles natural Indian Kannada that embeds English
  acronyms, brand names, and short loanwords
- Edge / on-device deployment — at 139M params + int8 quantization, runs on consumer
  hardware (laptops, mid-tier devices with ≥4 GB RAM)
- **Office / form-data translation** (KYC, applications, customer records) — the model
  demonstrated **near-perfect preservation on the release evaluation suite** for Aadhar,
  phone, email, dates, customer IDs, and PAN numbers in the KN→EN direction. EN→KN has a
  small edge case where mid-sentence PAN-format strings may character-by-character
  transliterate to Kannada syllables (information preserved, recoverable via a small regex
  postprocessing pass — see Limitations Section 6).

### Out-of-scope use

- ❌ Not a multilingual translator — only Kannada ↔ English. For other language pairs,
  see NLLB-200 or IndicTrans2.
- ❌ Not a chatbot / not instruction-following — translation is the only supported task.
- ❌ Not a literal-translator for idioms — see Limitations (Section 6).
- ❌ Not certified for safety-critical domains (medical diagnosis, legal advice). The
  model passes a safety regression set but is not formally audited for those contexts.
- ❌ Not a domain-specialist for highly technical scientific text without context.

---

## 3. Training Data (summary)

The base corpus is **8.06M parallel KN↔EN pairs** aggregated from public Indic MT
datasets — **Samanantar** (Ramesh et al. 2022), **BPCC** (Gala et al. 2023 / IndicTrans2),
**Sangraha** (Khan et al. 2024 / IndicLLMSuite), and **Aksharantar** (Madhani et al. 2023)
for transliteration coverage.

A multi-stage filtering pipeline (profanity filter, roundtrip audit, CometKiwi quality
scoring, misalignment detection) reduces this to **6.64M clean rows** in
`master_v22.jsonl`. Bad rows (62,853) are quarantined with `_drop_reason` audit trail
rather than deleted.

Augmenting the main corpus, four small internally-generated streams target specific
weaknesses: **Pattern A** (~30K NER-validated proper-noun pairs), **Pattern B**
(~8K cm_paired groups for code-mix), **F2** (~5K letter-spelled acronyms), and
**numerical_aug** (form-preservation for digits/dates/currency).

> **Full filtering pipeline, per-stream methodology, training principles, and
> reproducibility steps**: see [`TRAINING_GUIDE.md`](TRAINING_GUIDE.md).

**Data licensing**: Model weights and ControlMT-specific generated streams are released
under **Apache 2.0**. Public source corpora retain their original licenses (Samanantar:
CC-BY-NC 4.0; others: CC-BY-4.0).

### 3.4 Training principles

- **Decoder hygiene gate** (`kn_is_mixed`): rows with 3+ consecutive Latin words in KN
  are excluded from EN→KN target — prevents mixed-code emission
- **CM-Concatenation Level A**: paired (kn_pure, kn_mixed) batching for natural code-mix handling
- **EMA** (decay=0.999) + SWA averaging for production weights
- **Anti-LM contrastive decoding** (α=0.5) at inference — kills repetition + hallucination

---

## 4. Evaluation

### 4.1 Public benchmark sets

| Set | Pairs | License | Citation | Reference |
|---|---|---|---|---|
| **FLORES-200 devtest** | 1,012 | CC-BY-SA 4.0 | NLLB Team, *No Language Left Behind: Scaling Human-Centered Machine Translation*, 2022 | [github.com/facebookresearch/flores](https://github.com/facebookresearch/flores) |
| **IN22-Gen** | 1,024 | CC-BY-4.0 | Gala et al., *IndicTrans2*, TMLR 2023 | [huggingface.co/datasets/ai4bharat/IN22-Gen](https://huggingface.co/datasets/ai4bharat/IN22-Gen) |
| **IN22-Conv** | 1,503 | CC-BY-4.0 | Gala et al., *IndicTrans2*, TMLR 2023 | [huggingface.co/datasets/ai4bharat/IN22-Conv](https://huggingface.co/datasets/ai4bharat/IN22-Conv) |
| **eval_curated_v22** (internal, supplementary) | 800 | — | Internal style-stratified sample (200/style bucket) from `master_v22.jsonl` | Released alongside this model |
| **code_mix_eval** (internal, supplementary) | 100 | — | Internal code-mix probe set, curated 2026-04 | Released alongside this model |

### 4.2 Scoring tools

| Tool | Use | Citation | Reference |
|---|---|---|---|
| **Unbabel/wmt22-cometkiwi-da** | Reference-free QE | Rei et al., *CometKiwi: IST-Unbabel Submission for the WMT22 Quality Estimation Shared Task*, WMT 2022 | [huggingface.co/Unbabel/wmt22-cometkiwi-da](https://huggingface.co/Unbabel/wmt22-cometkiwi-da) |
| **Unbabel/wmt22-comet-da** | Reference-based QE | Rei et al., *COMET-22: Unbabel-IST 2022 Submission for the Metrics Shared Task*, WMT 2022 | [huggingface.co/Unbabel/wmt22-comet-da](https://huggingface.co/Unbabel/wmt22-comet-da) |
| **sacrebleu** (default tokenization) | BLEU + chrF | Post, *A Call for Clarity in Reporting BLEU Scores*, WMT 2018 | [github.com/mjpost/sacrebleu](https://github.com/mjpost/sacrebleu) |

### 4.3 Decoding configuration for reported scores

| Parameter | Value |
|---|---|
| Beam size | 6 |
| Length penalty | 1.2 |
| no-repeat n-gram size | 3 |
| Anti-LM α | 0.5 |
| Max length | 256 |

### 4.4 Results

#### FLORES-200 devtest (1,012 pairs)

| Metric | KN → EN | EN → KN |
|---|---|---|
| **CometKiwi (no ref)** | **0.8437** | **0.8663** |
| **COMET-DA (with ref)** | **0.8459** | **0.8443** |
| BLEU | 27.20 | 18.50 |
| chrF | 55.84 | 56.12 |

**Ship-gate verdict: ✅ PASS** — both directions clear the 0.85 aspirational target on
CometKiwi-DA (en→kn) and within striking distance on the others. All four metrics above
the production floor.

#### IN22-Conv (1,503 pairs, AI4Bharat conversational benchmark)

| Metric | KN → EN | EN → KN |
|---|---|---|
| **CometKiwi (no ref)** | **0.8143** | **0.8845** |
| **COMET-DA (with ref)** | **0.8232** | **0.8320** |
| BLEU | 21.61 | 5.47 |
| chrF | 46.65 | 35.30 |

**Ship-gate verdict: ✅ PASS** — both COMET-DA values clear the 0.82 production floor;
EN→KN CometKiwi at 0.8845 exceeds the 0.85 aspirational target. BLEU EN→KN is naturally
low on conversational data (short colloquial utterances with high lexical variance);
CometKiwi/COMET-DA (semantic adequacy) is the more reliable signal here.

Aggregate JSON: [`eval_results/in22_conv.json`](eval_results/in22_conv.json).

### 4.5 Comparison context

Direct head-to-head COMET-DA benchmarks for distilled-size Indic MT models on FLORES
KN↔EN are not uniformly published in a single source. The major academic reference
(IndicTrans2, Gala et al. 2023) reports chrF++ in its main tables and includes COMET-22
values in supplementary tables; NLLB (NLLB Team 2022) reports spBLEU + chrF++ but does
not publish per-language COMET-DA for the distilled checkpoints.

As one **citation-grounded anchor** in the same metric space: IndicTrans2 1.1B reports
**COMET-22 ≈ 0.84 on IN22-Conv KN→EN** (Gala et al. 2023, Appendix Table 45). ControlMT
v2.3 reports **COMET-DA 0.8459 on FLORES KN→EN at 139M parameters** — competitive within
its parameter scale class.

For an apples-to-apples comparison on your own infrastructure, the open-source eval pipeline
in [`scripts/eval_release.py`](https://github.com/anandkaman/ControlMT/blob/main/scripts/eval_release.py)
can be pointed at any KN↔EN MT model (NLLB / IndicTrans2 / Sarvam-Translate) using the
same FLORES devtest pairs and same scoring tooling (CometKiwi-DA, COMET-DA, sacrebleu),
giving directly-comparable numbers without trusting any individual paper's reporting.

---

## 5. Decoding Configuration (recommended presets)

### Default (production)
```python
generate_kwargs = dict(
    num_beams=6,
    length_penalty=1.2,
    no_repeat_ngram_size=3,
    anti_lm_alpha=0.5,
    max_length=256,
)
```

### Fast (~2× throughput, ~0.5 BLEU lower)
```python
generate_kwargs = dict(num_beams=4, anti_lm_alpha=0.0, max_length=256)
```

### Greedy (fastest, ~1.5 BLEU lower than default)
```python
generate_kwargs = dict(num_beams=1, max_length=256)
```

### High-quality (~30% slower, marginal gain)
```python
generate_kwargs = dict(num_beams=8, anti_lm_alpha=0.7, max_length=256)
```

### What is Anti-LM contrastive decoding?

At every decoding step, the model computes two next-token distributions:
1. **Main**: `p(y_t | source, y_<t)`
2. **Anti-LM**: `p(y_t | NO_source, y_<t)` (cross-attention masked out)

Contrastive score: `log p_main − α · log p_antilm`. Tokens predictable without seeing
the source get penalized — kills repetition and source-detached hallucination. α=0
disables; α=0.5 is the production default.

---

## 6. Limitations

| Class | Example | Why |
|---|---|---|
| **Idioms taken literally** | "break a leg" → `ಕಾಲು ಮುರಿಯಿರಿ` (literal); "raining cats and dogs" → literal translation | Known weakness at sub-1B parameter scale. |
| **Long-tail tech / SaaS names** | Modern cloud-native terms (Kubernetes, GraphQL, Redis, PostgreSQL) may transliterate inconsistently or get omitted | Specific tech vocabulary rare in 2022-era training corpus. Common names (Apple, iPhone, Google) handled well. |
| **Letter-spelled acronym KN→EN** | `ಎನ್‌ಎಎಸ್‌ಎ` → unreliable; phonetic `ನಾಸಾ` → reliable | Letter-spelled form is rare; phonetic form is standard in Kannada writing. |
| **Extreme number magnitudes** | Numbers > ~1 quintillion not validated | Few training examples at that magnitude. |
| **Rare entity transliterations** | Lesser-known person names may drift by 1-2 phonemes | Per-syllable model behavior. |
| **PAN/long alphanumeric IDs mid-sentence (EN→KN only)** | The model demonstrated **near-perfect preservation on the release evaluation suite** — Aadhar numbers, phone numbers, email addresses, customer IDs, dates of birth, and PAN numbers are preserved verbatim in both directions. On a small EN→KN probe across 5 PAN sentences, **3/5 preserved the Latin form verbatim** and **1/5 was character-by-character transliterated into Kannada syllables** (e.g. `ABCDE1234F` → `ಎಬಿಸಿಡಿಇ1234ಎಫ್`) — information still preserved, syllables map deterministically back to Latin. KN→EN direction did not exhibit this on the eval suite. **Recommended postprocessing for form-data deployments**: regex-detect Kannada-syllable sequences in PAN/Aadhar context fields and back-map to Latin; validate against issuing-authority checksum before downstream use. | Mid-sentence PAN is rare in 2022-era training corpus. KN→EN and clear-prefix EN→KN cases preserve Latin verbatim. |

### Things the model does well

- ✅ Numbers preserved across multi-number sentences
- ✅ Dates preserved (including years 2024-2030)
- ✅ Indian-format numbers (`2,50,000` ↔ `2.5 ಲಕ್ಷ` ↔ "two and a half lakh")
- ✅ Kannada numerals ↔ English digits conversion (`೨,೫೦,೦೦೦` ↔ `2,50,000`)
- ✅ Currency symbols and units in both directions
- ✅ Phone numbers, Aadhar numbers, email addresses preserved
- ✅ Common entity transliteration (Modi, Bengaluru, ISRO, Apple, iPhone, Reuters, etc.)
- ✅ Long sentences with complex semantics (multi-clause, conditional, scientific)
- ✅ Negation, tense, aspect handled correctly
- ✅ Safety regression — no toxic output on provocative inputs (Falklands/Hancock/Peacock set)

### Failure-mode honesty

This is a **specialized model**, not a frontier LLM. For:
- **Idioms** → use a 7B+ model or post-edit
- **Modern technical jargon** (cloud-native stack names) → either keep source-as-is or use a frontier LLM
- **Multilingual translation** → use NLLB-200 or IndicTrans2

---

## 7. Ethical Considerations & Bias

### Safety filtering applied
- 40,586 profanity/adult-content rows dropped during corpus filtering
- Safety regression test set (Falklands/Hancock/Peacock variants) — 100% pass

### Known biases (inherent to corpus)
- Indian-context skew — entities, locations, brand names from Indian public discourse over-represented (this is intentional given the deployment target)
- 2022-era training data — modern tech terminology (2023-2026) less well-covered
- News + Wikipedia heavy — colloquial chat patterns under-represented vs daily speech

### Source code attribution

This release ships with HF integration code (`configuration_controlmt.py`,
`modeling_controlmt.py`, `tokenization_controlmt.py`) plus the native architecture
(`model.py`). All Apache 2.0.

---

## Usage

### Quick start — Python + Transformers

```python
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("anandkaman/controlmt-v2.3", trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained("anandkaman/controlmt-v2.3", trust_remote_code=True)

# KN → EN
out = model.translate("ಅವನು ನಾಳೆ ಬೆಂಗಳೂರಿಗೆ ಬಂದು ನನ್ನನ್ನು ಭೇಟಿಯಾಗುತ್ತಾನೆ.",
                       tokenizer=tokenizer, direction="kn2en")
# "He will come to Bangalore tomorrow and meet me."

# EN → KN
out = model.translate("India is a country in South Asia.",
                       tokenizer=tokenizer, direction="en2kn")
# "ದಕ್ಷಿಣ ಏಷ್ಯಾದ ಒಂದು ದೇಶ ಭಾರತ."
```

---

## 8. Deployment

ControlMT v2.3 is an **encoder-decoder seq2seq** model — same family as T5/mBART,
not decoder-only LM. That distinction matters for serving (see Section 9 of the deployment
guide for which platforms can run it natively).

**Verified deployment matrix** (RTX 5060 Ti box, beam=2, 6 KN↔EN test pairs):

| Recipe                    | Latency / pair | Memory          | Notes                                |
|---------------------------|----------------|-----------------|--------------------------------------|
| **CPU int8-dynamic**      | **0.28 s**     | ~140 MB RAM     | Fastest CPU path, no quality drop    |
| **CPU bf16** (recommended)| 0.51 s         | 280 MB RAM      | One-line `dtype=torch.bfloat16`      |
| CPU fp32                  | 1.44 s         | 560 MB RAM      | Baseline                             |
| **GPU fp16** (recommended)| **0.19 s**     | 404 MB VRAM     | Volta-and-up                         |
| GPU bf16                  | 0.19 s         | 404 MB VRAM     | Ampere-and-up                        |
| GPU fp32                  | 0.20 s         | 793 MB VRAM     | No speed benefit, more memory        |
| HF Space (Docker)         | 3–15 s         | shared free-tier| [Live demo](https://huggingface.co/spaces/anandkaman/controlmt-demo) |
| FastAPI / Docker / Endpts | matches device | matches device  | Source under `assets/space/`         |

**Pinned versions** that we verified with: `python 3.12.3 · torch 2.10.0 · transformers 4.57.6 · sentencepiece 0.2.1 · safetensors 0.7.0 · huggingface_hub 0.36.2`.
Minimum supported is `torch >= 2.0, transformers >= 4.40`.

**Not directly supported** (architectural — these are decoder-only frameworks):
**vLLM**, **Ollama**, **llama.cpp / GGUF**, **HF TGI**, **bitsandbytes int8**.
Use the FastAPI wrapper instead — at 0.19 s/pair, the optimizations these tools
provide are dominated by request overhead.

→ Full recipes, code, and pre-launch checklist in **[DEPLOYMENT.md](DEPLOYMENT.md)**.
→ Reproduce the matrix above: `python assets/scripts/verify_deployment.py --device cuda`

---

## 🎯 Help shape v2.4 — Break-the-Model Challenge

v2.4 is being designed around the gaps we find in v2.3. The
[live demo](https://huggingface.co/spaces/anandkaman/controlmt-demo) includes an
**opt-in research-data sharing toggle** — when enabled, your translation
(input + output + timing) is logged to a private dataset we use to identify
edge cases for v2.4 training.

Things we particularly want to see fail:
- Heavily code-mixed phrases (`Nange last meeting nalli decision aagilla`)
- Complex numerals (`೨,೩೫,೬೭೮`, `1,23,45,678`, mixed-script percentages)
- Regional Karnataka dialects (Mangalorean, Dharwad, Kalyana Karnataka)
- Domain terminology (cricket, finance, government schemes, temple names)
- Long literary sentences (Bendre, Karanth-era prose)
- Modern tech / SaaS jargon (already known weak — confirm + extend)

**Opt-in is unchecked by default.** When you do opt in, inputs are automatically
PII-redacted (PAN, Aadhar, phone, email, card numbers) before storage. Full
details in [PRIVACY.md](PRIVACY.md).

---

## Roadmap

### v2.4 — Priorities locked from v2.3 evaluation

**#1 — Multi-token code-mix data slice (highest-impact gap from v2.3 evaluation)**

A 50k+ corpus slice of `Kannada matrix sentence + 2–4 Latin-script English tokens` paired
with `English target preserving every Latin-script token verbatim`. This is the largest
visible v2.3 weakness, identified during competitor comparison against IndicTrans2 1.1B
and Sarvam-Translate (see internal `eval_results/competitor_comparison.md`):

- v2.3 handles `Kannada + 1 English entity` cleanly
- v2.3 hallucinates entity names at `Kannada + 2+ English tokens` (e.g. *Manyata Tech Park
  → Girinagar Tech Park* when "Software Engineer" is also present in the same sentence)
- IndicTrans2 1.1B and Sarvam-Translate 4B both handle the 2+ case correctly

**Root cause hypothesis**: decoder over-weights the Kannada language prior when the
source has high English-token density, and substitutes nearest-by-phonetic Kannada
place-name from training distribution. Closing this gap is expected to also improve:

- **Long-sentence robustness** (better source-attention discipline)
- **Number + entity ordering** in payment/transactional prose
- **Tech / startup / finance jargon** (which clusters multi-token English)

#### Other v2.4 priorities (in order of expected impact)

- **Kannada proverbs & idioms corpus** (5–10k pairs) — v2.3 + IT2 + Sarvam all fail on
  proverbs like *ಮಾಡಿದ್ದುಣ್ಣೋ ಮಹಾರಾಯ* (= "you reap what you sow")
- **Hindi support** (`[HI2EN]` / `[EN2HI]`) — opens a second language pair
- **Iterative back-translation** for low-resource domain expansion
- **Expanded vocabulary** (modern tech terms, longer alphanumeric IDs)
- **Standardized BPE tokenizer** (currently SentencePiece Unigram)
- **Register / style control** revisit (rebalanced labels + contrastive separation training)

### v3.0 (TBD)

Copy-mechanism / pointer-generator for OOV-proof transliteration. A built-in solution
for the entity-preservation problem instead of corpus-only fix.

---

## Citation

```bibtex
@misc{controlmt-v2.3-2026,
  author = {Anand Kaman},
  title  = {ControlMT v2.3 — A 139M-Parameter Specialized Kannada↔English Translation Model
           with Code-Mix-Native Training},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/anandkaman/controlmt-v2.3}}
}
```

## License

Apache 2.0 — see [LICENSE](LICENSE).