---
license: mit
language:
- en
- km
library_name: onnx
pipeline_tag: translation
tags:
- translation
- khmer
- english
- onnx
- netra-lab
datasets:
- Netra-Lab/bilingual-en-km
---

<p align="center">
  <img src="https://huggingface.co/Netra-Lab/Netra-NMT/resolve/main/assets/header.png" width="260" alt="Netra Lab" />
</p>

<hr>

<p align="center">
  <a href="https://github.com/netra-ai-lab/Netra-NMT"><b>GitHub</b></a> |
  <a href="https://pypi.org/project/netra-nmt/"><b>PyPI</b></a> |
  <a href="https://huggingface.co/spaces/Darayut/Netra-NMT"><b>Demo</b></a> |
  <a href="https://huggingface.co/datasets/Netra-Lab/bilingual-en-km"><b>Training data</b></a> |
  <a href="https://huggingface.co/Netra-Lab/Netra-NMT/tree/main/pytorch"><b>PyTorch weights</b></a>
</p>

# Netra NMT

A **77M-parameter** English↔Khmer translation model: one encoder-decoder transformer for both directions, trained from scratch on 220 million tokens of parallel text. It runs on ONNX Runtime with the `netra-nmt` package, on a CPU, without PyTorch.

**Code:** [github.com/netra-ai-lab/Netra-NMT](https://github.com/netra-ai-lab/Netra-NMT), with the inference package and the full training recipe.

## Results

ALT test split (`mutiyama/alt`, 1,018 pairs), greedy decoding; spBLEU with the FLORES-200 tokenizer, higher is better.

| Model | Params | EN → KM | KM → EN |
|---|---:|---:|---:|
| NLLB-350M (EN-KM) | 350M | 18.89 | 0.26 |
| NLLB-200 distilled 600M | 600M | 9.24 | 31.28 |
| NLLB-200 distilled 1.3B | 1.3B | 16.62 | 35.15 |
| MADLAD-400 | 3B | 16.19 | 36.13 |
| Google Translate | — | 28.93 | 34.86 |
| **Netra NMT** | **77M** | **25.83** | **19.47** |

- **Where it wins:** English→Khmer, the best spBLEU after Google Translate and the best TER of all systems (60.47), and on CPU 65.6 ms per sentence greedy against 550.8 for NLLB-350M.
- **Where it loses:** Khmer→English, 15.7 spBLEU behind NLLB-200 1.3B.

<p align="center"><img src="https://huggingface.co/Netra-Lab/Netra-NMT/resolve/main/assets/metrics_benchmark.png" width="100%" alt="spBLEU, chrF++, TER, COMET and BERTScore on ALT"></p>

*spBLEU, chrF++, TER, COMET (wmt22-comet-da) and BERTScore F1 (xlm-roberta-base) on ALT, both directions.*

<p align="center"><img src="https://huggingface.co/Netra-Lab/Netra-NMT/resolve/main/assets/speed_benchmark.png" width="100%" alt="Speed against NLLB on CPU and GPU"></p>

*English→Khmer, median ms per sentence over 20 sentences, Intel i7-13700KF (16 threads) and RTX 3060, PyTorch.*

## Architecture

<p align="center"><img src="https://huggingface.co/Netra-Lab/Netra-NMT/resolve/main/assets/model_architecture.png" width="70%" alt="Encoder-decoder architecture"></p>

- **Shape:** 12 encoder layers, 2 decoder layers (deep encoder, shallow decoder), d_model 512, 8 heads, FFN 2048, vocabulary 32K.
- **Components:** rotary position embeddings, RMSNorm (pre-norm), SwiGLU, one embedding table shared by the encoder, the decoder and the tied output head, a KV cache.
- **Input:** a target-language marker (`<2km>` or `<2en>`) then the source. English sources are NFC-normalised and lowercased; Khmer is word-segmented with khmercut. The package does this for you.

## How it was made

1. **Data.** About 2.4M unique pairs from four sources (synthetic and ParaCrawl), deduplicated, length-filtered, then used in both directions: 4.2M examples.
2. **Tokenizer.** A 32K SentencePiece unigram model shared by both languages, trained on Khmer segmented with khmercut.
3. **Training.** From scratch, with label smoothing and a warm-up + cosine learning-rate schedule (settings in `training/netra_nmt_train/scripts/recipe/09_train_student.py`); the checkpoint with the best validation chrF is released.
4. **Export.** The encoder and a one-step decoder with a KV cache as ONNX graphs, weights stored as fp16. The exported folder generates the same tokens as the PyTorch model on the ALT test set (greedy and beam-5, both directions).

## Usage

```bash
pip install "netra-nmt[cpu]"        # or [gpu]
```

```python
from netra_nmt import translate

translate("Hello, how are you?")                         # English -> Khmer
translate("ខ្ញុំស្រលាញ់ប្រទេសកម្ពុជា។", direction="km2en")    # -> "I love Cambodia."
```

```bash
netra-nmt --text "Hello, how are you?" --mode beam
netra-nmt serve                       # web app + REST API (extra: server)
```

The package downloads `model/` from this repo once, at the commit it was tested with.

## Files

| File | Contents |
|---|---|
| `model/netra-model.json` | settings: graphs, decoder shape, special ids, language markers, sha256 of every file |
| `model/encoder.onnx` | tokens → cross-attention keys/values for every decoder layer (136 MB) |
| `model/decoder_step.onnx` | one decoding step with the self-attention KV cache (52 MB) |
| `model/spm_32k.model` | the SentencePiece tokenizer |
| `pytorch/model.safetensors` | fp16 weights for fine-tuning (`lm_head` tied to the embeddings, not stored) |
| `pytorch/config.json`, `pytorch/spm_32k.model` | architecture and tokenizer for the PyTorch model |
| `assets/` | the figures on this card |

To fine-tune from `pytorch/`: `bash training/finetune.sh my_pairs.tsv` in the GitHub repo.

## License and attribution

- **Weights:** MIT, trained from scratch (no pretrained weights).
- **Training data:** see the [dataset card](https://huggingface.co/datasets/Netra-Lab/bilingual-en-km) and its sources; check their terms before commercial use.
- **Evaluation data:** ALT (NICT), CC BY 4.0.