Paar · Udgam Labs

UDGAM LABS · Translator · English → Bengali · 17.3M parameters, 17.7 MB at int8

Paar English → Bengali

Paar English → Bengali translates English sentences into Bengali (Bengali script). It has 17.3M parameters: the int8 model file is 17.7 MB (19.8 MB with its vocabulary), and it runs offline on an ordinary CPU, about 31 ms a sentence on one core of a Mac mini, or in a web browser with transformers.js. It was distilled from IndicTrans2: it learned from English sentences that IndicTrans2 dist-200M translated. Against Firefox's offline English-to-Bengali model (17.1 MB, a similar size), it is behind by 0.93 chrF++ on FLORES+ (news-style text) and ahead by 1.32 on IN22-Conv (everyday conversation). Against indicMT, a model about 8.6 times larger to download, it is behind by 0.58 on FLORES+ and behind by 2.33 on IN22-Conv. All differences are paired, with 95% intervals (Results below).

It is one of twelve, English to: Hindi, Bengali, Marathi, Telugu, Tamil, Gujarati, Urdu, Kannada, Malayalam, Punjabi, Odia, Assamese.

Try it: in your browser · code and fine-tuning recipe on GitHub

Quick start

Transformers (the weights as trained):

# pip install transformers sentencepiece torch
from transformers import MarianMTModel, MarianTokenizer

repo = "udgamlabs/Paar-en-bn"
tok = MarianTokenizer.from_pretrained(repo)
model = MarianMTModel.from_pretrained(repo)

batch = tok(["Can you send me the invoice for the last three months?"], return_tensors="pt", padding=True, truncation=True, max_length=128)
out = model.generate(**batch, num_beams=1, max_new_tokens=200)
print(tok.batch_decode(out, skip_special_tokens=True)[0])
# আপনি কি আমাকে গত তিন মাস ধরে চালানটি পাঠাতে পারেন?

CTranslate2 int8 (the small, fast build in ct2-int8/):

# pip install ctranslate2 sentencepiece huggingface_hub
import ctranslate2, sentencepiece as spm
from huggingface_hub import snapshot_download

path = snapshot_download("udgamlabs/Paar-en-bn", allow_patterns=["config.json", "ct2-int8/*"])
sp = spm.SentencePieceProcessor(model_file=f"{path}/ct2-int8/source.spm")
tr = ctranslate2.Translator(f"{path}/ct2-int8", device="cpu", compute_type="int8")

def translate(sentences):
    pieces = [p[:127] + ["</s>"] for p in sp.encode(sentences, out_type=str)]
    out = tr.translate_batch(pieces, beam_size=1, max_decoding_length=200)
    return [sp.decode([t for t in r.hypotheses[0] if t != "</s>"]) for r in out]

print(translate(["Can you send me the invoice for the last three months?"])[0])

JavaScript, in the browser or Node, with transformers.js (the ONNX build in onnx/, int8):

// npm install @huggingface/transformers
import { pipeline } from "@huggingface/transformers";

const translate = await pipeline("translation", "udgamlabs/Paar-en-bn", { dtype: "q8" });
const [out] = await translate("Can you send me the invoice for the last three months?", { max_new_tokens: 200 });
console.log(out.translation_text);
// আপনি কি আমাকে গত তিন মাস ধরে চালান?
  • One sentence per input. Split paragraphs into sentences first; inputs are cut at 127 pieces.
  • Greedy decoding (num_beams=1) is what we scored. Beam search was not tested.
  • The CTranslate2 int8 build scores 0.06 chrF++ from the fp32 weights on FLORES+, and 75% of its translations are word for word the same. The ONNX int8 build (for transformers.js) is within 0.08 chrF++ of the fp32 weights on both test sets. In the browser, the int8 files are 28.8 MB, downloaded once.
English Paar English → Bengali (from the Transformers quick start)
The train leaves at 6:45 tomorrow morning, so please be at the station by 6:15. ট্রেনটি আগামীকাল সকাল 6টা 45 মিনিটে চলে যায়, তাই দয়া করে 6টা 15 মিনিটের মধ্যে স্টেশনে থাকে।
Can you send me the invoice for the last three months? আপনি কি আমাকে গত তিন মাস ধরে চালানটি পাঠাতে পারেন?
My phone battery dies by the afternoon even though I charge it overnight. আমি রাতারাতি চার্জ করার পরেও বিকেলের মধ্যে আমার ফোন ব্যাটারি মারা যায়।

These are unedited outputs, mistakes included, and no native speaker has reviewed them.

Results

chrF++ (higher is better), NFC on both sides, our harness, the same sentences for every system. Ours greedy; indicMT int8 with beam 5 as its card says; Firefox's offline model greedy, as Firefox runs it. Differences (ours minus theirs) are paired: 1,000 bootstrap resamples, 95% interval. Measured 3 to 16 November 2026. Our scores are of the fp32 weights; the int8 build is within 0.07 chrF++ of them, and the comparison with Firefox, re-run on the int8 build, gives the same result.

Test set (sentences) Paar (17.3M) indicMT (153.8M) IndicTrans2 dist-200M (211.8M) Firefox's offline model (17.1 MB)
FLORES+ devtest (1,012) 50.41 50.99 (-0.58 [-1.04, -0.12]) 53.11 51.34 (-0.93 [-1.41, -0.43])
IN22-Conv (1,503) 44.88 47.21 (-2.33 [-2.93, -1.76]) 48.90 43.57 (+1.32 [+0.72, +1.97])
  • IN22-Gen: 47.08. Numbers kept (sentences whose every number survives), FLORES+ / IN22-Conv: 91.8% / 95.2%.
  • Size: the int8 folder is 19.8 MB; the browser (ONNX int8) files are 28.8 MB. indicMT's int8 files are 169.2 MB.
  • Speed (Mac mini, FLORES+ devtest): about 31 ms a sentence on one CPU thread (CTranslate2 int8, median), about 227 sentences a second batched on four threads. In a browser (transformers.js, int8, one thread) about 40 ms a sentence in Chrome and Safari on the same Mac (measured for English → Gujarati). Phones are not measured.

Where it falls short

  • It is behind the bigger models: indicMT by 0.6 (FLORES+) and 2.3 (IN22-Conv) chrF++, IndicTrans2 dist-200M, its teacher, by 2.7 on FLORES+. If you can run a 150-200M model, those translate better.
  • Firefox's offline English-to-Bengali model (17.1 MB) is ahead of it by 0.9 (FLORES+) chrF++.
  • Some letters are missing from its vocabulary (ঈ, ঊ, ঋ and ঔ). They come out as "⁇" (<unk> in transformers.js), or are dropped by the Transformers decoder. 1.7% of FLORES+ outputs carry one.
  • Idioms and short phrases are often translated word for word, and small words such as "by" (a deadline) can take the wrong sense.
  • Names are often mangled, and Latin letters inside names often become "⁇".
  • Long formal sentences lose meaning more often than short everyday ones.
  • English to Bengali only. It does not translate into English.
  • No native-speaker review of its outputs yet.
  • Speed is measured on one Mac mini (its CPU, and Chrome and Safari for the browser build), not on phones.

How it was trained

A Marian encoder-decoder (6 encoder and 2 decoder layers, width 256, one shared 32,000-piece SentencePiece vocabulary), trained from random weights. Its training targets are IndicTrans2 dist-200M's translations (beam 4) of English sentences from IndicCorp v2 (CC0). The sentences were cleaned and deduplicated, about a third were rewritten in a spoken style, and every sentence overlapping FLORES+, NTREX or IN22 was removed before training. Scores are on the test sets, scored once after training.

Intended use and limitations

For translating short English text into Bengali where a large model is too heavy or the text should not leave the device: apps, web pages, offline tools, browser extensions, a first draft for a human translator. It is not meant for legal, medical or official text without a human check. Check names and numbers in the output.

Fine-tuning

The GitHub repository has a recipe (finetune/) for training it further on your own sentence pairs with Transformers. If you publish a fine-tuned version, set base_model: udgamlabs/Paar-en-bn in its card so it links back here.

Licence

Apache-2.0. See NOTICE: IndicTrans2 (AI4Bharat, MIT) wrote the training targets, and the English sentences come from IndicCorp v2 (AI4Bharat, CC0). No third-party weights are included.

Citation

@misc{udgamlabs2026paarbn,
  title  = {Paar: a 17 MB English-to-Bengali translator},
  author = {Prasad, Jitesh K},
  year   = {2026},
  howpublished = {Udgam Labs},
  url    = {https://huggingface.co/udgamlabs/Paar-en-bn}
}

Please also cite IndicTrans2 (Gala et al., 2023, TMLR; arXiv:2305.16307), whose translations it learned from.


About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. udgamlabs.com

Downloads last month
30
Safetensors
Model size
17.2M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train udgamlabs/Paar-en-bn

Space using udgamlabs/Paar-en-bn 1

Collection including udgamlabs/Paar-en-bn

Paper for udgamlabs/Paar-en-bn