--- license: apache-2.0 language: - hi - en pipeline_tag: translation tags: - transliteration - hindi - devanagari - romanized-hindi - on-device - browser - javascript - tiny - indic - input-method - keyboard - offline - aksharalp datasets: - ai4bharat/Aksharantar - ai4bharat/IndicCorpV2 metrics: - accuracy - cer model-index: - name: AksharaLP-1M-v2 results: - task: type: translation name: Romanized Hindi → Devanagari transliteration (word level) dataset: type: ai4bharat/Aksharantar name: Aksharantar (Hindi test) split: test metrics: - type: accuracy name: Top-1 accuracy value: 70.04 - type: accuracy name: Top-3 accuracy value: 88.31 - type: accuracy name: Top-8 accuracy value: 93.63 - task: type: translation name: Romanized Hindi → Devanagari transliteration (sentence level) dataset: type: google/dakshina name: Dakshina (Hindi romanized sentences) split: test metrics: - type: accuracy name: Word accuracy value: 89.02 - type: exact_match name: Sentence exact match value: 25.70 - type: cer name: CER value: 4.07 ---

# AksharaLP-1M-v2

The world’s 3rd most spoken language, typed on the world’s most used keyboard. In under 8 MB.

Hindi typed in English letters → Devanagari, fully offline.
Starts typing at 0.6 MB; the dictionary pack streams in after and is cached.

### Why it matters | 🌏 **#3** | 🗣️ **611M** | 📦 **< 8 MB** | |:---:|:---:|:---:| | most spoken language in the world | people speak Hindi (first or second language) | is all it takes to type it — no server, no GPU | _{Source: Ethnologue 200 (Ethnologue: Languages of the World, 29th edition, SIL Global), 2026 — https://www.ethnologue.com/insights/ethnologue200/ (accessed 2026-10-31). Counts first- and second-language speakers.} On a Latin (QWERTY) keyboard, Hindi is often typed in English letters: *“kal meeting hai, kaise ho?”*. AksharaLP-1M-v2 turns that into proper Devanagari — accurately, in context, and entirely offline in the browser. It transliterates (changes the script); it does not translate. AksharaLP-1M-v2 is the 989K-parameter member of the **AksharaLP** family of tiny transliteration models. For *“kal meeting hai, kaise ho?”* AksharaLP writes **कल मीटिंग है, कैसे हो?**, the way a Hindi speaker would. It ships in two tiers: a **0.57 MB core** (character Transformer with 989K parameters, reranker and runtime) that works on its own, and a **7.35 MB dictionary pack** (3M-word lexicon and sentence language model) that the runtime fetches on first use and caches. The complete package is **under 8 MB** to download (7.92 MB gzipped). The complete package is **2.8× smaller** than the model weights of IndicXlit, the most widely used open Indic transliteration model, and **24× smaller** than IndicXlit with its rescoring dictionary — while **beating IndicXlit’s model on top-1 accuracy** (70.04% vs 59.16% on Aksharantar) and on par with the full package (69.33%; paired gap +0.71 points, 95% CI -0.04 to 1.40).

_{Recorded from the live demo: inference runs in a Web Worker in the browser tab.}

## Versions **v2 (this repo) is the recommended release.** [v1](https://huggingface.co/sourojitdotcom/AksharaLP-1M) remains available ([v1 demo](https://huggingface.co/spaces/sourojitdotcom/AksharaLP-demo)). | | v1 | **v2 (this repo)** | |---|---|---| | Dictionary pack | 1.5M words | **3M words** | | Complete download (gzipped) | **4.53 MB** | 7.92 MB | | Aksharantar top-1 / top-3 / top-8 % | **70.37** / 87.66 / 92.79 | 70.04 / **88.31** / **93.63** | | Core-only top-1 % | 53.36 | **56.01** | | Dakshina word acc / sentence EM % | 87.55 / 21.66 | **89.02** / **25.70** | | Chat-spelling sentences, word acc % (480 held-out) | 80.16 | **91.54** | | Browser latency / word (p50) | 219.7 ms | **206.0 ms** | _{v1 numbers from the v1 model card. Aksharantar top-1 differs by 0.33 points with overlapping 95% CIs. Choose v1 only if the smaller download matters more than chat-spelling accuracy.} ## Best in class for its size

2.8× beats a model 2.8× its size _{+10.9 pts top-1 vs IndicXlit model only (46.0 MB) · paired 95% CI 9.9–11.8}	88.3% top-3 accuracy — higher than IndicXlit full package (85.4%) _{+2.9 pts [95% CI 2.3–3.5] at 24× smaller}	11.4 pts more accurate on casual chat Hindi than v1 _{whole sentences right 62.9% → 82.5% · 480 held-out chat sentences}
89.0% word accuracy on full Hindi sentences _{Dakshina, 4,976 test sentences · v1 87.5% (+1.5 pts, 95% CI 1.3–1.6)}	24× smaller than IndicXlit full package _{16.3 MB vs 404 MB on disk (fp32 weights + 358 MB rescoring dictionary)}	0 calls network calls — fully offline _{206 ms per word (p50) in the browser · nothing you type leaves the device}

All measured results (14 more claims that passed their rules)

| Result | What | Details | |---|---|---| | **88.3%** | top-3 accuracy — higher than IndicXlit model only (80.0%) | +8.4 pts [95% CI 7.7–9.1] at 2.8× smaller | | **93.6%** | top-8 accuracy — higher than IndicXlit model only (88.3%) | +5.3 pts [95% CI 4.8–5.9] at 2.8× smaller | | **93.6%** | top-8 accuracy — higher than IndicXlit full package (88.3%) | +5.3 pts [95% CI 4.8–5.9] at 24× smaller | | **2.65 pts** | core-only top-1 gain over v1 | 53.36% → 56.01% without the dictionary pack | | **2.8×** | smaller than IndicXlit model only | 16.3 MB vs 46.0 MB on disk (fp32 weights) | | **70.0%** | top-1 — higher than IndicXlit model only (59.2%) | at 2.8× smaller · Aksharantar Hindi test | | **70.0%** | top-1 — on par with IndicXlit full package (69.3%, overlapping 95% CIs) | at 24× smaller · Aksharantar Hindi test | | **0.65 pts** | top-3 gain over v1 | 87.66% → 88.31% · Aksharantar Hindi test | | **0.84 pts** | top-8 gain over v1 | 92.79% → 93.63% · Aksharantar Hindi test | | **4.3 pts/MB** | highest top-1 accuracy per MB among compared models | next: IndicXlit model only at 1.28 pts/MB | | **88.3%** | right word in the first 3 suggestions — vs 59.2% top-1 for IndicXlit model only | Aksharantar Hindi test | | **88.3%** | right word in the first 3 suggestions — vs 69.3% top-1 for IndicXlit full package | Aksharantar Hindi test | | **< 8 MB** | complete download | core + 3M-word dictionary pack · 989K parameters · 7.92 MB gzipped | | **0.57 MB** | to start typing | the core works on its own (56.0% top-1 without the pack); the dictionary pack streams in after and is cached |

**Accuracy per MB** (top-1 on Aksharantar Hindi test ÷ on-disk size; systems measured by us with the same harness): | # | System | On disk | Top-1 % | Top-1 pts per MB | |---|---|---|---|---| | 1 | **AksharaLP-1M-v2** | 16.3 MB | 70.04 | **4.30** | | 2 | IndicXlit model only | 46.0 MB | 59.16 | 1.28 | | 3 | IndicXlit full package | 404 MB | 69.33 | 0.17 | > 🏆 No compared model is both smaller and more accurate: AksharaLP-1M-v2 sits on the size–accuracy frontier. _{Every tile and every row above is generated by the release build from measured results and shown only if its rule holds (rules and outcomes are logged in the build report).} ## Real-world comparison: casual Hindi

Same input, typed casually: a phone keyboard screenshot next to AksharaLP v2's output

**Same input, typed casually** (measured 2026-11-01). | | Output for *“Boht ache h Bhai”* | |---|---| | A popular phone keyboard (author's screenshot) | **बोहट अ थे भाई** | | AksharaLP-1M-v2 (released runtime, offline, under 8 MB) | **बोहत अच्छे है भाई** | | Standard Hindi spelling (reference) | बहुत अच्छे है भाई | Words in standard spelling: the keyboard 1 of 4 (भाई); AksharaLP-1M-v2 3 of 4 (अच्छे, है, भाई). AksharaLP-1M-v2 writes बोहत (standard: बहुत). _{Screenshot supplied by the author, cropped to the input and output lines (keyboard name not shown; text unaltered). AksharaLP-1M-v2's output is from the released runtime on this exact input, measured 2026-11-01. This is one sentence, shown as a showcase only: it is not part of any test or tuning set and does not change the accuracy numbers above.} ## Highlights | | | |---|---| | 🪶 **Extremely small** | Under 8 MB complete: 0.57 MB core + 7.35 MB dictionary pack = 7.92 MB gzipped (16.3 MB on disk); 989K parameters, 4-bit weights. Fits in a keyboard app, a browser extension or a web page. | | 🎯 **Accurate** | With the dictionary pack: 70.04% top-1 / 88.31% top-3 on the Aksharantar Hindi test set (9,991 words); 89.02% word accuracy on Dakshina sentences. Core-only numbers are in [Variants](#variants-core-only-vs-core--dictionary). | | 🧠 **Context-aware** | A compact bigram language model picks the right word from its neighbours: the same *main* becomes मैं or में. | | 📥 **Starts fast, degrades gracefully** | The core is enough to start typing; the dictionary pack streams in after and is cached. If the pack cannot be fetched, the runtime keeps working with the core alone. | | 🔒 **Private & offline** | Pure JavaScript, zero dependencies, no server. Nothing you type leaves the device; works offline after first load. | | ⚡ **Fast** | 206.0 ms per word (p50) in the browser on an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux). | ## What it brings on-device Everything below happens on the user’s own phone or laptop: the model is a set of static files plus a dependency-free JavaScript runtime. Each example output was produced by the released runtime when this card was built.

### ✈️ Fully offline No network after the first load, so it keeps working in airplane mode, in a lift, or on a train. `flight mein hoon, land karke call karta hoon` → फ्लाइट में हूं, लैंड करके कॉल करता हूं	### 🔒 Private by design Text never leaves the device: no API call, no logging server. Suited to banking, health and personal chat. `mera account balance kitna hai` → मेरा अकाउंट बैलेंस कितना है `doctor ne din mein do baar dawai lene ko kaha` → डॉक्टर ने दिन में दो बार दवाई लेने को कहा	### 🌐 Runs wherever JavaScript runs A browser tab, an Android or iOS WebView, an installable PWA, or Node.js. CPU only: no GPU, no server, no inference bill. `namaste, aapka swagat hai` → नमस्ते, आपका स्वागत है
### ⚡ Real-time, as you type The sentence is re-decoded on every keystroke in a Web Worker, and earlier words are corrected as context arrives: `main` → मैं `main theek` → मैं ठीक `main theek hoon` → मैं ठीक हूं _{Browser latency: 206.0 ms per word (p50) on an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux).}	### ⌨️ Keyboard & input-box integration Wire it to any `` or ``: chat, search, forms, CRM notes, customer-support replies. One call per keystroke: `transliterate(text, {k})`. `aapka refund teen din mein aa jayega`<br/>→ आपका रिफंड तीन दिन में आ जाएगा </td> <td width="33%" valign="top"> ### 🏷️ Brands your way English words inside Hindi sentences are transliterated by the same model. Two writing styles for the same input: Default → कल व्हाट्सएप पे मीटिंग है<br/> Strict (brands stay in Latin) → कल whatsapp पे मीटिंग है </td> </tr> <tr> <td width="33%" valign="top"> ### ☰ Top-k suggestions Ranked candidates for every word, ready for a suggestion bar or candidate picker. Picking one re-decodes the rest of the sentence. `main` → <b>मैं</b> · में · मैन · मेन · माइन </td> <td width="33%" valign="top"> ### 📶 Built for Bharat A 0.57 MB core to start, under 8 MB in total, downloaded once and cached; after that, patchy or missing connectivity does not matter. Pure JavaScript on the CPU, with no GPU or native code, designed with entry-level Android phones in mind. `signal nahi aa raha, baad mein baat karte hain`<br/>→ सिग्नल नहीं आ रहा, बाद में बात करते हैं </td> <td width="33%" valign="top"> ### ₹0 per call at scale Inference runs on your users’ devices, so there are no per-request API fees and no GPU servers to scale. Serving is static files from any CDN. Apache-2.0: free for commercial use. `mera order kab tak deliver hoga`<br/>→ मेरा ऑर्डर कब तक डिलिवर होगा </td> </tr> </table> ## What it does — real outputs Every output below was produced by the released runtime (`aksharalp.mjs`) and the files in this repository when this card was built. \| Feature \| You type \| AksharaLP writes \| \|---\|---\|---\| \| Everyday Hindi \| `kaise ho` \| कैसे हो \| \| Everyday Hindi \| `mujhe bhookh lagi hai` \| मुझे भूख लगी है \| \| Context — main = “I” \| `main theek hoon` \| मैं ठीक हूं \| \| Context — main = “in” \| `ghar main kaun hai` \| घर में कौन है \| \| English words in Hindi \| `kal office mein meeting hai` \| कल ऑफिस में मीटिंग है \| \| English words in Hindi \| `whatsapp pe message karo` \| व्हाट्सएप पे मैसेज करो \| \| Acronyms \| `sbi` \| एसबीआई \| \| Acronyms \| `SBI ka ATM kahan hai` \| एसबीआई का एटीएम कहां है \| \| English words \| `please mujhe call karna` \| प्लीज मुझे कॉल करना \| \| Kept as-is \| `@rahul 5 baje milte hain, link: www.example.com` \| @rahul 5 बजे मिलते हैं, लिंक: www.example.com \| \| Chat spelling \| `Ache ho na?` \| अच्छे हो ना? \| \| Chat spelling \| `nhi yr, bht kaam h` \| नहीं यार, बहुत काम है \| \| Chat spelling \| `main thk hu, tm kaise ho` \| मैं ठीक हूं, तुम कैसे हो \| \| Chat spelling \| `kya kr rhe ho` \| क्या कर रहे हो \| \| Chat spelling \| `pls jldi aao` \| प्लीज़ जल्दी आओ \| \| Chat spelling \| `bahuuut accha laga` \| बहुत अच्छा लगा \| \| Chat spelling \| `mujhy pta nai` \| मुझे पता नहीं \| \| Chat spelling \| `kb aaoge` \| कब आओगे \| \| Chat spelling \| `bht acha h` \| बहुत अच्छा है \| \| Chat spelling \| `mai thk hu` \| मैं ठीक हूं \| \| Chat spelling \| `kr rha hu` \| कर रहा हूं \| ### Two writing styles The same sentence in each mode: \| Mode \| `kal whatsapp pe meeting ka link bhej dena` \| \|---\|---\| \| Default \| कल व्हाट्सएप पे मीटिंग का लिंक भेज देना \| \| Strict \| कल whatsapp पे मीटिंग का लिंक भेज देना \| * Default — every word, English ones included, is written in Devanagari. * Strict — brand and product names stay in Latin script. ### Top-k suggestions Every word comes with ranked alternatives, ready for a keyboard suggestion bar: \| Input \| 1 \| 2 \| 3 \| 4 \| 5 \| \|---\|---\|---\|---\|---\|---\| \| `kaise` \| कैसे \| केसे \| कइसे \| कैस \| कईसे \| \| `main` \| मैं \| में \| मैन \| मेन \| माइन \| \| `dhanyavaad` \| धन्यवाद \| धन्यावाद \| धन्यवाद् \| धान्यवाद \| धनयवाद \| ### Core only vs. core + dictionary pack The same inputs through the core alone (pack not loaded) and with the pack, from the released runtime: \| You type \| Core only \| Core + dictionary pack \| \|---\|---\|---\| \| `main theek hoon` \| मैं थीक हूं \| मैं ठीक हूं \| \| `ghar main kaun hai` \| घर मैं कौं है \| घर में कौन है \| \| `mujhe bhookh lagi hai` \| मुझे भूख लगी है \| मुझे भूख लगी है \| \| `whatsapp pe message karo` \| वॉट्सएप पीई मेसेज करो \| व्हाट्सएप पे मैसेज करो \| \| `doctor ne dawai lene ko kaha` \| डॉक्टर ने दवाई लेने को कहा \| डॉक्टर ने दवाई लेने को कहा \| ## Chat-style spelling Hand-written chat-style Hindi sentences (WhatsApp-style spellings such as h, nhi, bht, thk, dropped vowels and stretched letters), held-out half of an internal dev set (never used for fitting). Word accuracy % per spelling pattern, v1 vs v2, same scorer. \| Spelling pattern \| Sentences \| Before (v1) word % \| After word % \| Before sentence % \| After sentence % \| \|---\|---\|---\|---\|---\|---\| \| Single letter for a doubled consonant \| 26 \| 79.35 \| 92.39 \| 38.46 \| 73.08 \| \| Extra doubled letters \| 25 \| 95.16 \| 98.39 \| 88.0 \| 96.0 \| \| Missing h in kh / th / bh \| 26 \| 54.24 \| 71.19 \| 23.08 \| 42.31 \| \| Extra h \| 25 \| 76.36 \| 85.45 \| 56.0 \| 72.0 \| \| Long / short vowel spelling \| 26 \| 85.71 \| 96.43 \| 73.08 \| 92.31 \| \| e for ai \| 25 \| 89.29 \| 96.43 \| 76.0 \| 92.0 \| \| o for au \| 25 \| 70.45 \| 84.09 \| 56.0 \| 72.0 \| \| Dropped or added a \| 25 \| 76.92 \| 97.44 \| 68.0 \| 96.0 \| \| Dropped vowels \| 25 \| 62.5 \| 90.0 \| 48.0 \| 84.0 \| \| Missing nasal (n) \| 25 \| 70.0 \| 86.0 \| 52.0 \| 72.0 \| \| w / v, z / j, f / ph \| 26 \| 97.78 \| 100.0 \| 96.15 \| 100.0 \| \| s for sh \| 25 \| 72.73 \| 86.36 \| 52.0 \| 76.0 \| \| ri / ru for ऋ \| 26 \| 96.43 \| 100.0 \| 96.15 \| 100.0 \| \| Word-final y / h endings \| 25 \| 55.32 \| 82.98 \| 20.0 \| 68.0 \| \| Stretched letters \| 25 \| 75.0 \| 82.14 \| 72.0 \| 80.0 \| \| Chat abbreviations (h, nhi, yr) \| 25 \| 45.83 \| 91.67 \| 12.0 \| 84.0 \| \| English words in Hindi chat \| 25 \| 96.77 \| 98.39 \| 92.0 \| 96.0 \| \| Ambiguous short words \| 25 \| 94.87 \| 93.59 \| 84.0 \| 80.0 \| \| Standard spelling (control) \| 25 \| 97.89 \| 97.89 \| 92.0 \| 92.0 \| \| All chat-spelling sentences \| 480 \| 80.16 \| 91.54 \| 62.92 \| 82.5 \| <sub>480 sentences. Overall word accuracy 80.16% → 91.54% (paired bootstrap gain +11.38 points, 95% CI 9.14 to 13.71); whole-sentence exact match 62.92% → 82.50%. Internal set written by the author; small per-pattern counts, so per-row numbers are indicative.</sub> ## Small by design <p align="center"><img src="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/assets/chart_size.png" alt="Size comparison" width="92%"/></p> What you download: \| File \| What it is \| On disk \| gzipped \| \|---\|---\|---\|---\| \| `xlit.q4` \| core: transliteration Transformer, 4-bit weights \| 589 KB \| 544 KB \| \| `params.json` \| core: reranker weights + decoding settings \| 15 KB \| 6 KB \| \| `aksharalp.mjs` \| core: JavaScript runtime (zero dependencies) \| 62 KB \| 20 KB \| \| Core total \| works on its own \| 0.67 MB \| 0.57 MB \| \| `dict/lex.bin` \| dictionary pack: Hindi lexicon (trie + unigram scores) \| 11,675 KB \| 6,473 KB \| \| `dict/lm.bin` \| dictionary pack: word-bigram language model, 4-bit \| 3,913 KB \| 869 KB \| \| `dict/params.json` \| dictionary pack: reranker weights + decoding settings used with the pack \| 28 KB \| 12 KB \| \| Dictionary pack total \| fetched on first use, then cached \| 15.6 MB \| 7.35 MB \| \| Complete package \| core + pack \| 16.3 MB \| 7.92 MB \| For comparison, IndicXlit’s fp32 model weights are 46.0 MB and its official rescoring dictionary adds another 358 MB. General-purpose LLMs that can transliterate are larger still, with hundreds of millions of parameters or more, and typically need a server or a GPU. ## GGUF at a glance The model's weights are also published as GGUF files, for anyone who wants them as one self-describing container rather than our custom `.q4` format. \| File \| Size \| What it holds \| Accuracy vs. the shipped model \| \|---\|---\|---\|---\| \| `aksharalp-1m-v2-q4.gguf` \| 593 KB (546 KB gzipped) \| the 4-bit `xlit.q4` weights, carried byte-identically \| 400/400 words identical — same top-1, same candidate sets, same rankings \| \| `aksharalp-1m-v2-f16.gguf` \| 1.99 MB (1.33 MB gzipped) \| the same weights dequantised to half precision, plus the reranker matrices \| 399/400 words identical top-1; the one miss (`pratinidheeyo`) slips out of the 16-candidate beam once scores move by ~0.27 \| The q4 file is the one to use if you want the published numbers exactly. At 593 KB it is far smaller than the 7.92 MB complete package because it carries only the network's weights — the 3M-word dictionary pack is what makes up the rest. These files do not load in llama.cpp, Ollama or LM Studio, and they are not a runnable transliterator on their own; see [GGUF files](#gguf-files) below for the reader and exactly what does run. ## Benchmarks <p align="center"><img src="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/assets/chart_accuracy.png" alt="Accuracy comparison" width="92%"/></p> \| System \| Download \| Aksharantar top-1 % \| top-3 % \| top-8 % \| Dakshina word acc % \| Dakshina sentence EM % \| Latency / word (p50) \| \|---\|---\|---\|---\|---\|---\|---\|---\| \| AksharaLP-1M-v2 (core + dictionary pack) \| 7.92 MB \| 70.04 <sub>[69.16, 70.90]</sub> \| 88.31 \| 93.63 \| 89.02 \| 25.70 \| 206.00 ms <sub>browser</sub> \| \| IndicXlit model only \| 46.0 MB <sub>fp32 weights</sub> \| 59.16 <sub>[58.14, 60.13]</sub> \| 79.95 \| 88.29 \| 67.55 \| 6.29 \| 48.4 ms <sub>PyTorch CPU</sub> \| \| IndicXlit full package \| 404 MB <sub>fp32 weights + 358 MB rescoring dictionary</sub> \| 69.33 <sub>[68.44, 70.23]</sub> \| 85.40 \| 88.29 \| — \| — \| — \| <p align="center"><img src="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/assets/chart_tradeoff.png" alt="Accuracy versus size" width="80%"/></p> ## Variants: core only vs. core + dictionary The dictionary pack is optional. Each row below was scored once on the Aksharantar Hindi test set (9,991 words, beam 8); settings and the size of the dictionary, the reranker fits, the sentence weights and the core were chosen on validation and internal dev data only. The test-words-removed column shows the same system with every test reference word deleted from the dictionary. \| Variant \| Shipped \| Download (gz) \| On disk \| Test top-1 % <sub>[95% CI]</sub> \| top-3 % \| top-8 % \| Val top-1 % <sub>(2-fold CV)</sub> \| Test top-1 % with test words removed from the dictionary \| Dakshina sentences word acc % \| \|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\| \| Core only \| yes \| 0.57 MB \| 0.67 MB \| 56.01 <sub>[55.06, 56.96]</sub> \| 76.91 \| 85.09 \| 55.79 \| — \| 70.23 \| \| Core + 3M-word dictionary pack \| yes \| 7.92 MB \| 16.3 MB \| 70.04 <sub>[69.16, 70.90]</sub> \| 88.31 \| 93.63 \| 69.69 \| 16.37 \| 89.02 \| <sub>The “test words removed” column re-runs the same system with every Aksharantar test reference word deleted from the dictionary, so the dictionary cannot simply look up the answer; it shows how much of the gain comes from dictionary coverage (see the caveat below). Dakshina sentences: 4,976 romanized sentences.</sub> How we measured. All systems were run by us on the same machine with the same scoring script (Unicode NFC, punctuation and danda normalised). Word-level: Aksharantar Hindi test set, 9,991 unique romanized words, beam 8 for every system; top-k = the reference appears among the first k candidates. Sentence-level: Dakshina romanized Hindi sentences (4,976 sentences); word accuracy, sentence exact match, CER and WER. IndicXlit “model only” is the released transformer checkpoint; “full package” adds rescoring with the official 4.2M-word dictionary (α = 0.9; our re-implementation of the released engine, which reproduces the published scores). Browser latency is per word with the dictionary pack loaded, single tab, measured in headless Chrome on an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux); IndicXlit latency is PyTorch fp32 on one CPU thread and is shown for reference only (different runtime). > Caveat. IndicXlit was trained on Aksharantar, so its Aksharantar test scores may benefit from train/test overlap. Our training pairs contain no Aksharantar test or validation native words (checked). The 3M-word dictionary pack is built from IndicCorp v2 text and naturally contains most test words: with every test reference word removed from the dictionary the same system scores 16.37% top-1 (42.51% top-3, 70.33% top-8), and the core alone scores 56.01% top-1. All tuning, the reranker fits, the dictionary size and the sentence weights used validation and internal dev data only; each test number comes from a single run. ## Quickstart ### Browser (ES module) ```html <script type="module"> import { AksharaLP } from "https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/aksharalp.mjs"; const base = "https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/"; // core first (0.57 MB): usable immediately const aklp = await AksharaLP.load(base, { dict: false }); // then the dictionary pack (7.35 MB): fetched once, kept in the Cache API; false = stays core-only const ok = await aklp.loadDict(); const r = aklp.transliterate("kal meeting hai, kaise ho?"); console.log(r.output); // कल मीटिंग है, कैसे हो? (with the pack) console.log(r.tokens[0].candidates); // ranked alternatives for the first word </script> ``` `AksharaLP.load(base)` without options loads the core and the pack together. For production, copy the files (including `dict/`) into your app and load them from your own origin (they are static and cache well); run the model in a Web Worker as the demo Space does. ### Node.js (≥ 18) ```bash hf download sourojitdotcom/AksharaLP-1M-v2 --local-dir aksharalp ``` ```js import { loadFromDir } from "./aksharalp/aksharalp.mjs"; const aklp = await loadFromDir("./aksharalp/"); // core + dict/ pack; { dict: false } for core only console.log(aklp.transliterate("main theek hoon", { mode: "conversational" }).output); // मैं ठीक हूं console.log(aklp.word("kaise", 5)); // top-5 candidates with scores ``` ### Python (reference implementation) ```python from huggingface_hub import snapshot_download import sys path = snapshot_download("sourojitdotcom/AksharaLP-1M-v2") sys.path.insert(0, f"{path}/python") from bhc.translit2 import BhcTranslit2 # needs numpy + torch (CPU) xl = BhcTranslit2(path) # use_dict=False for core only print(xl.convert("kal meeting hai, kaise ho?")) print(xl.translit_words(["kaise", "namaste"], k=5)) ``` The Python package is the reference used for training and evaluation; the JavaScript runtime reproduces it exactly (JS/Python parity test — core only: identical candidate sets and top-1 for 221/221 words, max feature difference 1.4e-05; with the dictionary pack: identical candidate sets and top-1 for 221/221 words, max feature difference 2.8e-05). ### API at a glance \| Call \| Returns \| \|---\|---\| \| `AksharaLP.load(baseUrl, {dict, cache})` \| engine (fetches `manifest.json` and the core; with `dict: true`, the default, also the dictionary pack) \| \| `loadDict({cache})` \| fetches the dictionary pack (Cache API name `"aksharalp-dict"` by default) and attaches it; returns `false` and keeps the core working if it fails \| \| `attachDict({lex, lm, params})` \| attaches a pack you fetched yourself (e.g. from IndexedDB or app assets) \| \| `transliterate(text, {mode, context, k, overrides})` \| `{output, tokens[], ms}`; each token has `text, output, source, candidates[]`; `mode` is `"conversational"` (default) or `"strict"` \| \| `word(romanWord, k)` \| top-k `{text, score}` for one word \| \| `info()` \| parameter count, lexicon size, bytes on disk / gzipped, `hasDict`, `dictError` \| `overrides` pins a user-chosen suggestion (`{tokenStartOffset: "मैं"}`) and re-decodes the rest of the sentence around it. ## How it works <p align="center"><img src="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/assets/architecture.png" alt="Architecture" width="100%"/></p> ```mermaid flowchart LR A["Romanized Hindi text"] --> T["① Tokenizer<br/>URLs, @mentions, numbers kept"] T -->\|Strict mode: brand\| P["kept in Latin"] T -->\|word\| X["② Char Transformer (core)<br/>d=128, 3+2 layers, 4 heads, FFN 384 · 4-bit · 989K"] X --> C["Candidates<br/>free beam + acronyms<br/>(+ lexicon-trie beam + skeleton lookups with the pack)"] C --> R["Learned reranker<br/>30-feature MLP"] R --> V["③ Sentence Viterbi (pack)<br/>bigram LM · 3M words"] P --> V V --> O["Devanagari + top-k"] ``` 1. Tokenizer. Splits words, numbers and punctuation; URLs, e-mail addresses and @mentions are kept as typed. 2. Transliteration (core). A character-level encoder–decoder Transformer (d=128, 3+2 layers, 4 heads, FFN 384) proposes spellings with a free beam; letter-name acronyms are added and scored by the same model. With the dictionary pack, a second beam constrained to the 3M-word Hindi lexicon trie and consonant-skeleton lookups add in-vocabulary candidates. A small learned reranker orders the candidates: with the pack, 30 features → 32 hidden units → 1, fitted on validation data plus synthetic and internal chat-style spellings (so h, nhi, bht map to है, नहीं, बहुत); the core alone has its own 30 → 16 → 1 reranker fitted on the validation set. 3. Sentence context (pack). A 4-bit quantized word-bigram language model (300K bigrams) runs Viterbi over the candidate lattice to choose the most fluent sentence. Core-only mode picks each word independently. English words inside Hindi sentences go through the same model (Default) or, for brand names, stay in Latin script (Strict). ### Files \| File \| Contents \| \|---\|---\| \| `xlit.q4` \| core: transliteration Transformer, 4-bit weights \| \| `params.json` \| core: reranker weights + decoding settings \| \| `dict/lex.bin` \| dictionary pack: Hindi lexicon (trie + unigram scores) \| \| `dict/lm.bin` \| dictionary pack: word-bigram language model, 4-bit \| \| `dict/params.json` \| dictionary pack: reranker weights + decoding settings used with the pack \| \| `manifest.json` \| file list, sizes, checksums (core + pack) \| \| `aksharalp.mjs` \| core: JavaScript runtime (zero dependencies) \| \| `aksharalp.min.mjs` \| same runtime, minified (attribution header preserved) \| \| `python/` \| reference Python implementation (numpy + PyTorch) \| \| `aksharalp-1m-v2-q4.gguf` \| GGUF container: the `xlit.q4` weights as one blob, for `gguf_read.py` \| \| `aksharalp-1m-v2-f16.gguf` \| GGUF container: the same weights as plain F16 tensors plus the reranker, for `gguf_read.py` \| \| `gguf_read.py` \| dependency-light GGUF reader (numpy only); decodes either file to the reference weight tensors \| \| `LICENSE`, `NOTICE` \| Apache-2.0 and third-party attributions \| ### GGUF files These GGUF files do not run in llama.cpp, Ollama, LM Studio or any other llama.cpp-based tool. AksharaLP-1M-v2 is not a LLaMA-style model: it is a custom character-level encoder-decoder Transformer (d=128, 4 heads, 3 encoder + 2 decoder layers, FFN 384, 988,928 parameters) wrapped in a bespoke decoding stack - beam search over the model, a lexicon-trie candidate generator, 30 hand-built candidate features, gates, and a linear/MLP reranker. llama.cpp has no architecture for that, so it cannot load these files, and nothing here is a llama.cpp checkpoint. What GGUF is used for here: a portable single-file tensor container with typed metadata. The weights, their configuration and the decoding settings travel as one self-describing file, readable by the small pure-Python reader shipped next to it (`gguf_read.py`, numpy only - no torch, no llama.cpp, no gguf-py), with no dependency on the custom `.q4` codec. It carries weights and config, not a runnable transliterator: the lexicon, feature pipeline, gates and reranker are code, so use the JavaScript runtime or the Python reference implementation above to actually transliterate. \| File \| Contents \| Raw bytes \| Gzipped \| SHA-256 \| \|---\|---\|---\|---\|---\| \| `aksharalp-1m-v2-q4.gguf` \| the `xlit.q4` file carried byte-identically as one I8 tensor (`aksharalp.q4.blob`), plus metadata describing the 4-bit layout \| 593,184 \| 545,820 \| `458fd07d…8efebc0` \| \| `aksharalp-1m-v2-f16.gguf` \| every weight under its reference name at F16, plus the core and pack reranker matrices \| 1,989,504 \| 1,332,735 \| `9cb7ab3a…55822ec4` \| \| `gguf_read.py` \| reads either file: magic and version check, typed metadata, named tensors, and the weight object the reference implementation expects \| 14,828 \| 4,808 \| `213255f3…eb1eab4755` \| Both are GGUF v3, little-endian, tensor data 32-byte aligned. The metadata carries the model name and version, the parameter count, the layer configuration, the quantisation description, the full candidate-feature list, the beam settings, the licence, and `aksharalp.requires_bundled_decoder = true` with `aksharalp.runs_in_llama_cpp = false`. ```python import gguf_read # the reader shipped in this repo w = gguf_read.load("aksharalp-1m-v2-f16.gguf") # numpy only w.cfg # {'d': 128, 'h': 4, 'enc': 3, 'dec': 2, 'ff': 384, 'p': 0.1} (p stored as float32) w.state_dict["semb.weight"].shape # (30, 128), float32 w.n_params # 988928 import torch, sys # into the reference implementation sys.path.insert(0, "python") from bhc.xlit_model import XlitTransformer m = XlitTransformer(w.cfg) m.load_state_dict({k: torch.from_numpy(v) for k, v in w.state_dict.items()}) m.eval() ``` `gguf_read.py <file>` prints a structural summary; `--json` dumps metadata and the tensor table. `gguf_read.q4_blob(path)` returns the carried `xlit.q4` bytes from the compact file: 588,891 bytes, byte-identical to the original (sha256 `652eafe0c1e2b46fec85663825621284f0a5911f2d7cc6e13b1182ce47cd9803`). Verification.** Both files were run back through the reference decoder on a fixed sample of 400 words (the first 400 distinct alphabetic romanised words of the Aksharantar Hindi test split), with the dictionary pack, comparing against the shipped `xlit.q4`: * `aksharalp-1m-v2-q4.gguf`: identical top-1, identical candidate sets and identical score-ranked candidate lists for 400/400 words. The blob is carried unmodified, so the weights are bit-identical to `xlit.q4`. * `aksharalp-1m-v2-f16.gguf`: identical top-1 for 399/400 words, identical candidate sets for 394/400 and identical ranked lists for 377/400. Every difference comes from rounding the dequantised float32 weights to F16 (max absolute weight difference 9.2e-04): candidate scores move by up to 0.27, which reorders near-tied candidates and occasionally drops one out of the 16-item beam. The one top-1 change is pratinidheeyo -> प्रतिनिधियो instead of प्रतिनिधियों: under F16, प्रतिनिधियों falls just outside the 16 surviving beam hypotheses (the beam's 16th score is -8.5693 against -8.5691 for q4), so the reranker never sees it and the runner-up wins. An F32 control built the same way from the same dequantised weights reproduced 400/400 top-1, candidate sets and ranked lists, which isolates the difference to F16 rounding rather than the container. For output identical to the published numbers, use `xlit.q4` or the q4 GGUF. Structurally both pass: magic, version 3, tensor count against the header, 32-byte alignment, offsets in bounds, a fresh-process re-parse, and an independent read with the upstream `gguf` package (1 and 102 tensors, shapes round-trip). ## Intended use * Hindi input methods: mobile and desktop keyboards, browser extensions, chat and messaging apps, search boxes. * Converting romanized Hindi text (chat logs, social posts, transcripts) to Devanagari for downstream NLP. * Privacy-sensitive or offline settings where text must not leave the device. ## Limitations * Hindi only. Other Indic languages and scripts are not supported. * Names and rare words. Proper nouns, rare or very long words, and unusual spellings are the main source of errors. Check the top-k list. * Ambiguity. Some romanizations have several valid Devanagari spellings (e.g. हूं / हूँ). The model follows the most common written form, which may differ from a reference. * Short context. The language model sees one word on each side (bigram), so long-range agreement is not modelled. * Core only is less accurate. Without the dictionary pack, top-1 accuracy drops substantially (see [Variants](#variants-core-only-vs-core--dictionary)). Load the pack when you can. * Dictionary coverage. Much of the accuracy with the pack comes from the lexicon: with every test reference word removed from it, top-1 falls well below the core-only score. Words outside the lexicon (new names, slang, rare spellings) are harder. * English words and language tags. This release has no word-level language identifier and no loanword lexicon: English words are transliterated phonetically by the same model and can look unusual, and tokens are not tagged Hindi / English. * Short ambiguous words. Without a loanword lexicon, short words that are both English and Hindi can go the wrong way in context (for example, in our tests call sometimes came out as कल and do as डॉ). Check the top-k list. * Chat-speak. Common chat spellings (h, nhi, bht, thk, kr, rhe) are handled well with the dictionary pack, but rarer abbreviations and some pronoun/verb pairs still go wrong (for example mai ghr pe hu gives मैं घर पे हुआ instead of मैं घर पे हूं). The core alone handles chat spellings much less well. * Speed depends on the device. Very old phones may see higher per-word latency than reported here. ## License & attribution Commercial use is allowed. Code and model weights are released under the Apache License 2.0 (see `LICENSE`), Copyright 2026 Sourojit Dhua. You may use, modify and ship AksharaLP-1M-v2 in commercial products, free of charge. Attribution is required when you redistribute it. Apache-2.0 §4(d) requires every redistribution of the model or a derivative (shipping the files in an app, web page, browser extension or SDK; publishing a fine-tuned, quantized or converted version) to include the `NOTICE` file, and the JavaScript runtime keeps its `/! … /` attribution header, including the minified `aksharalp.min.mjs`. Please use this attribution text: ```text Transliteration by AksharaLP (Sourojit Dhua) — https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2 ``` Recommended in-app credit (About, Settings or Licenses screen): ```html Hindi typing powered by <a href="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2">AksharaLP by Sourojit Dhua</a> ``` Third-party data used to build the model is credited in `NOTICE`: \| Source \| License \| Used for \| \|---\|---\|---\| \| [Aksharantar](https://huggingface.co/datasets/ai4bharat/Aksharantar), AI4Bharat \| CC0 (mined data) / CC-BY (manually collected data) \| romanized ↔ Devanagari training pairs \| \| [IndicXlit](https://github.com/AI4Bharat/IndicXlit), AI4Bharat \| MIT \| teacher model for distillation \| \| [IndicCorp v2](https://github.com/AI4Bharat/IndicBERT/tree/main#indiccorp-v2), AI4Bharat \| CC0 \| Hindi lexicon and bigram language model \| \| [CMUdict](https://github.com/cmusphinx/cmudict), Carnegie Mellon University \| BSD-2-Clause-style \| English word cues \| Fonts in the demo Space (Inter, Noto Sans Devanagari) are under the SIL Open Font License 1.1. ## Citation ```bibtex @software{dhua_aksharalp_2026, author = {Dhua, Sourojit}, title = {{AksharaLP}: Tiny, Context-Aware Hindi Transliteration for the Browser}, year = {2026}, version = {2.0.0}, url = {https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2}, license = {Apache-2.0} } ``` <sub>AksharaLP · v2.0.0 · built 2026-11-01 · Questions and feedback: open a discussion on this repository.</sub>

### ✈️ Fully offline No network after the first load, so it keeps working in airplane mode, in a lift, or on a train. `flight mein hoon, land karke call karta hoon` → फ्लाइट में हूं, लैंड करके कॉल करता हूं	### 🔒 Private by design Text never leaves the device: no API call, no logging server. Suited to banking, health and personal chat. `mera account balance kitna hai` → मेरा अकाउंट बैलेंस कितना है `doctor ne din mein do baar dawai lene ko kaha` → डॉक्टर ने दिन में दो बार दवाई लेने को कहा	### 🌐 Runs wherever JavaScript runs A browser tab, an Android or iOS WebView, an installable PWA, or Node.js. CPU only: no GPU, no server, no inference bill. `namaste, aapka swagat hai` → नमस्ते, आपका स्वागत है
### ⚡ Real-time, as you type The sentence is re-decoded on every keystroke in a Web Worker, and earlier words are corrected as context arrives: `main` → मैं `main theek` → मैं ठीक `main theek hoon` → मैं ठीक हूं _{Browser latency: 206.0 ms per word (p50) on an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux).}	### ⌨️ Keyboard & input-box integration Wire it to any `` or ``: chat, search, forms, CRM notes, customer-support replies. One call per keystroke: `transliterate(text, {k})`. `aapka refund teen din mein aa jayega`<br/>→ आपका रिफंड तीन दिन में आ जाएगा </td> <td width="33%" valign="top"> ### 🏷️ Brands your way English words inside Hindi sentences are transliterated by the same model. Two writing styles for the same input: Default → कल व्हाट्सएप पे मीटिंग है<br/> Strict (brands stay in Latin) → कल whatsapp पे मीटिंग है </td> </tr> <tr> <td width="33%" valign="top"> ### ☰ Top-k suggestions Ranked candidates for every word, ready for a suggestion bar or candidate picker. Picking one re-decodes the rest of the sentence. `main` → <b>मैं</b> · में · मैन · मेन · माइन </td> <td width="33%" valign="top"> ### 📶 Built for Bharat A 0.57 MB core to start, under 8 MB in total, downloaded once and cached; after that, patchy or missing connectivity does not matter. Pure JavaScript on the CPU, with no GPU or native code, designed with entry-level Android phones in mind. `signal nahi aa raha, baad mein baat karte hain`<br/>→ सिग्नल नहीं आ रहा, बाद में बात करते हैं </td> <td width="33%" valign="top"> ### ₹0 per call at scale Inference runs on your users’ devices, so there are no per-request API fees and no GPU servers to scale. Serving is static files from any CDN. Apache-2.0: free for commercial use. `mera order kab tak deliver hoga`<br/>→ मेरा ऑर्डर कब तक डिलिवर होगा </td> </tr> </table> ## What it does — real outputs Every output below was produced by the released runtime (`aksharalp.mjs`) and the files in this repository when this card was built. \| Feature \| You type \| AksharaLP writes \| \|---\|---\|---\| \| Everyday Hindi \| `kaise ho` \| कैसे हो \| \| Everyday Hindi \| `mujhe bhookh lagi hai` \| मुझे भूख लगी है \| \| Context — main = “I” \| `main theek hoon` \| मैं ठीक हूं \| \| Context — main = “in” \| `ghar main kaun hai` \| घर में कौन है \| \| English words in Hindi \| `kal office mein meeting hai` \| कल ऑफिस में मीटिंग है \| \| English words in Hindi \| `whatsapp pe message karo` \| व्हाट्सएप पे मैसेज करो \| \| Acronyms \| `sbi` \| एसबीआई \| \| Acronyms \| `SBI ka ATM kahan hai` \| एसबीआई का एटीएम कहां है \| \| English words \| `please mujhe call karna` \| प्लीज मुझे कॉल करना \| \| Kept as-is \| `@rahul 5 baje milte hain, link: www.example.com` \| @rahul 5 बजे मिलते हैं, लिंक: www.example.com \| \| Chat spelling \| `Ache ho na?` \| अच्छे हो ना? \| \| Chat spelling \| `nhi yr, bht kaam h` \| नहीं यार, बहुत काम है \| \| Chat spelling \| `main thk hu, tm kaise ho` \| मैं ठीक हूं, तुम कैसे हो \| \| Chat spelling \| `kya kr rhe ho` \| क्या कर रहे हो \| \| Chat spelling \| `pls jldi aao` \| प्लीज़ जल्दी आओ \| \| Chat spelling \| `bahuuut accha laga` \| बहुत अच्छा लगा \| \| Chat spelling \| `mujhy pta nai` \| मुझे पता नहीं \| \| Chat spelling \| `kb aaoge` \| कब आओगे \| \| Chat spelling \| `bht acha h` \| बहुत अच्छा है \| \| Chat spelling \| `mai thk hu` \| मैं ठीक हूं \| \| Chat spelling \| `kr rha hu` \| कर रहा हूं \| ### Two writing styles The same sentence in each mode: \| Mode \| `kal whatsapp pe meeting ka link bhej dena` \| \|---\|---\| \| Default \| कल व्हाट्सएप पे मीटिंग का लिंक भेज देना \| \| Strict \| कल whatsapp पे मीटिंग का लिंक भेज देना \| * Default — every word, English ones included, is written in Devanagari. * Strict — brand and product names stay in Latin script. ### Top-k suggestions Every word comes with ranked alternatives, ready for a keyboard suggestion bar: \| Input \| 1 \| 2 \| 3 \| 4 \| 5 \| \|---\|---\|---\|---\|---\|---\| \| `kaise` \| कैसे \| केसे \| कइसे \| कैस \| कईसे \| \| `main` \| मैं \| में \| मैन \| मेन \| माइन \| \| `dhanyavaad` \| धन्यवाद \| धन्यावाद \| धन्यवाद् \| धान्यवाद \| धनयवाद \| ### Core only vs. core + dictionary pack The same inputs through the core alone (pack not loaded) and with the pack, from the released runtime: \| You type \| Core only \| Core + dictionary pack \| \|---\|---\|---\| \| `main theek hoon` \| मैं थीक हूं \| मैं ठीक हूं \| \| `ghar main kaun hai` \| घर मैं कौं है \| घर में कौन है \| \| `mujhe bhookh lagi hai` \| मुझे भूख लगी है \| मुझे भूख लगी है \| \| `whatsapp pe message karo` \| वॉट्सएप पीई मेसेज करो \| व्हाट्सएप पे मैसेज करो \| \| `doctor ne dawai lene ko kaha` \| डॉक्टर ने दवाई लेने को कहा \| डॉक्टर ने दवाई लेने को कहा \| ## Chat-style spelling Hand-written chat-style Hindi sentences (WhatsApp-style spellings such as h, nhi, bht, thk, dropped vowels and stretched letters), held-out half of an internal dev set (never used for fitting). Word accuracy % per spelling pattern, v1 vs v2, same scorer. \| Spelling pattern \| Sentences \| Before (v1) word % \| After word % \| Before sentence % \| After sentence % \| \|---\|---\|---\|---\|---\|---\| \| Single letter for a doubled consonant \| 26 \| 79.35 \| 92.39 \| 38.46 \| 73.08 \| \| Extra doubled letters \| 25 \| 95.16 \| 98.39 \| 88.0 \| 96.0 \| \| Missing h in kh / th / bh \| 26 \| 54.24 \| 71.19 \| 23.08 \| 42.31 \| \| Extra h \| 25 \| 76.36 \| 85.45 \| 56.0 \| 72.0 \| \| Long / short vowel spelling \| 26 \| 85.71 \| 96.43 \| 73.08 \| 92.31 \| \| e for ai \| 25 \| 89.29 \| 96.43 \| 76.0 \| 92.0 \| \| o for au \| 25 \| 70.45 \| 84.09 \| 56.0 \| 72.0 \| \| Dropped or added a \| 25 \| 76.92 \| 97.44 \| 68.0 \| 96.0 \| \| Dropped vowels \| 25 \| 62.5 \| 90.0 \| 48.0 \| 84.0 \| \| Missing nasal (n) \| 25 \| 70.0 \| 86.0 \| 52.0 \| 72.0 \| \| w / v, z / j, f / ph \| 26 \| 97.78 \| 100.0 \| 96.15 \| 100.0 \| \| s for sh \| 25 \| 72.73 \| 86.36 \| 52.0 \| 76.0 \| \| ri / ru for ऋ \| 26 \| 96.43 \| 100.0 \| 96.15 \| 100.0 \| \| Word-final y / h endings \| 25 \| 55.32 \| 82.98 \| 20.0 \| 68.0 \| \| Stretched letters \| 25 \| 75.0 \| 82.14 \| 72.0 \| 80.0 \| \| Chat abbreviations (h, nhi, yr) \| 25 \| 45.83 \| 91.67 \| 12.0 \| 84.0 \| \| English words in Hindi chat \| 25 \| 96.77 \| 98.39 \| 92.0 \| 96.0 \| \| Ambiguous short words \| 25 \| 94.87 \| 93.59 \| 84.0 \| 80.0 \| \| Standard spelling (control) \| 25 \| 97.89 \| 97.89 \| 92.0 \| 92.0 \| \| All chat-spelling sentences \| 480 \| 80.16 \| 91.54 \| 62.92 \| 82.5 \| <sub>480 sentences. Overall word accuracy 80.16% → 91.54% (paired bootstrap gain +11.38 points, 95% CI 9.14 to 13.71); whole-sentence exact match 62.92% → 82.50%. Internal set written by the author; small per-pattern counts, so per-row numbers are indicative.</sub> ## Small by design <p align="center"><img src="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/assets/chart_size.png" alt="Size comparison" width="92%"/></p> What you download: \| File \| What it is \| On disk \| gzipped \| \|---\|---\|---\|---\| \| `xlit.q4` \| core: transliteration Transformer, 4-bit weights \| 589 KB \| 544 KB \| \| `params.json` \| core: reranker weights + decoding settings \| 15 KB \| 6 KB \| \| `aksharalp.mjs` \| core: JavaScript runtime (zero dependencies) \| 62 KB \| 20 KB \| \| Core total \| works on its own \| 0.67 MB \| 0.57 MB \| \| `dict/lex.bin` \| dictionary pack: Hindi lexicon (trie + unigram scores) \| 11,675 KB \| 6,473 KB \| \| `dict/lm.bin` \| dictionary pack: word-bigram language model, 4-bit \| 3,913 KB \| 869 KB \| \| `dict/params.json` \| dictionary pack: reranker weights + decoding settings used with the pack \| 28 KB \| 12 KB \| \| Dictionary pack total \| fetched on first use, then cached \| 15.6 MB \| 7.35 MB \| \| Complete package \| core + pack \| 16.3 MB \| 7.92 MB \| For comparison, IndicXlit’s fp32 model weights are 46.0 MB and its official rescoring dictionary adds another 358 MB. General-purpose LLMs that can transliterate are larger still, with hundreds of millions of parameters or more, and typically need a server or a GPU. ## GGUF at a glance The model's weights are also published as GGUF files, for anyone who wants them as one self-describing container rather than our custom `.q4` format. \| File \| Size \| What it holds \| Accuracy vs. the shipped model \| \|---\|---\|---\|---\| \| `aksharalp-1m-v2-q4.gguf` \| 593 KB (546 KB gzipped) \| the 4-bit `xlit.q4` weights, carried byte-identically \| 400/400 words identical — same top-1, same candidate sets, same rankings \| \| `aksharalp-1m-v2-f16.gguf` \| 1.99 MB (1.33 MB gzipped) \| the same weights dequantised to half precision, plus the reranker matrices \| 399/400 words identical top-1; the one miss (`pratinidheeyo`) slips out of the 16-candidate beam once scores move by ~0.27 \| The q4 file is the one to use if you want the published numbers exactly. At 593 KB it is far smaller than the 7.92 MB complete package because it carries only the network's weights — the 3M-word dictionary pack is what makes up the rest. These files do not load in llama.cpp, Ollama or LM Studio, and they are not a runnable transliterator on their own; see [GGUF files](#gguf-files) below for the reader and exactly what does run. ## Benchmarks <p align="center"><img src="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/assets/chart_accuracy.png" alt="Accuracy comparison" width="92%"/></p> \| System \| Download \| Aksharantar top-1 % \| top-3 % \| top-8 % \| Dakshina word acc % \| Dakshina sentence EM % \| Latency / word (p50) \| \|---\|---\|---\|---\|---\|---\|---\|---\| \| AksharaLP-1M-v2 (core + dictionary pack) \| 7.92 MB \| 70.04 <sub>[69.16, 70.90]</sub> \| 88.31 \| 93.63 \| 89.02 \| 25.70 \| 206.00 ms <sub>browser</sub> \| \| IndicXlit model only \| 46.0 MB <sub>fp32 weights</sub> \| 59.16 <sub>[58.14, 60.13]</sub> \| 79.95 \| 88.29 \| 67.55 \| 6.29 \| 48.4 ms <sub>PyTorch CPU</sub> \| \| IndicXlit full package \| 404 MB <sub>fp32 weights + 358 MB rescoring dictionary</sub> \| 69.33 <sub>[68.44, 70.23]</sub> \| 85.40 \| 88.29 \| — \| — \| — \| <p align="center"><img src="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/assets/chart_tradeoff.png" alt="Accuracy versus size" width="80%"/></p> ## Variants: core only vs. core + dictionary The dictionary pack is optional. Each row below was scored once on the Aksharantar Hindi test set (9,991 words, beam 8); settings and the size of the dictionary, the reranker fits, the sentence weights and the core were chosen on validation and internal dev data only. The test-words-removed column shows the same system with every test reference word deleted from the dictionary. \| Variant \| Shipped \| Download (gz) \| On disk \| Test top-1 % <sub>[95% CI]</sub> \| top-3 % \| top-8 % \| Val top-1 % <sub>(2-fold CV)</sub> \| Test top-1 % with test words removed from the dictionary \| Dakshina sentences word acc % \| \|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\| \| Core only \| yes \| 0.57 MB \| 0.67 MB \| 56.01 <sub>[55.06, 56.96]</sub> \| 76.91 \| 85.09 \| 55.79 \| — \| 70.23 \| \| Core + 3M-word dictionary pack \| yes \| 7.92 MB \| 16.3 MB \| 70.04 <sub>[69.16, 70.90]</sub> \| 88.31 \| 93.63 \| 69.69 \| 16.37 \| 89.02 \| <sub>The “test words removed” column re-runs the same system with every Aksharantar test reference word deleted from the dictionary, so the dictionary cannot simply look up the answer; it shows how much of the gain comes from dictionary coverage (see the caveat below). Dakshina sentences: 4,976 romanized sentences.</sub> How we measured. All systems were run by us on the same machine with the same scoring script (Unicode NFC, punctuation and danda normalised). Word-level: Aksharantar Hindi test set, 9,991 unique romanized words, beam 8 for every system; top-k = the reference appears among the first k candidates. Sentence-level: Dakshina romanized Hindi sentences (4,976 sentences); word accuracy, sentence exact match, CER and WER. IndicXlit “model only” is the released transformer checkpoint; “full package” adds rescoring with the official 4.2M-word dictionary (α = 0.9; our re-implementation of the released engine, which reproduces the published scores). Browser latency is per word with the dictionary pack loaded, single tab, measured in headless Chrome on an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux); IndicXlit latency is PyTorch fp32 on one CPU thread and is shown for reference only (different runtime). > Caveat. IndicXlit was trained on Aksharantar, so its Aksharantar test scores may benefit from train/test overlap. Our training pairs contain no Aksharantar test or validation native words (checked). The 3M-word dictionary pack is built from IndicCorp v2 text and naturally contains most test words: with every test reference word removed from the dictionary the same system scores 16.37% top-1 (42.51% top-3, 70.33% top-8), and the core alone scores 56.01% top-1. All tuning, the reranker fits, the dictionary size and the sentence weights used validation and internal dev data only; each test number comes from a single run. ## Quickstart ### Browser (ES module) ```html <script type="module"> import { AksharaLP } from "https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/aksharalp.mjs"; const base = "https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/"; // core first (0.57 MB): usable immediately const aklp = await AksharaLP.load(base, { dict: false }); // then the dictionary pack (7.35 MB): fetched once, kept in the Cache API; false = stays core-only const ok = await aklp.loadDict(); const r = aklp.transliterate("kal meeting hai, kaise ho?"); console.log(r.output); // कल मीटिंग है, कैसे हो? (with the pack) console.log(r.tokens[0].candidates); // ranked alternatives for the first word </script> ``` `AksharaLP.load(base)` without options loads the core and the pack together. For production, copy the files (including `dict/`) into your app and load them from your own origin (they are static and cache well); run the model in a Web Worker as the demo Space does. ### Node.js (≥ 18) ```bash hf download sourojitdotcom/AksharaLP-1M-v2 --local-dir aksharalp ``` ```js import { loadFromDir } from "./aksharalp/aksharalp.mjs"; const aklp = await loadFromDir("./aksharalp/"); // core + dict/ pack; { dict: false } for core only console.log(aklp.transliterate("main theek hoon", { mode: "conversational" }).output); // मैं ठीक हूं console.log(aklp.word("kaise", 5)); // top-5 candidates with scores ``` ### Python (reference implementation) ```python from huggingface_hub import snapshot_download import sys path = snapshot_download("sourojitdotcom/AksharaLP-1M-v2") sys.path.insert(0, f"{path}/python") from bhc.translit2 import BhcTranslit2 # needs numpy + torch (CPU) xl = BhcTranslit2(path) # use_dict=False for core only print(xl.convert("kal meeting hai, kaise ho?")) print(xl.translit_words(["kaise", "namaste"], k=5)) ``` The Python package is the reference used for training and evaluation; the JavaScript runtime reproduces it exactly (JS/Python parity test — core only: identical candidate sets and top-1 for 221/221 words, max feature difference 1.4e-05; with the dictionary pack: identical candidate sets and top-1 for 221/221 words, max feature difference 2.8e-05). ### API at a glance \| Call \| Returns \| \|---\|---\| \| `AksharaLP.load(baseUrl, {dict, cache})` \| engine (fetches `manifest.json` and the core; with `dict: true`, the default, also the dictionary pack) \| \| `loadDict({cache})` \| fetches the dictionary pack (Cache API name `"aksharalp-dict"` by default) and attaches it; returns `false` and keeps the core working if it fails \| \| `attachDict({lex, lm, params})` \| attaches a pack you fetched yourself (e.g. from IndexedDB or app assets) \| \| `transliterate(text, {mode, context, k, overrides})` \| `{output, tokens[], ms}`; each token has `text, output, source, candidates[]`; `mode` is `"conversational"` (default) or `"strict"` \| \| `word(romanWord, k)` \| top-k `{text, score}` for one word \| \| `info()` \| parameter count, lexicon size, bytes on disk / gzipped, `hasDict`, `dictError` \| `overrides` pins a user-chosen suggestion (`{tokenStartOffset: "मैं"}`) and re-decodes the rest of the sentence around it. ## How it works <p align="center"><img src="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/assets/architecture.png" alt="Architecture" width="100%"/></p> ```mermaid flowchart LR A["Romanized Hindi text"] --> T["① Tokenizer<br/>URLs, @mentions, numbers kept"] T -->\|Strict mode: brand\| P["kept in Latin"] T -->\|word\| X["② Char Transformer (core)<br/>d=128, 3+2 layers, 4 heads, FFN 384 · 4-bit · 989K"] X --> C["Candidates<br/>free beam + acronyms<br/>(+ lexicon-trie beam + skeleton lookups with the pack)"] C --> R["Learned reranker<br/>30-feature MLP"] R --> V["③ Sentence Viterbi (pack)<br/>bigram LM · 3M words"] P --> V V --> O["Devanagari + top-k"] ``` 1. Tokenizer. Splits words, numbers and punctuation; URLs, e-mail addresses and @mentions are kept as typed. 2. Transliteration (core). A character-level encoder–decoder Transformer (d=128, 3+2 layers, 4 heads, FFN 384) proposes spellings with a free beam; letter-name acronyms are added and scored by the same model. With the dictionary pack, a second beam constrained to the 3M-word Hindi lexicon trie and consonant-skeleton lookups add in-vocabulary candidates. A small learned reranker orders the candidates: with the pack, 30 features → 32 hidden units → 1, fitted on validation data plus synthetic and internal chat-style spellings (so h, nhi, bht map to है, नहीं, बहुत); the core alone has its own 30 → 16 → 1 reranker fitted on the validation set. 3. Sentence context (pack). A 4-bit quantized word-bigram language model (300K bigrams) runs Viterbi over the candidate lattice to choose the most fluent sentence. Core-only mode picks each word independently. English words inside Hindi sentences go through the same model (Default) or, for brand names, stay in Latin script (Strict). ### Files \| File \| Contents \| \|---\|---\| \| `xlit.q4` \| core: transliteration Transformer, 4-bit weights \| \| `params.json` \| core: reranker weights + decoding settings \| \| `dict/lex.bin` \| dictionary pack: Hindi lexicon (trie + unigram scores) \| \| `dict/lm.bin` \| dictionary pack: word-bigram language model, 4-bit \| \| `dict/params.json` \| dictionary pack: reranker weights + decoding settings used with the pack \| \| `manifest.json` \| file list, sizes, checksums (core + pack) \| \| `aksharalp.mjs` \| core: JavaScript runtime (zero dependencies) \| \| `aksharalp.min.mjs` \| same runtime, minified (attribution header preserved) \| \| `python/` \| reference Python implementation (numpy + PyTorch) \| \| `aksharalp-1m-v2-q4.gguf` \| GGUF container: the `xlit.q4` weights as one blob, for `gguf_read.py` \| \| `aksharalp-1m-v2-f16.gguf` \| GGUF container: the same weights as plain F16 tensors plus the reranker, for `gguf_read.py` \| \| `gguf_read.py` \| dependency-light GGUF reader (numpy only); decodes either file to the reference weight tensors \| \| `LICENSE`, `NOTICE` \| Apache-2.0 and third-party attributions \| ### GGUF files These GGUF files do not run in llama.cpp, Ollama, LM Studio or any other llama.cpp-based tool. AksharaLP-1M-v2 is not a LLaMA-style model: it is a custom character-level encoder-decoder Transformer (d=128, 4 heads, 3 encoder + 2 decoder layers, FFN 384, 988,928 parameters) wrapped in a bespoke decoding stack - beam search over the model, a lexicon-trie candidate generator, 30 hand-built candidate features, gates, and a linear/MLP reranker. llama.cpp has no architecture for that, so it cannot load these files, and nothing here is a llama.cpp checkpoint. What GGUF is used for here: a portable single-file tensor container with typed metadata. The weights, their configuration and the decoding settings travel as one self-describing file, readable by the small pure-Python reader shipped next to it (`gguf_read.py`, numpy only - no torch, no llama.cpp, no gguf-py), with no dependency on the custom `.q4` codec. It carries weights and config, not a runnable transliterator: the lexicon, feature pipeline, gates and reranker are code, so use the JavaScript runtime or the Python reference implementation above to actually transliterate. \| File \| Contents \| Raw bytes \| Gzipped \| SHA-256 \| \|---\|---\|---\|---\|---\| \| `aksharalp-1m-v2-q4.gguf` \| the `xlit.q4` file carried byte-identically as one I8 tensor (`aksharalp.q4.blob`), plus metadata describing the 4-bit layout \| 593,184 \| 545,820 \| `458fd07d…8efebc0` \| \| `aksharalp-1m-v2-f16.gguf` \| every weight under its reference name at F16, plus the core and pack reranker matrices \| 1,989,504 \| 1,332,735 \| `9cb7ab3a…55822ec4` \| \| `gguf_read.py` \| reads either file: magic and version check, typed metadata, named tensors, and the weight object the reference implementation expects \| 14,828 \| 4,808 \| `213255f3…eb1eab4755` \| Both are GGUF v3, little-endian, tensor data 32-byte aligned. The metadata carries the model name and version, the parameter count, the layer configuration, the quantisation description, the full candidate-feature list, the beam settings, the licence, and `aksharalp.requires_bundled_decoder = true` with `aksharalp.runs_in_llama_cpp = false`. ```python import gguf_read # the reader shipped in this repo w = gguf_read.load("aksharalp-1m-v2-f16.gguf") # numpy only w.cfg # {'d': 128, 'h': 4, 'enc': 3, 'dec': 2, 'ff': 384, 'p': 0.1} (p stored as float32) w.state_dict["semb.weight"].shape # (30, 128), float32 w.n_params # 988928 import torch, sys # into the reference implementation sys.path.insert(0, "python") from bhc.xlit_model import XlitTransformer m = XlitTransformer(w.cfg) m.load_state_dict({k: torch.from_numpy(v) for k, v in w.state_dict.items()}) m.eval() ``` `gguf_read.py <file>` prints a structural summary; `--json` dumps metadata and the tensor table. `gguf_read.q4_blob(path)` returns the carried `xlit.q4` bytes from the compact file: 588,891 bytes, byte-identical to the original (sha256 `652eafe0c1e2b46fec85663825621284f0a5911f2d7cc6e13b1182ce47cd9803`). Verification.** Both files were run back through the reference decoder on a fixed sample of 400 words (the first 400 distinct alphabetic romanised words of the Aksharantar Hindi test split), with the dictionary pack, comparing against the shipped `xlit.q4`: * `aksharalp-1m-v2-q4.gguf`: identical top-1, identical candidate sets and identical score-ranked candidate lists for 400/400 words. The blob is carried unmodified, so the weights are bit-identical to `xlit.q4`. * `aksharalp-1m-v2-f16.gguf`: identical top-1 for 399/400 words, identical candidate sets for 394/400 and identical ranked lists for 377/400. Every difference comes from rounding the dequantised float32 weights to F16 (max absolute weight difference 9.2e-04): candidate scores move by up to 0.27, which reorders near-tied candidates and occasionally drops one out of the 16-item beam. The one top-1 change is pratinidheeyo -> प्रतिनिधियो instead of प्रतिनिधियों: under F16, प्रतिनिधियों falls just outside the 16 surviving beam hypotheses (the beam's 16th score is -8.5693 against -8.5691 for q4), so the reranker never sees it and the runner-up wins. An F32 control built the same way from the same dequantised weights reproduced 400/400 top-1, candidate sets and ranked lists, which isolates the difference to F16 rounding rather than the container. For output identical to the published numbers, use `xlit.q4` or the q4 GGUF. Structurally both pass: magic, version 3, tensor count against the header, 32-byte alignment, offsets in bounds, a fresh-process re-parse, and an independent read with the upstream `gguf` package (1 and 102 tensors, shapes round-trip). ## Intended use * Hindi input methods: mobile and desktop keyboards, browser extensions, chat and messaging apps, search boxes. * Converting romanized Hindi text (chat logs, social posts, transcripts) to Devanagari for downstream NLP. * Privacy-sensitive or offline settings where text must not leave the device. ## Limitations * Hindi only. Other Indic languages and scripts are not supported. * Names and rare words. Proper nouns, rare or very long words, and unusual spellings are the main source of errors. Check the top-k list. * Ambiguity. Some romanizations have several valid Devanagari spellings (e.g. हूं / हूँ). The model follows the most common written form, which may differ from a reference. * Short context. The language model sees one word on each side (bigram), so long-range agreement is not modelled. * Core only is less accurate. Without the dictionary pack, top-1 accuracy drops substantially (see [Variants](#variants-core-only-vs-core--dictionary)). Load the pack when you can. * Dictionary coverage. Much of the accuracy with the pack comes from the lexicon: with every test reference word removed from it, top-1 falls well below the core-only score. Words outside the lexicon (new names, slang, rare spellings) are harder. * English words and language tags. This release has no word-level language identifier and no loanword lexicon: English words are transliterated phonetically by the same model and can look unusual, and tokens are not tagged Hindi / English. * Short ambiguous words. Without a loanword lexicon, short words that are both English and Hindi can go the wrong way in context (for example, in our tests call sometimes came out as कल and do as डॉ). Check the top-k list. * Chat-speak. Common chat spellings (h, nhi, bht, thk, kr, rhe) are handled well with the dictionary pack, but rarer abbreviations and some pronoun/verb pairs still go wrong (for example mai ghr pe hu gives मैं घर पे हुआ instead of मैं घर पे हूं). The core alone handles chat spellings much less well. * Speed depends on the device. Very old phones may see higher per-word latency than reported here. ## License & attribution Commercial use is allowed. Code and model weights are released under the Apache License 2.0 (see `LICENSE`), Copyright 2026 Sourojit Dhua. You may use, modify and ship AksharaLP-1M-v2 in commercial products, free of charge. Attribution is required when you redistribute it. Apache-2.0 §4(d) requires every redistribution of the model or a derivative (shipping the files in an app, web page, browser extension or SDK; publishing a fine-tuned, quantized or converted version) to include the `NOTICE` file, and the JavaScript runtime keeps its `/! … /` attribution header, including the minified `aksharalp.min.mjs`. Please use this attribution text: ```text Transliteration by AksharaLP (Sourojit Dhua) — https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2 ``` Recommended in-app credit (About, Settings or Licenses screen): ```html Hindi typing powered by <a href="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2">AksharaLP by Sourojit Dhua</a> ``` Third-party data used to build the model is credited in `NOTICE`: \| Source \| License \| Used for \| \|---\|---\|---\| \| [Aksharantar](https://huggingface.co/datasets/ai4bharat/Aksharantar), AI4Bharat \| CC0 (mined data) / CC-BY (manually collected data) \| romanized ↔ Devanagari training pairs \| \| [IndicXlit](https://github.com/AI4Bharat/IndicXlit), AI4Bharat \| MIT \| teacher model for distillation \| \| [IndicCorp v2](https://github.com/AI4Bharat/IndicBERT/tree/main#indiccorp-v2), AI4Bharat \| CC0 \| Hindi lexicon and bigram language model \| \| [CMUdict](https://github.com/cmusphinx/cmudict), Carnegie Mellon University \| BSD-2-Clause-style \| English word cues \| Fonts in the demo Space (Inter, Noto Sans Devanagari) are under the SIL Open Font License 1.1. ## Citation ```bibtex @software{dhua_aksharalp_2026, author = {Dhua, Sourojit}, title = {{AksharaLP}: Tiny, Context-Aware Hindi Transliteration for the Browser}, year = {2026}, version = {2.0.0}, url = {https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2}, license = {Apache-2.0} } ``` <sub>AksharaLP · v2.0.0 · built 2026-11-01 · Questions and feedback: open a discussion on this repository.</sub>

The world’s 3rd most spoken language, typed on the world’s most used keyboard. In under 8 MB.

2.8×

88.3%

11.4 pts

89.0%

24×

0 calls