AksharaLP-1M-v2

The world’s 3rd most spoken language, typed on the world’s most used keyboard. In under 8 MB.

Hindi typed in English letters → Devanagari, fully offline.
Starts typing at 0.6 MB; the dictionary pack streams in after and is cached.

Why it matters

🌏 #3	🗣️ 611M	📦 < 8 MB
most spoken language in the world	people speak Hindi (first or second language)	is all it takes to type it — no server, no GPU

_{Source: Ethnologue 200 (Ethnologue: Languages of the World, 29th edition, SIL Global), 2026 — https://www.ethnologue.com/insights/ethnologue200/ (accessed 2026-10-31). Counts first- and second-language speakers.}

On a Latin (QWERTY) keyboard, Hindi is often typed in English letters: “kal meeting hai, kaise ho?”. AksharaLP-1M-v2 turns that into proper Devanagari — accurately, in context, and entirely offline in the browser. It transliterates (changes the script); it does not translate.

AksharaLP-1M-v2 is the 989K-parameter member of the AksharaLP family of tiny transliteration models.

For “kal meeting hai, kaise ho?” AksharaLP writes कल मीटिंग है, कैसे हो?, the way a Hindi speaker would. It ships in two tiers: a 0.57 MB core (character Transformer with 989K parameters, reranker and runtime) that works on its own, and a 7.35 MB dictionary pack (3M-word lexicon and sentence language model) that the runtime fetches on first use and caches. The complete package is under 8 MB to download (7.92 MB gzipped). The complete package is 2.8× smaller than the model weights of IndicXlit, the most widely used open Indic transliteration model, and 24× smaller than IndicXlit with its rescoring dictionary — while beating IndicXlit’s model on top-1 accuracy (70.04% vs 59.16% on Aksharantar) and on par with the full package (69.33%; paired gap +0.71 points, 95% CI -0.04 to 1.40).

_{Recorded from the live demo: inference runs in a Web Worker in the browser tab.}

Versions

v2 (this repo) is the recommended release. v1 remains available (v1 demo).

	v1	v2 (this repo)
Dictionary pack	1.5M words	3M words
Complete download (gzipped)	4.53 MB	7.92 MB
Aksharantar top-1 / top-3 / top-8 %	70.37 / 87.66 / 92.79	70.04 / 88.31 / 93.63
Core-only top-1 %	53.36	56.01
Dakshina word acc / sentence EM %	87.55 / 21.66	89.02 / 25.70
Chat-spelling sentences, word acc % (480 held-out)	80.16	91.54
Browser latency / word (p50)	219.7 ms	206.0 ms

_{v1 numbers from the v1 model card. Aksharantar top-1 differs by 0.33 points with overlapping 95% CIs. Choose v1 only if the smaller download matters more than chat-spelling accuracy.}

Best in class for its size

2.8× beats a model 2.8× its size _{+10.9 pts top-1 vs IndicXlit model only (46.0 MB) · paired 95% CI 9.9–11.8}	88.3% top-3 accuracy — higher than IndicXlit full package (85.4%) _{+2.9 pts [95% CI 2.3–3.5] at 24× smaller}	11.4 pts more accurate on casual chat Hindi than v1 _{whole sentences right 62.9% → 82.5% · 480 held-out chat sentences}
89.0% word accuracy on full Hindi sentences _{Dakshina, 4,976 test sentences · v1 87.5% (+1.5 pts, 95% CI 1.3–1.6)}	24× smaller than IndicXlit full package _{16.3 MB vs 404 MB on disk (fp32 weights + 358 MB rescoring dictionary)}	0 calls network calls — fully offline _{206 ms per word (p50) in the browser · nothing you type leaves the device}

All measured results (14 more claims that passed their rules)

Result	What	Details
88.3%	top-3 accuracy — higher than IndicXlit model only (80.0%)	+8.4 pts [95% CI 7.7–9.1] at 2.8× smaller
93.6%	top-8 accuracy — higher than IndicXlit model only (88.3%)	+5.3 pts [95% CI 4.8–5.9] at 2.8× smaller
93.6%	top-8 accuracy — higher than IndicXlit full package (88.3%)	+5.3 pts [95% CI 4.8–5.9] at 24× smaller
2.65 pts	core-only top-1 gain over v1	53.36% → 56.01% without the dictionary pack
2.8×	smaller than IndicXlit model only	16.3 MB vs 46.0 MB on disk (fp32 weights)
70.0%	top-1 — higher than IndicXlit model only (59.2%)	at 2.8× smaller · Aksharantar Hindi test
70.0%	top-1 — on par with IndicXlit full package (69.3%, overlapping 95% CIs)	at 24× smaller · Aksharantar Hindi test
0.65 pts	top-3 gain over v1	87.66% → 88.31% · Aksharantar Hindi test
0.84 pts	top-8 gain over v1	92.79% → 93.63% · Aksharantar Hindi test
4.3 pts/MB	highest top-1 accuracy per MB among compared models	next: IndicXlit model only at 1.28 pts/MB
88.3%	right word in the first 3 suggestions — vs 59.2% top-1 for IndicXlit model only	Aksharantar Hindi test
88.3%	right word in the first 3 suggestions — vs 69.3% top-1 for IndicXlit full package	Aksharantar Hindi test
< 8 MB	complete download	core + 3M-word dictionary pack · 989K parameters · 7.92 MB gzipped
0.57 MB	to start typing	the core works on its own (56.0% top-1 without the pack); the dictionary pack streams in after and is cached

Accuracy per MB (top-1 on Aksharantar Hindi test ÷ on-disk size; systems measured by us with the same harness):

#	System	On disk	Top-1 %	Top-1 pts per MB
1	AksharaLP-1M-v2	16.3 MB	70.04	4.30
2	IndicXlit model only	46.0 MB	59.16	1.28
3	IndicXlit full package	404 MB	69.33	0.17

🏆 No compared model is both smaller and more accurate: AksharaLP-1M-v2 sits on the size–accuracy frontier.

_{Every tile and every row above is generated by the release build from measured results and shown only if its rule holds (rules and outcomes are logged in the build report).}

Real-world comparison: casual Hindi

Same input, typed casually: a phone keyboard screenshot next to AksharaLP v2's output

Same input, typed casually (measured 2026-11-01).

	Output for “Boht ache h Bhai”
A popular phone keyboard (author's screenshot)	बोहट अ थे भाई
AksharaLP-1M-v2 (released runtime, offline, under 8 MB)	बोहत अच्छे है भाई
Standard Hindi spelling (reference)	बहुत अच्छे है भाई

Words in standard spelling: the keyboard 1 of 4 (भाई); AksharaLP-1M-v2 3 of 4 (अच्छे, है, भाई). AksharaLP-1M-v2 writes बोहत (standard: बहुत).

_{Screenshot supplied by the author, cropped to the input and output lines (keyboard name not shown; text unaltered). AksharaLP-1M-v2's output is from the released runtime on this exact input, measured 2026-11-01. This is one sentence, shown as a showcase only: it is not part of any test or tuning set and does not change the accuracy numbers above.}

Highlights


🪶 Extremely small	Under 8 MB complete: 0.57 MB core + 7.35 MB dictionary pack = 7.92 MB gzipped (16.3 MB on disk); 989K parameters, 4-bit weights. Fits in a keyboard app, a browser extension or a web page.
🎯 Accurate	With the dictionary pack: 70.04% top-1 / 88.31% top-3 on the Aksharantar Hindi test set (9,991 words); 89.02% word accuracy on Dakshina sentences. Core-only numbers are in Variants.
🧠 Context-aware	A compact bigram language model picks the right word from its neighbours: the same main becomes मैं or में.
📥 Starts fast, degrades gracefully	The core is enough to start typing; the dictionary pack streams in after and is cached. If the pack cannot be fetched, the runtime keeps working with the core alone.
🔒 Private & offline	Pure JavaScript, zero dependencies, no server. Nothing you type leaves the device; works offline after first load.
⚡ Fast	206.0 ms per word (p50) in the browser on an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux).

What it brings on-device

Everything below happens on the user’s own phone or laptop: the model is a set of static files plus a dependency-free JavaScript runtime. Each example output was produced by the released runtime when this card was built.

✈️ Fully offline

No network after the first load, so it keeps working in airplane mode, in a lift, or on a train.

flight mein hoon, land karke call karta hoon
→ फ्लाइट में हूं, लैंड करके कॉल करता हूं

🔒 Private by design

Text never leaves the device: no API call, no logging server. Suited to banking, health and personal chat.

mera account balance kitna hai
→ मेरा अकाउंट बैलेंस कितना है
doctor ne din mein do baar dawai lene ko kaha
→ डॉक्टर ने दिन में दो बार दवाई लेने को कहा

🌐 Runs wherever JavaScript runs

A browser tab, an Android or iOS WebView, an installable PWA, or Node.js. CPU only: no GPU, no server, no inference bill.

namaste, aapka swagat hai
→ नमस्ते, आपका स्वागत है

⚡ Real-time, as you type

The sentence is re-decoded on every keystroke in a Web Worker, and earlier words are corrected as context arrives:

main → मैं
main theek → मैं ठीक
main theek hoon → मैं ठीक हूं

_{Browser latency: 206.0 ms per word (p50) on an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux).}

⌨️ Keyboard & input-box integration

Wire it to any <input> or <textarea>: chat, search, forms, CRM notes, customer-support replies. One call per keystroke: transliterate(text, {k}).

aapka refund teen din mein aa jayega
→ आपका रिफंड तीन दिन में आ जाएगा

🏷️ Brands your way

English words inside Hindi sentences are transliterated by the same model. Two writing styles for the same input:

Default → कल व्हाट्सएप पे मीटिंग है
Strict (brands stay in Latin) → कल whatsapp पे मीटिंग है

☰ Top-k suggestions

Ranked candidates for every word, ready for a suggestion bar or candidate picker. Picking one re-decodes the rest of the sentence.

main → मैं · में · मैन · मेन · माइन

📶 Built for Bharat

A 0.57 MB core to start, under 8 MB in total, downloaded once and cached; after that, patchy or missing connectivity does not matter. Pure JavaScript on the CPU, with no GPU or native code, designed with entry-level Android phones in mind.

signal nahi aa raha, baad mein baat karte hain
→ सिग्नल नहीं आ रहा, बाद में बात करते हैं

₹0 per call at scale

Inference runs on your users’ devices, so there are no per-request API fees and no GPU servers to scale. Serving is static files from any CDN. Apache-2.0: free for commercial use.

mera order kab tak deliver hoga
→ मेरा ऑर्डर कब तक डिलिवर होगा

What it does — real outputs

Every output below was produced by the released runtime (aksharalp.mjs) and the files in this repository when this card was built.

Feature	You type	AksharaLP writes
Everyday Hindi	`kaise ho`	कैसे हो
Everyday Hindi	`mujhe bhookh lagi hai`	मुझे भूख लगी है
Context — main = “I”	`main theek hoon`	मैं ठीक हूं
Context — main = “in”	`ghar main kaun hai`	घर में कौन है
English words in Hindi	`kal office mein meeting hai`	कल ऑफिस में मीटिंग है
English words in Hindi	`whatsapp pe message karo`	व्हाट्सएप पे मैसेज करो
Acronyms	`sbi`	एसबीआई
Acronyms	`SBI ka ATM kahan hai`	एसबीआई का एटीएम कहां है
English words	`please mujhe call karna`	प्लीज मुझे कॉल करना
Kept as-is	`@rahul 5 baje milte hain, link: www.example.com`	@rahul 5 बजे मिलते हैं, लिंक: www.example.com
Chat spelling	`Ache ho na?`	अच्छे हो ना?
Chat spelling	`nhi yr, bht kaam h`	नहीं यार, बहुत काम है
Chat spelling	`main thk hu, tm kaise ho`	मैं ठीक हूं, तुम कैसे हो
Chat spelling	`kya kr rhe ho`	क्या कर रहे हो
Chat spelling	`pls jldi aao`	प्लीज़ जल्दी आओ
Chat spelling	`bahuuut accha laga`	बहुत अच्छा लगा
Chat spelling	`mujhy pta nai`	मुझे पता नहीं
Chat spelling	`kb aaoge`	कब आओगे
Chat spelling	`bht acha h`	बहुत अच्छा है
Chat spelling	`mai thk hu`	मैं ठीक हूं
Chat spelling	`kr rha hu`	कर रहा हूं

Two writing styles

The same sentence in each mode:

Mode	`kal whatsapp pe meeting ka link bhej dena`
Default	कल व्हाट्सएप पे मीटिंग का लिंक भेज देना
Strict	कल whatsapp पे मीटिंग का लिंक भेज देना

Default — every word, English ones included, is written in Devanagari.
Strict — brand and product names stay in Latin script.

Top-k suggestions

Every word comes with ranked alternatives, ready for a keyboard suggestion bar:

Input	1	2	3	4	5
`kaise`	कैसे	केसे	कइसे	कैस	कईसे
`main`	मैं	में	मैन	मेन	माइन
`dhanyavaad`	धन्यवाद	धन्यावाद	धन्यवाद्	धान्यवाद	धनयवाद

Core only vs. core + dictionary pack

The same inputs through the core alone (pack not loaded) and with the pack, from the released runtime:

You type	Core only	Core + dictionary pack
`main theek hoon`	मैं थीक हूं	मैं ठीक हूं
`ghar main kaun hai`	घर मैं कौं है	घर में कौन है
`mujhe bhookh lagi hai`	मुझे भूख लगी है	मुझे भूख लगी है
`whatsapp pe message karo`	वॉट्सएप पीई मेसेज करो	व्हाट्सएप पे मैसेज करो
`doctor ne dawai lene ko kaha`	डॉक्टर ने दवाई लेने को कहा	डॉक्टर ने दवाई लेने को कहा

Chat-style spelling

Hand-written chat-style Hindi sentences (WhatsApp-style spellings such as h, nhi, bht, thk, dropped vowels and stretched letters), held-out half of an internal dev set (never used for fitting). Word accuracy % per spelling pattern, v1 vs v2, same scorer.

Spelling pattern	Sentences	Before (v1) word %	After word %	Before sentence %	After sentence %
Single letter for a doubled consonant	26	79.35	92.39	38.46	73.08
Extra doubled letters	25	95.16	98.39	88.0	96.0
Missing h in kh / th / bh	26	54.24	71.19	23.08	42.31
Extra h	25	76.36	85.45	56.0	72.0
Long / short vowel spelling	26	85.71	96.43	73.08	92.31
e for ai	25	89.29	96.43	76.0	92.0
o for au	25	70.45	84.09	56.0	72.0
Dropped or added a	25	76.92	97.44	68.0	96.0
Dropped vowels	25	62.5	90.0	48.0	84.0
Missing nasal (n)	25	70.0	86.0	52.0	72.0
w / v, z / j, f / ph	26	97.78	100.0	96.15	100.0
s for sh	25	72.73	86.36	52.0	76.0
ri / ru for ऋ	26	96.43	100.0	96.15	100.0
Word-final y / h endings	25	55.32	82.98	20.0	68.0
Stretched letters	25	75.0	82.14	72.0	80.0
Chat abbreviations (h, nhi, yr)	25	45.83	91.67	12.0	84.0
English words in Hindi chat	25	96.77	98.39	92.0	96.0
Ambiguous short words	25	94.87	93.59	84.0	80.0
Standard spelling (control)	25	97.89	97.89	92.0	92.0
All chat-spelling sentences	480	80.16	91.54	62.92	82.5

_{480 sentences. Overall word accuracy 80.16% → 91.54% (paired bootstrap gain +11.38 points, 95% CI 9.14 to 13.71); whole-sentence exact match 62.92% → 82.50%. Internal set written by the author; small per-pattern counts, so per-row numbers are indicative.}

Small by design

Size comparison

What you download:

File	What it is	On disk	gzipped
`xlit.q4`	core: transliteration Transformer, 4-bit weights	589 KB	544 KB
`params.json`	core: reranker weights + decoding settings	15 KB	6 KB
`aksharalp.mjs`	core: JavaScript runtime (zero dependencies)	62 KB	20 KB
Core total	works on its own	0.67 MB	0.57 MB
`dict/lex.bin`	dictionary pack: Hindi lexicon (trie + unigram scores)	11,675 KB	6,473 KB
`dict/lm.bin`	dictionary pack: word-bigram language model, 4-bit	3,913 KB	869 KB
`dict/params.json`	dictionary pack: reranker weights + decoding settings used with the pack	28 KB	12 KB
Dictionary pack total	fetched on first use, then cached	15.6 MB	7.35 MB
Complete package	core + pack	16.3 MB	7.92 MB

For comparison, IndicXlit’s fp32 model weights are 46.0 MB and its official rescoring dictionary adds another 358 MB. General-purpose LLMs that can transliterate are larger still, with hundreds of millions of parameters or more, and typically need a server or a GPU.

GGUF at a glance

The model's weights are also published as GGUF files, for anyone who wants them as one self-describing container rather than our custom .q4 format.

File	Size	What it holds	Accuracy vs. the shipped model
`aksharalp-1m-v2-q4.gguf`	593 KB (546 KB gzipped)	the 4-bit `xlit.q4` weights, carried byte-identically	400/400 words identical — same top-1, same candidate sets, same rankings
`aksharalp-1m-v2-f16.gguf`	1.99 MB (1.33 MB gzipped)	the same weights dequantised to half precision, plus the reranker matrices	399/400 words identical top-1; the one miss (`pratinidheeyo`) slips out of the 16-candidate beam once scores move by ~0.27

The q4 file is the one to use if you want the published numbers exactly. At 593 KB it is far smaller than the 7.92 MB complete package because it carries only the network's weights — the 3M-word dictionary pack is what makes up the rest. These files do not load in llama.cpp, Ollama or LM Studio, and they are not a runnable transliterator on their own; see GGUF files below for the reader and exactly what does run.

Benchmarks

Accuracy comparison

System	Download	Aksharantar top-1 %	top-3 %	top-8 %	Dakshina word acc %	Dakshina sentence EM %	Latency / word (p50)
AksharaLP-1M-v2 (core + dictionary pack)	7.92 MB	70.04 _{[69.16, 70.90]}	88.31	93.63	89.02	25.70	206.00 ms _browser
IndicXlit model only	46.0 MB _{fp32 weights}	59.16 _{[58.14, 60.13]}	79.95	88.29	67.55	6.29	48.4 ms _{PyTorch CPU}
IndicXlit full package	404 MB _{fp32 weights + 358 MB rescoring dictionary}	69.33 _{[68.44, 70.23]}	85.40	88.29	—	—	—

Accuracy versus size

Variants: core only vs. core + dictionary

The dictionary pack is optional. Each row below was scored once on the Aksharantar Hindi test set (9,991 words, beam 8); settings and the size of the dictionary, the reranker fits, the sentence weights and the core were chosen on validation and internal dev data only. The test-words-removed column shows the same system with every test reference word deleted from the dictionary.

Variant	Shipped	Download (gz)	On disk	Test top-1 % _{[95% CI]}	top-3 %	top-8 %	Val top-1 % _{(2-fold CV)}	Test top-1 % with test words removed from the dictionary	Dakshina sentences word acc %
Core only	yes	0.57 MB	0.67 MB	56.01 _{[55.06, 56.96]}	76.91	85.09	55.79	—	70.23
Core + 3M-word dictionary pack	yes	7.92 MB	16.3 MB	70.04 _{[69.16, 70.90]}	88.31	93.63	69.69	16.37	89.02

_{The “test words removed” column re-runs the same system with every Aksharantar test reference word deleted from the dictionary, so the dictionary cannot
simply look up the answer; it shows how much of the gain comes from dictionary coverage (see the caveat below). Dakshina sentences: 4,976 romanized sentences.}

How we measured. All systems were run by us on the same machine with the same scoring script (Unicode NFC, punctuation and danda normalised). Word-level: Aksharantar Hindi test set, 9,991 unique romanized words, beam 8 for every system; top-k = the reference appears among the first k candidates. Sentence-level: Dakshina romanized Hindi sentences (4,976 sentences); word accuracy, sentence exact match, CER and WER. IndicXlit “model only” is the released transformer checkpoint; “full package” adds rescoring with the official 4.2M-word dictionary (α = 0.9; our re-implementation of the released engine, which reproduces the published scores). Browser latency is per word with the dictionary pack loaded, single tab, measured in headless Chrome on an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux); IndicXlit latency is PyTorch fp32 on one CPU thread and is shown for reference only (different runtime).

Caveat. IndicXlit was trained on Aksharantar, so its Aksharantar test scores may benefit from train/test overlap. Our training pairs contain no Aksharantar test or validation native words (checked). The 3M-word dictionary pack is built from IndicCorp v2 text and naturally contains most test words: with every test reference word removed from the dictionary the same system scores 16.37% top-1 (42.51% top-3, 70.33% top-8), and the core alone scores 56.01% top-1. All tuning, the reranker fits, the dictionary size and the sentence weights used validation and internal dev data only; each test number comes from a single run.

Quickstart

Browser (ES module)

<script type="module">
  import { AksharaLP } from "https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/aksharalp.mjs";

  const base = "https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2/resolve/main/";
  // core first (0.57 MB): usable immediately
  const aklp = await AksharaLP.load(base, { dict: false });
  // then the dictionary pack (7.35 MB): fetched once, kept in the Cache API; false = stays core-only
  const ok = await aklp.loadDict();
  const r = aklp.transliterate("kal meeting hai, kaise ho?");
  console.log(r.output);                       // कल मीटिंग है, कैसे हो? (with the pack)
  console.log(r.tokens[0].candidates);         // ranked alternatives for the first word
</script>

AksharaLP.load(base) without options loads the core and the pack together. For production, copy the files (including dict/) into your app and load them from your own origin (they are static and cache well); run the model in a Web Worker as the demo Space does.

Node.js (≥ 18)

hf download sourojitdotcom/AksharaLP-1M-v2 --local-dir aksharalp

import { loadFromDir } from "./aksharalp/aksharalp.mjs";

const aklp = await loadFromDir("./aksharalp/");            // core + dict/ pack; { dict: false } for core only
console.log(aklp.transliterate("main theek hoon", { mode: "conversational" }).output);   // मैं ठीक हूं
console.log(aklp.word("kaise", 5));                                                      // top-5 candidates with scores

Python (reference implementation)

from huggingface_hub import snapshot_download
import sys

path = snapshot_download("sourojitdotcom/AksharaLP-1M-v2")
sys.path.insert(0, f"{path}/python")
from bhc.translit2 import BhcTranslit2          # needs numpy + torch (CPU)

xl = BhcTranslit2(path)                         # use_dict=False for core only
print(xl.convert("kal meeting hai, kaise ho?"))
print(xl.translit_words(["kaise", "namaste"], k=5))

The Python package is the reference used for training and evaluation; the JavaScript runtime reproduces it exactly (JS/Python parity test — core only: identical candidate sets and top-1 for 221/221 words, max feature difference 1.4e-05; with the dictionary pack: identical candidate sets and top-1 for 221/221 words, max feature difference 2.8e-05).

API at a glance

Call	Returns
`AksharaLP.load(baseUrl, {dict, cache})`	engine (fetches `manifest.json` and the core; with `dict: true`, the default, also the dictionary pack)
`loadDict({cache})`	fetches the dictionary pack (Cache API name `"aksharalp-dict"` by default) and attaches it; returns `false` and keeps the core working if it fails
`attachDict({lex, lm, params})`	attaches a pack you fetched yourself (e.g. from IndexedDB or app assets)
`transliterate(text, {mode, context, k, overrides})`	`{output, tokens[], ms}`; each token has `text, output, source, candidates[]`; `mode` is `"conversational"` (default) or `"strict"`
`word(romanWord, k)`	top-k `{text, score}` for one word
`info()`	parameter count, lexicon size, bytes on disk / gzipped, `hasDict`, `dictError`

overrides pins a user-chosen suggestion ({tokenStartOffset: "मैं"}) and re-decodes the rest of the sentence around it.

How it works

Architecture

flowchart LR
    A["Romanized Hindi text"] --> T["① Tokenizer<br/>URLs, @mentions, numbers kept"]
    T -->|Strict mode: brand| P["kept in Latin"]
    T -->|word| X["② Char Transformer (core)<br/>d=128, 3+2 layers, 4 heads, FFN 384 · 4-bit · 989K"]
    X --> C["Candidates<br/>free beam + acronyms<br/>(+ lexicon-trie beam + skeleton lookups with the pack)"]
    C --> R["Learned reranker<br/>30-feature MLP"]
    R --> V["③ Sentence Viterbi (pack)<br/>bigram LM · 3M words"]
    P --> V
    V --> O["Devanagari + top-k"]

Tokenizer. Splits words, numbers and punctuation; URLs, e-mail addresses and @mentions are kept as typed.
Transliteration (core). A character-level encoder–decoder Transformer (d=128, 3+2 layers, 4 heads, FFN 384) proposes spellings with a free beam; letter-name acronyms are added and scored by the same model. With the dictionary pack, a second beam constrained to the 3M-word Hindi lexicon trie and consonant-skeleton lookups add in-vocabulary candidates. A small learned reranker orders the candidates: with the pack, 30 features → 32 hidden units → 1, fitted on validation data plus synthetic and internal chat-style spellings (so h, nhi, bht map to है, नहीं, बहुत); the core alone has its own 30 → 16 → 1 reranker fitted on the validation set.
Sentence context (pack). A 4-bit quantized word-bigram language model (300K bigrams) runs Viterbi over the candidate lattice to choose the most fluent sentence. Core-only mode picks each word independently.

English words inside Hindi sentences go through the same model (Default) or, for brand names, stay in Latin script (Strict).

Files

File	Contents
`xlit.q4`	core: transliteration Transformer, 4-bit weights
`params.json`	core: reranker weights + decoding settings
`dict/lex.bin`	dictionary pack: Hindi lexicon (trie + unigram scores)
`dict/lm.bin`	dictionary pack: word-bigram language model, 4-bit
`dict/params.json`	dictionary pack: reranker weights + decoding settings used with the pack
`manifest.json`	file list, sizes, checksums (core + pack)
`aksharalp.mjs`	core: JavaScript runtime (zero dependencies)
`aksharalp.min.mjs`	same runtime, minified (attribution header preserved)
`python/`	reference Python implementation (numpy + PyTorch)
`aksharalp-1m-v2-q4.gguf`	GGUF container: the `xlit.q4` weights as one blob, for `gguf_read.py`
`aksharalp-1m-v2-f16.gguf`	GGUF container: the same weights as plain F16 tensors plus the reranker, for `gguf_read.py`
`gguf_read.py`	dependency-light GGUF reader (numpy only); decodes either file to the reference weight tensors
`LICENSE`, `NOTICE`	Apache-2.0 and third-party attributions

GGUF files

These GGUF files do not run in llama.cpp, Ollama, LM Studio or any other llama.cpp-based tool. AksharaLP-1M-v2 is not a LLaMA-style model: it is a custom character-level encoder-decoder Transformer (d=128, 4 heads, 3 encoder + 2 decoder layers, FFN 384, 988,928 parameters) wrapped in a bespoke decoding stack - beam search over the model, a lexicon-trie candidate generator, 30 hand-built candidate features, gates, and a linear/MLP reranker. llama.cpp has no architecture for that, so it cannot load these files, and nothing here is a llama.cpp checkpoint.

What GGUF is used for here: a portable single-file tensor container with typed metadata. The weights, their configuration and the decoding settings travel as one self-describing file, readable by the small pure-Python reader shipped next to it (gguf_read.py, numpy only - no torch, no llama.cpp, no gguf-py), with no dependency on the custom .q4 codec. It carries weights and config, not a runnable transliterator: the lexicon, feature pipeline, gates and reranker are code, so use the JavaScript runtime or the Python reference implementation above to actually transliterate.

File	Contents	Raw bytes	Gzipped	SHA-256
`aksharalp-1m-v2-q4.gguf`	the `xlit.q4` file carried byte-identically as one I8 tensor (`aksharalp.q4.blob`), plus metadata describing the 4-bit layout	593,184	545,820	`458fd07d…8efebc0`
`aksharalp-1m-v2-f16.gguf`	every weight under its reference name at F16, plus the core and pack reranker matrices	1,989,504	1,332,735	`9cb7ab3a…55822ec4`
`gguf_read.py`	reads either file: magic and version check, typed metadata, named tensors, and the weight object the reference implementation expects	14,828	4,808	`213255f3…eb1eab4755`

Both are GGUF v3, little-endian, tensor data 32-byte aligned. The metadata carries the model name and version, the parameter count, the layer configuration, the quantisation description, the full candidate-feature list, the beam settings, the licence, and aksharalp.requires_bundled_decoder = true with aksharalp.runs_in_llama_cpp = false.

import gguf_read                                     # the reader shipped in this repo
w = gguf_read.load("aksharalp-1m-v2-f16.gguf")       # numpy only
w.cfg                                                # {'d': 128, 'h': 4, 'enc': 3, 'dec': 2, 'ff': 384, 'p': 0.1} (p stored as float32)
w.state_dict["semb.weight"].shape                    # (30, 128), float32
w.n_params                                           # 988928

import torch, sys                                    # into the reference implementation
sys.path.insert(0, "python")
from bhc.xlit_model import XlitTransformer
m = XlitTransformer(**w.cfg)
m.load_state_dict({k: torch.from_numpy(v) for k, v in w.state_dict.items()})
m.eval()

gguf_read.py <file> prints a structural summary; --json dumps metadata and the tensor table. gguf_read.q4_blob(path) returns the carried xlit.q4 bytes from the compact file: 588,891 bytes, byte-identical to the original (sha256 652eafe0c1e2b46fec85663825621284f0a5911f2d7cc6e13b1182ce47cd9803).

Verification. Both files were run back through the reference decoder on a fixed sample of 400 words (the first 400 distinct alphabetic romanised words of the Aksharantar Hindi test split), with the dictionary pack, comparing against the shipped xlit.q4:

aksharalp-1m-v2-q4.gguf: identical top-1, identical candidate sets and identical score-ranked candidate lists for 400/400 words. The blob is carried unmodified, so the weights are bit-identical to xlit.q4.
aksharalp-1m-v2-f16.gguf: identical top-1 for 399/400 words, identical candidate sets for 394/400 and identical ranked lists for 377/400. Every difference comes from rounding the dequantised float32 weights to F16 (max absolute weight difference 9.2e-04): candidate scores move by up to 0.27, which reorders near-tied candidates and occasionally drops one out of the 16-item beam. The one top-1 change is pratinidheeyo -> प्रतिनिधियो instead of प्रतिनिधियों: under F16, प्रतिनिधियों falls just outside the 16 surviving beam hypotheses (the beam's 16th score is -8.5693 against -8.5691 for q4), so the reranker never sees it and the runner-up wins. An F32 control built the same way from the same dequantised weights reproduced 400/400 top-1, candidate sets and ranked lists, which isolates the difference to F16 rounding rather than the container. For output identical to the published numbers, use xlit.q4 or the q4 GGUF.

Structurally both pass: magic, version 3, tensor count against the header, 32-byte alignment, offsets in bounds, a fresh-process re-parse, and an independent read with the upstream gguf package (1 and 102 tensors, shapes round-trip).

Intended use

Hindi input methods: mobile and desktop keyboards, browser extensions, chat and messaging apps, search boxes.
Converting romanized Hindi text (chat logs, social posts, transcripts) to Devanagari for downstream NLP.
Privacy-sensitive or offline settings where text must not leave the device.

Limitations

Hindi only. Other Indic languages and scripts are not supported.
Names and rare words. Proper nouns, rare or very long words, and unusual spellings are the main source of errors. Check the top-k list.
Ambiguity. Some romanizations have several valid Devanagari spellings (e.g. हूं / हूँ). The model follows the most common written form, which may differ from a reference.
Short context. The language model sees one word on each side (bigram), so long-range agreement is not modelled.
Core only is less accurate. Without the dictionary pack, top-1 accuracy drops substantially (see Variants). Load the pack when you can.
Dictionary coverage. Much of the accuracy with the pack comes from the lexicon: with every test reference word removed from it, top-1 falls well below the core-only score. Words outside the lexicon (new names, slang, rare spellings) are harder.
English words and language tags. This release has no word-level language identifier and no loanword lexicon: English words are transliterated phonetically by the same model and can look unusual, and tokens are not tagged Hindi / English.
Short ambiguous words. Without a loanword lexicon, short words that are both English and Hindi can go the wrong way in context (for example, in our tests call sometimes came out as कल and do as डॉ). Check the top-k list.
Chat-speak. Common chat spellings (h, nhi, bht, thk, kr, rhe) are handled well with the dictionary pack, but rarer abbreviations and some pronoun/verb pairs still go wrong (for example mai ghr pe hu gives मैं घर पे हुआ instead of मैं घर पे हूं). The core alone handles chat spellings much less well.
Speed depends on the device. Very old phones may see higher per-word latency than reported here.

License & attribution

Commercial use is allowed. Code and model weights are released under the Apache License 2.0 (see LICENSE), Copyright 2026 Sourojit Dhua. You may use, modify and ship AksharaLP-1M-v2 in commercial products, free of charge.

Attribution is required when you redistribute it. Apache-2.0 §4(d) requires every redistribution of the model or a derivative (shipping the files in an app, web page, browser extension or SDK; publishing a fine-tuned, quantized or converted version) to include the NOTICE file, and the JavaScript runtime keeps its /*! … */ attribution header, including the minified aksharalp.min.mjs. Please use this attribution text:

Transliteration by AksharaLP (Sourojit Dhua) — https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2

Recommended in-app credit (About, Settings or Licenses screen):

Hindi typing powered by <a href="https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2">AksharaLP by Sourojit Dhua</a>

Third-party data used to build the model is credited in NOTICE:

Source	License	Used for
Aksharantar, AI4Bharat	CC0 (mined data) / CC-BY (manually collected data)	romanized ↔ Devanagari training pairs
IndicXlit, AI4Bharat	MIT	teacher model for distillation
IndicCorp v2, AI4Bharat	CC0	Hindi lexicon and bigram language model
CMUdict, Carnegie Mellon University	BSD-2-Clause-style	English word cues

Fonts in the demo Space (Inter, Noto Sans Devanagari) are under the SIL Open Font License 1.1.

Citation

@software{dhua_aksharalp_2026,
  author    = {Dhua, Sourojit},
  title     = {{AksharaLP}: Tiny, Context-Aware Hindi Transliteration for the Browser},
  year      = {2026},
  version   = {2.0.0},
  url       = {https://huggingface.co/sourojitdotcom/AksharaLP-1M-v2},
  license   = {Apache-2.0}
}

_{AksharaLP · v2.0.0 · built 2026-11-01 · Questions and feedback: open a discussion on this repository.}

Downloads last month: 28

GGUF

Model size

990k params