Translation
Keras
Khmer
bigru-additive-attention-seq2seq
transliteration
khmer
romanization
seq2seq
gru
attention
Eval Results (legacy)
Instructions to use Netra-Lab/AkaraAlpha with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Keras
How to use Netra-Lab/AkaraAlpha with Keras:
# !pip install -U keras tensorflow huggingface_hub # Keras needs TensorFlow installed to read "hf://" paths, so the tensorflow backend is selected here; # "jax" and "torch" also work for computation once TensorFlow is installed. import os os.environ["KERAS_BACKEND"] = "tensorflow" import keras model = keras.saving.load_model("hf://Netra-Lab/AkaraAlpha") - Notebooks
- Google Colab
- Kaggle
File size: 9,631 Bytes
d728d26 36e7d31 d728d26 36e7d31 d728d26 36e7d31 d728d26 6c79074 d728d26 36e7d31 d728d26 36e7d31 d728d26 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 | ---
license: mit
language:
- km
library_name: keras
pipeline_tag: translation
tags:
- transliteration
- khmer
- romanization
- seq2seq
- gru
- attention
- keras
metrics:
- cer
- accuracy
model-index:
- name: AkaraAlpha
results:
- task:
type: translation
name: Romanized Khmer to Khmer transliteration
dataset:
type: khmer-text-transliteration
name: Khmer Text Transliteration Dataset (20% held out)
metrics:
- type: cer
value: 15.07
name: CER (%), model alone
- type: accuracy
value: 73.6
name: Top-1 word accuracy (%), beam 5 with dictionary
---
# AkaraAlpha
AkaraAlpha turns Khmer typed in the Latin alphabet back into Khmer script: `brodae` β ααααα. It is a character-level encoder-decoder with **78,705 parameters**. A bidirectional GRU reads the romanized word, and a GRU with additive attention writes it in Khmer, one character at a time. A Khmer dictionary of 38,743 words then checks its suggestions.
- **Accuracy.** The model alone has a character error rate of 15.07% on held-out words, lower than every baseline trained on the same data, including a Transformer with three times as many parameters.
- **Dictionary.** Checking five beam candidates against the dictionary makes the first suggestion right 73.6% of the time, against 44.4% for the model's single best guess.
- **Speed.** The whole system takes 94 ms per word with five candidates.
The code is on [GitHub](https://github.com/netra-ai-lab/Romanized-Khmer-to-Khmer-Transliteration) and the package is [`netra-transliterate`](https://pypi.org/project/netra-transliterate/) on PyPI, both under the MIT licence.
## Architecture
<p align="center"><img src="https://huggingface.co/Netra-Lab/AkaraAlpha/resolve/main/assets/architecture.png" width="100%" alt="AkaraAlpha turning brodae into Khmer: a bidirectional GRU encoder reading upwards and a GRU decoder with additive attention reading downwards"></p>
*AkaraAlpha turning `brodae` into ααααα.* The encoder reads upwards: each letter becomes a 32-dimensional embedding, a bidirectional GRU reads the letters in both directions, and a projection brings each state to 64 dimensions, the keys and values for attention. The decoder reads downwards. At step 4, with α α α written, its GRU output is the query for additive attention, and a softmax layer over 81 ids gives α with probability 0.997. The chosen character joins the input and the loop repeats until the end marker.
| Part | Setting |
| --- | --- |
| Input | Romanized word, up to 25 characters: 26 Latin letters |
| Output | Khmer word, up to 24 characters: 77 Khmer characters |
| Character embeddings | 32 dimensions, separate for input and output |
| Encoder | Bidirectional GRU, 64 units each way, then a projection to 64 |
| Decoder | GRU, 64 units, starting from zeros |
| Attention | Additive (Keras `AdditiveAttention`, learned scale); keys and values are the projected encoder states |
| Output layer | Softmax over 81 ids, from the GRU output and the context |
| Parameters | 78,705 (encoder 46,784, decoder 31,921) |
| Dictionary | 38,743 Khmer words |
The decoder doesn't start from the encoder's final state, so everything it knows about the input reaches it through attention.
## Dictionary check
The model proposes *k* candidates by beam search, and each is normalised with [khmerspell](https://pypi.org/project/khmerspell/). Candidates that are dictionary words come first. Each remaining candidate is replaced by the closest dictionary word within two edits (Levenshtein distance), unless that word is already in the list. If no candidate is near a dictionary word, the raw candidates are returned.
<p align="center"><img src="https://huggingface.co/Netra-Lab/AkaraAlpha/resolve/main/assets/pipeline.png" width="100%" alt="The full pipeline on konhchreng: preprocessing, encoder and decoder, three beam candidates, the dictionary check, Levenshtein correction and the merged suggestions"></p>
*The full system on `konhchreng`, a held-out word, with k = 3.* Left, reading upwards: the word is cleaned, turned into letter ids and padded, and the model proposes three candidates by beam search. Right: candidates that are dictionary words are kept, and the others are replaced by their nearest dictionary word within two edits; a replacement that is already kept is dropped (dashed). The model ranked the correct word, αααα
αααα, second; the dictionary puts it first.
## Results
Character error rate of the models alone on the 5,714 held-out words. Lower is better.
| Model | Parameters | CER (%) |
| --- | ---: | ---: |
| **AkaraAlpha** | 78,705 | **15.07** |
| Transformer | 248,337 | 17.78 |
| Attention BiLSTM | 96,753 | 18.26 |
| LSTM | 58,417 | 31.78 |
| GRU | 46,385 | 51.64 |
| RNN | 42,609 | 102.41 |
Word-level accuracy and latency of the full system, with the dictionary check. Top-1 is the share of words where the first suggestion is exactly right; Top-K the share where the right word is anywhere in the list.
| Decoding | Top-1 | Top-K | Latency per word |
| --- | ---: | ---: | ---: |
| Greedy | 44.44% | 44.44% | 21.92 ms |
| Beam, k = 3 | 70.06% | 78.72% | 66.03 ms |
| **Beam, k = 5 (recommended)** | **73.60%** | **83.41%** | **94.15 ms** |
| Beam, k = 7 | 74.87% | 85.60% | 122.10 ms |
| Beam, k = 10 | 75.85% | 87.37% | 167.21 ms |
## Usage
### With the package
The package includes beam search and the dictionary check:
```bash
pip install netra-transliterate
```
```python
from khmer_transliterator import transliterate, transliterate_with_dict
transliterate("brodae") # 'ααααα'
transliterate_with_dict("brodae", n=5) # dictionary-checked suggestions
```
`netra-transliterate --serve` opens a typing interface in the browser that suggests Khmer words as you type romanized ones.
### From this repository
The Keras model and a plain JSON copy of its tokenizers are enough for greedy decoding (TensorFlow 2.17 or later):
```python
import json, re
import numpy as np
import keras
from huggingface_hub import snapshot_download
path = snapshot_download("Netra-Lab/AkaraAlpha")
model = keras.saving.load_model(f"{path}/bi_gru_attention_khmer_transliterator.keras")
tok = json.load(open(f"{path}/tokenizers.json", encoding="utf-8"))
latin, khmer = tok["latin"], tok["khmer"]
id_to_khmer = {i: c for c, i in khmer["word_index"].items()}
start, end = khmer["word_index"]["\t"], khmer["word_index"]["\n"]
def transliterate(word: str) -> str:
letters = re.sub("[^a-z]", "", word.lower())[: latin["max_len"]]
src = [latin["word_index"].get(c, latin["word_index"]["<unk>"]) for c in letters]
src = np.array([src + [0] * (latin["max_len"] - len(src))])
out = [start]
for _ in range(khmer["max_len"] + 1):
probs = model.predict([src, np.array([out])], verbose=0)[0, -1]
nxt = int(probs.argmax())
if nxt == end:
break
out.append(nxt)
return "".join(id_to_khmer[i] for i in out[1:])
print(transliterate("brodae")) # ααααα
```
## Files
| File | What it is |
| --- | --- |
| `bi_gru_attention_khmer_transliterator.keras` | The trained model (Keras 3) |
| `tokenizers.json` | Character-to-id maps for both scripts, maximum lengths and special tokens |
| `gru_transliteration_assets.pkl` | The same tokenizers as pickled Keras `Tokenizer` objects, as the package loads them. Only unpickle files you trust |
| `khmer_dictionary.txt` | The Khmer word list used by the dictionary check, one word per line |
| `config.json` | Architecture and decoding settings |
## Training
- **Data.** The [Khmer Text Transliteration Dataset](https://github.com/Chhunneng/khmer-text-transliteration) (Chhunneng, 2023): 28,569 single words in romanized form and in Khmer script. The last 20% (5,714 words) was held out; the model trained on the other 22,855.
- **Objective.** Teacher forcing with sparse categorical cross-entropy.
- **Optimiser.** Adam at 0.001, batch size 64, dropout 0.3 on the inputs and recurrent states of both GRUs.
- **Schedule.** 50 epochs. The learning rate was halved when the validation loss stopped improving for three epochs, which happened once, after epoch 42. The weights are from epoch 48, which had the lowest validation loss (0.1034).
## Limitations
- **Words, not sentences.** The model transliterates one word at a time, so it can't use the surrounding sentence to choose between two words with the same romanized spelling.
- **No separate test set.** The held-out words were also used to adjust the learning rate and choose the epoch, so the scores may be slightly optimistic.
- **One way of romanizing.** All the training data comes from one dataset. People who spell Khmer differently in Latin letters may get worse results.
- **Ties.** When several dictionary words are equally close to a candidate, the package keeps whichever it meets first, and that order can change between runs, so later suggestions can differ from run to run.
- **The dictionary has edges.** A name or new word that isn't in the dictionary may be "corrected" into a real word within two edits.
- **Latency depends on hardware.** The timings were measured on one machine; read them as the relative cost of each setting.
## Citation
```bibtex
@software{akaraalpha2026,
title = {AkaraAlpha: Romanized Khmer to Khmer Script Transliteration},
author = {Nhem, Darayut},
year = {2026},
publisher = {Netra Lab},
url = {https://huggingface.co/Netra-Lab/AkaraAlpha}
}
```
## Licence
MIT.
|