uzmorph: Uzbek morphological analyzer and generator
This is an improved version of the Apertium Uzbek transducer (apertium-uzb, upstream commit 00522e3), packaged with a Python layer:
- a confident guesser for unknown words;
- a proper-noun layer;
- a corpus-trained reading ranker that replaces Constraint Grammar
-1; - UD-style output (lemma, UPOS, features);
- input in official Latin, Cyrillic or the 2026 Latin alphabet.
Code, documentation, evaluation protocol and issues: https://github.com/javohirmat/uzmorph (release v0.9.0).
Files in this repository
| path | contents |
|---|---|
dist/uzmorph-0.9.0-py3-none-any.whl |
the Python package (includes everything in data/) |
dist/uzmorph-0.9.0.tar.gz |
sdist: the full source, including the modified apertium-uzb/ (Corresponding Source of the transducers) |
data/uzb.automorf.bin, uzb.autogen.bin, uzb_guesser.automorf.bin |
compiled lttoolbox transducers: analyser, generator, guesser |
data/*.json.gz, data/name_sufprior.json |
ranker, guesser and capitalisation models, and the proper-noun suffix prior |
data/names_list.tsv |
proper-noun stems verified against Wikidata (ids included) |
data/SOURCE.md |
where the Corresponding Source is, and the written offer |
The models are counts only. They hold word types, Apertium tags and suffix strings with frequencies or probabilities, derived from tahrirchi/uz-books-v2 (dataset card: MIT). They contain no sentences or running text. Everything here is licensed GPL-3.0-only, the same as the package and upstream apertium-uzb.
Usage
sudo apt install lttoolbox
pip install https://huggingface.co/javohirmat/uzmorph/resolve/main/dist/uzmorph-0.9.0-py3-none-any.whl
import uzmorph
for t in uzmorph.analyze("Kecha doʻstlarim bilan Samarqandga bordik."):
if t.is_word:
print(t.text, t.best.lemma, t.best.upos, t.best.feats)
print(uzmorph.generate("kitob", "<n><pl><px1pl><abl>")) # ['kitoblarimizdan']
Results
The numbers are the same as in the GitHub README; every number there comes from a file in the repository.
Top-1 accuracy, held-out test split (156 sentences, 1,731 scored tokens, four genres; an unanalysed token counts as wrong):
| system | lemma+feats | lemma | UPOS* |
|---|---|---|---|
upstream apertium-uzb 00522e3 + CG -1 |
79.7 | 86.5 | 75.4 |
previous build (v8d + CG -1 + guesser) |
83.9 | 90.4 | 77.1 |
| uzmorph 0.9 | 93.5 | 96.1 | 79.1 |
| uzmorph 0.9, dictionary words only | 88.0 | 90.5 | 74.1 |
| ZirhUzMorph 0.1.1, all analyses | 87.8 | 92.2 | 57.3 |
| ZirhUzMorph 0.1.1, known stems only | 83.5 | 87.9 | 56.9 |
*UPOS is not comparable across tools: ZirhUzMorph only distinguishes N and V.
On the dev split (1,062 tokens), uzmorph scores 94.1, Zirh 87.5 and upstream + CG 78.1.
Coverage (token coverage of the uz-books-v2 word-frequency list, anchored to the upstream 87.51):
- dictionary: 90.69;
- with the confident guesser: 94.80.
Non-words (299 lower-case non-words; lower is better):
- 1% are accepted by the dictionary;
- 17% with the confident guesser;
- ZirhUzMorph accepts 15% (known stems).
Generation: 0 of 462 checked verb forms give a spurious -ish- variant (previous build: 275).
Limitations
- The evaluation gold is mostly model-built:
- 1,525 labels were auto-accepted where the tools agreed (150-token spot-check);
- about 1,120 were AI-adjudicated and never reviewed by a person;
- 250 rows were checked by a native speaker.
- Conventions: Finite 3rd-person verbs default to Number=Sing. Without that ranker rule, dev drops to 87.5 (equal to Zirh).
- UPOS is weak (79%).
- Guesser readings are often wrong: 57–63% are fully right.
- The proper-noun layer accepts any capitalised unknown word in mid-sentence.
- The evaluation sentences are private (copyrighted books, news, Telegram) and are not in this repository or on GitHub. Researchers can request them from the maintainer.
Citation and contact
Please cite the GitHub repository (see its CITATION.cff) and Apertium.
Contact: Javohir Matniyazov, javohirmatniyazovv@gmail.com.