Horizon-Labs/multilingual-punctuation-truecase-small

Multilingual Punctuation + Truecasing (small, 140M): 88 languages

Restores punctuation and capital letters in lowercase text in one pass, for example the output of a speech-recognition (ASR) system, subtitles or chat messages. For each word it predicts the mark that follows it (. , ? : - or nothing) and its casing (lower, Capitalized or UPPER). Trained on 88 languages. Built on mmBERT-small, Apache-2.0, ONNX files for CPU and the browser (transformers.js) included. Try it in the browser. A larger, more accurate version is available as multilingual-punctuation-truecase-base.

Actual output of this model (from the release check):

in : my name is anna i live in berlin and work for the united nations in new york
out: My name is Anna. I live in Berlin and work for the United Nations in New York.
in : ich heiße thomas und wohne in münchen was machst du am wochenende
out: Ich heiße Thomas und wohne in München. Was machst du am Wochenende?

• A standard token-classification model with 18 labels <mark>|<case> (0|lower, .|Cap, ,|UPPER, ...), so it runs with the transformers pipeline and transformers.js. The included truecase.py turns the labels into text. It also writes ，。？： in Chinese and Japanese (no word segmentation needed), । in Hindi and Bengali, ، ؟ in Arabic script, ; for questions in Greek and ። ፣ in Amharic.
• Input is lowercased first: the model was trained on lowercase text and does not use casing you already have. If your text is already cased and only lacks punctuation, use punctuation-restoration-small instead.
• onnx/model_quantized.onnx (int8 embeddings, 268 MB) predicts the same label as fp32 for 99.91% of tokens on lowercased held-out text.

Usage

With the included helper (any language; long texts are processed in overlapping chunks):

from huggingface_hub import hf_hub_download
import importlib.util
spec = importlib.util.spec_from_file_location("truecase", hf_hub_download("Horizon-Labs/multilingual-punctuation-truecase-small", "truecase.py"))
truecase = importlib.util.module_from_spec(spec); spec.loader.exec_module(truecase)

t = truecase.Truecaser("Horizon-Labs/multilingual-punctuation-truecase-small")
print(t("my name is anna i live in berlin and work for the united nations in new york"))
print(t("今天天气很好我们去公园散步吧你觉得怎么样"))

Plain transformers: the label of each word is the label of its last token. Lowercase the text first.

from transformers import pipeline
tagger = pipeline("token-classification", model="Horizon-Labs/multilingual-punctuation-truecase-small")
tagger("the nasa mission launched on monday from florida")

transformers.js:

import { pipeline } from "@huggingface/transformers";
const tagger = await pipeline("token-classification", "Horizon-Labs/multilingual-punctuation-truecase-small", { dtype: "q8" });
console.log(await tagger("the nasa mission launched on monday from florida"));

Evaluation

Every system gets the same input: a paragraph lowercased, with . , ? : ; ! - removed. The test sets were used only for evaluation:

• Europarl: European Parliament proceedings, 5 consecutive sentences, 200 paragraphs in each of 12 languages.
• TED talks (TED2020 via OPUS): transcripts and their translations, 5 consecutive sentences, 200 paragraphs in each of 25 languages. Spoken style.
• FLORES-200 devtest: professionally translated news, travel and wiki text, one paragraph per article. Columns: our 88 training languages, and all 199 FLORES languages (the extra 111 were never trained on).

Metrics are averaged over languages:

• Casing F1: over the words that should be Capitalized or UPPER. A word counts only with the exact class, and capitals predicted on words that should be lower case count as false positives. Languages are included when their test text contains capitals (for Chinese, Japanese, Arabic and other scripts without case, only the few Latin-script words count).
• Casing accuracy: all words, including the lower-case ones.
• Punctuation F1: as on the punctuation-restoration cards. Micro-F1 over the five marks, and a mark counts only at the right position.

Casing F1:

model	Europarl, 12 languages	TED talks, 25 languages	FLORES, our 88 languages	FLORES, all 199 languages
this model (140M)	0.887	0.805	0.883	0.784
multilingual-punctuation-truecase-base (307M)	0.897	0.816	0.886	0.796
punctuation-restoration-small + rule casing	0.541	0.589	0.482	0.389
xlm-roberta_punctuation_fullstop_truecase	0.830	0.774	0.802	0.658

Punctuation F1 (lowercased input):

model	Europarl, 12 languages	TED talks, 25 languages	FLORES, our 88 languages	FLORES, all 199 languages
this model (140M)	0.826	0.706	0.794	0.584
multilingual-punctuation-truecase-base (307M)	0.836	0.723	0.803	0.602
punctuation-restoration-small + rule casing	0.830	0.706	0.796	0.588
xlm-roberta_punctuation_fullstop_truecase	0.750	0.644	0.689	0.483

Casing accuracy:

model	Europarl, 12 languages	TED talks, 25 languages	FLORES, our 88 languages	FLORES, all 199 languages
this model (140M)	0.978	0.975	0.982	0.966
multilingual-punctuation-truecase-base (307M)	0.980	0.977	0.984	0.969
punctuation-restoration-small + rule casing	0.922	0.949	0.939	0.934
xlm-roberta_punctuation_fullstop_truecase	0.966	0.967	0.961	0.912

• + rule casing is our punctuation-only model punctuation-restoration-small, followed by capitalizing the first word and every word after . or ? (and English "i"). This is what you get by adding capitals to the output of a punctuation-only model. Its punctuation F1 is that model's own lowercased-input score on these paragraphs.
• xlm-roberta_punctuation_fullstop_truecase (Apache-2.0, 47 languages) was run through the punctuators package on lowercased input, with its sentence splitting turned off, and its output was re-aligned to the input words. 2.3% of words could not be aligned and count as "no mark, lower case", which lowers its scores somewhat. The FLORES columns include languages it was not trained on.
• Two training seeds per size were evaluated before release. Mean casing F1 / punctuation F1 over Europarl, TED and FLORES-199: small .824 / .705, base .834 / .720.

Per language (FLORES, our 88 training languages)

language	casing F1, this model	casing F1, xlm-roberta_punctuation_fullstop_truecase	punctuation F1, this model	punctuation F1, xlm-roberta_punctuation_fullstop_truecase
Afrikaans	0.907	0.879	0.808	0.768
Albanian	0.902	0.852	0.835	0.769
Amharic	0.803	0.504	0.761	0.720
Arabic	0.835	0.752	0.734	0.652
Armenian	0.891	0.837	0.577	0.551
Azerbaijani	0.907	0.855	0.818	0.750
Basque	0.875	0.779	0.789	0.544
Belarusian	0.899	0.849	0.877	0.808
Bengali	0.887	0.797	0.739	0.628
Bosnian	0.920	0.894	0.861	0.806
Bulgarian	0.925	0.897	0.886	0.849
Catalan	0.916	0.853	0.843	0.751
Central Kurdish	0.891	0.214	0.623	0.147
Chinese (Simplified)	0.310	0.923	0.829	0.771
Croatian	0.926	0.898	0.853	0.798
Czech	0.928	0.888	0.916	0.839
Danish	0.947	0.914	0.888	0.751
Dutch	0.947	0.919	0.900	0.835
English	0.928	0.893	0.826	0.781
Esperanto	0.894	0.838	0.849	0.713
Estonian	0.929	0.900	0.905	0.847
Faroese	0.923	0.682	0.826	0.466
Filipino	0.879	0.804	0.770	0.658
Finnish	0.932	0.917	0.896	0.865
French	0.934	0.882	0.871	0.806
Galician	0.914	0.863	0.849	0.752
Georgian	0.876	0.095	0.779	0.652
German	0.985	0.978	0.924	0.900
Greek	0.890	0.865	0.811	0.750
Gujarati	0.902	0.865	0.750	0.692
Hausa	0.792	0.692	0.605	0.512
Hebrew	0.858	0.832	0.789	0.612
Hindi	0.822	0.732	0.792	0.709
Hungarian	0.900	0.871	0.867	0.819
Icelandic	0.928	0.909	0.828	0.780
Indonesian	0.907	0.881	0.805	0.769
Irish	0.844	0.703	0.729	0.584
Italian	0.907	0.859	0.835	0.775
Japanese	0.557	0.899	0.905	0.806
Javanese	0.828	0.763	0.659	0.536
Kannada	0.859	0.793	0.749	0.709
Kazakh	0.918	0.882	0.842	0.752
Kinyarwanda	0.796	0.768	0.570	0.571
Korean	0.907	0.913	0.875	0.859
Kyrgyz	0.906	0.880	0.855	0.792
Latvian	0.914	0.893	0.908	0.862
Lithuanian	0.911	0.886	0.868	0.816
Luxembourgish	0.978	0.702	0.832	0.266
Macedonian	0.908	0.873	0.839	0.777
Malay	0.891	0.850	0.773	0.707
Malayalam	0.848	0.738	0.693	0.660
Maltese	0.896	0.391	0.755	0.247
Marathi	0.938	0.881	0.752	0.709
Mongolian	0.876	0.849	0.839	0.750
Nepali	0.953	0.917	0.813	0.686
Norwegian Bokmål	0.943	0.905	0.845	0.786
Odia	0.859	0.767	0.618	0.714
Pashto	0.942	0.929	0.728	0.653
Persian	0.894	0.882	0.815	0.678
Polish	0.919	0.902	0.888	0.857
Portuguese	0.916	0.872	0.853	0.793
Punjabi	0.896	0.797	0.759	0.685
Romanian	0.921	0.885	0.861	0.793
Russian	0.928	0.890	0.913	0.850
Serbian	0.912	0.892	0.851	0.798
Sindhi	0.922	0.808	0.721	0.648
Sinhala	0.873	0.838	0.716	0.731
Slovak	0.931	0.889	0.913	0.848
Slovenian	0.932	0.888	0.919	0.764
Somali	0.774	0.764	0.596	0.583
Spanish	0.922	0.871	0.839	0.778
Sundanese	0.831	0.735	0.638	0.489
Swahili	0.845	0.822	0.716	0.672
Swedish	0.951	0.908	0.859	0.797
Tajik	0.905	0.250	0.883	0.220
Tamil	0.892	0.839	0.711	0.692
Tatar	0.893	0.541	0.804	0.293
Telugu	0.932	0.918	0.721	0.699
Turkish	0.932	0.893	0.801	0.750
Ukrainian	0.908	0.884	0.892	0.849
Urdu	0.923	0.780	0.769	0.702
Uyghur	0.902	0.824	0.753	0.757
Uzbek	0.891	0.826	0.837	0.733
Vietnamese	0.866	0.792	0.765	0.668
Welsh	0.870	0.755	0.750	0.649
Xhosa	0.794	0.572	0.655	0.387
Yoruba	0.717	0.521	0.467	0.259
Zulu	0.763	0.520	0.643	0.365

Training

• Data: the training text of punctuation-restoration v1.1: 1,168,626 windows (119M words) of FineWeb-2 and FineWeb web text (ODC-BY) in 88 languages, plus 49,144 synthetic spoken-style transcripts in 69 languages, written by Qwen3.8-27B (Apache-2.0) for this purpose. Any transcript that shared a sentence with a test set was removed.
• Labels come from the original text. Punctuation is normalized as for punctuation-restoration. The casing classes are: Cap (first cased letter upper case), UPPER (two or more cased letters, all upper case) and lower (everything else, including words in scripts without case). The model always sees the text lowercased.
• Model: mmBERT-small token classification, 18 joint labels on each word's last token, 2 epochs, learning rate 5e-5 (the punctuation-restoration-small recipe). The checkpoint and the released seed were chosen on held-out web text by the mean of punctuation F1 and casing F1.
• Release criteria, fixed before training: punctuation F1 no more than .01 below punctuation-restoration-small on lowercased input; casing F1 at least .05 above the rule baseline on every test set and .10 above on average; and higher casing F1 than xlm-roberta_punctuation_fullstop_truecase on Europarl + TED. Both sizes passed.
• Code: code/ in this repository.

Limitations

• Only three casing classes. Mixed-case words come out wrong: iphone becomes Iphone, and mcdonald's becomes Mcdonald's. All-caps words are rare (0.3% of the words in the test text), so acronyms the model has not seen may stay lower case.
• Capitals on names depend on the model having seen the name or its context. This is weaker in low-resource languages (see the per-language table).
• The mark after a word and the casing of the next word are separate labels, so the model sometimes writes a comma followed by a capital ("Hello, How are you") where a full stop belongs or the next word should be lower case. On the test sets this happens for about 1 in 1,000 words or fewer, and most comma + capital predictions are right (a name after a comma). A decoding that re-checks these boundaries did not help on held-out text (eval/boundary_decoding.json), so truecase.py uses the plain predictions.
• Your casing is ignored: the input is lowercased first.
• Only . , ? : - are predicted. Exclamation marks, semicolons, quotes and brackets are not restored, and : and - are much less reliable than . , ?. Thai, Lao, Khmer, Burmese and Tibetan are not supported.
• Trained mostly on written web text, plus synthetic spoken-style transcripts. Disfluent speech (fillers, false starts, repetitions) is harder.