license: mit
language:
- en
- de
- fr
- it
- nl
- multilingual
base_model: oliverguhr/fullstop-punctuation-multilingual-base
pipeline_tag: token-classification
library_name: coreml
tags:
- coreml
- punctuation-restoration
- token-classification
- xlm-roberta
- on-device
fullstop-coreml
oliverguhr/fullstop-punctuation-multilingual-base (XLM-RoBERTa token-classification,
~0.3 B) converted to CoreML (fp16) for on-device punctuation restoration —
downloaded on first use and run locally through CoreML + swift-transformers.
This is a format conversion only — the weights are unchanged from the upstream
PyTorch model. The model is a verbatim token classifier: it emits one punctuation
label per token, so it can add punctuation (. , ? - :) but can never reword the
input. Capitalization is handled separately (a heuristic restorer is typically chained
after it).
Label map (id2label)
| id | label | meaning |
|---|---|---|
| 0 | "" |
no punctuation |
| 1 | . |
period |
| 2 | , |
comma |
| 3 | ? |
question mark |
| 4 | - |
dash |
| 5 | : |
colon |
Files (repo layout)
The repo root is the folder a loader consumes directly:
| File | Purpose |
|---|---|
Fullstop.mlpackage/ |
CoreML model bundle (compile before load) |
tokenizer.json |
XLM-R Unigram tokenizer (HuggingFace tokenizers) |
tokenizer_config.json |
tokenizer config |
special_tokens_map.json |
XLM-R special tokens (<s>, </s>, <pad>, <unk>, <mask>) |
config.json |
model config incl. id2label / label2id |
Loading (Swift)
// 1. Snapshot the repo (swift-transformers Hub):
// Hub.snapshot(from: Repo(id: "gregbarbosa/fullstop-coreml")) -> localDir
// 2. Compile + load the CoreML model, build the tokenizer from the same folder:
let compiled = try await MLModel.compileModel(at: localDir.appendingPathComponent("Fullstop.mlpackage"))
let model = try MLModel(contentsOf: compiled)
let tokenizer = try await AutoTokenizer.from(modelFolder: localDir)
Inference contract: feed input_ids (shape [1, seq], natural length — no padding)
and attention_mask; the output logits are [1, seq, 6] → argmax per token yields the
label id above. Attach each word's punctuation from the label on its final subword
(token boundaries marked by the XLM-R ▁ U+2581 prefix); skip special tokens.
Provenance
- Source model:
oliverguhr/fullstop-punctuation-multilingual-base(MIT) - Conversion: PyTorch → ONNX → CoreML (fp16). Parity verified 15/15 labels identical to the PyTorch reference across the conversion fixtures; tokenizer parity re-verified on-device 2026-06-26.
- Conversion harness:
restore-bench(standalone repo).
License
MIT — same as the upstream model. Conversion produced by Greg Barbosa.
Citation
@article{guhr-EtAl:2021:fullstop,
title={FullStop: Multilingual Deep Models for Punctuation Prediction},
author={Guhr, Oliver and Schumann, Anne-Kathrin and Bahrmann, Frank and B{\"o}hme, Hans Joachim},
booktitle={Proceedings of the Swiss Text Analytics Conference 2021},
month={June},
year={2021},
address={Winterthur, Switzerland},
publisher={CEUR Workshop Proceedings},
url={http://ceur-ws.org/Vol-2957/sepp_paper4.pdf}
}