Hugging Face's logo

ProMeText
/
aquilign-multilingual-segmenter

Token Classification
Transformers
Safetensors
bert
medieval-texts
phrase-segmentation
multilingual
Model card Files
xet
 Community
2
carolisteia commited on
Commit
e0ac941
·
verified ·
1 Parent(s): c5ad4f9

Add model card

Browse files
Files changed (1) hide show
  1. README.md +114 -0
README.md ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-sa-4.0
3
+ language:
4
+ - la
5
+ - fr
6
+ - en
7
+ - pt
8
+ - ca
9
+ - es
10
+ - it
11
+ pipeline_tag: token-classification
12
+ library_name: transformers
13
+ tags:
14
+ - medieval-texts
15
+ - phrase-segmentation
16
+ - multilingual
17
+ ---
18
+
19
+ ---
20
+ language:
21
+ - la
22
+ - fr
23
+ - es
24
+ - pt
25
+ - ca
26
+ - en
27
+ - it
28
+ tags:
29
+ - token-classification
30
+ - sentence-segmentation
31
+ - phrase-segmentation
32
+ - historical-texts
33
+ - medieval-texts
34
+ - multilingual
35
+ - digital-humanities
36
+ - computational-humanities
37
+ license: cc-by-nc-sa-4.0
38
+ library_name: transformers
39
+ pipeline_tag: token-classification
40
+ ---
41
+
42
+ # Aquilign Multilingual Segmenter
43
+
44
+ **Aquilign Multilingual Segmenter** is a token-classification model for phrase-level segmentation of medieval and historical texts.
45
+
46
+ The model is designed to detect custom segmentation delimiters in multilingual historical corpora and is used as part of the [Aquilign](https://github.com/ProMeText/Aquilign) alignment workflow.
47
+
48
+ ## Model Description
49
+
50
+ The segmenter is based on a trainable `BertForTokenClassification` model from Hugging Face’s `transformers` library.
51
+
52
+ It was fine-tuned on historical prose from the [Multilingual Segmentation Dataset](https://github.com/ProMeText/multilingual-segmentation-dataset) to identify phrase-level segmentation boundaries.
53
+
54
+ ## Supported Languages
55
+
56
+ - Latin
57
+ - French
58
+ - Castilian
59
+ - Portuguese
60
+ - Catalan
61
+ - English
62
+ - Italian
63
+
64
+ ## Intended Use
65
+
66
+ This model is intended for:
67
+
68
+ - phrase-level segmentation of **medieval texts**
69
+ - preprocessing parallel corpora before alignment
70
+ - multilingual medieval text alignment workflows
71
+ - digital philology and computational humanities research
72
+
73
+ It is especially designed to be used with [Aquilign](https://github.com/ProMeText/Aquilign).
74
+
75
+
76
+ ## Related Resources
77
+
78
+ - [Aquilign alignment tool](https://github.com/ProMeText/Aquilign)
79
+ - [Multilingual Segmentation Dataset](https://github.com/ProMeText/multilingual-segmentation-dataset)
80
+ - [ProMeTEXT GitHub organization](https://github.com/ProMeText)
81
+
82
+ ## Citation
83
+
84
+ If you use this model, please cite the related dataset and publication.
85
+
86
+ ### Dataset
87
+
88
+ ```bibtex
89
+ @dataset{ing2025multilingual,
90
+ author = {Ing, L. and Gille Levenson, M. and Macedo, C.},
91
+ title = {Multilingual Segmentation Dataset for Historical Prose (13th--16th c.)},
92
+ year = {2025},
93
+ publisher = {Zenodo},
94
+ version = {1.0},
95
+ doi = {10.5281/zenodo.16992629},
96
+ url = {https://doi.org/10.5281/zenodo.16992629},
97
+ license = {Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International}
98
+ }
99
+ ```
100
+
101
+ ### Related Publication
102
+ ```bibtex
103
+ @inproceedings{ing-etal-2026-phrase,
104
+ title = {Phrase-Level Segmentation on Medieval Corpora for Aligning Multilingual Texts},
105
+ author = {Ing, Lucence and Gille Levenson, Matthias and Macedo, Carolina},
106
+ booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026)},
107
+ month = {May},
108
+ year = {2026},
109
+ pages = {936--946},
110
+ address = {Palma, Mallorca, Spain},
111
+ publisher = {European Language Resources Association (ELRA)},
112
+ doi = {10.63317/32huzuuokpfr}
113
+ }
114
+ ```