Medieval Latin Abbreviation Expander (abbreviationes-v2)

This model is a specialized Seq2Seq transformer designed to expand medieval scribal abbreviations (brevigraphs and suspensions) into their full forms. It was specifically trained to handle the complexities of Latin manuscripts based on a fixed set of special characters used in Burchards Dekret Digital .

The model was developed as part of the projects "Embedding the Past" (LOEWE-Exploration, TU Darmstadt) and "Burchards Dekret Digital" (Akademie der Wissenschaften und der Literatur | Mainz).

Model Logic

Unlike traditional token-based models, this model utilizes ByT5, which operates on raw UTF-8 bytes. This is critical for Medieval Latin, as it allows the model to see and correctly process non-standard Unicode characters such as macrons, brevigraphs, tironian notes or special ligatures, that are often discarded by standard subword tokenizers.

• Input: Abbreviated text lines extracted from PageXML (e.g., ep̅i, conꝓuinciales).
• Output: Fully expanded Unicode text (e.g., episcopi, conprouinciales).

Training & Technical Details

• Architecture: ByT5-Base (encoder-decoder).
• Data Source: ~32,800 paired lines (Abbr/Expan) from the Decretum Burchardi.
• Hardware: Optimized for NVIDIA Blackwell (TF32/BF16 training).
• Training Regime: 15 epochs with a Cosine learning rate scheduler (LR 2e-4).

Performance (Test Set)

Metric	Value
Character Error Rate (CER)	0.45%
Word-Level F1-Score	98.75%
Eval Loss	0.00064

Usage

You can use this model via the Hugging Face pipeline interface for quick inference:

from transformers import pipeline

# Load the expander
expander = pipeline("text2text-generation", model="mschonhardt/abbreviationes-v2")

# Example: "Vt ep̅i conꝓuinciales peregrina iu¬" abbreviated
text = "Vt ep̅i conꝓuinciales peregrina iu¬"
result = expander(text, max_length=512)

print(f"Source: {text}")
print(f"Expanded: {result[0]['generated_text']}")

Citation

If you use this model in your research, please cite the project and the underlying architecture:

@software{schonhardt_michael_2026_expansion,
  author = "Schonhardt, Michael",
  title = "Medieval Latin Abbreviation Expander (abbreviationes-v2)",
  year = "2026",
  publisher = "Zenodo",
  doi = "10.5281/zenodo.18411989",
  url = "[https://doi.org/10.5281/zenodo.18411989](https://doi.org/10.5281/zenodo.18411989)"
}

@article{xue-etal-2022-byt5,
    title = "{B}y{T}5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models",
    author = "Xue, Linting  and
      Barua, Aditya  and
      Constant, Noah  and
      Al-Rfou, Rami  and
      Narang, Sharan  and
      Kale, Mihir  and
      Roberts, Adam  and
      Raffel, Colin",
    editor = "Roark, Brian  and
      Nenkova, Ani",
    journal = "Transactions of the Association for Computational Linguistics",
    volume = "10",
    year = "2022",
    address = "Cambridge, MA",
    publisher = "MIT Press",
    url = "https://aclanthology.org/2022.tacl-1.17/",
    doi = "10.1162/tacl_a_00461",
    pages = "291--306"}