---
language: la
library_name: transformers
license: cc-by-sa-4.0
base_model: google/byt5-base
pipeline_tag: text2text-generation
tags:
- latin
- medieval-latin
- legal-history
- digital-humanities
- ocr-postprocessing
- expansion
- pagexml
- htr
widget:
- text: "Vt ep̅i conꝓuinciales peregrina iu¬"
---

# Medieval Latin Abbreviation Expander (abbreviationes-v2)

This model is a specialized Seq2Seq transformer designed to expand medieval scribal abbreviations (brevigraphs and suspensions) into their full forms. It was specifically trained to handle the complexities of Latin manuscripts based on a fixed set of special characters used in [Burchards Dekret Digital](www.burchards-dekret-digital.de) .

The model was developed as part of the projects **"Embedding the Past"** (LOEWE-Exploration, TU Darmstadt) and **"Burchards Dekret Digital"** (Akademie der Wissenschaften und der Literatur | Mainz).

## Model Logic
Unlike traditional token-based models, this model utilizes **ByT5**, which operates on raw UTF-8 bytes. This is critical for Medieval Latin, as it allows the model to see and correctly process non-standard Unicode characters such as macrons, brevigraphs, tironian notes or special ligatures, that are often discarded by standard subword tokenizers.

- **Input:** Abbreviated text lines extracted from PageXML (e.g., `ep̅i`, `conꝓuinciales`).
- **Output:** Fully expanded Unicode text (e.g., `episcopi`, `conprouinciales`).

## Training & Technical Details
- **Architecture:** [ByT5-Base](https://huggingface.co/google/byt5-base) (encoder-decoder).
- **Data Source:** ~32,800 paired lines (Abbr/Expan) from the *Decretum Burchardi*.
- **Hardware:** Optimized for NVIDIA Blackwell (TF32/BF16 training).
- **Training Regime:** 15 epochs with a Cosine learning rate scheduler (LR 2e-4).

### Performance (Test Set)
| Metric | Value |
| :--- | :--- |
| **Character Error Rate (CER)** | **0.45%** |
| **Word-Level F1-Score** | **98.75%** |
| **Eval Loss** | 0.00064 |

## Usage

You can use this model via the Hugging Face `pipeline` interface for quick inference:

```python
from transformers import pipeline

# Load the expander
expander = pipeline("text2text-generation", model="mschonhardt/abbreviationes-v2")

# Example: "Vt ep̅i conꝓuinciales peregrina iu¬" abbreviated
text = "Vt ep̅i conꝓuinciales peregrina iu¬"
result = expander(text, max_length=512)

print(f"Source: {text}")
print(f"Expanded: {result[0]['generated_text']}")
```

## Citation

If you use this model in your research, please cite the project and the underlying architecture:

```bibtex
@software{schonhardt_michael_2026_expansion,
  author = "Schonhardt, Michael",
  title = "Medieval Latin Abbreviation Expander (abbreviationes-v2)",
  year = "2026",
  publisher = "Zenodo",
  doi = "10.5281/zenodo.18411989",
  url = "[https://doi.org/10.5281/zenodo.18411989](https://doi.org/10.5281/zenodo.18411989)"
}

@article{xue-etal-2022-byt5,
    title = "{B}y{T}5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models",
    author = "Xue, Linting  and
      Barua, Aditya  and
      Constant, Noah  and
      Al-Rfou, Rami  and
      Narang, Sharan  and
      Kale, Mihir  and
      Roberts, Adam  and
      Raffel, Colin",
    editor = "Roark, Brian  and
      Nenkova, Ani",
    journal = "Transactions of the Association for Computational Linguistics",
    volume = "10",
    year = "2022",
    address = "Cambridge, MA",
    publisher = "MIT Press",
    url = "https://aclanthology.org/2022.tacl-1.17/",
    doi = "10.1162/tacl_a_00461",
    pages = "291--306"}
    
```