|
|
---
|
|
|
language: la
|
|
|
library_name: transformers
|
|
|
license: cc-by-sa-4.0
|
|
|
base_model: google/byt5-large
|
|
|
pipeline_tag: text2text-generation
|
|
|
tags:
|
|
|
- latin
|
|
|
- medieval-latin
|
|
|
- normalization
|
|
|
- legal-history
|
|
|
- digital-humanities
|
|
|
- ocr-postprocessing
|
|
|
widget:
|
|
|
- text: "viiii vt in sabbato sancto ieiunium ante noctis initium non soluatur"
|
|
|
example_title: "Medieval Legal Latin"
|
|
|
---
|
|
|
|
|
|
# Medieval Latin Normalizer (ByT5-Large)
|
|
|
|
|
|
This model is a **ByT5-Large** transformer fine-tuned to normalize medieval Latin text. It transforms diplomatic transcriptions or noisy HTR/OCR output into a standardized normalized orthography, facilitating better downstream processing such as POS tagging, lemmatization, and linguistic analysis. The model was developed as part of the following research projects **"Embedding the Past"** (LOEWE-Exploration, TU Darmstadt) and **"Burchards Dekret Digital"** (Academy of Sciences and Literature | Mainz).
|
|
|
|
|
|
## Model Logic
|
|
|
Medieval Latin normalization involves handling inconsistent orthography (e.g., `u/v`, `i/j`, or `ae/e` variations) and resolving phonetic spellings common in legal and ecclesiastical manuscripts.
|
|
|
|
|
|
By using **ByT5-Large**, the model operates directly on **UTF-8 bytes**. This is a significant advantage for Medieval Latin, as it allows the model to process non-standard characters without the information loss typical of subword tokenizers (like BERT or standard T5).
|
|
|
|
|
|
- **Input:** Raw/Diplomatic medieval Latin text.
|
|
|
- **Output:** Standardized/Normalized Latin text.
|
|
|
|
|
|
## Technical Specifications
|
|
|
- **Architecture:** [ByT5-Large](https://huggingface.co/google/byt5-large) (~1.2B parameters).
|
|
|
- **Hardware:** Trained on NVIDIA Blackwell GPUs using `bf16` precision and `adamw_torch_fused` optimization.
|
|
|
- **Training Parameters:**
|
|
|
- **Learning Rate:** 2e-4
|
|
|
- **Epochs:** 20
|
|
|
- **Label Smoothing:** 0.1 (to improve robustness against transcription noise).
|
|
|
- **Batch Size:** 48.
|
|
|
|
|
|
## Performance (Test Set)
|
|
|
The model was evaluated on a held-out test set (85 samples) from medieval legal corpora:
|
|
|
|
|
|
| Metric | Value |
|
|
|
| :--- | :--- |
|
|
|
| **Character Error Rate (CER)** | **1.62%** |
|
|
|
| **Word-Level F1-Score** | **94.12%** |
|
|
|
| **Evaluation Loss** | 0.143 |
|
|
|
|
|
|
## Usage
|
|
|
You can utilize this model through the Hugging Face `pipeline` API:
|
|
|
|
|
|
```python
|
|
|
from transformers import pipeline
|
|
|
|
|
|
# Initialize the normalizer
|
|
|
normalizer = pipeline("text2text-generation", model="mschonhardt/latin-normalizer")
|
|
|
|
|
|
# Example input
|
|
|
raw_text = "viiii vt in sabbato sancto ieiunium ante noctis initium non soluatur"
|
|
|
result = normalizer(raw_text, max_length=128)
|
|
|
|
|
|
print(f"Normalized: {result[0]['generated_text']}")
|
|
|
|
|
|
```
|
|
|
|
|
|
## Citation
|
|
|
|
|
|
If you use this model in your research, please cite:
|
|
|
|
|
|
```bibtex
|
|
|
@software{schonhardt_michael_2026_normalization,
|
|
|
author = "Schonhardt, Michael",
|
|
|
title = "Medieval Latin Normalizer",
|
|
|
year = "2026",
|
|
|
publisher = "Zenodo",
|
|
|
doi = "10.5281/zenodo.18416639",
|
|
|
url = "https://doi.org/10.5281/zenodo.18416639"
|
|
|
}
|
|
|
|
|
|
@article{xue-etal-2022-byt5,
|
|
|
title = "{B}y{T}5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models",
|
|
|
author = "Xue, Linting and
|
|
|
Barua, Aditya and
|
|
|
Constant, Noah and
|
|
|
Al-Rfou, Rami and
|
|
|
Narang, Sharan and
|
|
|
Kale, Mihir and
|
|
|
Roberts, Adam and
|
|
|
Raffel, Colin",
|
|
|
editor = "Roark, Brian and
|
|
|
Nenkova, Ani",
|
|
|
journal = "Transactions of the Association for Computational Linguistics",
|
|
|
volume = "10",
|
|
|
year = "2022",
|
|
|
address = "Cambridge, MA",
|
|
|
publisher = "MIT Press",
|
|
|
url = "https://aclanthology.org/2022.tacl-1.17/",
|
|
|
doi = "10.1162/tacl_a_00461",
|
|
|
pages = "291--306"}
|
|
|
|
|
|
``` |
