README.md

---
language:
- ang
license: cc-by-4.0
library_name: spacy
pipeline_tag: token-classification
tags:
- spacy
- token-classification
- old-english
- anglo-saxon
- universal-dependencies
- dependency-parsing
- lemmatization
- historical-languages
- low-resource
model-index:
- name: en_matilda_oe_ud
  results:
  - task:
      type: token-classification
      name: Dependency Parsing
    dataset:
      name: MATILDA Old English UD treebank (held-out test)
      type: matilda-oe-ud
    metrics:
    - type: uas
      name: Unlabelled Attachment Score
      value: 0.8635
    - type: las
      name: Labelled Attachment Score
      value: 0.7943
  - task:
      type: token-classification
      name: Part-of-Speech Tagging
    dataset:
      name: MATILDA Old English UD treebank (held-out test)
      type: matilda-oe-ud
    metrics:
    - type: accuracy
      name: UPOS Accuracy
      value: 0.9510
    - type: accuracy
      name: XPOS Accuracy
      value: 0.9473
    - type: accuracy
      name: Morphological Features Accuracy
      value: 0.8542
    - type: accuracy
      name: Lemma Accuracy
      value: 0.8694
---

# MATILDA — Old English Universal Dependencies pipeline

A spaCy pipeline for **Old English** (Anglo-Saxon, c. 650–1150 CE) covering
part-of-speech tagging, morphological analysis, lemmatisation and Universal
Dependencies parsing.

On a matched evaluation set it improves on the published state of the art
(Martín Arista et al., 2025) by **+5.57 LAS** and **+3.42 UAS**.

## Usage

```bash
pip install https://huggingface.co/<org>/en_matilda_oe_ud/resolve/main/en_matilda_oe_ud-1.0.0-py3-none-any.whl
```

```python
import spacy

nlp = spacy.load("en_matilda_oe_ud")
doc = nlp("Hwæt we Gardena in geardagum þeodcyninga þrym gefrunon .")

for token in doc:
    print(token.text, token.lemma_, token.pos_, token.dep_, token.head.text)
```

The opening of *Beowulf* — "Lo, we have heard of the glory of the Spear-Danes
in days of old" — produces:

| TEXT | LEMMA | UPOS | DEP | HEAD |
|---|---|---|---|---|
| Hwæt | hwæt | ADV | advmod | gefrunon |
| we | we | PRON | nsubj | gefrunon |
| Gardena | Gardenā | ADJ | nsubj | gefrunon |
| in | in | ADP | case | geardagum |
| geardagum | geardeg | NOUN | obl:tmod | gefrunon |
| þeodcyninga | þeodcyning | NOUN | nmod | þrym |
| þrym | þrym | NOUN | obj | gefrunon |
| gefrunon | gefrunon | VERB | ROOT | gefrunon |
| . | . | PUNCT | punct | gefrunon |

This is verse and therefore out of domain, since the treebank is prose. The
analysis is nonetheless mostly right: `gefrunon` is correctly the root, `we` its
subject, `þrym` its object, and `þeodcyninga` a genitive modifier of `þrym`.
`Gardena` is the visible error, a genitive plural attached as a second subject
and tagged `ADJ`.

Note also that the input uses thorn ⟨þ⟩ while the training treebank is normalised
to eth ⟨ð⟩. The floret subword vectors absorb the difference, which is one of the
reasons they were chosen.

## Performance

Held-out test set, 561 sentences, decontaminated against the silver training data.
All figures are percentages.

| Metric | Score |
|---|---|
| TOK | 99.80 |
| XPOS (TAG) | 94.73 |
| UPOS (POS) | 95.10 |
| FEATS (MORPH) | 85.42 |
| LEMMA | 86.94 |
| **UAS** | **86.35** |
| **LAS** | **79.43** |
| Sentence F | 87.44 |

### Against the published baseline

The reference point is the best configuration of Martín Arista et al. (2025),
measured on a matched 285-sentence evaluation set with identical tokenisation:

| Metric | BDCC 2025 | This model | Δ |
|---|---|---|---|
| XPOS | 93.20 | 94.91 | +1.71 |
| UPOS | 92.96 | 95.02 | +2.06 |
| FEATS | 84.21 | 87.77 | +3.56 |
| UAS | 83.24 | 86.66 | +3.42 |
| LAS | 74.23 | 79.80 | +5.57 |
| SENT-F | 71.38 | 84.18 | +12.80 |

For wider context, the best cross-lingual result reported for Old English —
Brigada Villa and Giarda (2023), Old English paired with Icelandic — reached
68.44 UAS and 58.70 LAS.

Lemma accuracy is excluded from this comparison. The 2025 figure of 79.83 was
measured against lemma cells that still carried glosses and word-class markers
inherited from the source spreadsheets, so the two references differ and the
numbers are not comparable. The normalisation that removed those markers is
described below and analysed in the accompanying paper.

## Training data

| Source | Size | Role |
|---|---|---|
| MATILDA gold UD treebank | 87,993 tokens | supervised training |
| MATILDA silver treebanks | 212,471 tokens | supervised training, filtered |
| Dictionary of Old English Corpus | ~3.4 M tokens | floret vectors, tok2vec pretraining |
| Synthetic Old English | 1.73 M words | floret vectors, tok2vec pretraining |

The gold treebank derives from ParCorOEv3 and from doctoral work in the same
project, covering Ælfric's *Catholic Homilies*, the *Anglo-Saxon Chronicle*,
Anglo-Saxon laws, St Mark's Gospel, the Old English *Orosius*, Bede, charters,
martyrologies and Boethius.

Its LEMMA and XPOS columns were normalised before training. 31.7 % of lemma cells
carried a gloss or word-class marker (`and 'and' (CONJ)` rather than `and`), and
the tag column ran to 71 strings for roughly 25 categories. Because the reference
contradicted itself, a model that always guessed the most frequent lemma for each
(form, UPOS) pair could not have exceeded 79.83 % accuracy; after normalisation
that ceiling is 97.81 %. The same normalisation has been applied to the treebank
itself, so the published corpus and this model follow one set of conventions.

The silver portion is machine-annotated and was filtered before use: four
rule-completed treebanks and one heavily underspecified treebank were excluded
entirely, sentences carrying the `dep` placeholder were dropped, and 338 sentences
overlapping the evaluation sets were removed by 4-gram matching. What makes the
remainder valuable is its **manually revised morphology**; its trees came from an
earlier, weaker parser.

The synthetic portion is machine-translated Latin from works attested in pre-1100
Anglo-Saxon libraries. It is used only in the unlabelled channels — vectors and
pretraining — and never as supervised annotation.

## Architecture

```
tokenizer → tok2vec → tagger → morphologizer → trainable_lemmatizer → parser
```

- **Vectors**: floret subword embeddings, 50,000 rows × 300 dimensions, chosen for
  a zero out-of-vocabulary rate on a morphologically rich language with unstable
  spelling. A perfect word-level table built on the entire 3.4 M-token corpus
  still leaves 28 % of the types in the gold treebank unseen.
- **tok2vec**: `MultiHashEmbed` with `MaxoutWindowEncoder`, pretrained on the
  Dictionary of Old English Corpus plus the synthetic text, with orthography
  harmonised to eth before pretraining.
- **Parser**: `TransitionBasedParser` (arc-eager).

There is no transformer. The 2025 baseline study found MobileBERT trained from
scratch reached only 60.17 UAS and 45.51 LAS on this data, at a
parameter-to-token ratio of roughly 1250:1, and transfer from modern English
checkpoints is blocked by the graphemes ⟨æ⟩, ⟨ð⟩, ⟨þ⟩, ⟨ȝ⟩ and ⟨ƿ⟩.

## Limitations

- **The tokenizer is English.** spaCy provides none for Old English, so the
  pipeline's `lang` is `en` and tokenisation uses English rules, as in the
  baseline study. Every trained component is Old English only. This is why the
  package is named `en_matilda_oe_ud` while the repository is tagged `ang`.
- **Non-projective structures fail.** Crossing dependencies arising from Old
  English word order cannot be represented by a transition-based parser. This
  affects relative clauses, clausal modifiers and some coordination.
- **Long-distance relations are weak.** Local and morphologically marked
  relations parse well (`det:poss` 94.06 F1, `case` 93.38, `cc` 90.52); clausal
  complements and oblique variants much less so.
- **Genre skew.** The treebank is prose — homiletic, legal, historical and
  biblical. Poetry is under-represented.
- **Part of the training data is machine-annotated.** If you need a parser whose
  syntax derives solely from human annotation, use the
  `gold-parser-silver-morphology` variant, which costs 1.4 LAS.
- **The parser sometimes emits `dep`.** In Universal Dependencies `dep` means
  "unclassified", so it carries no information. It appears for two reasons. 490
  tokens of the treebank still carry it as an unresolved placeholder, almost all
  of them sentence-initial annal years and chapter numbers. And spaCy's arc-eager
  parser declares `dep` a builtin label and adds it as a backoff whatever the
  training data contains, so a model trained on data with no `dep` at all still
  predicts it. Treat `dep` in the output as "no relation assigned" rather than as
  a relation. Roughly 0.5 % of tokens are affected.

## Citation

Please cite the baseline study, which this model extends:

```bibtex
@article{martinarista2025parsing,
  title   = {Parsing Old English with Universal Dependencies---The Impacts
             of Model Architectures and Dataset Sizes},
  author  = {Mart{\'i}n Arista, Javier and Ojanguren L{\'o}pez, Ana Elvira
             and Dom{\'i}nguez Barrag{\'a}n, Sara},
  journal = {Big Data and Cognitive Computing},
  volume  = {9},
  number  = {8},
  pages   = {199},
  year    = {2025},
  issn    = {2504-2289},
  doi     = {10.3390/bdcc9080199}
}
```

## Licence and funding

CC BY 4.0. Funded by AEI /10.13039/501100011033, grant PID2023-149762NB-100 MCIN.
Nerthus Project, Universidad de La Rioja.