Create README.md

README.md

@@ -0,0 +1,243 @@
+---
+language:
+- ang
+license: cc-by-4.0
+library_name: spacy
+pipeline_tag: token-classification
+tags:
+- spacy
+- token-classification
+- old-english
+- anglo-saxon
+- universal-dependencies
+- dependency-parsing
+- lemmatization
+- historical-languages
+- low-resource
+model-index:
+- name: en_matilda_oe_ud
+  results:
+  - task:
+      type: token-classification
+      name: Dependency Parsing
+    dataset:
+      name: MATILDA Old English UD treebank (held-out test)
+      type: matilda-oe-ud
+    metrics:
+    - type: uas
+      name: Unlabelled Attachment Score
+      value: 0.8635
+    - type: las
+      name: Labelled Attachment Score
+      value: 0.7943
+  - task:
+      type: token-classification
+      name: Part-of-Speech Tagging
+    dataset:
+      name: MATILDA Old English UD treebank (held-out test)
+      type: matilda-oe-ud
+    metrics:
+    - type: accuracy
+      name: UPOS Accuracy
+      value: 0.9510
+    - type: accuracy
+      name: XPOS Accuracy
+      value: 0.9473
+    - type: accuracy
+      name: Morphological Features Accuracy
+      value: 0.8542
+    - type: accuracy
+      name: Lemma Accuracy
+      value: 0.8694
+---
+
+# MATILDA — Old English Universal Dependencies pipeline
+
+A spaCy pipeline for **Old English** (Anglo-Saxon, c. 650–1150 CE) covering
+part-of-speech tagging, morphological analysis, lemmatisation and Universal
+Dependencies parsing.
+
+On a matched evaluation set it improves on the published state of the art
+(Martín Arista et al., 2025) by **+5.57 LAS** and **+3.42 UAS**.
+
+## Usage
+
+```bash
+pip install https://huggingface.co/<org>/en_matilda_oe_ud/resolve/main/en_matilda_oe_ud-1.0.0-py3-none-any.whl
+```
+
+```python
+import spacy
+
+nlp = spacy.load("en_matilda_oe_ud")
+doc = nlp("Hwæt we Gardena in geardagum þeodcyninga þrym gefrunon .")
+
+for token in doc:
+    print(token.text, token.lemma_, token.pos_, token.dep_, token.head.text)
+```
+
+The opening of *Beowulf* — "Lo, we have heard of the glory of the Spear-Danes
+in days of old" — produces:
+
+| TEXT | LEMMA | UPOS | DEP | HEAD |
+|---|---|---|---|---|
+| Hwæt | hwæt | ADV | advmod | gefrunon |
+| we | we | PRON | nsubj | gefrunon |
+| Gardena | Gardenā | ADJ | nsubj | gefrunon |
+| in | in | ADP | case | geardagum |
+| geardagum | geardeg | NOUN | obl:tmod | gefrunon |
+| þeodcyninga | þeodcyning | NOUN | nmod | þrym |
+| þrym | þrym | NOUN | obj | gefrunon |
+| gefrunon | gefrunon | VERB | ROOT | gefrunon |
+| . | . | PUNCT | punct | gefrunon |
+
+This is verse and therefore out of domain, since the treebank is prose. The
+analysis is nonetheless mostly right: `gefrunon` is correctly the root, `we` its
+subject, `þrym` its object, and `þeodcyninga` a genitive modifier of `þrym`.
+`Gardena` is the visible error, a genitive plural attached as a second subject
+and tagged `ADJ`.
+
+Note also that the input uses thorn ⟨þ⟩ while the training treebank is normalised
+to eth ⟨ð⟩. The floret subword vectors absorb the difference, which is one of the
+reasons they were chosen.
+
+## Performance
+
+Held-out test set, 561 sentences, decontaminated against the silver training data.
+All figures are percentages.
+
+| Metric | Score |
+|---|---|
+| TOK | 99.80 |
+| XPOS (TAG) | 94.73 |
+| UPOS (POS) | 95.10 |
+| FEATS (MORPH) | 85.42 |
+| LEMMA | 86.94 |
+| **UAS** | **86.35** |
+| **LAS** | **79.43** |
+| Sentence F | 87.44 |
+
+### Against the published baseline
+
+The reference point is the best configuration of Martín Arista et al. (2025),
+measured on a matched 285-sentence evaluation set with identical tokenisation:
+
+| Metric | BDCC 2025 | This model | Δ |
+|---|---|---|---|
+| XPOS | 93.20 | 94.91 | +1.71 |
+| UPOS | 92.96 | 95.02 | +2.06 |
+| FEATS | 84.21 | 87.77 | +3.56 |
+| UAS | 83.24 | 86.66 | +3.42 |
+| LAS | 74.23 | 79.80 | +5.57 |
+| SENT-F | 71.38 | 84.18 | +12.80 |
+
+For wider context, the best cross-lingual result reported for Old English —
+Brigada Villa and Giarda (2023), Old English paired with Icelandic — reached
+68.44 UAS and 58.70 LAS.
+
+Lemma accuracy is excluded from this comparison. The 2025 figure of 79.83 was
+measured against lemma cells that still carried glosses and word-class markers
+inherited from the source spreadsheets, so the two references differ and the
+numbers are not comparable. The normalisation that removed those markers is
+described below and analysed in the accompanying paper.
+
+## Training data
+
+| Source | Size | Role |
+|---|---|---|
+| MATILDA gold UD treebank | 87,993 tokens | supervised training |
+| MATILDA silver treebanks | 212,471 tokens | supervised training, filtered |
+| Dictionary of Old English Corpus | ~3.4 M tokens | floret vectors, tok2vec pretraining |
+| Synthetic Old English | 1.73 M words | floret vectors, tok2vec pretraining |
+
+The gold treebank derives from ParCorOEv3 and from doctoral work in the same
+project, covering Ælfric's *Catholic Homilies*, the *Anglo-Saxon Chronicle*,
+Anglo-Saxon laws, St Mark's Gospel, the Old English *Orosius*, Bede, charters,
+martyrologies and Boethius.
+
+Its LEMMA and XPOS columns were normalised before training. 31.7 % of lemma cells
+carried a gloss or word-class marker (`and 'and' (CONJ)` rather than `and`), and
+the tag column ran to 71 strings for roughly 25 categories. Because the reference
+contradicted itself, a model that always guessed the most frequent lemma for each
+(form, UPOS) pair could not have exceeded 79.83 % accuracy; after normalisation
+that ceiling is 97.81 %. The same normalisation has been applied to the treebank
+itself, so the published corpus and this model follow one set of conventions.
+
+The silver portion is machine-annotated and was filtered before use: four
+rule-completed treebanks and one heavily underspecified treebank were excluded
+entirely, sentences carrying the `dep` placeholder were dropped, and 338 sentences
+overlapping the evaluation sets were removed by 4-gram matching. What makes the
+remainder valuable is its **manually revised morphology**; its trees came from an
+earlier, weaker parser.
+
+The synthetic portion is machine-translated Latin from works attested in pre-1100
+Anglo-Saxon libraries. It is used only in the unlabelled channels — vectors and
+pretraining — and never as supervised annotation.
+
+## Architecture
+
+```
+tokenizer → tok2vec → tagger → morphologizer → trainable_lemmatizer → parser
+```
+
+- **Vectors**: floret subword embeddings, 50,000 rows × 300 dimensions, chosen for
+  a zero out-of-vocabulary rate on a morphologically rich language with unstable
+  spelling. A perfect word-level table built on the entire 3.4 M-token corpus
+  still leaves 28 % of the types in the gold treebank unseen.
+- **tok2vec**: `MultiHashEmbed` with `MaxoutWindowEncoder`, pretrained on the
+  Dictionary of Old English Corpus plus the synthetic text, with orthography
+  harmonised to eth before pretraining.
+- **Parser**: `TransitionBasedParser` (arc-eager).
+
+There is no transformer. The 2025 baseline study found MobileBERT trained from
+scratch reached only 60.17 UAS and 45.51 LAS on this data, at a
+parameter-to-token ratio of roughly 1250:1, and transfer from modern English
+checkpoints is blocked by the graphemes ⟨æ⟩, ⟨ð⟩, ⟨þ⟩, ⟨ȝ⟩ and ⟨ƿ⟩.
+
+## Limitations
+
+- **The tokenizer is English.** spaCy provides none for Old English, so the
+  pipeline's `lang` is `en` and tokenisation uses English rules, as in the
+  baseline study. Every trained component is Old English only. This is why the
+  package is named `en_matilda_oe_ud` while the repository is tagged `ang`.
+- **Non-projective structures fail.** Crossing dependencies arising from Old
+  English word order cannot be represented by a transition-based parser. This
+  affects relative clauses, clausal modifiers and some coordination.
+- **Long-distance relations are weak.** Local and morphologically marked
+  relations parse well (`det:poss` 94.06 F1, `case` 93.38, `cc` 90.52); clausal
+  complements and oblique variants much less so.
+- **Genre skew.** The treebank is prose — homiletic, legal, historical and
+  biblical. Poetry is not represented.
+- **The parser sometimes emits `dep`.** In Universal Dependencies `dep` means
+  "unclassified", so it carries no information. It appears for two reasons. 490
+  tokens of the treebank still carry it as an unresolved placeholder, almost all
+  of them sentence-initial annal years and chapter numbers. And spaCy's arc-eager
+  parser declares `dep` a builtin label and adds it as a backoff whatever the
+  training data contains, so a model trained on data with no `dep` at all still
+  predicts it. Treat `dep` in the output as "no relation assigned" rather than as
+  a relation. Roughly 0.5 % of tokens are affected.
+
+## Citation
+
+Please cite the baseline study, which this model extends:
+
+```bibtex
+@article{martinarista2025parsing,
+  title   = {Parsing Old English with Universal Dependencies---The Impacts
+             of Model Architectures and Dataset Sizes},
+  author  = {Mart{\'i}n Arista, Javier and Ojanguren L{\'o}pez, Ana Elvira
+             and Dom{\'i}nguez Barrag{\'a}n, Sara},
+  journal = {Big Data and Cognitive Computing},
+  volume  = {9},
+  number  = {8},
+  pages   = {199},
+  year    = {2025},
+  issn    = {2504-2289},
+  doi     = {10.3390/bdcc9080199}
+}
+```
+
+## Licence and funding
+
+CC BY 4.0. Funded by AEI /10.13039/501100011033, grant PID2023-149762NB-100 MCIN.
+Nerthus Project, Universidad de La Rioja.