File size: 2,101 Bytes
79dab27 6e1b78f 79dab27 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
license: mit
language:
- es
tags:
- morphology
- phonology
- character-level
- transformer
- fairseq
- probing
- interpretability
library_name: pytorch
---
# Checkpoints: Probing Character-level Transformers for the Spanish L-shaped Morphome
Trained model checkpoints for the paper [*Probing Character-level Transformers
for the Spanish L-shaped Morphome*](https://arxiv.org/abs/2608.03452).
Code and result data: https://github.com/hhuslamlab/probing-morphome
Five character-level inflection transformer architectures (4 encoder + 4
decoder layers, embed dim 256, FFN 1024, 4 heads), trained on the 10L_90NL
split; 12 runs per architecture (run id `X_Y`: X = data split / held-out test
set, Y = training seed).
## Layout
The repo mirrors the `checkpoints/` subtree the probing pipeline expects:
```
checkpoints/
vanilla/fixed_checkpoints/10L_90NL_<run>-models/checkpoint_best.pt # fairseq 0.10.2 state_dict
char_sep/seperate_char_checkpoints/10L_90NL_<run>-models/checkpoint_best.pt
feature_onehot/independentfeature_fixed/10L_90NL_<run>.nll_0.0000.epoch_103
feature_invariant/10L_90NL_<run> # pickled transformer.Transformer
feature_geometric/10L_90NL_<run>
```
Note: `char_sep` run `1_1` has `checkpoint_last.pt` instead of
`checkpoint_best.pt` (no best checkpoint was saved for that run); the
reproduction script handles this.
## Loading
The `vanilla` and `char_sep` checkpoints are fairseq `transformer`
checkpoints, but fairseq is not required: the probing repo loads them with a
pure-PyTorch reimplementation of the fairseq forward pass
(`probing/extract_representations_vanilla.py`,
`probing/extract_representations_char_sep.py`). The other three architectures
are pickled `transformer.Transformer` objects that need the model source code
from the training repo (available from the authors).
To use with the probing pipeline:
```bash
hf download akki2825/probing-morphome-checkpoints --local-dir "$FEATURE_INFORMED_ROOT"
```
## Citation
See the [GitHub repository](https://github.com/hhuslamlab/probing-morphome)
for citation information.
|