license: mit
language:
- es
tags:
- morphology
- phonology
- character-level
- transformer
- fairseq
- probing
- interpretability
library_name: pytorch
Checkpoints: Probing Character-level Transformers for the Spanish L-shaped Morphome
Trained model checkpoints for the paper Probing Character-level Transformers for the Spanish L-shaped Morphome. Code and result data: https://github.com/hhuslamlab/probing-morphome
Five character-level inflection transformer architectures (4 encoder + 4
decoder layers, embed dim 256, FFN 1024, 4 heads), trained on the 10L_90NL
split; 12 runs per architecture (run id X_Y: X = data split / held-out test
set, Y = training seed).
Layout
The repo mirrors the checkpoints/ subtree the probing pipeline expects:
checkpoints/
vanilla/fixed_checkpoints/10L_90NL_<run>-models/checkpoint_best.pt # fairseq 0.10.2 state_dict
char_sep/seperate_char_checkpoints/10L_90NL_<run>-models/checkpoint_best.pt
feature_onehot/independentfeature_fixed/10L_90NL_<run>.nll_0.0000.epoch_103
feature_invariant/10L_90NL_<run> # pickled transformer.Transformer
feature_geometric/10L_90NL_<run>
Note: char_sep run 1_1 has checkpoint_last.pt instead of
checkpoint_best.pt (no best checkpoint was saved for that run); the
reproduction script handles this.
Loading
The vanilla and char_sep checkpoints are fairseq transformer
checkpoints, but fairseq is not required: the probing repo loads them with a
pure-PyTorch reimplementation of the fairseq forward pass
(probing/extract_representations_vanilla.py,
probing/extract_representations_char_sep.py). The other three architectures
are pickled transformer.Transformer objects that need the model source code
from the training repo (available from the authors).
To use with the probing pipeline:
hf download akki2825/probing-morphome-checkpoints --local-dir "$FEATURE_INFORMED_ROOT"
Citation
See the GitHub repository for citation information.