README.md

---
license: mit
language:
  - es
tags:
  - morphology
  - phonology
  - character-level
  - transformer
  - fairseq
  - probing
  - interpretability
library_name: pytorch
---

# Checkpoints: Probing Character-level Transformers for the Spanish L-shaped Morphome

Trained model checkpoints for the paper [*Probing Character-level Transformers
for the Spanish L-shaped Morphome*](https://arxiv.org/abs/2608.03452).
Code and result data: https://github.com/hhuslamlab/probing-morphome

Five character-level inflection transformer architectures (4 encoder + 4
decoder layers, embed dim 256, FFN 1024, 4 heads), trained on the 10L_90NL
split; 12 runs per architecture (run id `X_Y`: X = data split / held-out test
set, Y = training seed).

## Layout

The repo mirrors the `checkpoints/` subtree the probing pipeline expects:

```
checkpoints/
  vanilla/fixed_checkpoints/10L_90NL_<run>-models/checkpoint_best.pt   # fairseq 0.10.2 state_dict
  char_sep/seperate_char_checkpoints/10L_90NL_<run>-models/checkpoint_best.pt
  feature_onehot/independentfeature_fixed/10L_90NL_<run>.nll_0.0000.epoch_103
  feature_invariant/10L_90NL_<run>          # pickled transformer.Transformer
  feature_geometric/10L_90NL_<run>
```

Note: `char_sep` run `1_1` has `checkpoint_last.pt` instead of
`checkpoint_best.pt` (no best checkpoint was saved for that run); the
reproduction script handles this.

## Loading

The `vanilla` and `char_sep` checkpoints are fairseq `transformer`
checkpoints, but fairseq is not required: the probing repo loads them with a
pure-PyTorch reimplementation of the fairseq forward pass
(`probing/extract_representations_vanilla.py`,
`probing/extract_representations_char_sep.py`). The other three architectures
are pickled `transformer.Transformer` objects that need the model source code
from the training repo (available from the authors).

To use with the probing pipeline:

```bash
hf download akki2825/probing-morphome-checkpoints --local-dir "$FEATURE_INFORMED_ROOT"
```

## Citation

See the [GitHub repository](https://github.com/hhuslamlab/probing-morphome)
for citation information.