| --- |
| library_name: transformers |
| tags: [model-merging, mergeability, training-free, quotient-merge-distance] |
| --- |
| |
| # polypythia-14m-s1s2__average__aligned |
|
|
| Training-free merged checkpoint from the **Mergeability** sweep |
| (`benchmark/emit_lm.py --real`), produced by weight-space merging of two independently |
| trained parents. No gradient steps were taken. |
|
|
| | field | value | |
| |---|---| |
| | pair_id | `polypythia-14m-s1s2` | |
| | parent_a | `EleutherAI/pythia-14m-seed1` | |
| | parent_b | `EleutherAI/pythia-14m-seed2` | |
| | ceiling | `(floor-relative)` | |
| | operator | `average` | |
| | alignment | `aligned` | |
| | align_method | `permutation` | |
| | regime | `indep_seed` | |
| | eval_langs | `eng` | |
| | nll_merge | `12.0372` | |
| | nll_floor | `5.1597` | |
| | param_coverage | `1.0` | |
| | MS | `None` | |
|
|
| ## How it was made |
|
|
| Parents were loaded, activations extracted on a shared calibration corpus, and the merge applied |
| either **naive** (parents combined in their own coordinates) or **aligned** (parent B carried into |
| parent A's residual-stream basis via `common.alignment.residual_basis_map` before merging — |
| permutation for same-width pairs, orthogonal/rectangular for cross-width). |
|
|
| `MS` is the recovery score from `common.eval.mergeability_score` (merged vs. floor vs. ceiling), the |
| same normalisation used by Zhou et al., so it is comparable across rows of the sweep. |
|
|
| ## Caveats |
|
|
| Sub-1B merges are noisy; an aligned signal where the naive one is noise is the finding, not a bug. |
| Rows without a joint ceiling are floor-relative and must not be read as absolute recovery. |
|
|
| Generated automatically — see the [mergeability repo](https://github.com/suchirsalhan/mergeability). |
|
|
