| --- |
| library_name: transformers |
| tags: [model-merging, mergeability, training-free, quotient-merge-distance] |
| --- |
| |
| # pythia-en-zh-14m |
|
|
| Training-free merged checkpoint from the **Mergeability** sweep |
| (`benchmark/emit_lm.py --real`), produced by weight-space merging of two independently |
| trained parents. No gradient steps were taken. |
|
|
| | field | value | |
| |---|---| |
| | pair_id | `pythia-en-zh-14m` | |
| | merges_in_repo | `average__aligned, average__naive, task_arithmetic__aligned, task_arithmetic__naive, ties__aligned, ties__naive` | |
| | note | `card backfilled by mergeschool.tools.org_tidy` | |
| |
| ## How it was made |
| |
| Parents were loaded, activations extracted on a shared calibration corpus, and the merge applied |
| either **naive** (parents combined in their own coordinates) or **aligned** (parent B carried into |
| parent A's residual-stream basis via `common.alignment.residual_basis_map` before merging — |
| permutation for same-width pairs, orthogonal/rectangular for cross-width). |
| |
| `MS` is the recovery score from `common.eval.mergeability_score` (merged vs. floor vs. ceiling), the |
| same normalisation used by Zhou et al., so it is comparable across rows of the sweep. |
| |
| ## Caveats |
| |
| Sub-1B merges are noisy; an aligned signal where the naive one is noise is the finding, not a bug. |
| Rows without a joint ceiling are floor-relative and must not be read as absolute recovery. |
| |
| Generated automatically — see the [mergeschool repo](https://github.com/suchirsalhan/merge-school). |
| |
