---
license: other
license_name: see-each-subdirectory
tags: [tensor-decomposition, svd, low-rank, model-compression, knowledge-distillation]
---

# TR-3 study models: main compressed models and matched-budget baselines

Layout: **<method> / <stage> / <model>**. Method families are `tr3` (our 3-site tensor ring +
sparse residual), `tt3` (3-site tensor train ablation), `svd_llm`, `svd_llm_v2` and `asvd_llm`
(matched-budget baselines). Stages are `heal`
(KD-distilled, deployable) and `unheal` (raw post-compression, for measuring the healing delta).

| method | stage | model | params (B) | licence | KD result (probe) |
|---|---|---|---:|---|---|
| tr3 | heal | [`qwen2.5-7b`](tr3/heal/qwen2.5-7b) | 5.187 | qwen-research (non-commercial) | see dir |
| tt3 | heal | [`llama2-7b`](tt3/heal/llama2-7b) | 4.021 | llama2 | see dir |
| tt3 | heal | [`qwen2.5-3b`](tt3/heal/qwen2.5-3b) | 2.388 | qwen-research (non-commercial) | see dir |
| tt3 | unheal | [`llama2-7b`](tt3/unheal/llama2-7b) | 4.021 | llama2 | — |
| tt3 | unheal | [`qwen2.5-3b`](tt3/unheal/qwen2.5-3b) | 2.388 | qwen-research (non-commercial) | — |
| tr3 | heal | [`llama2-7b`](tr3/heal/llama2-7b) | 4.023 | llama2 | MMLU 43.21* |
| tr3 | heal | [`llama2-13b`](tr3/heal/llama2-13b) | 7.740 | llama2 | see dir |
| tr3 | heal | [`qwen2.5-3b`](tr3/heal/qwen2.5-3b) | 2.078 | qwen-research (non-commercial) | see dir |
| tr3 | unheal | [`llama2-13b`](tr3/unheal/llama2-13b) | 7.740 | llama2 | — |
| tr3 | unheal | [`llama2-7b`](tr3/unheal/llama2-7b) | (off-cluster archive) | llama2 | — |
| svd_llm | heal | [`llama2-7b`](svd_llm/heal/llama2-7b) | 4.022 | llama2 | MMLU 39.47 |
| svd_llm | heal | [`llama2-13b`](svd_llm/heal/llama2-13b) | 7.738 | llama2 | MMLU 46.54 |
| svd_llm | unheal | [`llama2-7b`](svd_llm/unheal/llama2-7b) | 4.022 | llama2 | — |
| svd_llm | unheal | [`llama2-13b`](svd_llm/unheal/llama2-13b) | 7.738 | llama2 | — |
| svd_llm_v2 ‡ | heal | [`llama2-7b`](svd_llm2/heal/llama2-7b) | 4.023 | llama2 | MMLU 40.22 |
| svd_llm_v2 ‡ | heal | [`llama2-13b`](svd_llm2/heal/llama2-13b) | 7.740 | llama2 | MMLU 44.96 |
| svd_llm_v2 ‡ | unheal | [`llama2-7b`](svd_llm2/unheal/llama2-7b) | 4.023 | llama2 | — |
| svd_llm_v2 ‡ | unheal | [`llama2-13b`](svd_llm2/unheal/llama2-13b) | 7.740 | llama2 | — |
| asvd_llm | heal | [`llama2-7b`](asvd_llm/heal/llama2-7b) | 4.023 | llama2 | MMLU 35.09 |
| asvd_llm | heal | [`llama2-13b`](asvd_llm/heal/llama2-13b) | 7.736 | llama2 | MMLU 42.46 |
| asvd_llm | unheal | [`llama2-7b`](asvd_llm/unheal/llama2-7b) | 4.023 | llama2 | — |
| asvd_llm | unheal | [`llama2-13b`](asvd_llm/unheal/llama2-13b) | 7.736 | llama2 | — |

\* full-benchmark lm-eval 0.4.12 result; SVD probe columns are limit-40 healing probes here —
each dir's README/provenance carries the precise protocol.

‡ Official SVD-LLM V2 code was never released; `svd_llm_v2` artifacts are our faithful
implementation of the published algorithms (per-layer ranks document the allocation).

## ⚠️ These are NOT standard `transformers` checkpoints

`tr3/**` stores **ring cores** (load with each dir's `tensor_formats.py`); `svd_llm*/**` stores
**low-rank factor pairs** (load with each dir's `lowrank_linear.py`). Weight files are
uniformly named `state_dict_fp16.pt`. `from_pretrained()` will not work on this repo.