Index the two unhealed TT-3 topology controls
Browse files
README.md
CHANGED
|
@@ -53,6 +53,23 @@ and at equal size and equal rate the Qwen2.5 architecture is harder to compress
|
|
| 53 |
(2.0204 vs 0.6915, both ~7B at ~40%). Scale mitigates the architecture effect but does not
|
| 54 |
remove it. Each subdirectory's README carries the budget sweep behind its own rate.
|
| 55 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
## ⚠️ These are NOT standard `transformers` checkpoints
|
| 57 |
|
| 58 |
`state_dict_fp16.pt` holds **ring cores**, not dense `nn.Linear` weights. Loading requires
|
|
|
|
| 53 |
(2.0204 vs 0.6915, both ~7B at ~40%). Scale mitigates the architecture effect but does not
|
| 54 |
remove it. Each subdirectory's README carries the budget sweep behind its own rate.
|
| 55 |
|
| 56 |
+
## Matched-budget TT controls — **UNHEALED**
|
| 57 |
+
|
| 58 |
+
The same 3-site parameterisation with the boundary bond pinned to `r_0 = 1`, i.e. an open
|
| 59 |
+
tensor TRAIN, at the *same per-layer* budget as the ring directories above. They isolate one
|
| 60 |
+
variable — whether closing the ring helps at equal parameters — and nothing else.
|
| 61 |
+
|
| 62 |
+
| Directory | Base model | Params | vs the ring | Licence |
|
| 63 |
+
|---|---|---:|---:|---|
|
| 64 |
+
| [`tt3-llama2-7b/`](./tt3-llama2-7b) | NousResearch/Llama-2-7b-chat-hf | 4.0207B | 4.0231B (−0.06%) | `llama2` |
|
| 65 |
+
| [`tt3-qwen2.5-3b/`](./tt3-qwen2.5-3b) | Qwen/Qwen2.5-3B-Instruct | 2.0766B | 2.0783B (−0.08%) | `qwen-research` — **non-commercial** |
|
| 66 |
+
|
| 67 |
+
> **These two received NO knowledge distillation.** Every ring directory above was healed on
|
| 68 |
+
> 60M tokens; these were not. Do not place their scores in the same table as the healed rows,
|
| 69 |
+
> or next to a dense teacher — that compares two different things. They are control arms,
|
| 70 |
+
> published so the ring-versus-train comparison can be reproduced, not models to deploy. Each
|
| 71 |
+
> README states which layers could not afford ALS and what that costs.
|
| 72 |
+
|
| 73 |
## ⚠️ These are NOT standard `transformers` checkpoints
|
| 74 |
|
| 75 |
`state_dict_fp16.pt` holds **ring cores**, not dense `nn.Linear` weights. Loading requires
|