Rhett1 commited on
Commit
1ca2ab0
·
verified ·
1 Parent(s): aad004d

Index the two unhealed TT-3 topology controls

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -53,6 +53,23 @@ and at equal size and equal rate the Qwen2.5 architecture is harder to compress
53
  (2.0204 vs 0.6915, both ~7B at ~40%). Scale mitigates the architecture effect but does not
54
  remove it. Each subdirectory's README carries the budget sweep behind its own rate.
55
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
56
  ## ⚠️ These are NOT standard `transformers` checkpoints
57
 
58
  `state_dict_fp16.pt` holds **ring cores**, not dense `nn.Linear` weights. Loading requires
 
53
  (2.0204 vs 0.6915, both ~7B at ~40%). Scale mitigates the architecture effect but does not
54
  remove it. Each subdirectory's README carries the budget sweep behind its own rate.
55
 
56
+ ## Matched-budget TT controls — **UNHEALED**
57
+
58
+ The same 3-site parameterisation with the boundary bond pinned to `r_0 = 1`, i.e. an open
59
+ tensor TRAIN, at the *same per-layer* budget as the ring directories above. They isolate one
60
+ variable — whether closing the ring helps at equal parameters — and nothing else.
61
+
62
+ | Directory | Base model | Params | vs the ring | Licence |
63
+ |---|---|---:|---:|---|
64
+ | [`tt3-llama2-7b/`](./tt3-llama2-7b) | NousResearch/Llama-2-7b-chat-hf | 4.0207B | 4.0231B (−0.06%) | `llama2` |
65
+ | [`tt3-qwen2.5-3b/`](./tt3-qwen2.5-3b) | Qwen/Qwen2.5-3B-Instruct | 2.0766B | 2.0783B (−0.08%) | `qwen-research` — **non-commercial** |
66
+
67
+ > **These two received NO knowledge distillation.** Every ring directory above was healed on
68
+ > 60M tokens; these were not. Do not place their scores in the same table as the healed rows,
69
+ > or next to a dense teacher — that compares two different things. They are control arms,
70
+ > published so the ring-versus-train comparison can be reproduced, not models to deploy. Each
71
+ > README states which layers could not afford ALS and what that costs.
72
+
73
  ## ⚠️ These are NOT standard `transformers` checkpoints
74
 
75
  `state_dict_fp16.pt` holds **ring cores**, not dense `nn.Linear` weights. Loading requires