Rhett1 commited on
Commit
29f34b1
·
verified ·
1 Parent(s): b9f25d3

Upload asvd_llm/unheal/llama2-7b-base/README.md with huggingface_hub

Browse files
asvd_llm/unheal/llama2-7b-base/README.md ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama2
3
+ base_model: NousResearch/Llama-2-7b-hf
4
+ tags: [asvd, low-rank, model-compression, llama2, negative-result]
5
+ ---
6
+
7
+ # ⚠️ ASVD @ 4.02B, UNHEALED (LLaMA-2-7B base) — DEGENERATE OUTPUT (archived negative result)
8
+
9
+ **DO NOT USE THIS CHECKPOINT FOR INFERENCE.** The compression run that produced it yielded a
10
+ **degenerate model**: probe MMLU (limit 40/subtask, n=2280, seed 1234, fp16) = **23.42%**,
11
+ below the 25% chance level, and a pre-heal health gate initially refused it with KL=NaN.
12
+ Even 60M tokens of KD-healing could not restore capability (26.27% — see
13
+ `../heal/llama2-7b-base/README.md`).
14
+
15
+ Archived as a documented negative result. The ASVD chat instance
16
+ (`asvd_llm/unheal/llama2-7b/`) is healthy; the failure is specific to this base run
17
+ (abs_mean scaling, alpha=0.5, seed=233, n_calib=256, ppl sensitivity, ratio target 0.589).
18
+
19
+ - Factored params: **4,020,677,888** (factored 3,532,823,808 + untouched 487,854,080;
20
+ budget 4.0231B, matched to the TR-3 llama2-7b row)
21
+
22
+ Load with `lowrank_linear.py` in this dir (identical recipe to the sibling dirs).
23
+ Compression provenance is in `provenance.json`.