Upload svd_llm2/heal/llama2-7b-base/README.md with huggingface_hub

svd_llm2/heal/llama2-7b-base/README.md

@@ -0,0 +1,55 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-7b-hf
+tags:
+  - svd
+  - low-rank
+  - model-compression
+  - llama2
+---
+
+# SVD-LLM V2 (NAACL 2025): LLaMA-2-7B base @ 4.02B, KD-healed
+
+SVD-LLM V2: heterogeneous compression-ratio allocation from theoretical truncation loss (Algorithm 1) + loss-optimized weight truncation (Algorithm 2). Official code was never released; this is our faithful implementation of the published algorithms (per-layer ranks in ranks.json show the allocation).
+
+- **4.023B params** (4,023,134,464; down from 6.74B, ~40% reduction)
+- Healed with pure forward-KL distillation from the dense 7B base teacher: **60M tokens**,
+  held-out KL **2.0105 -> 0.1935**, probe MMLU (limit 40/subtask) **24.34% -> 35.66%**
+- ⚠️ **Probe-only evaluation**: full-benchmark evaluation (full MMLU / HellaSwag / BoolQ /
+  TriviaQA / GSM8K) was intentionally NOT run for the base-model series. All numbers below
+  come from the same probe harness (lm-eval 0.4.12, MMLU limit=40 questions/subtask,
+  n=2280, seed 1234, fp16) and are mutually comparable.
+
+| Reference (same probe harness) | probe MMLU |
+|---|---:|
+| unhealed 4.02B (same structure, unhealed sibling dir) | 24.34% |
+| **this checkpoint (healed 60M)** | **35.66%** |
+| TR-3 base (same 4.0231B budget + same healing recipe) | 41.97% |
+| FP16 dense 6.74B base teacher | 47.41%* |
+
+\* teacher number as measured by this heal run's own pre-eval; the TR-3 run's pre-eval of the
+same probe set gave 47.50% (0.09pp run-to-run wobble, same harness and protocol file
+`eval_protocol_llama2_7b_base.json`).
+
+## How to load
+
+```python
+import json
+from lowrank_linear import build, materialize_all
+
+model = build("NousResearch/Llama-2-7b-hf",
+              json.load(open("ranks.json")), "state_dict_fp16.pt",
+              device="cuda")
+# for generation/eval: fuse the factors first (inference-only)
+materialize_all(model)
+```
+
+`ranks.json` maps each compressed projection path to its integer rank; untouched weights are
+stock LLaMA-2-7B base tensors. The full healing-provenance record (data mixture & revisions,
+decontamination, SHA256 of inputs/checkpoint, 5M milestone trajectory) is in `provenance.json`.
+
+## License
+
+Derived from Meta's LLaMA 2 (`NousResearch/Llama-2-7b-hf`, an unofficial redistribution of
+the same weights). Subject to the **LLaMA 2 Community License** — see
+https://ai.meta.com/llama/license/.