Upload svd_llm/heal/llama2-7b-base/README.md with huggingface_hub

svd_llm/heal/llama2-7b-base/README.md

@@ -0,0 +1,56 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-7b-hf
+tags:
+  - svd
+  - low-rank
+  - model-compression
+  - llama2
+---
+
+# SVD-LLM (ICLR 2025): LLaMA-2-7B base @ 4.02B, KD-healed
+
+SVD-LLM: truncation-aware data whitening (256 wikitext2 samples) + closed-form parameter update, official code ([AIoT-MLSys-Lab/SVD-LLM @ 7538cca](https://github.com/AIoT-MLSys-Lab/SVD-LLM)), uniform 41.93% removal across all 7 projection types of all 32 layers.
+
+- **4.022B params** (4,021,989,376; down from 6.74B, ~40% reduction)
+- Healed with pure forward-KL distillation from the dense 7B base teacher: **60M tokens**,
+  held-out KL **2.0439 -> 0.2089**, probe MMLU (limit 40/subtask) **25.61% -> 34.12%**
+- ⚠️ **Probe-only evaluation**: full-benchmark evaluation (full MMLU / HellaSwag / BoolQ /
+  TriviaQA / GSM8K) was intentionally NOT run for the base-model series. All numbers below
+  come from the same probe harness (lm-eval 0.4.12, MMLU limit=40 questions/subtask,
+  n=2280, seed 1234, fp16) and are mutually comparable.
+
+| Reference (same probe harness) | probe MMLU |
+|---|---:|
+| unhealed 4.02B (same structure, unhealed sibling dir) | 25.61% |
+| **this checkpoint (healed 60M)** | **34.12%** |
+| SVD-LLM V2 base (same budget + same healing recipe) | 35.66% |
+| TR-3 base (same budget + same healing recipe) | 41.97% |
+| FP16 dense 6.74B base teacher | 47.41%* |
+
+\* teacher number as measured by this heal run's own pre-eval; the TR-3 run's pre-eval of the
+same probe set gave 47.50% (0.09pp run-to-run wobble, same harness and protocol file
+`eval_protocol_llama2_7b_base.json`).
+
+## How to load
+
+```python
+import json
+from lowrank_linear import build, materialize_all
+
+model = build("NousResearch/Llama-2-7b-hf",
+              json.load(open("ranks.json")), "state_dict_fp16.pt",
+              device="cuda")
+# for generation/eval: fuse the factors first (inference-only)
+materialize_all(model)
+```
+
+`ranks.json` maps each compressed projection path to its integer rank; untouched weights are
+stock LLaMA-2-7B base tensors. The full healing-provenance record (data mixture & revisions,
+decontamination, SHA256 of inputs/checkpoint, 5M milestone trajectory) is in `provenance.json`.
+
+## License
+
+Derived from Meta's LLaMA 2 (`NousResearch/Llama-2-7b-hf`, an unofficial redistribution of
+the same weights). Subject to the **LLaMA 2 Community License** — see
+https://ai.meta.com/llama/license/.