Upload README.md with huggingface_hub

README.md

@@ -23,9 +23,9 @@ tags:
 # BaiHu-V1-Flash
 
 **BaiHu-V1-Flash** is an SSA (Sparse-attention + SubQ) retrofit of `Qwen/Qwen3-0.6B-Base`,
-fine-tuned on **1,294 bilingual multi-turn dialogues** whose purpose is **举一反三** — the
-model is shown one worked rule / format / method in the first turn, and later turns ask it to
-reuse that rule on a *new* case it has never seen.
+fine-tuned on **1,294 bilingual multi-turn dialogues** whose purpose is **generalizing from a
+single worked example** — the model is shown one worked rule / format / method in the first
+turn, and later turns ask it to reuse that rule on a *new* case it has never seen.
 
 - Base model: `Qwen/Qwen3-0.6B-Base` (28 layers / 16 Q heads / 8 KV heads / head_dim 128 / 32K context / tied embeddings)
 - Parameters: **598.8M** (2.75M are SSA-only modules)
@@ -149,7 +149,42 @@ The improvement holds in **all 8 slices** (2 languages × 6 kinds + overall).
   is correct on most samples (`B12A`, `100, 50, 25, 12.5, …`, format rewrites). Arithmetic
   errors remain — a 0.6B model with 1.3K training dialogues still slips on multi-step math.
 
-### 4.3 Effect of the P0 revision
+### 4.3 Standard benchmarks (lm-evaluation-harness)
+
+Both checkpoints were scored with `lm-evaluation-harness` **0.4.13** in one environment, with
+the same prompts, batch size and dtype (`bfloat16`), over the **entire evaluation split** of
+every task — 2,376 arc_easy / 1,172 arc_challenge / 10,042 hellaswag / 1,838 piqa / 1,267
+winogrande examples, with no `--limit` subsampling — so the two columns are like-for-like. The
+metrics are the harness's 0-shot numbers, scored in raw-completion mode (no chat template),
+which is what a base checkpoint supports.
+
+| Task | Metric | Qwen3-0.6B-Base | **BaiHu-V1-Flash** | Δ |
+|---|---|---|---|---|
+| arc_easy | acc_norm | 0.5791 ±0.0101 | **0.5939 ±0.0101** | +0.0148 |
+| arc_challenge | acc_norm | 0.3848 ±0.0142 | **0.3882 ±0.0142** | +0.0034 |
+| hellaswag | acc_norm | 0.5385 ±0.0050 | **0.5507 ±0.0050** | +0.0122 |
+| piqa | acc_norm | 0.6997 ±0.0107 | **0.7084 ±0.0106** | +0.0087 |
+| winogrande | acc | 0.5856 ±0.0138 | **0.6062 ±0.0137** | +0.0206 |
+| unweighted mean | | 0.5575 | **0.5695** | +0.0119 |
+
+**How to read this.** All five deltas are positive, but each is between 0.3σ and 1.5σ of its
+own standard error, so the defensible claim is **"no regression in general capability"**, not
+"the fine-tune made the model smarter". The movement is also not an SSA effect: a control run
+of the *dense* `Qwen3-0.6B-Base` fine-tuned with the identical data and recipe lands within
+±0.005 of BaiHu-V1-Flash on every one of these tasks (arc_easy 0.5951, arc_challenge 0.3908,
+hellaswag 0.5511, piqa 0.7111, winogrande 0.6014). What this release did buy is the transfer
+behaviour of §4.1–§4.2 — a 63% perplexity drop on held-out dialogues and no degenerate
+generations.
+
+Caveats, stated plainly:
+
+- These are **English** benchmarks. This harness build ships no C-Eval / CMMLU / C3 tasks, so
+  Chinese capability is not measured above; the held-out split of §4.1 (which contains both
+  languages) is the only Chinese-side evidence in this card.
+- The Hub checkpoint stores float32 weights and the harness ran it at `bfloat16` (see §5.2).
+  The two agree to within 0.002 on every task listed, so precision is not driving the table.
+
+### 4.4 Effect of the P0 revision
 
 Measured with the same data, recipe and seed, P0 vs the old single-vector shared branch:
 
@@ -174,8 +209,10 @@ been run yet — see Limitations.
 `generate()` (greedy or temperature/top-k; no beam search).
 
 The implementation lives in the project repository (not on the Hub). **This checkpoint
-requires the P0 revision** of `modeling_baihu_ssa.py` — an older copy loads without error but
-computes a different shared branch.
+requires two revisions** of that code: the P0 revision of `modeling_baihu_ssa.py` (an older
+copy loads without error but computes a different shared branch), and the dtype-aware loader
+in `model_baihu_ssa.py` — before it, `from_pretrained(..., dtype=...)` was silently ignored
+and the model always came back float32.
 
 ### 5.2 Minimal working example
 
@@ -203,9 +240,10 @@ tok = AutoTokenizer.from_pretrained(REPO)
 model = AutoModelForCausalLM.from_pretrained(REPO, dtype=dtype).to(device).eval()
 
 messages = [
-    {"role": "user", "content": "这个数列的规律是什么？1、4、9、16、25"},
-    {"role": "assistant", "content": "相邻两项的差是 3、5、7、9，每次加 2，所以第 n 项是 n 的平方。"},
-    {"role": "user", "content": "按同样的规律，下一项是多少？36、49 之后呢？"},
+    {"role": "user", "content": "What is the rule behind this sequence? 1, 4, 9, 16, 25"},
+    {"role": "assistant", "content": "The gaps between consecutive terms are 3, 5, 7, 9 "
+                                     "(+2 each time), so the n-th term is n squared."},
+    {"role": "user", "content": "Using that same rule, what comes after 36 and 49?"},
 ]
 ids = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
                               enable_thinking=False)
@@ -222,6 +260,10 @@ print(text.strip())
 Training used `add_generation_prompt=True, enable_thinking=False` (the generation prompt then
 ends with `<|im_start|>assistant\n<think>\n\n</think>\n\n`). Keep that format for best results.
 
+The Hub checkpoint stores **float32** weights (2.4 GB). Passing `dtype=torch.bfloat16`, as
+above, loads them at **1.20 GB** and leaves the (float32) rotary tables untouched; omitting
+`dtype` gives the 2.40 GB float32 model. Both score identically on the benchmarks in §4.3.
+
 ### 5.3 Troubleshooting
 
 | Error | Cause | Fix |