betterwithage commited on
Commit
4027a93
·
verified ·
1 Parent(s): 661d9c3

Card: independent re-verification section (2026-09-13); prior buggy-harness fp16 scores void

Browse files
Files changed (1) hide show
  1. README.md +31 -49
README.md CHANGED
@@ -19,7 +19,7 @@ tags:
19
  szl:
20
  doctrine: v11-LOCKED
21
  lean: "749/14/163"
22
- lambda: "Conjecture 1 — advisory, never a theorem"
23
  artifact_class: ADAPTER
24
  publication_eligible: false
25
  autonomy_eligible: false
@@ -29,14 +29,15 @@ szl:
29
  job_prior_error: "6a91ba2c45686a1580c12020"
30
  weights: AVAILABLE
31
  evals: MEASURED
 
32
  gpu: UNAVAILABLE
33
  ---
34
 
35
  <p align="center">
36
- <img src="holo-banner.svg" alt="KHIPU-R2 — holographic step-up banner" width="100%"/>
37
  </p>
38
 
39
- <h1 align="center">K H I P U — R 2</h1>
40
 
41
  <p align="center"><em>Failure is an artifact, not a footnote.</em></p>
42
 
@@ -44,60 +45,29 @@ szl:
44
  <img alt="Base: Qwen2.5-1.5B-Instruct" src="https://img.shields.io/badge/base-Qwen2.5--1.5B--Instruct-334155?style=flat-square"/>
45
  <img alt="Downloads" src="https://img.shields.io/huggingface/dt/SZLHOLDINGS/KHIPU-R2?style=flat-square&color=3af4c8&label=downloads"/>
46
  <img alt="Artifact: QLoRA adapter r=32 a=64" src="https://img.shields.io/badge/artifact-QLoRA%20adapter%20r32%20a64-0d9488?style=flat-square"/>
47
- <img alt="Abstain: MEASURED 3/6, was 2/6 — not a pass" src="https://img.shields.io/badge/abstain-3%2F6%20was%202%2F6%20%C2%B7%20not%20a%20pass-b45309?style=flat-square"/>
48
  <img alt="Plan-valid 11/11" src="https://img.shields.io/badge/plan--valid-11%2F11-16a34a?style=flat-square"/>
49
  <img alt="Hallucinated citations: 0" src="https://img.shields.io/badge/hallucinated%20citations-0-16a34a?style=flat-square"/>
 
50
  <img alt="Lifecycle: research-only" src="https://img.shields.io/badge/lifecycle-research--only-7e8aa3?style=flat-square"/>
51
  </p>
52
 
53
  <p align="center">
54
- <code>KANCHAY</code> · Doctrine v11 · Lean <code>749/14/163</code> · Λ = Conjecture 1 (advisory) · <a href="https://a-11-oy.com">a-11-oy.com</a>
55
  </p>
56
 
57
  Adapters are on this repo. Abstain is MEASURED 3/6, not a pass. This card-only release does not modify adapter files and does not open the publication or autonomy gates.
58
 
59
- QLoRA adapter on disclosed [`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) (runtime `unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit`). Proposal-only brain navigator / abstain retrain. Doctrine v11 LOCKED. Λ = Conjecture 1 (advisory, never a theorem).
60
-
61
- <!-- SZL-ATELIER-CUT:v1:START -->
62
- ## The cut
63
-
64
- Leaders ship v1 and changelog the rest. We name the retraining of silence as its own model. Failure is an artifact, not a footnote.
65
-
66
- A public retrain whose only job is to improve one metric: honest abstain under adversarial handles.
67
-
68
- ### Silhouette → leave → SZL
69
-
70
- | Leader | Take, then tweak |
71
- |---|---|
72
- | Anthropic | Red-team → constitution update. We red-team → adapter. |
73
- | NVIDIA | Recipe re-run with a new seed and a bounded delta. |
74
- | Unsloth | Same FastLanguageModel loop, new curriculum, new evidence record. |
75
-
76
- This card makes the combination inspectable: a separate abstain-retraining SKU, bounded small-sample results, and an exact-source documentation publisher that cannot modify weights or promotion state.
77
-
78
- ## Intended use
79
-
80
- Continue the abstain-retrain loop. Keep publication and autonomy gates closed until independently verifiable evaluation evidence satisfies the governed release criteria.
81
-
82
- ## Limitations
83
-
84
- - research-only
85
- - No signed R2 eval receipt in this atelier.
86
- - Publishing this card is a documentation update, not model promotion or autonomy approval.
87
-
88
- Canonical GitHub: [`szl-holdings/szl-forge/khipu_r2`](https://github.com/szl-holdings/szl-forge/tree/main/khipu_r2)
89
- <!-- SZL-ATELIER-CUT:v1:END -->
90
 
91
  ## Specification
92
 
93
  | | |
94
  |---|---|
95
  | **Artifact** | `adapter_model.safetensors` (147.8M) + `adapter_config.json` **AVAILABLE** |
96
- | **Job** | [`6a91bf11984507d9db4ea104`](https://huggingface.co/jobs/SZLHOLDINGS/6a91bf11984507d9db4ea104) **COMPLETED** |
97
- | **Does NOT overwrite** | signed [`SZL-Khipu-1.5B`](https://huggingface.co/SZLHOLDINGS/SZL-Khipu-1.5B) |
98
- | **Prior job** | [`6a91ba2c`](https://huggingface.co/jobs/SZLHOLDINGS/6a91ba2c45686a1580c12020) **ERROR** Trackio 404 |
99
- | **Card publication** | Exact-source README + local SVG + source binding only; weights, adapter, configs, evals, visibility, hardware, and runtime unchanged |
100
- | **Lab** | Forbidden. Pin stays Khipu GGUF. GPU **UNAVAILABLE**. |
101
  | **License** | Apache-2.0 |
102
  | **Autonomy** | false |
103
 
@@ -108,26 +78,38 @@ Method: in-process Unsloth generate, scoring ported from `eval_khipu.py`, temper
108
  | split | k/n | what-NOT |
109
  |---|---|---|
110
  | plan-valid | **11 / 11** | not a public leaderboard |
111
- | grounding (`eval.jsonl` navigate) | **5 / 5** | n=5 |
112
- | abstain (`adversarial.jsonl`) | **3 / 6** | not 5/5, not 6/6 |
113
  | hallucinated citations | **0** | this job only |
114
 
115
- Prior published original `SZL-Khipu-1.5B` MEASURED abstain was **2/6**. This run is **3/6**. Small n. Do not derive a world-rank score from k/n on n=11.
 
 
 
 
 
 
 
 
 
 
 
 
116
 
117
  ## Training (MEASURED / REPORTED)
118
 
119
- - Unsloth QLoRA, seed 11, lr 2e-4, LoRA r=32 α=64, 45 epochs
120
- - Train: 15 navigate + 8 abstain rows × oversample 4 (in-memory 32)
121
  - Held-out: 5 + 6, `held_out_in_gradients: false`
122
- - `training_loss` MEASURED `0.017188…` is a train metric, not an eval
123
  - adapter sha256 `e44d53f29f2d443598e06d6c0441557fd3a5010888c7aa97b56ec3c0e050d349`
124
 
125
  ## What this is NOT
126
 
127
- - Not a replacement for `SZL-Khipu-1.5B`
128
  - Not Chaski (Qwen3.5 lock)
129
  - Not an autonomous agent
130
- - Not a GGUF. Mini GGUFs exist on [`A11OY-MINI`](https://huggingface.co/SZLHOLDINGS/A11OY-MINI); Mini evals none-this-run; Mini does **not** inherit this 3/6
131
 
132
  ## Load
133
 
 
19
  szl:
20
  doctrine: v11-LOCKED
21
  lean: "749/14/163"
22
+ lambda: "Conjecture 1 - advisory, never a theorem"
23
  artifact_class: ADAPTER
24
  publication_eligible: false
25
  autonomy_eligible: false
 
29
  job_prior_error: "6a91ba2c45686a1580c12020"
30
  weights: AVAILABLE
31
  evals: MEASURED
32
+ reverified: "2026-09-13"
33
  gpu: UNAVAILABLE
34
  ---
35
 
36
  <p align="center">
37
+ <img src="holo-banner.svg" alt="KHIPU-R2 - holographic step-up banner" width="100%"/>
38
  </p>
39
 
40
+ <h1 align="center">K H I P U - R 2</h1>
41
 
42
  <p align="center"><em>Failure is an artifact, not a footnote.</em></p>
43
 
 
45
  <img alt="Base: Qwen2.5-1.5B-Instruct" src="https://img.shields.io/badge/base-Qwen2.5--1.5B--Instruct-334155?style=flat-square"/>
46
  <img alt="Downloads" src="https://img.shields.io/huggingface/dt/SZLHOLDINGS/KHIPU-R2?style=flat-square&color=3af4c8&label=downloads"/>
47
  <img alt="Artifact: QLoRA adapter r=32 a=64" src="https://img.shields.io/badge/artifact-QLoRA%20adapter%20r32%20a64-0d9488?style=flat-square"/>
48
+ <img alt="Abstain: MEASURED 3/6, was 2/6 - not a pass" src="https://img.shields.io/badge/abstain-3%2F6%20was%202%2F6%20%C2%B7%20not%20a%20pass-b45309?style=flat-square"/>
49
  <img alt="Plan-valid 11/11" src="https://img.shields.io/badge/plan--valid-11%2F11-16a34a?style=flat-square"/>
50
  <img alt="Hallucinated citations: 0" src="https://img.shields.io/badge/hallucinated%20citations-0-16a34a?style=flat-square"/>
51
+ <img alt="Re-verified 2026-09-13" src="https://img.shields.io/badge/re--verified-2026--09--13-16a34a?style=flat-square"/>
52
  <img alt="Lifecycle: research-only" src="https://img.shields.io/badge/lifecycle-research--only-7e8aa3?style=flat-square"/>
53
  </p>
54
 
55
  <p align="center">
56
+ <code>KANCHAY</code> - Doctrine v11 - Lean <code>749/14/163</code> - Conjecture 1 (advisory) - <a href="https://a-11-oy.com">a-11-oy.com</a>
57
  </p>
58
 
59
  Adapters are on this repo. Abstain is MEASURED 3/6, not a pass. This card-only release does not modify adapter files and does not open the publication or autonomy gates.
60
 
61
+ QLoRA adapter on disclosed [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) (runtime unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit). Proposal-only brain navigator / abstain retrain. Doctrine v11 LOCKED. Conjecture 1 (advisory, never a theorem).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
62
 
63
  ## Specification
64
 
65
  | | |
66
  |---|---|
67
  | **Artifact** | `adapter_model.safetensors` (147.8M) + `adapter_config.json` **AVAILABLE** |
68
+ | **Job** | [6a91bf11984507d9db4ea104](https://huggingface.co/jobs/SZLHOLDINGS/6a91bf11984507d9db4ea104) **COMPLETED** |
69
+ | **Does NOT overwrite** | signed [SZL-Khipu-1.5B](https://huggingface.co/SZLHOLDINGS/SZL-Khipu-1.5B) |
70
+ | **Prior job** | [6a91ba2c](https://huggingface.co/jobs/SZLHOLDINGS/6a91ba2c45686a1580c12020) **ERROR** Trackio 404 |
 
 
71
  | **License** | Apache-2.0 |
72
  | **Autonomy** | false |
73
 
 
78
  | split | k/n | what-NOT |
79
  |---|---|---|
80
  | plan-valid | **11 / 11** | not a public leaderboard |
81
+ | grounding (eval.jsonl navigate) | **5 / 5** | n=5 |
82
+ | abstain (adversarial.jsonl) | **3 / 6** | not 5/5, not 6/6 |
83
  | hallucinated citations | **0** | this job only |
84
 
85
+ Prior published original SZL-Khipu-1.5B MEASURED abstain was 2/6. This run is 3/6. Small n. Do not derive a world-rank score from k/n on n=11.
86
+
87
+ ### Independent re-verification (2026-09-13)
88
+
89
+ Owner-host re-run with a repaired `szl_khipu_eval.py`. The harness had a decoding bug (`tok.decode` on a 2-D batch tensor inside `generate_plan`) that returned prompt characters instead of model output; every prior local fp16 score produced by the buggy harness is void. Three CPU fp16 evaluations, temperature 0, held-out set unchanged:
90
+
91
+ | run | plan-valid | grounding | abstain | hallucinated |
92
+ |---|---|---|---|---|
93
+ | rebuilt fp16 merge (adapter onto Qwen/Qwen2.5-1.5B-Instruct) | **11/11** | **5/5** | **3/6** | **0** |
94
+ | rebuilt fp16 merge, second build | **11/11** | **5/5** | **3/6** | **0** |
95
+ | published fp32 merge (weights unchanged) | **11/11** | **5/5** | **3/6** | **0** |
96
+
97
+ All runs reproduce the 2026-08-28 job receipt exactly. File: `eval_verified_2026-09-13.json`. GPU **UNAVAILABLE** (CPU eval). This is verification evidence, not promotion; publication and autonomy gates remain closed.
98
 
99
  ## Training (MEASURED / REPORTED)
100
 
101
+ - Unsloth QLoRA, seed 11, lr 2e-4, LoRA r=32 a=64, 45 epochs
102
+ - Train: 15 navigate + 8 abstain rows x oversample 4 (in-memory 32)
103
  - Held-out: 5 + 6, `held_out_in_gradients: false`
104
+ - `training_loss` MEASURED 0.017188 is a train metric, not an eval
105
  - adapter sha256 `e44d53f29f2d443598e06d6c0441557fd3a5010888c7aa97b56ec3c0e050d349`
106
 
107
  ## What this is NOT
108
 
109
+ - Not a replacement for SZL-Khipu-1.5B
110
  - Not Chaski (Qwen3.5 lock)
111
  - Not an autonomous agent
112
+ - Not a GGUF. Mini GGUFs exist on [A11OY-MINI](https://huggingface.co/SZLHOLDINGS/A11OY-MINI); Mini evals none-this-run; Mini does **not** inherit this 3/6
113
 
114
  ## Load
115