Correct v1 card: leakage audit, retired metric, v4 successor

README.md

@@ -10,46 +10,56 @@ tags:
 - cryptography
 - post-quantum
 - static-analysis
+- legacy
 widget:
 - text: "cipher = Cipher.getInstance(\"AES/ECB/PKCS5Padding\"); // usage under review"
 - text: "kem = OQS_KEM_ml_kem_768_new(); OQS_KEM_ml_kem_768_keypair(kem, pk, sk);"
 ---
 
-# Q-Trust CodeBERT — crypto-usage discovery classifier
+# Q-Trust CodeBERT (v1 — legacy checkpoint)
 
-Binary code classifier that answers **"does this code use cryptography, and which
-primitive?"** — the discovery layer of the Q-Trust post-quantum migration
-project (`humoge7502/q-trust` on GitHub). Fine-tuned from
-`huggingface/CodeBERTa-small-v1` on 12,462 real files (6,636 crypto-labeled,
-34 repos, deterministic seed-42 split, repo-disjoint held-out set).
+> **Superseded.** This is the first Q-Trust crypto-usage classifier, kept for lineage and for
+> reproducing earlier experiments. Its published held-out score came from a split that was later
+> found to leak: the 2026-10-23 audit measured **87.6 % of its evaluation paths inside its own
+> training data**. The weights are unchanged, but its F1 is **not a clean generalization estimate**
+> and should not be quoted as one.
+>
+> Current model: [`KRISHNAPURI/q-trust-crypto-detector`](https://huggingface.co/KRISHNAPURI/q-trust-crypto-detector) —
+> clean held-out **F1 0.9205** de-duplicated / **0.9235** native at the 0.50 gate, **0 % train/eval
+> path overlap**. See [`EXPERIMENTS.md`](https://github.com/humoge7502/q-trust/blob/main/EXPERIMENTS.md)
+> E-13 → E-17c for the audit, the retraining and the recorded negatives.
 
-## Measured results (never estimated)
+Binary code classifier that answers **"does this code use cryptography?"** — the discovery layer of
+the Q-Trust post-quantum migration pipeline (`humoge7502/q-trust` on GitHub).
 
-| Metric | Value |
+## What this checkpoint is
+
+| | |
+|---|---|
+| Base | `huggingface/CodeBERTa-small-v1`, 2-label sequence classification |
+| Training data | 12,462 real files (6,636 crypto-labelled) from 34 repositories |
+| Input | source code, truncated to 256 tokens |
+| Output | `LABEL_1` = crypto usage, `LABEL_0` = no crypto usage |
+| Retained for | v1-vs-v4 head-to-head, GNN ranker and RL experiments that reference these exact weights |
+
+## Corrected evaluation status
+
+| | |
 |---|---|
-| Precision / Recall / F1 | 0.9279 / 0.9514 / **0.9395** (n=2,415, corrected) |
-| Training | 4 epochs, CUDA, `scripts/train_qtrust_all.py --real --hf-epochs 4` |
-| Leakage audit | Split reproduced bit-identically; 0.4% cross-repo dupes, bounded well below the claimed gain |
-| Calibration | Temperature T=1.2142, ECE 0.1557 → 0.1387 cross-fitted (`artifacts/calibration.json` in the GitHub repo) |
-
-> **Metric correction (2026-10-23):** the originally published 0.9525 was never
-> reproducible — no threshold on these weights reaches it (best operating point
-> 0.9434 at τ = 0.67). The corrected held-out F1 is **0.9395**; the same
-> correction is applied across the GitHub repo's docs and is enforced there by
-> a CI metric-consistency gate. This card keeps the checkpoint unchanged — it
-> is the canonical v1 model.
-
-> **Successor:** [`KRISHNAPURI/q-trust-crypto-detector`](https://huggingface.co/KRISHNAPURI/q-trust-crypto-detector)
-> (v4: larger corpus, dedup-corrected held-out F1 0.9039 with a leakage-audited
-> repo-disjoint digest split) — with the honest caveat that no clean cross-model
-> v1-vs-v4 comparison exists (v1 trained on v4's eval files); see the GitHub
-> repo's EXPERIMENTS.md E-13/E-14/E-15 for the full program, including the
-> recorded negatives.
-
-Try it in the widget above, or in code:
+| Pre-audit held-out F1 | 0.9395 (leak-inflated: 87.6 % eval-path overlap) |
+| Leakage audit | [`eval_leakage_audit`](https://github.com/humoge7502/q-trust/blob/main/EXPERIMENTS.md) — 2026-10-23 |
+| Clean estimate for this checkpoint | none established — do not cite |
+| Recommended successor | v4 detector, F1 0.9205 de-duplicated / 0.9235 native at gate 0.50, 0 % overlap |
+
+The correction is enforced in the GitHub repository by `scripts/verify_metric_consistency.py`, so
+the retired figure cannot drift back into current-facing documentation. This card keeps the
+checkpoint unchanged: it is the canonical v1 artifact, not the current recommendation.
+
+## Usage
 
 ```python
 from transformers import pipeline
+
 clf = pipeline("text-classification", model="KRISHNAPURI/q-trust-codebert")
 clf("EVP_PKEY_assign_RSA(pkey, rsa);")
 # [{'label': 'LABEL_1', 'score': 0.99}]  # LABEL_1 = crypto usage
@@ -57,21 +67,30 @@ clf("EVP_PKEY_assign_RSA(pkey, rsa);")
 
 ```python
 from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
 tok = AutoTokenizer.from_pretrained("KRISHNAPURI/q-trust-codebert")
 model = AutoModelForSequenceClassification.from_pretrained("KRISHNAPURI/q-trust-codebert")
 ```
 
-Training data: [`KRISHNAPURI/q-trust-datasets`](https://huggingface.co/datasets/KRISHNAPURI/q-trust-datasets)
-(`qtrust_ai/artifacts/real_datasets/code_corpus.json`). Full lineage
-(config + seed + metrics): `reports/training_report_real.json` in
-[`KRISHNAPURI/q-trust-codebert`](https://huggingface.co/KRISHNAPURI/q-trust-codebert)
-sibling files, and `qtrust_ai/artifacts/benchmark_comparison.json` in the
-GitHub repo.
-
 ## Scope and limits
 
-This model finds crypto **usage** (recall 0.877 on CryptoAPI-Bench files); it is
-not a misuse detector and makes no SOTA claim. Sibling artifacts in this repo:
-GNN ranker (honest tie with the heuristic, τ-b 0.7168 vs 0.7210), RL agent
-(tie), side-channel detector (54 real trace sets). See `REPORTS` and the
-per-artifact notes in the repo card below.
+This model finds crypto **usage** (recall 0.877 on CryptoAPI-Bench files); it is not a misuse
+detector and makes no SOTA claim. Labels are regex-derived weak labels, so the attainable score is
+bounded by the labeler's error. It is not a vulnerability scanner and not a substitute for human
+review.
+
+Sibling artifacts in this repository, all reported at their measured values:
+
+- **GNN ranker** — τ-b 0.7168 vs 0.7210 for the doctrine heuristic: a tie.
+- **RL migration agent** — likewise a tie with the heuristic.
+- **Side-channel detector** — evaluated on 54 real trace sets.
+
+`onnx/model_quantized.onnx` is the original int8 export used by the first browser demo. The current
+[in-browser scanner](https://huggingface.co/spaces/KRISHNAPURI/q-trust-scanner) loads the v4 detector
+instead.
+
+## Provenance
+
+- Training corpus: [`KRISHNAPURI/q-trust-datasets`](https://huggingface.co/datasets/KRISHNAPURI/q-trust-datasets)
+- Metrics and audit trail: `reports/` in this repository and
+  [`humoge7502/q-trust`](https://github.com/humoge7502/q-trust)