Correct headline metrics to reproducible 0.9395 + successor pointer to v4 detector

README.md

@@ -1,62 +1,77 @@
----
-license: mit
-library_name: transformers
-pipeline_tag: text-classification
-datasets:
-- KRISHNAPURI/q-trust-datasets
-base_model: huggingface/CodeBERTa-small-v1
-tags:
-- code
-- cryptography
-- post-quantum
-- static-analysis
-widget:
-- text: "cipher = Cipher.getInstance(\"AES/ECB/PKCS5Padding\"); // usage under review"
-- text: "kem = OQS_KEM_ml_kem_768_new(); OQS_KEM_ml_kem_768_keypair(kem, pk, sk);"
----
-
-# Q-Trust CodeBERT — crypto-usage discovery classifier
-
-Binary code classifier that answers **"does this code use cryptography, and which
-primitive?"** — the discovery layer of the Q-Trust post-quantum migration
-project (`humoge7502/q-trust` on GitHub). Fine-tuned from
-`huggingface/CodeBERTa-small-v1` on 12,462 real files (6,636 crypto-labeled,
-34 repos, deterministic seed-42 split, repo-disjoint held-out set).
-
-## Measured results (never estimated)
-
-| Metric | Value |
-|---|---|
-| Precision / Recall / F1 | 0.952 / 0.953 / **0.9525** (n=2,415) |
-| Training | 4 epochs, CUDA, `scripts/train_qtrust_all.py --real --hf-epochs 3` |
-| Leakage audit | Split reproduced bit-identically; 0.4% cross-repo dupes, bounded below the +0.11 F1 gain |
-
-Try it in the widget above, or in code:
-
-```python
-from transformers import pipeline
-clf = pipeline("text-classification", model="KRISHNAPURI/q-trust-codebert")
-clf("EVP_PKEY_assign_RSA(pkey, rsa);")
-# [{'label': 'LABEL_1', 'score': 0.99}]  # LABEL_1 = crypto usage
-```
-
-```python
-from transformers import AutoTokenizer, AutoModelForSequenceClassification
-tok = AutoTokenizer.from_pretrained("KRISHNAPURI/q-trust-codebert")
-model = AutoModelForSequenceClassification.from_pretrained("KRISHNAPURI/q-trust-codebert")
-```
-
-Training data: [`KRISHNAPURI/q-trust-datasets`](https://huggingface.co/datasets/KRISHNAPURI/q-trust-datasets)
-(`qtrust_ai/artifacts/real_datasets/code_corpus.json`). Full lineage
-(config + seed + metrics): `reports/training_report_real.json` in
-[`KRISHNAPURI/q-trust-codebert`](https://huggingface.co/KRISHNAPURI/q-trust-codebert)
-sibling files, and `qtrust_ai/artifacts/benchmark_comparison.json` in the
-GitHub repo.
-
-## Scope and limits
-
-This model finds crypto **usage** (recall 0.877 on CryptoAPI-Bench files); it is
-not a misuse detector and makes no SOTA claim. Sibling artifacts in this repo:
-GNN ranker (honest tie with the heuristic, τ-b 0.7168 vs 0.7210), RL agent
-(tie), side-channel detector (54 real trace sets). See `REPORTS` and the
-per-artifact notes in the repo card below.
+---
+license: mit
+library_name: transformers
+pipeline_tag: text-classification
+datasets:
+- KRISHNAPURI/q-trust-datasets
+base_model: huggingface/CodeBERTa-small-v1
+tags:
+- code
+- cryptography
+- post-quantum
+- static-analysis
+widget:
+- text: "cipher = Cipher.getInstance(\"AES/ECB/PKCS5Padding\"); // usage under review"
+- text: "kem = OQS_KEM_ml_kem_768_new(); OQS_KEM_ml_kem_768_keypair(kem, pk, sk);"
+---
+
+# Q-Trust CodeBERT — crypto-usage discovery classifier
+
+Binary code classifier that answers **"does this code use cryptography, and which
+primitive?"** — the discovery layer of the Q-Trust post-quantum migration
+project (`humoge7502/q-trust` on GitHub). Fine-tuned from
+`huggingface/CodeBERTa-small-v1` on 12,462 real files (6,636 crypto-labeled,
+34 repos, deterministic seed-42 split, repo-disjoint held-out set).
+
+## Measured results (never estimated)
+
+| Metric | Value |
+|---|---|
+| Precision / Recall / F1 | 0.9279 / 0.9514 / **0.9395** (n=2,415, corrected) |
+| Training | 4 epochs, CUDA, `scripts/train_qtrust_all.py --real --hf-epochs 4` |
+| Leakage audit | Split reproduced bit-identically; 0.4% cross-repo dupes, bounded well below the claimed gain |
+| Calibration | Temperature T=1.2142, ECE 0.1557 → 0.1387 cross-fitted (`artifacts/calibration.json` in the GitHub repo) |
+
+> **Metric correction (2026-10-23):** the originally published 0.9525 was never
+> reproducible — no threshold on these weights reaches it (best operating point
+> 0.9434 at τ = 0.67). The corrected held-out F1 is **0.9395**; the same
+> correction is applied across the GitHub repo's docs and is enforced there by
+> a CI metric-consistency gate. This card keeps the checkpoint unchanged — it
+> is the canonical v1 model.
+
+> **Successor:** [`KRISHNAPURI/q-trust-crypto-detector`](https://huggingface.co/KRISHNAPURI/q-trust-crypto-detector)
+> (v4: larger corpus, dedup-corrected held-out F1 0.9039 with a leakage-audited
+> repo-disjoint digest split) — with the honest caveat that no clean cross-model
+> v1-vs-v4 comparison exists (v1 trained on v4's eval files); see the GitHub
+> repo's EXPERIMENTS.md E-13/E-14/E-15 for the full program, including the
+> recorded negatives.
+
+Try it in the widget above, or in code:
+
+```python
+from transformers import pipeline
+clf = pipeline("text-classification", model="KRISHNAPURI/q-trust-codebert")
+clf("EVP_PKEY_assign_RSA(pkey, rsa);")
+# [{'label': 'LABEL_1', 'score': 0.99}]  # LABEL_1 = crypto usage
+```
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+tok = AutoTokenizer.from_pretrained("KRISHNAPURI/q-trust-codebert")
+model = AutoModelForSequenceClassification.from_pretrained("KRISHNAPURI/q-trust-codebert")
+```
+
+Training data: [`KRISHNAPURI/q-trust-datasets`](https://huggingface.co/datasets/KRISHNAPURI/q-trust-datasets)
+(`qtrust_ai/artifacts/real_datasets/code_corpus.json`). Full lineage
+(config + seed + metrics): `reports/training_report_real.json` in
+[`KRISHNAPURI/q-trust-codebert`](https://huggingface.co/KRISHNAPURI/q-trust-codebert)
+sibling files, and `qtrust_ai/artifacts/benchmark_comparison.json` in the
+GitHub repo.
+
+## Scope and limits
+
+This model finds crypto **usage** (recall 0.877 on CryptoAPI-Bench files); it is
+not a misuse detector and makes no SOTA claim. Sibling artifacts in this repo:
+GNN ranker (honest tie with the heuristic, τ-b 0.7168 vs 0.7210), RL agent
+(tie), side-channel detector (54 real trace sets). See `REPORTS` and the
+per-artifact notes in the repo card below.