v5.0.0: Full fine-tune on data_v10 (1.14M lines, 195 negative patterns). Fixes CloudFormation Default: FNs (F1 0.0→1.0), Redis --requirepass FNs (F1 0.67→1.0), and all 14 confirmed FP categories from real-world scanner runs. F1=0.9999, Precision=1.0000, Recall=0.9998 on held-out eval.

README.md

@@ -60,28 +60,28 @@ model-index:
           type: text-classification
           name: Secret / Credential Detection
         dataset:
-          name: Synthetic + Real-World Labeled (data_v9 + scan_vuln.db)
+          name: Synthetic + Real-World Labeled (data_v10 + labeled_data.jsonl)
           type: custom
           split: test
         metrics:
           - type: f1
-            value: 0.9992
+            value: 0.9999
             name: F1 Score
           - type: precision
-            value: 0.9983
+            value: 1.0000
             name: Precision
           - type: recall
-            value: 1.0000
+            value: 0.9998
             name: Recall
           - type: accuracy
-            value: 0.9992
+            value: 0.9999
             name: Accuracy
 ---
 
-# Secrets Sentinel — v4.0.0
+# Secrets Sentinel — v5.0.0
 
 > **Context-aware AI secret detection for CI/CD pipelines, pre-commit hooks, and code scanners.**  
-> Fine-tuned DeBERTa-v3-base · F1 = 0.9992 · Precision = 1.0 · <500ms inference · Runs fully on-prem.
+> Fine-tuned DeBERTa-v3-base · F1 = 0.9999 · Precision = 1.0 · <500ms inference · Runs fully on-prem.
 
 [![HuggingFace](https://img.shields.io/badge/🤗-Model_Card-yellow)](https://huggingface.co/hypn05/secrets-sentinel)
 [![CPU Version](https://img.shields.io/badge/⚡-CPU_ONNX_INT8-blue)](https://huggingface.co/hypn05/secrets-sentinel-cpu)
@@ -89,13 +89,19 @@ model-index:
 
 ---
 
-## What's New in v4.0.0
+## What's New in v5.0.0
+
+v5 is a full fine-tune of all 184M parameters on data_v10 (1.14M training lines, 195 negative
+patterns), fixing the remaining false negatives from v4 and further reducing false positives:
+
+| Fix | Example |
+|---|---|
+| **CloudFormation `Default:` passwords** | `Default: MyS3cr3tPass!` now correctly flagged |
+| **Redis `--requirepass` in docker-compose** | `command: redis-server --requirepass s3cr3t` now correctly flagged |
 
-v4 significantly reduces false positives identified from scanning real-world open-source codebases
-and deliberate security testing repositories (OWASP WebGoat, OWASP juice-shop, OWASP wrongsecrets,
-TruffleHog fixtures, Gitleaks testdata, Plazmaz/leaky-repo, and more):
+Combined with all v4 false-positive fixes:
 
-| Pattern Fixed | Example (now correctly classified as safe) |
+| Pattern (safe, not flagged) | Example |
 |---|---|
 | **GitHub Actions pinned SHAs** | `uses: docker/login-action@5e57cd118...ef # v3.6.0` |
 | **`.env.example` placeholders** | `REDIS_PASSWORD=null`, `DB_PASSWORD=YOURPASSWORD` |
@@ -153,15 +159,13 @@ Secrets pushed to repositories create a critical, expensive security incident:
 | **GPU memory** | ~800 MB (inference) |
 | **License** | MIT |
 
-### Training Data (v4)
+### Training Data (v5)
 
 | Source | Lines | Label |
 |---|---|---|
-| Synthetic generator (data_v9) | ~135,000 | Mixed |
-| Real-world scan — OWASP apps | ~2,600 | Labeled |
-| Scanner fixture repos (TruffleHog, Gitleaks, leaky-repo, detect-secrets) | ~2,000 | Labeled |
-| Synthetic FP negatives (15 pattern categories) | ~2,700 | Negative |
-| **Total** | **~144,000** | — |
+| Synthetic generator (data_v10, 162 positive + 195 negative patterns) | ~1,106,000 | Mixed |
+| Real-world labeled examples (OWASP apps, scanner fixtures, vuln repos) | ~37,900 | Labeled |
+| **Total** | **~1,144,000** | — |
 
 ### Evaluation History
 
@@ -170,9 +174,10 @@ Secrets pushed to repositories create a critical, expensive security incident:
 | v1.0.0 | 0.9910 | 0.9905 | 0.9915 | 0.9920 | — |
 | v2.0.0 | 0.9976 | 0.9975 | 0.9977 | 0.9977 | 0.0112 |
 | v3.0.0 | 0.9994 | 0.9994 | 0.9994 | 0.9995 | 0.0051 |
-| **v4.0.0** | **0.9992** | **0.9983** | **1.0000** | **0.9992** | **~0.001** |
+| v4.0.0 | 0.9992 | 0.9983 | 1.0000 | 0.9992 | ~0.001 |
+| **v5.0.0** | **0.9999** | **1.0000** | **0.9998** | **0.9999** | **~0.000** |
 
-v4 achieves **Recall = 1.0** (zero missed secrets on the held-out eval set) while maintaining near-perfect precision, with the key improvement being elimination of false positives from real-world codebases (GitHub Actions pinned SHAs, `.env.example` templates, PHP config patterns, test fixtures).
+v5 achieves **Precision = 1.0** (zero false positives on the held-out eval set) with near-perfect recall, trained as a full fine-tune on data_v10 which incorporates all confirmed real-world FP and FN patterns at scale.
 
 ---
 
@@ -564,7 +569,7 @@ A: No. Run the model locally or on your own infrastructure. Inference is fully o
 A: `0.85` for pre-receive hooks (low FP, may miss some edge cases). `0.60` for retrospective scans where you want higher recall and can accept some FP for manual review.
 
 **Q: How often is the model updated?**  
-A: Continuously — each version is trained on newly identified FP and FN patterns from real-world scanning. v4 specifically addresses GitHub Actions workflow files, PHP config templates, and Laravel test fixtures.
+A: Continuously — each version is trained on newly identified FP and FN patterns from real-world scanning. v5 specifically addresses CloudFormation `Default:` password lines and Redis `--requirepass` docker-compose commands, while v4 addressed GitHub Actions workflow files, PHP config templates, and Laravel test fixtures.
 
 ---
 

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:24abae7bc410d6bcbfc8a8e2af5c02a9934ca362cb4c7c03dc4402ce43d36a90
+oid sha256:932fcb4d9809efec148bb0a1a89f4ad60e78d741d786e79aa932f05545003372
 size 737719272