dannyliv
/

agent-guard-deberta-pi-base

@@ -56,7 +56,7 @@ It is small (184M parameters), CPU-friendly, and Apache-2.0. It is warm-started
 This repo now ships the **V3.2** weights, replacing the prior `v1.x` release. V3.2 was retrained on a permissively-licensed corpus (no gated AI2 datasets) with a rebalanced benign side and nine literature red-team augmentation techniques.
-**What V3.2 changed, stated honestly:**
 - **Fixes the GCG adversarial-suffix weakness.** The prior release had a 100% precomputed-replay attack-success rate against GCG-style adversarial suffixes. V3.2 cuts that to **31.3%** for this model, a large improvement, though weaker than the V3.2 ModernBERT sister (2.4%). Fresh adaptive white-box GCG still succeeds at ~100% against the bare classifier; that is expected for a 184M encoder and is addressed by the SDK-layer perplexity pre-filter, not by this model alone.
 - **Benign false-positive rate REGRESSED versus the prior release.** This is the honest tradeoff and it must be stated plainly. The prior DeBERTa release had a benign FPR of **0.8%** at the canonical threshold 0.5, already production-safe. V3.2 DeBERTa measures **1.6%** at the same threshold. The FPR roughly doubled. The project's strict internal release gate was **≤0.8%**, so V3.2 DeBERTa **misses that gate** (and also misses the 1.0% gate at t=0.3, where it measures 6.4%). V3.2 is shipped anyway, as a deliberate owner-approved decision: shipping both sister models on the same V3.2 corpus, with the GCG fix and a higher JBB F1, was judged to outweigh the FPR regression. If a 0.8% FPR matters more to you than the GCG fix, the prior release was lower-FPR; this release is not.

 This repo now ships the **V3.2** weights, replacing the prior `v1.x` release. V3.2 was retrained on a permissively-licensed corpus (no gated AI2 datasets) with a rebalanced benign side and nine literature red-team augmentation techniques.
+**What V3.2 changed:**
 - **Fixes the GCG adversarial-suffix weakness.** The prior release had a 100% precomputed-replay attack-success rate against GCG-style adversarial suffixes. V3.2 cuts that to **31.3%** for this model, a large improvement, though weaker than the V3.2 ModernBERT sister (2.4%). Fresh adaptive white-box GCG still succeeds at ~100% against the bare classifier; that is expected for a 184M encoder and is addressed by the SDK-layer perplexity pre-filter, not by this model alone.
 - **Benign false-positive rate REGRESSED versus the prior release.** This is the honest tradeoff and it must be stated plainly. The prior DeBERTa release had a benign FPR of **0.8%** at the canonical threshold 0.5, already production-safe. V3.2 DeBERTa measures **1.6%** at the same threshold. The FPR roughly doubled. The project's strict internal release gate was **≤0.8%**, so V3.2 DeBERTa **misses that gate** (and also misses the 1.0% gate at t=0.3, where it measures 6.4%). V3.2 is shipped anyway, as a deliberate owner-approved decision: shipping both sister models on the same V3.2 corpus, with the GCG fix and a higher JBB F1, was judged to outweigh the FPR regression. If a 0.8% FPR matters more to you than the GCG fix, the prior release was lower-FPR; this release is not.