Hugging Face's logo
Activity Feed Request to join this org

AI & ML interests

None defined yet.
scthornton 
posted an update about 1 month ago
view post
Post
2674
SecureCode update: we went back and fact-checked our own security dataset and corrected what didn't hold up.

The original claim was "complete incident grounding, every example ties to a documented CVE." An adversarial re-audit found that it was overstated: many CVEs were misattributed, and many "incidents" were representative scenarios carrying invented statistics. So we fixed it.

- Grounding: re-verified every reference. Removed 802 misattributed CVEs on the web side, corrected or honestly relabeled the incident narratives, and confirmed the AI/ML conversation CVEs are real (EchoLeak CVE-2025-32711, EmailGPT CVE-2024-5184, and others).
- Fix-correctness: reviewed whether each "secure" example actually eliminates the vulnerability. Removed 28 that did not (a "secure" secret scanner whose entropy check always returned zero, an Angular example still using bypassSecurityTrustHtml, and more).
- Leakage: re-split so near-duplicates stay on one side. Test contamination went from 11.6% to zero.
- Viewer, schema, and metadata: rebuilt as parquet under a shared schema. All three viewers are live.
- Models: retrained the whole family on the corrected data so the fix reaches the weights, not just the cards. Now ten open models (3B to 26B), including two new Gemma 4 variants, refreshed locally on a DGX Spark GB10. The paper (arXiv:2512.18542) was revised to match.

Counts moved from 2,185 to 2,372 unified (web 1,625 + AI/ML 747). A slightly smaller, fully-checked dataset beats a larger one you have to take on faith. Full writeup and links in the article.

Datasets: scthornton/securecode, scthornton/securecode-web, scthornton/securecode-aiml

  • 8 replies
·
perfecXion 
posted an update 5 months ago
view post
Post
2574
# IntentGuard: Open-Source Vertical Intent Classifiers for LLM Guardrails

Three models published to the Hub:

- [perfecXion/intentguard-finance]( perfecXion/intentguard-finance)
- [perfecXion/intentguard-healthcare]( perfecXion/intentguard-healthcare)
- [perfecXion/intentguard-legal]( perfecXion/intentguard-legal)

DeBERTa-v3-xsmall fine-tuned for three-way classification: **allow**, **deny**, or **abstain**. ONNX + INT8 quantized, under 80MB, p99 <30ms on CPU. Margin-based thresholds (not argmax) — uncertain queries route to clarification instead of forcing a guess.

**Eval results (adversarial test sets, ~470-480 examples per vertical):**

| Vertical | Accuracy | Legit-Block Rate | Off-Topic-Pass Rate |
|----------|----------|------------------|---------------------|
| Finance | 99.6% | 0.00% | 0.00% |
| Healthcare | 98.9% | 0.00% | 0.98% |
| Legal | 97.9% | 0.00% | 0.50% |

docker run -p 8080:8080 ghcr.io/perfecxion/intentguard:finance-latest

curl -X POST http://localhost:8080/v1/classify \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "What are current mortgage rates?"}]}'


Apache 2.0. Full pipeline + Docker configs on [GitHub](https://github.com/perfecxion-ai/intentguard).

Feedback welcome on domain coverage, adversarial robustness, and multilingual demand.
scthornton 
posted an update 6 months ago
view post
Post
1986
# SecureCode Dataset Family Update: 2,185 Security Examples, Framework-Specific Patterns, Clean Parquet Loading

Hey y'all,

Quick update on the SecureCode dataset family. We've restructured things and fixed several issues:

**What changed:**

- The datasets are now properly split into three repos: [unified]( scthornton/securecode) (2,185), [web]( scthornton/securecode-web) (1,378), [AI/ML]( scthornton/securecode-aiml) (750)
- All repos now use Parquet format -- load_dataset() just works, no deprecated loading scripts
- SecureCode Web now includes 219 framework-specific examples (Express, Django, Spring Boot, Flask, Rails, Laravel, ASP.NET Core, FastAPI, NestJS)
- Data cards have been corrected and split sizes fixed

**Why it matters:**

With AI-generated code accounting for 60%+ of some codebases (Checkmarx 2025), security training data is more important than ever. Every example in SecureCode is grounded in a real CVE with 4-turn conversations that mirror actual developer-AI workflows.

If you're working on code generation models, I'd love to hear how you're approaching the security angle. Are there vulnerability categories or frameworks you'd like to see covered?

Paper: [arxiv.org/abs/2512.18542](https://arxiv.org/abs/2512.18542)
scthornton 
posted an update 7 months ago
view post
Post
2194
# SecureCode: Security-Aware Code Models

**A collection of 8 code models (3B–20B) trained to behave like a security reviewer.**

## The Problem

Code assistants frequently recommend patterns that pass tests but fail security review—string-built SQL, brittle auth logic, unsafe parsing, insecure defaults, and more. I built SecureCode to address this gap.

## What SecureCode Does

- **Identify vulnerable patterns** and explain why they're risky
- **Outline plausible abuse paths** (defensive framing)
- **Propose secure rewrites** (drop-in replacements where possible)
- **Include defense-in-depth guidance** + regression tests/checks

## Resources

| Resource | Link |
|----------|------|
| Models | https://huggingface.co/collections/scthornton/securecode |
| Dataset | scthornton/securecode (2,185 examples) |
| Paper | https://arxiv.org/abs/2512.18542 |

## How to Test It

Copy and paste this prompt with your code:

You are a senior application security engineer. Review the code below.

Output: 
(1) findings with severity, 
(2) likely exploit scenarios (high level),
(3) secure rewrite,
(4) defense-in-depth recommendations, 
(5) regression tests/checks.

Code: `...`


## Dataset Coverage

SecureCode covers both traditional and emerging security domains:
- **Traditional web security** (OWASP Top 10 2021)
- **AI/ML security** (OWASP LLM Top 10 2025): prompt injection, RAG poisoning, model extraction, agentic AI patterns

## We Want Your Feedback

We're looking for real-world contributions:

- **Real snippets**: Share code that "slipped through review once" (sanitized is fine)
- **False positives/negatives**: What didn't work as expected?
- **CVE-grounded examples**: New vulnerability patterns you've encountered

**Please include**: language/framework + what the correct remediation looks like in your environment.

---

**Have contributions or suggestions?** I'd be happy to hear them. Thanks for your support!
scthornton 
submitted a paper to Daily Papers 8 months ago