Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
scthornton 
posted an update 3 days ago
Post
2574
SecureCode update: we went back and fact-checked our own security dataset and corrected what didn't hold up.

The original claim was "complete incident grounding, every example ties to a documented CVE." An adversarial re-audit found that it was overstated: many CVEs were misattributed, and many "incidents" were representative scenarios carrying invented statistics. So we fixed it.

- Grounding: re-verified every reference. Removed 802 misattributed CVEs on the web side, corrected or honestly relabeled the incident narratives, and confirmed the AI/ML conversation CVEs are real (EchoLeak CVE-2025-32711, EmailGPT CVE-2024-5184, and others).
- Fix-correctness: reviewed whether each "secure" example actually eliminates the vulnerability. Removed 28 that did not (a "secure" secret scanner whose entropy check always returned zero, an Angular example still using bypassSecurityTrustHtml, and more).
- Leakage: re-split so near-duplicates stay on one side. Test contamination went from 11.6% to zero.
- Viewer, schema, and metadata: rebuilt as parquet under a shared schema. All three viewers are live.
- Models: retrained the whole family on the corrected data so the fix reaches the weights, not just the cards. Now ten open models (3B to 26B), including two new Gemma 4 variants, refreshed locally on a DGX Spark GB10. The paper (arXiv:2512.18542) was revised to match.

Counts moved from 2,185 to 2,372 unified (web 1,625 + AI/ML 747). A slightly smaller, fully-checked dataset beats a larger one you have to take on faith. Full writeup and links in the article.

Datasets: scthornton/securecode, scthornton/securecode-web, scthornton/securecode-aiml
scthornton

https://huggingface.co/blog/scthornton/securecode-audit-fixes
dipankarsarkar

The 802 is the safer number. The 28 is the one I would build around.

A misattributed CVE has an external referee. The database says yes or no, you re-check all 1,379, you drive it to zero, and it stays there. That work is finished.

The 28 are a different class. A SecretScanner that computes Shannon entropy and always returns 0.0 is not a wrong citation. It is code that looks like a fix and passes everything that never feeds it a real secret. Nothing outside the repo says it is wrong. A reader noticed.

You reviewed all 1,412, and setting the review up so a claim only survives if it can't be refuted is the right instinct. But zero disputed is a precision number. It says what review flagged held up. It says nothing about what review walked past. Full coverage is not full recall, and the one failure mode that matters has no false-negative estimate on it.

That part is mechanizable. Every secure example claims a specific vulnerability is gone. Make it prove it: feed the paired vulnerable input through the secure version and require the check to fire. The entropy-always-zero one dies on the first positive control, with no reviewer and no referee argument to have.

We ran the same move on LLM-generated GPU kernels. Mutation-scored the oracle instead of the artifact, and torch.allclose waved through buggy variants at a rate that made the oracle the actual problem.

So run positive controls across the examples you kept. How many fire? That number is the recall you don't have yet, and it's the only thing that tells you whether 28 was the count or just the start.

·
scthornton

Yep, you've named the real weakness, and you are right.

The 802 and the 28 are different epistemics. The CVEs had an external referee, so rechecking all 1,379 and driving it to zero is a final result that stays final. The 28 had no referee, and "zero disputed" is a precision number: it says what the review flagged held up, not what the review walked past. Human review cannot estimate its own false-negative rate, and "a reader noticed" is the honest account of how the entropy-always-zero scanner surfaced. A reader is not a guarantee.

Positive controls are the right fix, and I'm in the process of running them now. For each secure example that claims a specific vulnerability is eliminated, push the paired vulnerable input through the secure path and require the guard to fire. The entropy scanner dies on the first one with no argument to have. That turns "does the fix work" from a review opinion into an executable assertion, and the fraction that fires is the recall figure I do not currently have.

The honest constraint: a slice of the corpus is illustrative multi-turn snippets, not standalone programs with a defined input interface, so I can't mechanize it uniformly. But the failure mode you are pointing at lives in the executable subset, a scanner, a sanitizer, an auth gate, and those are exactly the ones I'm working to wrap in positive controls. Plan: isolate the runnable, secure paths; write a positive control per stated vulnerability; publish the fire rate as a real recall number; and call out the non-executable remainder separately rather than letting it hide inside a pass.

The oracle point is the sharpest part, and I hadn't framed it that way. Mutation-scoring the oracle instead of the artifact, torch.allclose waving through buggy variants, is the same disease as a review that flags nothing on a fake fix. Measuring the oracle is the move I skipped, and I am going to steal it. Thank you for that!

If 28 was just the count review that happened to catch, the positive-control pass will say so. I would rather publish that number than keep the precision one. Thank you sincerely for taking the time to share this.
dipankarsarkar

The non-executable slice is the part I would think hardest about, because it's about to inherit a number it didn't earn.

Once the positive-control pass gives you a fire rate on the runnable subset, that rate is going to read as the corpus's number. It isn't. It's the number for the examples that happened to be shaped like standalone programs.

And there's no reason to expect the remainder is better. If anything the opposite. An illustrative multi-turn snippet never had to run, so nothing ever pushed back on it. The entropy scanner survived because nobody fed it a secret. A snippet nobody can feed anything is that same condition, made permanent.

So I would publish it as two numbers, not one number plus a caveat. Measured recall on the executable subset. Then the remainder reported as unmeasured, carrying the executable rate over as the optimistic prior rather than leaving it as unknown. Unknown rounds to fine in a reader's head. An explicit prior doesn't.

How big is that slice, as a fraction of the 1,412?
In this post