TraceGRPO

Structured reasoning, program-analysis rewards, and the limits observed in paired vulnerability detection

TraceGRPO is a completed empirical study of C/C++ vulnerability detection with Qwen3-4B: teacher-trace supervised fine-tuning, verifiable-reward GRPO, controlled diagnostics, and classical baselines. This repository preserves the implementation, frozen measurements, failed hypotheses, checkpoints, and recovery archives needed to inspect the full scientific record.

Status: empirical program complete; archival maintenance only. Final TEST was consumed once per registered system and remains closed. No training, inference, model selection, or additional TEST measurements are part of repository maintenance.

Start here: master evidence report · experiment registry · reproduction · archive / recovery · decisions

What the experiments established

1. Labels are learnable in the small control. C-L1 reaches 100% fixed-prefix accuracy on the same 200 training rows, while free generation ends at 14.5%. This isolates label learnability from unsupervised format generation; it is not evidence of generalization. Frozen curve.
2. Teacher-reasoning imitation did not yield useful discrimination in the tested runs. Three SFT attempts failed their VALID gate. The teacher audit found 16/30 materially unsupported rationales, including 4/30 explicit metadata leaks. These are distinct findings, not a “53% metadata leak” claim. SFT chapter.
3. Reward design explains measurable behavior. Exact counterfactual replay covered 18,722/18,722 records. The vulnerable-label multiplier plus difficulty weighting produced the prior tilt; the SAFE empty-set R2 convention encouraged empty evidence emission. Counterfactual analysis.
4. Balanced rewards changed behavior without rescuing the registered discrimination endpoint. E4b step-200 VALID ΔAUC versus SFT-500 was −0.0077 [−0.0253, +0.0096], below the pre-registered +0.03 target. This is an efficacy-negative result with verified behavioral predictions. Frozen endpoint.
5. Oracle and deployable slicing must remain separate. The oracle bound is non-deployable; E1 predicted slicing failed its feasibility gate. Neither result establishes that all slicing methods fail. E1/E3 evidence.

Final chronological TEST results

2,526 rows / 1,263 pairs. Each neural system generated 8 completions per row (temperature 0.8, seed 42, maximum 1,536 completion tokens); the classical system used one deterministic pass. AUC uses the frozen score aggregation; CIs use pair-cluster bootstrap. Label accuracy uses the frozen vote rule. No threshold was tuned on TEST.

System	Label accuracy	SAFE accuracy	VULNERABLE accuracy	Row AUC [95% CI]	Statement IoU	P-Pass@1
Classical · TF-IDF + SGD	52.53%	52.89%	52.18%	0.5307 [0.5241, 0.5374]	0.0964	7.28%
SFT v3 · checkpoint 500	51.62%	56.77%	46.48%	0.5165 [0.5043, 0.5291]	0.1550	6.66%
Production GRPO · step 324	50.24%	18.13%	82.34%	0.5201 [0.5073, 0.5326]	0.1117	4.75%
Balanced GRPO E4b · step 200	51.39%	41.73%	61.05%	0.5172 [0.5041, 0.5302]	0.1228	6.32%

Source for every table cell: test_final_metrics.json. See final TEST report for Wilson intervals, paired comparisons, parse rates and protocol details. Trace R3 is unavailable, because 0/1,263 vulnerable TEST rows had eligible cached graph artifacts; unavailable is not a zero-quality score. The aggregation incident was resolved using persisted outputs, without another TEST read or inference pass.

Neural discrimination improvements were not established under these protocols. The classical result is modest, not production-grade security detection. Strong formatting and balanced predictions do not establish semantic correctness.

Why the controls matter

This animation reveals the actual recorded measurements every 25 optimizer steps with fixed axes. C-L1's fixed-prefix and free-generation metrics answer different questions. It does not show performance on held-out functions.

The reward-equilibrium check is a behavioral mechanism test. Training vote rates and component rewards are never treated as validation accuracy. E4b's balanced prior and evidence emission are consistent with the predicted mechanism; its negative AUC endpoint remains negative.

Repository map

tracegrpo/
├── README.md                 project overview and frozen headline results
├── src/                      data, program analysis, contracts, rewards, SFT/RL, evaluation
├── configs/                  original and frozen experiment configurations
├── scripts/                  pipeline entry points, diagnostics, export/recovery tools
├── tests/                    unit, contract, provenance and checkpoint-restore tests
├── docs/                     master report, architecture, reproduction, decisions, registries
│   ├── prompts/              two original prompts, byte-preserved
│   └── history/              one index of original reports and preservation records
├── figures/                  reproducible SVG/PNG scientific figures and provenance
│   └── animations/           measured C-L1 curve
├── data/                     frozen data at original paths; no cleanup relocation
├── models/                   persisted classical artifacts and model hash manifests
├── checkpoints/              frozen SFT/control adapters at original paths
├── results/                  original phase evidence, logs and registered measurements
├── audit/                    canonical grading packages and separate LLM-judge results
├── releases/                 immutable versioned bundles and verification records
└── archives/                 preserved historical recovery snapshots

Source map · script guide · configuration guide · model catalog · audit guide · all figure captions. Historical report-wrapper pages have been replaced by a single index; original scientific reports remain intact in results/.

Data and evaluation boundaries

• 27,314 corpus rows / 13,657 vulnerable-fixed pairs from PrimeVul and SecVulEval; chronological TRAIN has 8,232 pairs, VALID 1,104 pairs, TEST 1,263 pairs. Data / split summaries.
• Full annotation coverage: L1/L2 approximately 99.72%; trace GT approximately 70.67% of L1-covered vulnerable samples. Patch statements are sinks; backward def-use slices provide trace ground truth. Coverage.
• OOD is frozen at 134 pairs; collection/quarantine and contamination checks are documented. It is not the final chronological TEST and was not substituted into the audit. Funnel.
• Gold and silver teacher traces are distinct. Silver traces are an SFT data category, not reward ground truth. Teacher generation can see metadata; train/eval inference prompts cannot.
• The completed 150-card audit is an LLM-judge audit, not a human study: Claude Opus 5.5, high effort. Context carryover, a card-132 revision, same-VM blinding limitations and balanced-sample prevalence caveats are disclosed. Analysis. The separate human grading package is retained without claiming completed human grades.

Models and reproducibility

Neural artifacts are pinned base revision + adapter checkpoint, not self-contained merged models. The base is Qwen3-4B; the pinned revision and artifact hashes are in REPRODUCTION.md and CHECKPOINT_CATALOG.md. Production GRPO-324 has full state; step 399 is inference-only; no step-400 checkpoint is claimed. E4b step 200 is the registered balanced-reward endpoint. Prior failed and interrupted checkpoints remain preserved.

The captured stack includes Torch 2.8.0+cu128, Transformers 4.56.2, vLLM 0.10.2 and TRL 0.21.0. Per-run manifests and the environment freeze take precedence over this summary. The classical pipeline is loaded from persisted, hash-verified assets under models/classical_phase7_v1/.

Maintenance commands (no experiments):

source .venv/bin/activate
python -m pytest -q
python scripts/build_publication_assets.py
python scripts/verify_phase1_layout.py

The figure builder opens only registered aggregates and TRAIN diagnostic logs; it does not open datasets, split contents, checkpoints or raw TEST predictions. Do not execute historical training/evaluation commands merely to regenerate presentation assets.

Recovery and integrity

Use the complete snapshot, terminal-result overlays, archival report bundle and Phase 1 presentation overlay in the order specified in HF_ARCHIVE.md. Verify archive SHA-256 and file manifests before extraction. Existing large archives remain the recovery authority; this cleanup does not duplicate the research dataset or rewrite Git history.

The repository audit, Phase 1 completion report, preservation map and remote checksum evidence document every approved removal. Phase 2 preserves all 71 selected checkpoint/tokenizer paths using read-only payload hardlinks, reclaiming 1.28 GiB of local allocated space. Independent checkpoint state and all research bytes remain intact. Tokens, authentication files, answer keys and sealed joins are excluded from uploads. Local credentials are obtained through the CLI environment/auth store.

Reading the scientific record

The master report contains the complete SFT, GRPO, E1–E4b, baseline, audit and TEST chapters, compute ledger, deviations and supported/unsupported claims. The decision log records review authority. This repository supports an empirical analysis of the tested mechanisms and failures; it does not support claims that all RL, reasoning supervision or neural vulnerability detectors fail.

Archive-backed cache browsing

The HF tree keeps assembled datasets and frozen splits visible. Individually browsable API/graph caches and teacher batches are preserved in verified recovery bundles; local paths remain unchanged. See clutter classification and Phase 3 report and cache restore instructions. No scientific results were changed.