SFT_STALE.md

# Bucket data status — what's verified, what's pending

**Issue (root-caused 2026-05-02)**: HF repo `UCL-CSSB/PlasmidGPT-SFT` had two safetensors files; `AutoModelForCausalLM.from_pretrained` defaulted to a Base-clone duplicate. Cleanup commit `daeaabf` swapped them. SHA changed from `5748e6f9` → `daeaabf0`. `models/pinned_shas.csv` is updated.

## Verified clean (post-2026-05-04 audit)

- `evaluation/eight_prompt/{Base,SFT,RL}/` — analysis2 strict QC; 4.275 / 10.975 / 71.575
- `evaluation/eight_prompt/ablations/full_reward/` — GRPO @ T=0.95 = 66.875%
- `evaluation/eight_prompt/ablations/{cds_only,length_only,no_cassette_bonus,no_length_prior,no_repeat_penalty}/` — Table 7 rows 2-6
- `analysis/distribution_metrics.csv` + `analysis/distribution/per_seq_*.csv` — Table 6 source
- `continuation_benchmark/eval_set_656/` — 656 plasmids × 5 splits (PRIMARY Table 5 source)
- `continuation_benchmark/heldout_eng_r3/` — PLSDB-style F1–F6 NCBI queries
- `continuation_benchmark/{both_metric_eval, validation_eval, holdout30_non_addgene}/` — additional held-out evals
- `mfe/SFT_real/` — replaces stale; mean −0.148 (matches paper −0.149)
- `mfe/{SFT_circ10k_subset, SFT_temp_sweep, RL_t1.15_8prompt, RL_temp_sweep_2prompt}/` — additional MFE coverage
- `rejection_v3/{Base,SFT,GRPO}/` — 8-prompt × 1250 = 10K, analysis2 strict QC, sweep-optimal T
- `rejection_topK/` — M=50 attempts × K∈{1,4,16,64} success rates
- `plannotate/{RL, Base_t0.95, SFT_t0.95}/` — Table 8 sources
- `novelty_blastn/summary.csv` — Table 2
- `reference/addgene_500/`, `original_paper/`, `models/`, `code_snapshots/` — auxiliary

## SFT-stale files NOT yet replaced

- `rejection_sampling_v2/direct/SFT/` and `rejection_sampling_v2/best_of_16/SFT/` — the original Table 4 SFT cells. Kept in place for paper reproducibility but **superseded by `rejection_v3/SFT/`** if camera-ready uses the new 8-prompt protocol. Old numbers (7.15% / 32.4%) used pre-fix SFT checkpoint; new (10.87% / —) uses corrected checkpoint.
- `evaluation/temperature_sweep/SFT_t0.95/` — generations with broken checkpoint; appendix material only

The original `continuation_benchmark/{completion,surprisal}_benchmark.csv` (small 11-plasmid set) was kept as legacy data; numbers reproduce paper Table 5 (Base −12.449, RL −10.966) but the new `eval_set_656/` is much more rigorous.

## Unaffected — known good throughout

- `evaluation/eight_prompt/Base/`, `mfe/Base/`, `mfe/RL/` (= old `GRPO_temp1.0`), `mfe/ablations/*/`
- `rejection_sampling_v2/direct/{Base,GRPO}/`, `rejection_sampling_v2/best_of_16/{Base,GRPO}/`
- `evaluation/eight_prompt/ablations/*/` (ablation models from McClain/plasmidgpt-rl-*)
- `plannotate/RL/`, `novelty_blastn/`, `reference/`, `original_paper/`