README.md

# UCL-CSSB/PlasmidRL-ICML

Camera-ready artifacts for **"Effects of Structural Reward Shaping on Biophysical Properties in RL-Trained Plasmid Generators"** (ICML 2026, to appear).

See `INDEX.md` for the full per-folder navigation. See `SFT_STALE.md` for data-status flags. This README is the 30-second summary.

## Headline results

We apply Group Relative Policy Optimization (GRPO) to fine-tune `UCL-CSSB/PlasmidGPT` (Base) for whole-plasmid generation, evaluated across 8 prompts on 4,000 sequences each under analysis2 strict QC.

| Model | T | QC pass rate (8-prompt) |
|---|---:|---:|
| Base (`UCL-CSSB/PlasmidGPT`) | 1.0 | **4.275%** |
| SFT (`UCL-CSSB/PlasmidGPT-SFT`) | 1.0 | **10.975%** |
| RL = GRPO (`UCL-CSSB/PlasmidGPT-GRPO`) | 1.0 | **71.575%** |

Lift: ~16.7× over Base, ~6.5× over SFT.

Rejection sampling top-K (M=50 trials × 8 prompts):

| K | Base | SFT | GRPO |
|---:|---:|---:|---:|
| 1  | 4.25% | 9.75% | **76.75%** |
| 4  | 14.5% | 36.25% | **95.0%** |
| 16 | 38.75% | 76.25% | **99.0%** |
| 64 | 54.5% | 99.25% | **100%** |

## Lineage (parallel post-training paths)

```
Base = UCL-CSSB/PlasmidGPT  (= McClain/plasmidgpt-addgene-gpt2; same SHA, both public)
├─→ SFT next-token loss      → UCL-CSSB/PlasmidGPT-SFT  (sha daeaabf)
└─→ GRPO reward shaping       → UCL-CSSB/PlasmidGPT-GRPO  (sha db2462a)
```

Reward-component ablation models (`McClain/plasmidgpt-rl-{cds_only, length_only, no_cassette_bonus, no_length_prior, no_repeat_penalty}`) all branch from SFT.

## Where to look

- **Per-claim sources** — `INDEX.md` maps each paper Table/Figure to its bucket path
- **Continuation/surprisal benchmarks** — `continuation_benchmark/eval_set_656/` (primary, 656 plasmids × 5 splits)
- **Rejection sampling** — `rejection_topK/`, `rejection_v3/`, and the older `rejection_sampling_v2/` (Base+GRPO cells preserved; SFT cells moved to `deprecated/early_sft_checkpoint/` after model.safetensors fix)
- **MFE under DNA Mathews 2004** — `mfe/` with per-model + temperature-sweep folders
- **8-prompt eval** — `evaluation/eight_prompt/{Base, SFT, RL, ablations/...}/` with strict-QC artifacts
- **pLannotate ORI breakdown (Table 8 source)** — `plannotate/RL/`
- **Reference panel** — `reference/addgene_500/` (n=500)

## Reproducibility

- `models/pinned_shas.csv` — exact commit SHAs for the 8 surviving model repos
- `code_snapshots/{PlasmidRL, analysis2, plasmid-rl-paper-2}.sha` — paper repo + analysis pipeline + training repo HEADs
- Each per-cell `metadata.json` has the seed, sampling params, sha256 of outputs, and the analysis2 strict-QC pipeline name + thresholds
- W&B training runs: `ucl-cssb/PlasmidRL` (Nov 2025 GRPO production) + `ucl-cssb/plasmid-rl-icml-revision` (March 2026 ablations)

## License + citation

Bucket data: CC-BY-4.0 (TBD — confirm before public release).
Models: see individual repo cards.
Citation: TBD on paper acceptance.