INDEX.md

UCL-CSSB/PlasmidRL-ICML — index of canonical artifacts

ICML camera-ready artifacts for Effects of Structural Reward Shaping on Biophysical Properties in RL-Trained Plasmid Generators.

Lineage (parallel post-training paths from the same Base):

Base = UCL-CSSB/PlasmidGPT
├─→ SFT next-token loss      → UCL-CSSB/PlasmidGPT-SFT  (sha daeaabf, post-2026-05-02 cleanup)
└─→ GRPO reward shaping       → UCL-CSSB/PlasmidGPT-GRPO  (sha db2462a)

The 5 reward-ablation models branch from SFT, separately. McClain/PlasmidGPT-RL also branches from SFT — kept in deprecated/early_rl_lineage/ as appendix material.

Headline numbers (analysis2 strict QC, 8-prompt eval, T-matched)

Model	T=1.0 (n=4000)	T=0.95 (n=4000)	rejection_v3 (n=10K, 8-prompt @ sweep-optimal T)
Base	4.275%	—	3.99% (T=1.0)
SFT	10.975%	—	10.87% (T=1.0)
GRPO	71.575%	66.875%	78.66% (T=1.15)

Headline lift (GRPO/Base @ T=1.0): ~16.7×.

Layout

README.md                             interim
INDEX.md                              this file
SFT_STALE.md                          status flags

analysis/                             truth-set CSVs + distribution metrics + plannotate
├── distribution_metrics.csv          per-cell length/GC/ORF/JSD/Jaccard
├── distribution/per_seq_{Base,SFT,RL}.csv
├── distribution_report.html
└── (table1_*, table4_*, ... pending manifest build)

continuation_benchmark/               held-out continuation + surprisal benchmarks
├── eval_set_656/                     656 plasmids × 5 splits — primary eval
│   ├── summary.json
│   ├── per_split_{completion,surprisal}.csv
│   ├── per_plasmid_{completion,surprisal}.csv
│   ├── all_{completion,surprisal}.csv  (window-level)
│   ├── full_set.fasta
│   ├── metadata.tsv
│   └── report.html
├── heldout_eng_r3/                   PLSDB-style (F1-F6 NCBI queries) engineered held-out
│   ├── summary.json
│   ├── all_{completion,surprisal}.csv
│   └── report.html
├── both_metric_eval/                 47 archetype-matched (5 archetypes)
│   ├── summary.json
│   ├── joint_per_plasmid.csv
│   ├── per_plasmid_{completion,surprisal}.csv
│   ├── metadata.tsv
│   └── both_metric_candidates.fasta
├── validation_eval/                  80-plasmid regression test (3 strata)
│   ├── summary.json
│   ├── per_plasmid_{completion,surprisal}.csv
│   ├── metadata.tsv
│   └── validation_set.fasta
└── holdout30_non_addgene/            29 curated non-Addgene
    ├── holdout30_non_addgene.csv
    └── holdout30_non_addgene.fasta

evaluation/                           generation outputs + QC
├── eight_prompt/{Base,SFT,RL}/       Table 1 sources at T=1.0
└── eight_prompt/ablations/{full_reward,5×reward_ablations}/   Table 7

mfe/                                  MFE under DNA Mathews 2004 params
├── Base/                              n=4000 SFT-fixed Base, T=0.95 (paper original)
├── RL/                               n=4000 GRPO @ T=1.0 (= old GRPO_temp1.0)
├── ablations/{...}/                  5 reward-ablation models
├── SFT_real/                         n=4000 SFT @ T=1.0 (replaces stale mfe/SFT/) — mean −0.148
├── SFT_circ10k_subset/               96 stratified, circular-folding for short seqs — mean −0.172
├── SFT_temp_sweep/                   200/T at T={0.5, 0.8, 0.95, 1.0, 1.15, 1.3}
├── RL_t1.15_8prompt/                 GRPO @ T=1.15 (sweep-optimal) — mean −0.155
└── RL_temp_sweep_2prompt/            GRPO across T={0.5, ..., 1.3}, 2-prompt protocol

rejection_sampling_v2/                ORIGINAL paper Table 4 source (2-prompt, plasmidkit-loose QC)
├── direct/{Base,SFT,GRPO}/           SFT cell still uses pre-fix checkpoint — see SFT_STALE.md
└── best_of_16/{Base,SFT,GRPO}/

rejection_v3/                         NEW — 8-prompt × 1250 = 10K, analysis2 strict QC
├── Base/    metadata.json (3.99%)
├── SFT/     metadata.json (10.87%)
└── GRPO/    metadata.json (78.66% @ T=1.15)

rejection_topK/                       NEW — top-K-of-K sampling success rate (M=50 trials, 8 prompts)
├── summary.json                       success rate per (model, K∈{1,4,16,64})
├── success_per_model_K.csv
├── success_summary.csv               per-prompt breakdown
├── diversity.csv                     Jaccard similarity of kept samples
├── ori_usage.csv                     ORI breakdown of kept samples
├── amr_usage.csv                     AMR breakdown of kept samples
├── per_attempt.csv                   trial-level data
├── kept_samples.csv + .fasta         all selected samples

plannotate/                           Table 8 — pLannotate-detected ORI breakdown
├── RL/                               GRPO @ T=1.0
└── {Base_t0.95, SFT_t0.95}/          supplementary (T=0.95 versions)

novelty_blastn/summary.csv            Table 2 (n=22/28/30 BLAST against Addgene)

reference/addgene_500/                Reference panel: plasmids.csv + metrics.csv + 3mer_freqs.json

models/pinned_shas.csv                8 model commit SHAs (SFT updated to daeaabf)

code_snapshots/                       git SHAs (PlasmidRL, analysis2, plasmid-rl-paper-2)

manifests/                            pending — paper_v2_camera_ready.json + deprecated.json

deprecated/                           audit trail (v1 baselines, early RL lineage, old figures)

original_paper/                       frozen pre-revision data

Key new findings vs paper draft

1. Headline lift is ~16.7×, not 2.7× — from QC-pipeline tightening (analysis2 strict QC). The 71.6% RL number is unchanged; Base/SFT drop because the loose QC was overly permissive.

2. Alignment tax on continuation logprob is real and replicated: across 656/47/80-plasmid evals, RL is −2 to −3 nats per window worse than SFT on continuation. RL wins SFT in 0–12% of plasmids. Paper's "evidence too thin to claim alignment tax" should flip to "alignment tax is measurable; RL trades next-token prediction for QC pass rate".

3. Lineage is parallel, not serial: GRPO trained from Base, not from SFT. Abstract / §3.2 framing of "RL preserves the SFT-induced thermodynamic manifold" needs rewording — RL didn't inherit what it never saw. Both SFT and GRPO independently land near real-plasmid MFE / JSD / ORF length via different mechanisms.

4. SFT generates plasmid-scale sequences (mean length 5,441 bp, ORF 272 aa) — old SFT data showed Base-like 1,970 bp due to the model.safetensors checkpoint dispatch issue. SFT's MFE of −0.148 happens to match the paper's reported −0.149.

5. Diversity convention: the paper uses Jaccard distance (~1.0 = diverse). New distribution_metrics.csv reports Jaccard similarity (low = diverse). Convert via distance = 1 − similarity when reading. Paper's "RL diversity 0.573" = new "RL Jaccard similarity 0.426". Same signal.

Ablation table (T=1.15, post-2026-05-05)

evaluation/eight_prompt/ablations/manifest.json is the source of truth. Pass rates at sweep-optimal T=1.15 (matching rejection-sampling protocol):

Ablation	Pass%	MFE (kcal/mol/bp)	Diversity (1−Jaccard)	T=0.95 pass% (deprecated)
full_reward	78.35%	−0.165	0.585	66.88%
no_repeat_penalty	75.15%	−0.151	0.419	72.17%
no_length_prior	72.15%	−0.140	0.459	71.38%
no_cassette_bonus	44.52%	−0.170	0.369	19.80%
length_only	37.90%	−0.131	0.772	34.73%
cds_only	1.73%	−0.130	0.861	2.40%
Addgene baseline	—	—	0.925	—

• Pass rate from evaluation/eight_prompt/ablations/{cell}/qc/qc_summary.csv (n=4000 generations per cell, analysis2 strict QC).
• MFE from evaluation/eight_prompt/ablations/{cell}/mfe/mfe_summary.json (n=200 random subset per cell at T=1.15; cds_only n=69 because only 69 sequences pass strict QC; computed with ViennaRNA 2.7.2 / Mathews 2004 DNA params, 1000 bp window).
• Diversity: 1 − mean pairwise 21-mer Jaccard similarity (n=200 sampled from passing sequences, except cds_only n=69). Addgene baseline (n=200 from reference) = 0.9245.
• Per-cell metadata.json carries seed, sampling params, model SHA, and sha256 of every output file.

Cassette-bonus removal is still the largest single-component drop (78.35 → 44.52 = 33.8 pp). T=0.95 data preserved at deprecated/ablations_t0.95/ (bucket-state archive) and deprecated/ablations_t0.95_source/ (strict-QC source files at T=0.95).

Pending decisions

• Table 4 protocol: keep paper's 2-prompt v2 (plasmidkit-loose QC) or switch to 8-prompt rejection_v3 (analysis2 strict QC)?
• Table 5 source: smaller 11-plasmid (paper-original numbers reproduce) or larger 656-plasmid eval_set?
• MFE protocol: 8-prompt full or 2-prompt protocol for the camera-ready Table 6 row?
• Table 7 row 1 number: 66.875% (GRPO @ T=0.95) recommended; awaiting final paper-text confirmation.