# Experiments

Use fixed molecular splits and record all seeds. Tune response-head parameters, temperature, and success thresholds on the `tune` partition. Fit the uncertainty radius on `calibration` after choosing those parameters.

## Matched generation comparisons

`remedi evaluate` runs uniform, mean, Gibbs, marginal, absolute, Thompson, finite-scenario minimax, and ReMEDi on identical candidate pools. `per_query.csv` contains endpoint loss, regret, success, best-of-10, dose regret, and timing. `summary.csv` contains molecule-bootstrap intervals. The query compound is excluded from the candidate pool at every dose.

## Uncertainty ablations

Calibrate each uncertainty variant before evaluation:

```bash
for variant in none permuted diagonal no-floor; do
  remedi calibrate --data runs/tahoe/data --model runs/tahoe/model \
    --output runs/tahoe/cal-$variant --ablation $variant
  remedi evaluate --data runs/tahoe/data --model runs/tahoe/model \
    --calibration runs/tahoe/cal-$variant --output runs/tahoe/eval-$variant \
    --ablation $variant --tolerance 1
done
```

`permuted` breaks scenario alignment across candidates. `diagonal` retains marginal variances. `no-floor` removes the residual variance term. `no-target-sampling` is an evaluation ablation with fixed target/control means. Compare `tau` values and ensemble sizes through separate calibration/evaluation runs.

## Representations and response heads

Prepare PCA and frozen per-cell UCE features under the same molecular split. Train fingerprint and cached MolFormer inputs with ridge and MLP heads. Keep an independent expression-space metric for comparisons across cell encoders. `evaluate --metric-data PATH` accepts aligned condition records with that representation and training-derived metric scaling.

## Frozen Tahoe-to-sci-Plex transfer

For PCA, align the raw cohorts on common gene identifiers before fitting:

```bash
python scripts/align_genes.py --inputs tahoe.h5ad sciplex.h5ad \
  --gene-columns gene_symbol index --drop-ambiguous --output data/aligned
```

Fit the Tahoe projector on the aligned Tahoe training cells. Prepare sci-Plex using `--feature-model runs/tahoe/data/cell_feature_model.joblib`. Supply a sci-Plex split CSV containing `smiles,split` with `split=test`. Use an explicit structure mapping and harmonized gene IDs. Molecular overlaps are determined from canonical structures.

```bash
remedi prepare --h5ad data/aligned/1_sciplex.h5ad \
  --mapping configs/sciplex.yaml --structures sciplex_structures.csv \
  --splits sciplex_test.csv --feature-model runs/tahoe/data/cell_feature_model.joblib \
  --output runs/external/data
remedi evaluate --data runs/external/data --model runs/tahoe/model \
  --calibration runs/tahoe/calibration --external-track unseen \
  --output runs/external/evaluation --tolerance 1
```

For `unseen`, every test structure must be absent from Tahoe response-head training. For `shared`, every test structure must be present in that training set. Construct separate cohort files for the two tracks. Record exposure time and cell-line overlap. The scalar-dose head is fitted at the Tahoe exposure time. Transfer across a different exposure time measures an additional distribution shift.

Frozen UCE preparation requires the same checkpoint for both studies. External treated cells are used for queries and final scoring. Fit response parameters, feature transforms, and calibration on the source study.

## Dose, pool-size, and learning curves

The current API predicts at any positive concentration. Scientific validation should begin with the observed dose grids. Middle-dose holdout requires a dedicated condition split in the training data, while keeping outer-dose measurements of those training compounds. The default molecular split is intentionally stricter and does not implement this condition-level experiment.

For pool-size experiments, supply nested catalog CSVs and vary `--shortlist`. Record the candidate IDs, runtime, and solver residual for each run. For learning curves, reduce training compounds while preserving calibration and test compounds, then refit the response head and projector.

```bash
remedi plot-curve --csv scaling.csv --output figures/scaling.pdf \
  --x candidates --y seconds --hue method
```

## Additional manuscript comparisons

The manuscript also proposes CPA/chemCPA, scGen, scVI, transport models, COMs, and molecular-generator comparisons. Those models require their released implementations and compatible study splits. This package implements the ReMEDi computation and its matched generation rules. It does not include substitute implementations under those method names. Keep their measured predictions and resource usage in separate result files, with their checkpoint, version, and training split documented.

## Interpretation

A source-data smoke test establishes execution on measured RNA and dataset metadata. Performance conclusions require the full held-out study comparison. REAL candidates have predicted endpoints until those compounds have independent measured responses.