Experiments
Use fixed molecular splits and record all seeds. Tune response-head parameters, temperature, and success thresholds on the tune partition. Fit the uncertainty radius on calibration after choosing those parameters.
Matched generation comparisons
remedi evaluate runs uniform, mean, Gibbs, marginal, absolute, Thompson, finite-scenario minimax, and ReMEDi on identical candidate pools. per_query.csv contains endpoint loss, regret, success, best-of-10, dose regret, and timing. summary.csv contains molecule-bootstrap intervals. The query compound is excluded from the candidate pool at every dose.
Uncertainty ablations
Calibrate each uncertainty variant before evaluation:
for variant in none permuted diagonal no-floor; do
remedi calibrate --data runs/tahoe/data --model runs/tahoe/model \
--output runs/tahoe/cal-$variant --ablation $variant
remedi evaluate --data runs/tahoe/data --model runs/tahoe/model \
--calibration runs/tahoe/cal-$variant --output runs/tahoe/eval-$variant \
--ablation $variant --tolerance 1
done
permuted breaks scenario alignment across candidates. diagonal retains marginal variances. no-floor removes the residual variance term. no-target-sampling is an evaluation ablation with fixed target/control means. Compare tau values and ensemble sizes through separate calibration/evaluation runs.
Representations and response heads
Prepare PCA and frozen per-cell UCE features under the same molecular split. Train fingerprint and cached MolFormer inputs with ridge and MLP heads. Keep an independent expression-space metric for comparisons across cell encoders. evaluate --metric-data PATH accepts aligned condition records with that representation and training-derived metric scaling.
Frozen Tahoe-to-sci-Plex transfer
For PCA, align the raw cohorts on common gene identifiers before fitting:
python scripts/align_genes.py --inputs tahoe.h5ad sciplex.h5ad \
--gene-columns gene_symbol index --drop-ambiguous --output data/aligned
Fit the Tahoe projector on the aligned Tahoe training cells. Prepare sci-Plex using --feature-model runs/tahoe/data/cell_feature_model.joblib. Supply a sci-Plex split CSV containing smiles,split with split=test. Use an explicit structure mapping and harmonized gene IDs. Molecular overlaps are determined from canonical structures.
remedi prepare --h5ad data/aligned/1_sciplex.h5ad \
--mapping configs/sciplex.yaml --structures sciplex_structures.csv \
--splits sciplex_test.csv --feature-model runs/tahoe/data/cell_feature_model.joblib \
--output runs/external/data
remedi evaluate --data runs/external/data --model runs/tahoe/model \
--calibration runs/tahoe/calibration --external-track unseen \
--output runs/external/evaluation --tolerance 1
For unseen, every test structure must be absent from Tahoe response-head training. For shared, every test structure must be present in that training set. Construct separate cohort files for the two tracks. Record exposure time and cell-line overlap. The scalar-dose head is fitted at the Tahoe exposure time. Transfer across a different exposure time measures an additional distribution shift.
Frozen UCE preparation requires the same checkpoint for both studies. External treated cells are used for queries and final scoring. Fit response parameters, feature transforms, and calibration on the source study.
Dose, pool-size, and learning curves
The current API predicts at any positive concentration. Scientific validation should begin with the observed dose grids. Middle-dose holdout requires a dedicated condition split in the training data, while keeping outer-dose measurements of those training compounds. The default molecular split is intentionally stricter and does not implement this condition-level experiment.
For pool-size experiments, supply nested catalog CSVs and vary --shortlist. Record the candidate IDs, runtime, and solver residual for each run. For learning curves, reduce training compounds while preserving calibration and test compounds, then refit the response head and projector.
remedi plot-curve --csv scaling.csv --output figures/scaling.pdf \
--x candidates --y seconds --hue method
Additional manuscript comparisons
The manuscript also proposes CPA/chemCPA, scGen, scVI, transport models, COMs, and molecular-generator comparisons. Those models require their released implementations and compatible study splits. This package implements the ReMEDi computation and its matched generation rules. It does not include substitute implementations under those method names. Keep their measured predictions and resource usage in separate result files, with their checkpoint, version, and training split documented.
Interpretation
A source-data smoke test establishes execution on measured RNA and dataset metadata. Performance conclusions require the full held-out study comparison. REAL candidates have predicted endpoints until those compounds have independent measured responses.