PIVOT / docs /results.md
pranamanam's picture
Upload 176 files
6fa9282 verified
|
Raw
History Blame Contribute Delete
3.49 kB

Result provenance and manuscript changes

The original Hugging Face revision is recorded in reproducibility/upstream.json. Original experiment JSON files and trained checkpoints are preserved. The draft reports those measurements with their original preprocessing scope; corrected full-scale tables remain unfilled.

Manuscript comparison Source result records
Response accuracy and baseline comparison norman_benchmark.json, norman_distloss.json
Single-gene nomination norman_benchmark.json, norman_nom_improved.json
Pair retrieval and residual correlation norman_combinatorial.json, norman_extra_ablations.json, norman_timing_scaling.json
GEARS response and ranking pivot_vs_gears.json, gears_ranking.json
Loss components norman_ablation_components.json
Reward and guidance sweeps norman_ablation_reward.json, norman_ablation_guidance_steps.json
Larger single-gene catalog replogle_k562_nom_improved.json

Numerical records are in results/legacy/. The original source is in reproducibility/original_source.zip. It includes the historical GEARS workflows. Extract it into a separate directory when reproducing those experiments and use the corresponding original preprocessing. Historical model weights cannot be loaded with a corrected cache.

The manuscript now leads with population matching and held-out pair nomination. It retains additive ranking and the cell-conditional MLP because they are necessary comparisons for those claims. Secondary experiments and detailed implementation differences appear in the appendix.

Substantive corrections:

  1. PCA is a lossy projection with an affine reconstruction. Historical expression error and DE correlation used reconstructed observed expression as well as reconstructed predictions.
  2. Feature selection and PCA were fitted before splitting. Some evaluation routines pooled all cells carrying a test label, creating training-cell overlap in the cell-holdout analysis.
  3. The original semigroup branch used a state interpolated at a different starting time. The corrected branch recomputes the source state at its sampled time.
  4. The default historical DE correlation used 20 selected coordinates. It differs from GEARS Pearson correlation on absolute expression.
  5. The GEARS nomination task used 21 targets and 101 single-gene candidates; PIVOT used 22 and 105. Pair candidate pools were 128 and 131, respectively.
  6. Several earlier significance statements referred to Top-1 while the underlying calculations used Top-5. Two summary records disagreed on a random-comparison bootstrap result. The draft uses recovery counts and the stored interval without those significance claims.
  7. Effect clusters are expression-derived groups. They do not establish pathway recovery. Their high random-recovery rate appears in the paper.
  8. Greedy search returned one set. The duplicated historical Top-5 proxy has been removed.
  9. The recorded pair residual correlation subtracts the fitted additive baseline from both predicted and observed effects. A confirmatory residual analysis should use independent single-gene response cells, residual MSE, and zero/shuffled residual controls.
  10. Guidance initialization, backward evaluations, and projection costs must be counted. A best-candidate initialization includes an exhaustive pass.

The shared Figure 1 is schematic. No synthetic performance curves or invented experimental measurements are included.