PIVOT / docs /data.md
pranamanam's picture
Upload 176 files
6fa9282 verified
|
Raw
History Blame Contribute Delete
2.68 kB

Data and partitions

The full adapters read count matrices from scPerturb record 10044268. Norman contains K562 CRISPR activation experiments with single-gene and two-gene conditions. The Replogle adapter selects the K562 essential-gene interference screen. The source files are approximately 0.70 GB and 1.55 GB, respectively.

Required input: an AnnData .h5ad file with unique cell and feature IDs, a nonnegative count matrix, and obs.perturbation. The control label is control; underscore-separated gene names denote combinations. Optional context columns default to gemgroup and celltype; they can be set explicitly with CLI flags. For Replogle, pass --batch-col batch --celltype-col cell_line to preserve the source context columns. Missing context columns produce one documented dataset-wide context. A requested batch-matching operation fails if no allowed controls match a response cell.

The cache contains normalized sparse expression, PCA coordinates, basis and mean, feature order, cell metadata, and checksums. PCA is an affine reconstruction of retained components and is lossy. Metadata-only gene identities may include held-out genes. Their learned embedding rows receive no response supervision when those genes are absent from training.

Partition regime Unit assigned to a partition
cell Cells within every condition, 70/15/15
perturbation Complete intervention labels, approximately 70/10/20
combination Two-gene conditions, approximately 70/10/20; singles remain in training
gene Gene identities; every containing intervention follows the held-out gene

Controls are partitioned independently. Approximately 10% of each condition's cells are reserved as reference outcomes before fitting expression features. Reference cells do not fit the model, determine validation selection, or form target queries. They are used to measure the response quality of a nominated action. The independent experimental unit for summary intervals is a condition. Separate cells from one screen do not establish cross-study biological replication.

The included fixture has 920 measured Norman cells, 20 perturbed conditions, 120 controls, and 33,694 source features. The selection uses seed 17. provenance.json records the source checksum and retained cell IDs. The example selects approximately 200 variable genes; the dispersion threshold can retain ties, so the tested cache retained 201 features and 10 principal components.

For cross-study evaluation, freeze the source transformation and explicitly align genes before projection. No cross-cell-type or cross-study transfer result is reported in this release.