PANDA — reproducibility recipe
PANDA (Pan-tissue Adversarial Normalized Domain-invariant Anchored MLP) is a
compact prototype-anchored MLP classifier for scRNA-seq cell identity across skin,
hematopoietic, and pancreatic tissues, trained under a composite loss (supervised-
contrastive + VICReg + sub-center angular prototype-InfoNCE + gradient-reversal
dataset/depth adversary + HSIC depth-decorrelation + prototype-repulsion). Two
input variants ship out of the box: PANDA-PCA (PCA(50) -> trunk) and
PANDA-Marker ([PCA(50) || marker_expr] -> trunk); Marker beats PCA on 33/35
fold-comparisons across the three systems.
This README is a complete recipe to reproduce every result in PAPER.tex from a
clean clone. All commands are copy-pasteable and use absolute paths.
Repository layout:
panda/ model + losses + panda/markers.yaml
scripts/pan_skin/ skin pipeline: download -> corpus -> train -> CV -> zero-shot
scripts/pancreas/ pancreas pipeline (same shape)
scripts/hematopoiesis/ HSC pipeline (same shape)
scripts/common/ system-agnostic train / CV / zero-shot drivers
scripts/analysis/ downstream discovery + interpretability
scripts/figures/ paper + supplement figure builders
data/corpus/{sys}/ downloaded + harmonized data
data/raw/ per-dataset raw counts (not in git)
checkpoints/{sys}/{variant}/panda_final.pt
discovery/{sys}/{variant}/*.json,*.csv all quantitative artefacts
figures/ fig1_..fig6, PANDA_supplement.pdf, biology/*.pdf
1. Requirements
- Python: 3.9+ (project is tested on 3.10).
- CUDA: PyTorch 2.6.0 wheels — CUDA 12.1/12.4 runtime works.
- GPU: 4x A100 40GB used for the paper; 1 GPU works if you drop the
training batch size to
bs=64(default is 256 for 4-GPU DataParallel). - Disk: ~400 GB (raw GEO tars + harmonized corpora + checkpoints).
- RAM: ~64 GB (the pancreas HVG builder peaks near ~40 GB).
Pinned runtime dependencies (from pyproject.toml / requirements.txt):
torch==2.6.0 transformers==5.6.2 peft==0.18.1
scanpy==1.11.5 anndata==0.11.4 scvi-tools==1.3.3
harmonypy==0.2.0 numpy>=1.24,<3.0 scipy>=1.10
scikit-learn>=1.2 pandas>=1.5 matplotlib>=3.7
seaborn>=0.12 umap-learn>=0.5 pyyaml>=6.0
tqdm>=4.65 einops>=0.6 gdown>=5.0
GEOparse>=2.0 leidenalg>=0.10 pynndescent>=0.5
scikit-misc>=0.5
Optional extras: bayes (numpyro/jax for horseshoe), gpu (flash-attn 2.8.2),
viz (plotly), dev (pytest, ruff, mypy).
Install:
git clone <this-repo> /home/bcheng/PRISM
cd /home/bcheng/PRISM
pip install -e .
# or, editable dev install:
pip install -e ".[dev]"
1.1 LD_LIBRARY_PATH prefix (required for every PyTorch invocation)
PyTorch 2.6 sparse ops load libcusparseLt.so.0 which sits under the pip-installed
nvidia-cusparselt-cu12 package, and scanpy needs a modern libstdc++. Both
paths must be exported at the shell level, before Python starts:
export LD_LIBRARY_PATH="$(python -c "import site,os; print(os.path.join(site.getsitepackages()[0],'nvidia','cusparselt','lib'))"):$LD_LIBRARY_PATH"
If you also have a conda env that ships a newer libstdc++, prepend it:
# example — path is machine-specific; drop it if your system libstdc++ is >= 3.4.30
export LD_LIBRARY_PATH="/home/bcheng/.conda/pkgs/libstdcxx-15.2.0-h39759b7_7/lib:$LD_LIBRARY_PATH"
Every bash scripts/*/run_all.sh driver applies the same export automatically.
2. Data acquisition
Every URL below is a public GEO/ArrayExpress FTP link. Raw data is not
committed — it must be re-downloaded before anything else runs. Corpus builders
expect files at data/corpus/{system}/tier_{a,b,c,v2}/.
Pan-skin (6 studies, 45,387 cells)
| Study | GEO | Role |
|---|---|---|
| Sulic 2023 (E14.5 dorsal) | GSE212673 | anchor + held-out zero-shot |
| Dingwall 2024 (En1-cKO) | GSE220977 | discovery target (paired with Aldrich GSE214695) |
| Belote 2021 (human melanocyte) | GSE151091 | melanocyte anchor + held-out zero-shot |
| Haensel/Annusver 2020 | GSE142471 | adult homeostasis + wound |
| Joost 2016 | GSE67602 | Smart-seq2 platform anchor |
| Sennett 2015 (bulk RNA) | GSE70288 | placode/dermal-condensate marker reference |
| Han MCA 2018 (neonatal skin) | GSE108097 | Microwell-seq low-depth anchor |
| Merkel 2022 | GSE201447 | touch dome / volar biology |
| Aldrich 2023 (paired with Dingwall) | GSE214695 | En1-cKO snRNA-seq |
bash scripts/pan_skin/01_download_tier_a.sh # Aldrich, Ge/Gupta, Joost, Haensel
bash scripts/pan_skin/02_download_tier_b.sh # MCA, WIHN, Ge/Fuchs, Merkel
bash scripts/pan_skin/03_download_tier_c.sh # Sennett, Tie, Wiedemann (bulk + human)
# Dingwall / Sulic / Belote must be placed in data/raw/ manually — see repo notes
Pan-hematopoietic (3 studies used in the paper, 192,833 cells)
| Study | GEO | Role |
|---|---|---|
| Weinreb LARRY 2020 | GSE140802 | corpus anchor |
| Baccin whole-BM 2020 | GSE122465 | corpus (stromal + hematopoietic) |
| Tabula Muris Senis BM 2020 | GSE132042 | corpus (paper-labeled) |
| Nestorowa 2016 | GSE81682 | held-out zero-shot (Smart-seq2) |
| Dahlin 2018 | GSE107727 | discovery target (Kit-W41 mutant) |
| Paul 2015 (auxiliary) | GSE72857 | myeloid branch reference |
| Tusi 2018 (auxiliary) | GSE89754 | erythroid trajectory |
bash scripts/hematopoiesis/01_download.sh # Paul, Nestorowa, Tusi, Dahlin
bash scripts/hematopoiesis/02_download.sh # Baccin whole-BM, TMS bone marrow
Pan-pancreatic (6 studies, 120,611 cells)
| Study | GEO | Role |
|---|---|---|
| Baron 2016 | GSE84133 | corpus mouse-train half + held-out mouse-test half |
| Bastidas-Ponce 2019 (E15.5) | GSE132188 | corpus (endocrine progenitor time course) |
| Byrnes 2018 | GSE101099 | corpus (paper-labeled subset) |
| Yu 2021 | GSE139627 | corpus (paper-labeled Ngn3 lineage) |
| Hrovatin MIA 2023 | GSE211796 | corpus (adult islet, paper-labeled) |
| Veres 2019 | GSE114412 | 57,297 corpus + 12,297 held-out slice |
bash scripts/pancreas/01_download.sh # Baron, Muraro, Grun, Byrnes, Veres
bash scripts/pancreas/09_download.sh # Yu Ngn3 seq-EP, MIA 4-month adult islet
Wall-clock: 2-6 h depending on bandwidth (GSE108097 MCA tar is ~9 GB, GSE140802 Weinreb is ~14 GB, GSE114412 Veres is ~4 GB).
3. Corpus build (per system)
Each system builds a data/corpus/{system}/harmonized/corpus.h5ad plus a shared
HVG list, per-HVG mean/std, and a fitted PCA basis. Corpus is 100% paper-labeled;
every cell carries a label from its source paper's supplementary table.
Pan-skin
python scripts/pan_skin/06_build_per_dataset_h5ads.py
python scripts/pan_skin/07_build_shared_hvgs_and_pca.py
python scripts/pan_skin/08_assign_labels.py
python scripts/pan_skin/08b_curated_label_override.py
python scripts/pan_skin/10_build_corpus.py # canonical corpus.h5ad
python scripts/pan_skin/93_add_belote_anchor.py # +Belote melanocyte anchor
Wall-clock ~10-20 min (HVG + PCA is the expensive step).
Pan-hematopoietic
python scripts/hematopoiesis/02_build_per_dataset.py
python scripts/hematopoiesis/03_shared_hvgs_and_pca.py
python scripts/hematopoiesis/10_build_corpus.py # canonical corpus.h5ad
python scripts/hematopoiesis/11_filter_paper_only.py # enforce paper-labeled subset
python scripts/hematopoiesis/09_retrain_with_nestorowa_anchor.py # optional anchor
Wall-clock ~15-30 min.
Pan-pancreatic
python scripts/pancreas/02_build_per_dataset.py
python scripts/pancreas/03_shared_hvgs_and_pca.py
python scripts/pancreas/04_assign_labels.py
python scripts/pancreas/11_build_corpus.py # canonical corpus.h5ad
python scripts/pancreas/08_add_baron_split.py # 943-cell Baron test-half
Wall-clock ~30-60 min (peak ~40 GB RAM on the union HVG step).
Also generate the held-out labeled slices used for zero-shot:
python scripts/common/generate_missing_holdouts.py
writes data/corpus/hematopoiesis/held_out_labeled/nestorowa_GSE81682_test.h5ad
and data/corpus/pan_skin/held_out_labeled/sulic_GSE212673_test.h5ad.
4. Training
The canonical trainer is system-agnostic. It reads
data/corpus/{system}/harmonized/corpus.h5ad and writes
checkpoints/{system}/{variant}/panda_final.pt.
# 6 checkpoints total (3 systems x 2 variants). ~30-60 min each on 1x A100.
python -m scripts.common.train_panda pan_skin --variant pca --epochs 8
python -m scripts.common.train_panda pan_skin --variant marker --epochs 8
python -m scripts.common.train_panda hematopoiesis --variant pca --epochs 8
python -m scripts.common.train_panda hematopoiesis --variant marker --epochs 8
python -m scripts.common.train_panda pancreas --variant pca --epochs 8
python -m scripts.common.train_panda pancreas --variant marker --epochs 8
Legacy per-system entry points also exist and are functionally equivalent for
skin/HSC/pancreas single-variant training:
scripts/pan_skin/20_train_panda.py, scripts/hematopoiesis/05_train_panda.py,
scripts/pancreas/05_train_panda.py. Prefer scripts.common.train_panda.
5. Held-out 5-fold cross-validation (Table 1)
The paper's Table 1 CV block reads
discovery/{system}/{variant}/cv_5fold.json. Two drivers exist:
scripts/common/run_cv.py— canonical, 5 epochs per fold, matches paper numbers (mean acc / F1 / AUROC + per-class report).scripts/common/cv_holdout.py— same architecture but supports GroupKFold by dataset and a fuller 6-8 epoch curriculum; slower.
Both accept --systems and --variants:
# canonical 5-fold CV for all 3 systems x 2 variants
python -m scripts.common.run_cv --folds 5 --epochs 5
Per-system CV drivers also exist (scripts/pan_skin/40_heldout_5fold_cv.py,
scripts/hematopoiesis/07_heldout_5fold_cv.py,
scripts/pancreas/07_heldout_5fold_cv.py); they are single-variant, single-
system alternatives.
Multi-seed rigor (35 fold-comparisons)
The paper reports Marker beating PCA on 33/35 folds across seeded runs
(2 seeds for skin+pancreas, 3 for HSC). Re-run with different random_states;
outputs write to cv_5fold_seed{1,2}.json:
python -m scripts.common.run_cv --folds 5 --epochs 5 # seed 0
# The script's seed hook is the random_state passed to StratifiedKFold + model
# init; re-run with edits to `run_cv.py` main() (add `--seed N` arg) or wrap
# a small loop. See scripts/common/cv_holdout.py for the current seed plumbing.
6. Held-out labeled zero-shot targets (Section 5)
One driver runs every zero-shot target for both variants:
python -m scripts.common.run_all_zero_shot \
--systems pan_skin hematopoiesis pancreas \
--variants pca marker
writes discovery/{system}/{variant}/{target}_predictions.csv and
{target}_summary.json. Individual per-target scripts exist for finer-grained
control:
| Target | Script | Populates |
|---|---|---|
| Baron test-half (pancreas, 943 cells) | scripts/analysis/93_true_zero_shot_baron.py |
Table 1 Baron row + Sec 5.1 |
| Nestorowa Smart-seq2 (HSC, 66 LT-HSC gated) | scripts/analysis/94_true_zero_shot_nestorowa.py |
Sec 5.3 |
| Sulic E14.5 dorsal skin (4,183 cells) | scripts/common/run_all_zero_shot.py (target sulic) |
Sec 5.5 |
| Belote melanocyte (6,088 cells) | scripts/common/run_all_zero_shot.py (target belote) |
Sec 5.4 |
| Veres held-out slice (12,297 pancreas) | scripts/common/run_all_zero_shot.py (target veres) |
Sec 5.2 |
| Dingwall (25,344 skin, discovery) | scripts/pan_skin/30_zero_shot_aldrich.py |
Sec 6 |
| Dahlin (61,122 HSC, discovery) | scripts/common/run_all_zero_shot.py (target dahlin) |
Sec 7 |
Adult-beta canonical panel validation on Veres:
python scripts/analysis/95_adult_beta_validation.py
# -> discovery/pancreas/marker/95_adult_beta_validation.json
7. Discovery analyses
Grouped by paper section. All write to discovery/{system}/{variant}/.
7.1 Section 5 (held-out labeled) marker deep-dives
python scripts/analysis/90_dingwall_marker_deep_dive.py # skin -> 90_..._marker_deep_dive.csv
python scripts/analysis/91_veres_marker_deep_dive.py # pancreas
python scripts/analysis/92_dahlin_marker_deep_dive.py # HSC
7.2 Section 6 — Dingwall En1-cKO (skin)
python scripts/analysis/44_en1_cko_contrast.py # class-level cKO vs WT contrast
python scripts/analysis/45_marker_refinement.py # per-class marker refinement
python scripts/analysis/49_melanocyte_deep_dive.py # melanocyte 2x expansion
python scripts/analysis/57_multiclass_pathway_analysis.py # Dingwall pathway table
python scripts/analysis/57_pathway_analysis.py # symmetric 25+ module scoring, all systems
python scripts/analysis/99_en1_dual_role_analysis.py # spatial repressor / local activator
python scripts/analysis/106_melanoblast_neural_crest.py # Sec 6.5 MITF-axis vs NC reversion
python scripts/analysis/107_dingwall_class_deg_count.py # Sec 6.7 HF-placode DEG rank
# EDEN validation — three complementary lines of evidence (Sec 6.4)
python scripts/analysis/98_eden_posthoc_detection.py # Line A (null)
python scripts/analysis/103_replicate_dingwall_seurat_pipeline.py # Line B (Derm10 4.32x)
python scripts/analysis/104_train_on_dingwall_derm_labels.py # Line C (Derm10 5.20x)
# Primary EDEN (Derm2) discovery — Sec 6.4.1
python scripts/analysis/100_primary_eden_discovery.py # sub-cluster fibro predictions
python scripts/analysis/101_primary_eden_derm_scoring.py # score vs Data S1C panels
python scripts/analysis/105_primary_eden_full_dermal.py # on full dermal denominator
# Auxiliary: variant-A Dingwall training used for scope comparison
python scripts/analysis/102_train_on_dingwall_variantA.py
7.3 Section 7 — Dahlin Kit-mutant (hematopoiesis)
python scripts/analysis/66_dahlin_kit_mutant.py # class enrichment WT vs Kit-W41
python scripts/analysis/67_dahlin_within_class.py # within-class Wilcoxon DE
python scripts/analysis/73_novel_populations_dahlin.py # abstain-gated novel pops
python scripts/analysis/108_dahlin_lineage_metabolism.py # per-lineage OXPHOS/glycolysis
7.4 Section 7.4 — Veres held-out (pancreas)
python scripts/analysis/62_time_course_analysis.py # class fractions across LARRY days (HSC time-course template)
python scripts/analysis/109_veres_mature_beta.py # Sec 5.2 adult MAFA/UCN3 quadrant
python scripts/analysis/110_veres_polyhormonal_alpha.py # Sec 5.2 polyhormonal alpha cluster
7.5 Section 8 — cross-system prototype geometry + interpretability
python scripts/analysis/70_prototype_geometry.py # intra + cross-system cosine + eff-dim
python scripts/analysis/72_emergent_axes.py # within-class PCA of 128-d z
python scripts/analysis/80_prototype_gene_attribution.py # integrated gradients per prototype
python scripts/analysis/81_counterfactual_knockouts.py # per-gene KO delta on cosine
python scripts/analysis/82_gene_coattribution_modules.py # gene co-attribution modules
python scripts/analysis/83_prototype_training_trajectory.py # prototype drift across curriculum
python scripts/analysis/84_adversary_purification.py # test GRL adversary is at chance
python scripts/analysis/85_hessian_gene_interactions.py # second-order gene pair Hessian
python scripts/analysis/63_nestorowa_zero_shot.py # Nestorowa unlabeled discovery
8. Figures + supplement
Main-text figures (figures/fig{1,2,3,4}_*.pdf)
python scripts/figures/generate_paper_figures.py
# fig1_confusion_matrices.pdf 3-panel per-class F1 confusion matrices
# fig2_aldrich_volcano.pdf melanocyte cKO vs WT volcano
# fig3_dahlin_heatmap.pdf within-class module-score heatmap
# fig4_sharon_stage_stack.pdf Veres per-stage class fractions
Figures 5/6 (En1-cKO + Kit-W41 recap) are built by the biology page pipeline
below — the standalone regen_fig5_fig6.py referenced in older notes is not in
the current tree; use the biology pipeline instead.
Supplement (figures/PANDA_supplement.pdf)
python scripts/figures/build_pca_vs_marker_umaps.py # PCA vs Marker UMAPs per target
python scripts/figures/build_figure_supplement.py # combined supplement PDF
Biology deep-dive supplement pages
Cache UMAPs once, then build per-topic pages, then merge into the supplement:
python scripts/figures/biology_00_umap_cache.py
python scripts/figures/biology_01_dingwall_umap.py
python scripts/figures/biology_02_primary_eden.py
python scripts/figures/biology_03_melanoblast_mitf.py
python scripts/figures/biology_04_dahlin_metabolism.py
python scripts/figures/biology_05_dahlin_composition.py
python scripts/figures/biology_06_veres_beta_quadrant.py
python scripts/figures/biology_07_veres_polyhormonal.py
python scripts/figures/biology_08_prototype_geometry.py
python scripts/figures/biology_99_merge_supplement.py # appends into PANDA_supplement.pdf
Wall-clock: 20-40 min end-to-end (UMAPs dominate).
9. PDF build
The paper is a self-contained LaTeX document referencing PDFs in figures/:
cd /home/bcheng/PRISM
pdflatex -interaction=nonstopmode PAPER.tex # first pass (writes .aux)
pdflatex -interaction=nonstopmode PAPER.tex # second pass (resolves refs)
bibtex is not required — the paper uses an embedded thebibliography.
10. End-to-end make target
The provided Makefile covers the canonical skin pipeline end-to-end:
make install # pip install -e .
make run # bash scripts/pan_skin/run_all.sh — full skin pipeline
make test-heldout # scripts/pan_skin/40_heldout_5fold_cv.py
make clean # remove __pycache__ + *.pyc
For a full three-system reproduction, chain the per-section commands above. A minimal "everything" recipe:
# 1) data
bash scripts/pan_skin/01_download_tier_a.sh
bash scripts/pan_skin/02_download_tier_b.sh
bash scripts/pan_skin/03_download_tier_c.sh
bash scripts/hematopoiesis/01_download.sh
bash scripts/hematopoiesis/02_download.sh
bash scripts/pancreas/01_download.sh
bash scripts/pancreas/09_download.sh
# 2) corpora
bash scripts/pan_skin/run_all.sh # includes build + train + skin CV
python scripts/hematopoiesis/02_build_per_dataset.py
python scripts/hematopoiesis/03_shared_hvgs_and_pca.py
python scripts/hematopoiesis/10_build_corpus.py
python scripts/hematopoiesis/11_filter_paper_only.py
python scripts/pancreas/02_build_per_dataset.py
python scripts/pancreas/03_shared_hvgs_and_pca.py
python scripts/pancreas/04_assign_labels.py
python scripts/pancreas/11_build_corpus.py
python scripts/pancreas/08_add_baron_split.py
python scripts/common/generate_missing_holdouts.py
python scripts/pan_skin/93_add_belote_anchor.py
# 3) all 6 training runs
for sys in pan_skin hematopoiesis pancreas; do
for v in pca marker; do
python -m scripts.common.train_panda $sys --variant $v --epochs 8
done
done
# 4) CV + zero-shot
python -m scripts.common.run_cv --folds 5 --epochs 5
python -m scripts.common.run_all_zero_shot
# 5) discovery
bash scripts/common/rerun_all_discovery.sh # drives scripts/analysis/* end-to-end
python scripts/analysis/95_adult_beta_validation.py
# 6) figures + PDF
python scripts/figures/generate_paper_figures.py
python scripts/figures/build_pca_vs_marker_umaps.py
python scripts/figures/build_figure_supplement.py
python scripts/figures/biology_00_umap_cache.py
for i in 01 02 03 04 05 06 07 08; do
python scripts/figures/biology_${i}_*.py
done
python scripts/figures/biology_99_merge_supplement.py
pdflatex -interaction=nonstopmode PAPER.tex && pdflatex -interaction=nonstopmode PAPER.tex
11. Trouble-shooting
libcusparseLt.so.0: cannot open shared object file— you forgot to exportLD_LIBRARY_PATHbefore Python started. The pip-installednvidia-cusparselt-cu12provides the library; PyTorch does not add its path to the loader search. See section 1.1.GLIBCXX_3.4.30 not found— your systemlibstdc++is too old; prepend a newerlibstdc++.so.6's directory toLD_LIBRARY_PATH.- Dingwall GSM -> genotype mapping (frequent bug source): the correct map is
WT = {GSM6833478, GSM6833479, GSM6833480, GSM6833481},cKO = {GSM6833482, GSM6833483}. GSM6833480/481 arerttaControl(Cre-negative WT), not cKO. Getting this wrong flips every En1-cKO enrichment sign. - Pancreas HVG builder OOM —
scripts/pancreas/03_shared_hvgs_and_pca.pypeaks near 40 GB RAM on the 6-study union. Run on a node with >= 64 GB. data/rawnot in git — it is git-ignored (14+ GB of GEO tars). Rerun section 2 to repopulate.stratified split failureon rare classes — small-support classes (< 2 members per fold) are merged into the parentcanonical_label. If a fold still errors, check thatcorpus.h5ad'scanonical_labelcolumn has the expected vocabulary; the paper vocab is the union enumerated inPAPER.texSec 3.n_conditionsmismatch —PANDAEncoderreads it fromlen(datasets)in the checkpoint; regenerate the checkpoint if you have added/removed a dataset.ContrastiveSamplerin 1-condition data — auto-disabled when there is only one condition; no config change needed.- DataParallel batch-size — default
bs=256is calibrated for 4x A100 40 GB. Drop tobs=64for a single GPU or you will OOM inside the sub-center prototype attention. X_prismnot persisted — after20_train_panda.py/05_train_panda.pywrites the checkpoint, the embedding is re-projected on demand in every downstream analysis; if you want it cached, re-save the AnnData explicitly viaadata.write_h5ad().
12. Artefact index (from PAPER.tex Section 11)
Every quantitative claim traces to one of:
- Model checkpoints:
checkpoints/{system}/{variant}/panda_final.pt - Marker gene lists:
panda/markers.yaml - Corpus builders:
scripts/{pan_skin,hematopoiesis,pancreas}/ - Unified trainer:
scripts/common/train_panda.py - Unified 5-fold CV:
scripts/common/run_cv.py(orcv_holdout.py) - Zero-shot driver:
scripts/common/run_all_zero_shot.py - External label supplements:
data/external_labels/{dingwall_supp,haensel,joost2016,mca,mia,byrnes,yu,baccin,melanocyte_anchor}/ - CV outputs:
discovery/{system}/{variant}/cv_5fold{,_seed1,_seed2}.json - Zero-shot summaries:
discovery/pancreas/{pca,marker}/baron_summary.jsondiscovery/pancreas/{pca,marker}/veres_summary.jsondiscovery/hematopoiesis/{pca,marker}/nestorowa_summary.jsondiscovery/pan_skin/{pca,marker}/sulic_summary.jsondiscovery/pan_skin/{pca,marker}/belote_summary.json
- Adult-beta panel:
discovery/pancreas/marker/95_adult_beta_validation.json - Pathway modules:
discovery/{system}/marker/57_pathway_class_by_module_{padj,delta}.tsv - Dingwall EDEN validation:
Line A
discovery/pan_skin/marker/98_eden_summary.json; Line Bdata/processed/dingwall_replica/dingwall_replica.h5ad,replica_cluster_20_qc.json,replica_marker_matches.csv; Line Cdiscovery/pan_skin/marker/104_dingwall_derm_summary.json+ prediction CSVs. - Primary EDEN (Derm2):
discovery/pan_skin/marker/100_primary_eden_discovery.csv,100_primary_eden_summary.json,101_derm_identity_summary.json,101_derm_subcluster_scores.csv. - Dingwall other:
discovery/pan_skin/marker/57_pathway_analysis.csv,90_dingwall_marker_deep_dive.csv. - Dahlin Kit-mutant:
discovery/hematopoiesis/marker/92_dahlin_marker_deep_dive.csv,dahlin_summary.json. - Veres deep-dive:
discovery/pancreas/marker/91_veres_marker_deep_dive.csv.
Central architecture: panda/model.py. Composite loss lives in the same file
(supcon_loss, vicreg_loss, hsic_biased, subcenter_angular_infonce,
prototype_repulsion) and is imported as from panda import PANDAEncoder, ....
Data mirror
Full data (~195 GB corpus + raw + processed + external labels) is mirrored to Hugging Face at bryan7264/PANDA. Fetch with:
huggingface-cli download bryan7264/PANDA --local-dir . --include "data/corpus/pan_skin/**"
Priority folders (fetch these first for the minimum-reproducible pipeline):
data/corpus/{pan_skin,hematopoiesis,pancreas}/harmonized/— training corporadata/external_labels/— paper-supplement label filescheckpoints/{pan_skin,hematopoiesis,pancreas}/marker/— trained weights
Bulk (only needed to reproduce corpus builds from scratch):
data/raw/— GEO downloads (regenerable fromscripts/*/03_download*.sh)data/corpus/tier{1,2,3}/— pretraining tier datadata/processed/— intermediate build artefacts