ccst-spatial-clustering / data /DATA_PROVENANCE.md
hutaobo's picture
Publish bundled DST-GNN inputs and formal reproducibility outputs
91f5f1d verified
|
Raw
History Blame Contribute Delete
2.65 kB
# Data Provenance
This repository bundles the direct DST-GNN input table and a formal release of the cleaned public DST-GNN outputs so the analysis can be rerun end-to-end from the repository contents alone.
## Bundled input
- `data/inputs/cophenetic_distances_searcher_D_score_in_all_samples.csv`
This table is the direct input to the public DST-GNN pipeline. It contains the flattened COSTE/SSS-style pairwise spatial relationship values used to build temporal cell-type graphs.
Expected columns:
- `row`
- `column`
- `value`
- `sample`
- `group`
The cohort-level structure matches the manuscript description:
- `45` total samples
- `47` predefined cell types
- `HD -> T1` (`healthy`)
- `LA -> T2` (`less affected`)
- `MA -> T3` (`more affected`)
## Upstream origin
The bundled CSV was recovered from the lung fibrosis analysis directory and traced to the manuscript-era COSTE workflow:
- recovered notebook using the CSV:
- `Y:/long/publication_datasets/Vannan_2023_Lung_Fibrosis/notebook/GNN modelling.ipynb`
- recovered notebook generating the CSV:
- `Y:/long/publication_datasets/Vannan_2023_Lung_Fibrosis/notebook/Expression Distance Similarity.ipynb`
- mirrored A100 location:
- `/mnt/taobo.hu/long/publication_datasets/Vannan_2023_Lung_Fibrosis/`
The generation flow is:
1. For each sample, COSTE-style cophenetic / Searcher's D matrices were computed from annotated spatial data.
2. Each per-sample matrix was flattened into `(row, column, value)` records.
3. Sample identifiers and stage labels were attached.
4. The concatenated table was saved as `cophenetic_distances_searcher_D_score_in_all_samples.csv`.
## Bundled formal output
- `data/outputs/formal_release/`
This directory contains the standard public output release generated with the cleaned DST-GNN implementation in this repository, not the raw historical notebook outputs.
Formal release settings:
- `device=cpu`
- `seed=0`
- `hidden_channels=32`
- `dropout=0.0`
- `lr=0.01`
- `weight_decay=5e-4`
- `epochs=400`
- `top_k=20`
- `run_explainer=true`
The bundled output directory includes:
- observed stage-level SSS matrices
- predicted later-stage SSS matrices
- ranked dynamic nodes
- ranked changing edges
- explainer edge scores for the `T2 -> T3` transition
- structured metadata in `summary.json` and `run_config.json`
- `checksums.sha256` covering the input CSV and generated release files
## Scope note
This repository does not redistribute the raw Xenium data or the original full spatial omics study inputs. It only provides the direct DST-GNN input table needed for the public cleaned pipeline plus the corresponding formal release outputs.