ccst-spatial-clustering / data /DATA_PROVENANCE.md
hutaobo's picture
Publish bundled DST-GNN inputs and formal reproducibility outputs
91f5f1d verified
|
Raw
History Blame Contribute Delete
2.65 kB

Data Provenance

This repository bundles the direct DST-GNN input table and a formal release of the cleaned public DST-GNN outputs so the analysis can be rerun end-to-end from the repository contents alone.

Bundled input

  • data/inputs/cophenetic_distances_searcher_D_score_in_all_samples.csv

This table is the direct input to the public DST-GNN pipeline. It contains the flattened COSTE/SSS-style pairwise spatial relationship values used to build temporal cell-type graphs.

Expected columns:

  • row
  • column
  • value
  • sample
  • group

The cohort-level structure matches the manuscript description:

  • 45 total samples
  • 47 predefined cell types
  • HD -> T1 (healthy)
  • LA -> T2 (less affected)
  • MA -> T3 (more affected)

Upstream origin

The bundled CSV was recovered from the lung fibrosis analysis directory and traced to the manuscript-era COSTE workflow:

  • recovered notebook using the CSV:
    • Y:/long/publication_datasets/Vannan_2023_Lung_Fibrosis/notebook/GNN modelling.ipynb
  • recovered notebook generating the CSV:
    • Y:/long/publication_datasets/Vannan_2023_Lung_Fibrosis/notebook/Expression Distance Similarity.ipynb
  • mirrored A100 location:
    • /mnt/taobo.hu/long/publication_datasets/Vannan_2023_Lung_Fibrosis/

The generation flow is:

  1. For each sample, COSTE-style cophenetic / Searcher's D matrices were computed from annotated spatial data.
  2. Each per-sample matrix was flattened into (row, column, value) records.
  3. Sample identifiers and stage labels were attached.
  4. The concatenated table was saved as cophenetic_distances_searcher_D_score_in_all_samples.csv.

Bundled formal output

  • data/outputs/formal_release/

This directory contains the standard public output release generated with the cleaned DST-GNN implementation in this repository, not the raw historical notebook outputs.

Formal release settings:

  • device=cpu
  • seed=0
  • hidden_channels=32
  • dropout=0.0
  • lr=0.01
  • weight_decay=5e-4
  • epochs=400
  • top_k=20
  • run_explainer=true

The bundled output directory includes:

  • observed stage-level SSS matrices
  • predicted later-stage SSS matrices
  • ranked dynamic nodes
  • ranked changing edges
  • explainer edge scores for the T2 -> T3 transition
  • structured metadata in summary.json and run_config.json
  • checksums.sha256 covering the input CSV and generated release files

Scope note

This repository does not redistribute the raw Xenium data or the original full spatial omics study inputs. It only provides the direct DST-GNN input table needed for the public cleaned pipeline plus the corresponding formal release outputs.