Use BioMod model weights naming across documentation
Browse filesUpdate eight Markdown documents to use BioMod model weights terminology. Refresh stale source-guide examples and remove historical model-size comparisons and unverified benchmark values. Actual weight filenames, paired configuration, code, and datasets are unchanged.
- BioMod/README.md +30 -42
- BioMod/drug_combination/README.md +5 -4
- BioMod/gene_perturbation/README.md +11 -9
- BioMod/interpretability/README.md +11 -10
- BioMod/interpretability/attention/README.md +1 -1
- BioMod/interpretability/module_text_alignment/DESIGN.md +1 -1
- BioMod/zero_shot_batch/README.md +18 -13
- README.md +9 -9
BioMod/README.md
CHANGED
|
@@ -171,10 +171,9 @@ process group, with training data and paths configured for the target machine.
|
|
| 171 |
|
| 172 |
## Shared Assets
|
| 173 |
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
and update any server-specific paths for the local environment.
|
| 178 |
|
| 179 |
Tutorials expect a pretrained BioMod/PerformerModule checkpoint and shared
|
| 180 |
gene assets. Set `BIOMODULEFM_ASSET_ROOT` to the directory containing
|
|
@@ -187,7 +186,7 @@ export BIOMODULEFM_ASSET_ROOT=/path/to/BioMod
|
|
| 187 |
Current local asset files:
|
| 188 |
|
| 189 |
```text
|
| 190 |
-
$BIOMODULEFM_ASSET_ROOT/weight/0810model/ #
|
| 191 |
├── mae_tost_70M_75mask_ec2.pkl
|
| 192 |
└── mae_tost_70M_75mask_ec2_epoch0.pth
|
| 193 |
|
|
@@ -222,21 +221,13 @@ BioMod/module_data_all/ # Module definitions, m
|
|
| 222 |
└── merged_modules.json
|
| 223 |
```
|
| 224 |
|
| 225 |
-
##
|
| 226 |
|
| 227 |
-
|
| 228 |
-
|
| 229 |
-
|
| 230 |
-
|
| 231 |
-
|
| 232 |
-
`-- mae_tost_70M_75mask_ec2.pkl
|
| 233 |
-
```
|
| 234 |
-
|
| 235 |
-
It uses a 2-layer encoder with the same 27,874-gene axis and 660 module
|
| 236 |
-
definitions as the primary 32-layer model. Use its checkpoint and PKL together
|
| 237 |
-
by replacing the two `weight/0629model/` paths in the commands below with the
|
| 238 |
-
matching `weight/0810model/` files. The code restores `enc_depth=2` and the
|
| 239 |
-
other architecture settings from the PKL automatically.
|
| 240 |
|
| 241 |
## Tutorials
|
| 242 |
|
|
@@ -255,9 +246,11 @@ notebook.
|
|
| 255 |
|
| 256 |
## Data Layout
|
| 257 |
|
| 258 |
-
|
| 259 |
-
|
| 260 |
-
|
|
|
|
|
|
|
| 261 |
|
| 262 |
### Single-Drug Response
|
| 263 |
|
|
@@ -293,20 +286,20 @@ files for ROC, PR, and confusion-matrix visualization.
|
|
| 293 |
|
| 294 |
## Example Commands
|
| 295 |
|
| 296 |
-
Drug response prediction with the
|
| 297 |
|
| 298 |
```bash
|
| 299 |
python drug_response/train_drug_performer_module.py \
|
| 300 |
--gpu_id 0 \
|
| 301 |
--data_path /path/to/single_drug_response/data \
|
| 302 |
-
--checkpoint_path $BIOMODULEFM_ASSET_ROOT/weight/
|
| 303 |
-
--hyper_params_path $BIOMODULEFM_ASSET_ROOT/weight/
|
| 304 |
--module_data_dir ./module_data_all \
|
| 305 |
--gene_list_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/selected_genes_27k.txt \
|
| 306 |
--gene_mapping_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/gene2ens.csv \
|
| 307 |
--gene2vec_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/selected_gene2vec_27k.npy \
|
| 308 |
--symbol2ens_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/gene2ens.csv \
|
| 309 |
-
--save_dir ./results/
|
| 310 |
--epochs 300 \
|
| 311 |
--batch_size 32 \
|
| 312 |
--pool_method max
|
|
@@ -316,15 +309,15 @@ Drug-combination synergy prediction:
|
|
| 316 |
|
| 317 |
```bash
|
| 318 |
python drug_combination/train_drug_combination_performer_module.py \
|
| 319 |
-
--checkpoint_path $BIOMODULEFM_ASSET_ROOT/weight/
|
| 320 |
-
--hyper_params_path $BIOMODULEFM_ASSET_ROOT/weight/
|
| 321 |
--module_data_dir ./module_data_all \
|
| 322 |
--gene_list_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/selected_genes_27k.txt \
|
| 323 |
--gene_mapping_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/gene2ens.csv \
|
| 324 |
--gene2vec_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/selected_gene2vec_27k.npy \
|
| 325 |
--symbol2ens_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/gene2ens.csv \
|
| 326 |
--data_dir ./drug_combination/data_summary \
|
| 327 |
-
--output_dir ./results/
|
| 328 |
--epochs 500 \
|
| 329 |
--batch_size 256 \
|
| 330 |
--lr 1e-4 \
|
|
@@ -348,17 +341,12 @@ include:
|
|
| 348 |
For GitHub display, use PNG files. For manuscripts, use the PDF/SVG exports.
|
| 349 |
The plotting cells use embedded, editable PDF fonts where supported.
|
| 350 |
|
| 351 |
-
##
|
| 352 |
-
|
| 353 |
-
These values are retained as run-level reference points and should be refreshed
|
| 354 |
-
from final locked runs before manuscript submission.
|
| 355 |
|
| 356 |
-
|
| 357 |
-
|
| 358 |
-
|
| 359 |
-
|
| 360 |
-
| Zero-shot integration | HP, BMMC, Kim Lung, Kim Lung no-malignant | Notebook computes avgBatch and avgBio across repeated runs |
|
| 361 |
-
| Cell annotation | Seven benchmark datasets | Notebook reports accuracy, macro-F1, and confusion matrices |
|
| 362 |
|
| 363 |
## Reproducibility Conventions
|
| 364 |
|
|
@@ -379,9 +367,9 @@ files and shows which entries are missing.
|
|
| 379 |
|
| 380 |
### Missing model assets
|
| 381 |
|
| 382 |
-
Check `BIOMODULEFM_ASSET_ROOT` and verify that the
|
| 383 |
-
gene2vec, gene list, gene mapping, and
|
| 384 |
-
exist.
|
| 385 |
|
| 386 |
### No space left on device
|
| 387 |
|
|
|
|
| 171 |
|
| 172 |
## Shared Assets
|
| 173 |
|
| 174 |
+
The BioMod model weights and matching configuration are stored in
|
| 175 |
+
`weight/0810model/`. Use the supplied `.pth` and `.pkl` together, and update
|
| 176 |
+
server-specific paths for the local environment.
|
|
|
|
| 177 |
|
| 178 |
Tutorials expect a pretrained BioMod/PerformerModule checkpoint and shared
|
| 179 |
gene assets. Set `BIOMODULEFM_ASSET_ROOT` to the directory containing
|
|
|
|
| 186 |
Current local asset files:
|
| 187 |
|
| 188 |
```text
|
| 189 |
+
$BIOMODULEFM_ASSET_ROOT/weight/0810model/ # BioMod model weights and matching configuration
|
| 190 |
├── mae_tost_70M_75mask_ec2.pkl
|
| 191 |
└── mae_tost_70M_75mask_ec2_epoch0.pth
|
| 192 |
|
|
|
|
| 221 |
└── merged_modules.json
|
| 222 |
```
|
| 223 |
|
| 224 |
+
## BioMod model weights
|
| 225 |
|
| 226 |
+
Use `mae_tost_70M_75mask_ec2_epoch0.pth` with
|
| 227 |
+
`mae_tost_70M_75mask_ec2.pkl` from the shared-assets directory above. The supplied
|
| 228 |
+
configuration specifies a 2-layer encoder, a 27,874-gene axis, and 660 module
|
| 229 |
+
definitions. Keep this configuration paired with the weights so that task
|
| 230 |
+
runners can restore the corresponding architecture settings.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 231 |
|
| 232 |
## Tutorials
|
| 233 |
|
|
|
|
| 246 |
|
| 247 |
## Data Layout
|
| 248 |
|
| 249 |
+
The Hugging Face repository includes datasets under the project-root `data/`
|
| 250 |
+
directory. Individual tasks may require additional inputs or a different data
|
| 251 |
+
layout. Check each task's required files, then configure its data paths for the
|
| 252 |
+
local environment; the presence of `data/` does not imply that every tutorial
|
| 253 |
+
has all of its inputs.
|
| 254 |
|
| 255 |
### Single-Drug Response
|
| 256 |
|
|
|
|
| 286 |
|
| 287 |
## Example Commands
|
| 288 |
|
| 289 |
+
Drug response prediction with the BioMod model weights:
|
| 290 |
|
| 291 |
```bash
|
| 292 |
python drug_response/train_drug_performer_module.py \
|
| 293 |
--gpu_id 0 \
|
| 294 |
--data_path /path/to/single_drug_response/data \
|
| 295 |
+
--checkpoint_path $BIOMODULEFM_ASSET_ROOT/weight/0810model/mae_tost_70M_75mask_ec2_epoch0.pth \
|
| 296 |
+
--hyper_params_path $BIOMODULEFM_ASSET_ROOT/weight/0810model/mae_tost_70M_75mask_ec2.pkl \
|
| 297 |
--module_data_dir ./module_data_all \
|
| 298 |
--gene_list_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/selected_genes_27k.txt \
|
| 299 |
--gene_mapping_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/gene2ens.csv \
|
| 300 |
--gene2vec_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/selected_gene2vec_27k.npy \
|
| 301 |
--symbol2ens_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/gene2ens.csv \
|
| 302 |
+
--save_dir ./results/drug_response \
|
| 303 |
--epochs 300 \
|
| 304 |
--batch_size 32 \
|
| 305 |
--pool_method max
|
|
|
|
| 309 |
|
| 310 |
```bash
|
| 311 |
python drug_combination/train_drug_combination_performer_module.py \
|
| 312 |
+
--checkpoint_path $BIOMODULEFM_ASSET_ROOT/weight/0810model/mae_tost_70M_75mask_ec2_epoch0.pth \
|
| 313 |
+
--hyper_params_path $BIOMODULEFM_ASSET_ROOT/weight/0810model/mae_tost_70M_75mask_ec2.pkl \
|
| 314 |
--module_data_dir ./module_data_all \
|
| 315 |
--gene_list_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/selected_genes_27k.txt \
|
| 316 |
--gene_mapping_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/gene2ens.csv \
|
| 317 |
--gene2vec_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/selected_gene2vec_27k.npy \
|
| 318 |
--symbol2ens_path $BIOMODULEFM_ASSET_ROOT/model_dependencies/gene2ens.csv \
|
| 319 |
--data_dir ./drug_combination/data_summary \
|
| 320 |
+
--output_dir ./results/drug_combination \
|
| 321 |
--epochs 500 \
|
| 322 |
--batch_size 256 \
|
| 323 |
--lr 1e-4 \
|
|
|
|
| 341 |
For GitHub display, use PNG files. For manuscripts, use the PDF/SVG exports.
|
| 342 |
The plotting cells use embedded, editable PDF fonts where supported.
|
| 343 |
|
| 344 |
+
## Evaluation Results
|
|
|
|
|
|
|
|
|
|
| 345 |
|
| 346 |
+
This README does not provide verified benchmark scores for the bundled BioMod
|
| 347 |
+
model weights. Run the relevant task with a recorded data split and configuration
|
| 348 |
+
before reporting performance. Notebook plotting cells and historical outputs
|
| 349 |
+
alone do not establish performance for the supplied weights.
|
|
|
|
|
|
|
| 350 |
|
| 351 |
## Reproducibility Conventions
|
| 352 |
|
|
|
|
| 367 |
|
| 368 |
### Missing model assets
|
| 369 |
|
| 370 |
+
Check `BIOMODULEFM_ASSET_ROOT` and verify that the BioMod model weights, matching
|
| 371 |
+
`.pkl` configuration, gene2vec file, gene list, gene mapping, and
|
| 372 |
+
`performer_pytorch_cont/` directory exist at the paths listed under Shared Assets.
|
| 373 |
|
| 374 |
### No space left on device
|
| 375 |
|
BioMod/drug_combination/README.md
CHANGED
|
@@ -1,7 +1,8 @@
|
|
| 1 |
-
# Drug Combination Synergy Prediction
|
| 2 |
|
| 3 |
-
This task predicts drug-combination synergy using
|
| 4 |
-
|
|
|
|
| 5 |
|
| 6 |
It is adapted from [scLong/drug_combination](../../../drug_combination)
|
| 7 |
(DeepDDs), replacing the scfm/geneformer cell-line representation with
|
|
@@ -105,7 +106,7 @@ python train_drug_combination_performer_module.py \
|
|
| 105 |
|
| 106 |
| Argument | Default | Description |
|
| 107 |
| --- | --- | --- |
|
| 108 |
-
| `--checkpoint_path` | Required | Path to the
|
| 109 |
| `--hyper_params_path` | Required | Path to the PerformerModule hyper-parameter pkl file |
|
| 110 |
| `--module_data_dir` | None | Directory containing module data files |
|
| 111 |
| `--data_dir` | `./data_summary` | Data directory |
|
|
|
|
| 1 |
+
# Drug Combination Synergy Prediction with BioMod
|
| 2 |
|
| 3 |
+
This task predicts drug-combination synergy using BioMod cell embeddings from
|
| 4 |
+
the PerformerModule encoder and GCN-based drug molecular graph encoders. Use the
|
| 5 |
+
[BioMod model weights and matching configuration](../../README.md#biomod-model-weights).
|
| 6 |
|
| 7 |
It is adapted from [scLong/drug_combination](../../../drug_combination)
|
| 8 |
(DeepDDs), replacing the scfm/geneformer cell-line representation with
|
|
|
|
| 106 |
|
| 107 |
| Argument | Default | Description |
|
| 108 |
| --- | --- | --- |
|
| 109 |
+
| `--checkpoint_path` | Required | Path to the BioMod model weights |
|
| 110 |
| `--hyper_params_path` | Required | Path to the PerformerModule hyper-parameter pkl file |
|
| 111 |
| `--module_data_dir` | None | Directory containing module data files |
|
| 112 |
| `--data_dir` | `./data_summary` | Data directory |
|
BioMod/gene_perturbation/README.md
CHANGED
|
@@ -1,7 +1,9 @@
|
|
| 1 |
-
#
|
| 2 |
|
| 3 |
-
This document
|
| 4 |
-
|
|
|
|
|
|
|
| 5 |
|
| 6 |
## Overview
|
| 7 |
|
|
@@ -19,7 +21,7 @@ are:
|
|
| 19 |
|
| 20 |
`SCFM_Module` wraps `PerformerEncoderOnly_Module` and provides these functions:
|
| 21 |
|
| 22 |
-
- **Input handling**: reindex input gene expression values to the
|
| 23 |
- **Module encoding**: obtain module-level encodings from `PerformerEncoderOnly_Module`.
|
| 24 |
- **Gene embedding recovery**: recover gene-level embeddings from module encodings.
|
| 25 |
- For each gene, find all modules that contain it.
|
|
@@ -65,7 +67,7 @@ The training script is based on `GEARS/train_dual_ddp_reduce_dec.py`. Main chang
|
|
| 65 |
Make sure the following files exist:
|
| 66 |
|
| 67 |
- Perturbation data, such as Norman, Adamson, or Dixit.
|
| 68 |
-
-
|
| 69 |
- Module data in JSON format.
|
| 70 |
- Gene mapping file, such as `gene2ens.csv`.
|
| 71 |
|
|
@@ -94,14 +96,14 @@ python -m torch.distributed.launch --nproc_per_node=4 \
|
|
| 94 |
|
| 95 |
**Required arguments**:
|
| 96 |
|
| 97 |
-
- `--scfm_hyper_params_path`: path to the
|
| 98 |
-
- `--scfm_ckpt_path`: path to the
|
| 99 |
- `--module_data_path` or `--module_data_dir`: module data path; provide one of them.
|
| 100 |
|
| 101 |
**Recommended arguments**:
|
| 102 |
|
| 103 |
- `--gene_mapping_path`: gene mapping file, especially when module data use gene symbols.
|
| 104 |
-
- `--scfm_genes_list_path`:
|
| 105 |
- `--scfm_gene2vec_file`: path to the gene2vec file.
|
| 106 |
|
| 107 |
**Optional arguments**:
|
|
@@ -200,7 +202,7 @@ BioMod/gene_perturbation/ # Perturbation response
|
|
| 200 |
|
| 201 |
## Notes
|
| 202 |
|
| 203 |
-
1. **Module data format**: make sure the gene indices or names in the module data match the
|
| 204 |
2. **Gene mapping**: provide `gene_mapping_path` if the module data use gene symbols.
|
| 205 |
3. **Memory usage**: the module version may require more memory because it maintains the module-to-gene mapping.
|
| 206 |
4. **Training stability**: use the same learning rate and training strategy as original sclong when possible.
|
|
|
|
| 1 |
+
# Gene Perturbation Prediction with BioMod
|
| 2 |
|
| 3 |
+
This document describes the BioMod gene perturbation workflow (GEARS), using
|
| 4 |
+
the `PerformerEncoderOnly_Module` encoder. The
|
| 5 |
+
[BioMod model weights and matching configuration](../../README.md#biomod-model-weights)
|
| 6 |
+
are shared with the other downstream tasks.
|
| 7 |
|
| 8 |
## Overview
|
| 9 |
|
|
|
|
| 21 |
|
| 22 |
`SCFM_Module` wraps `PerformerEncoderOnly_Module` and provides these functions:
|
| 23 |
|
| 24 |
+
- **Input handling**: reindex input gene expression values to the BioMod gene order.
|
| 25 |
- **Module encoding**: obtain module-level encodings from `PerformerEncoderOnly_Module`.
|
| 26 |
- **Gene embedding recovery**: recover gene-level embeddings from module encodings.
|
| 27 |
- For each gene, find all modules that contain it.
|
|
|
|
| 67 |
Make sure the following files exist:
|
| 68 |
|
| 69 |
- Perturbation data, such as Norman, Adamson, or Dixit.
|
| 70 |
+
- BioMod model weights and their matching configuration file.
|
| 71 |
- Module data in JSON format.
|
| 72 |
- Gene mapping file, such as `gene2ens.csv`.
|
| 73 |
|
|
|
|
| 96 |
|
| 97 |
**Required arguments**:
|
| 98 |
|
| 99 |
+
- `--scfm_hyper_params_path`: path to the BioMod model configuration file.
|
| 100 |
+
- `--scfm_ckpt_path`: path to the BioMod model weights.
|
| 101 |
- `--module_data_path` or `--module_data_dir`: module data path; provide one of them.
|
| 102 |
|
| 103 |
**Recommended arguments**:
|
| 104 |
|
| 105 |
- `--gene_mapping_path`: gene mapping file, especially when module data use gene symbols.
|
| 106 |
+
- `--scfm_genes_list_path`: BioMod gene list file.
|
| 107 |
- `--scfm_gene2vec_file`: path to the gene2vec file.
|
| 108 |
|
| 109 |
**Optional arguments**:
|
|
|
|
| 202 |
|
| 203 |
## Notes
|
| 204 |
|
| 205 |
+
1. **Module data format**: make sure the gene indices or names in the module data match the BioMod gene list.
|
| 206 |
2. **Gene mapping**: provide `gene_mapping_path` if the module data use gene symbols.
|
| 207 |
3. **Memory usage**: the module version may require more memory because it maintains the module-to-gene mapping.
|
| 208 |
4. **Training stability**: use the same learning rate and training strategy as original sclong when possible.
|
BioMod/interpretability/README.md
CHANGED
|
@@ -1,10 +1,11 @@
|
|
| 1 |
-
# Module Interpretability Experiments for
|
| 2 |
|
| 3 |
-
This
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
|
|
|
| 8 |
|
| 9 |
## Experiment Roadmap, Ordered by Priority
|
| 10 |
|
|
@@ -13,7 +14,7 @@ scFoundation.
|
|
| 13 |
| 1 | `module_geometry.py` | Does the module latent space self-organize by pathway family, | Completed |
|
| 14 |
| 2 | `module_text_alignment/module_text_alignment.py` | Do module embeddings align with natural-language pathway descriptions, | Implemented |
|
| 15 |
| 3 | `module_importance/module_importance.py` | Does task-driven module importance match known pathway-cell-type or pathway-disease associations, | Implemented |
|
| 16 |
-
| 4 | `module_vs_gene.py` | Is
|
| 17 |
| 6 | `attention/module_crosstalk.py` | Do module interactions match Reactome/KEGG pathway relationships, | Implemented |
|
| 18 |
|
| 19 |
Experiments 1 and 2 are the core differentiators because gene-level baselines
|
|
@@ -25,7 +26,7 @@ be pursued after the core experiments are complete.
|
|
| 25 |
|
| 26 |
## Experiment 1: Intrinsic Geometry of Module Embeddings (`module_geometry.py`)
|
| 27 |
|
| 28 |
-
**Hypothesis:** the
|
| 29 |
reconstruction task, but it should still encode semantically related modules,
|
| 30 |
such as modules from the same pathway family or MSigDB hallmark biological
|
| 31 |
process, into nearby regions of latent space, even though family labels are
|
|
@@ -46,7 +47,7 @@ never seen during training.
|
|
| 46 |
**Differentiation from gene-level baselines:** gene-level foundation models embed
|
| 47 |
single genes rather than pathway groups. To reproduce this experiment, they would
|
| 48 |
first have to manually aggregate gene embeddings by pathway, which compresses
|
| 49 |
-
gene-level signals and introduces a hand-designed step.
|
| 50 |
aggregation end to end during pretraining.
|
| 51 |
|
| 52 |
## File Structure
|
|
@@ -345,7 +346,7 @@ necessary.
|
|
| 345 |
|
| 346 |
## Experiment 2: Module Embedding to Pathway Text Alignment (`module_text_alignment/`)
|
| 347 |
|
| 348 |
-
**Hypothesis:** the structure of the
|
| 349 |
aligns with natural-language pathway semantics. Semantically similar pathways
|
| 350 |
are also encoded nearby by the model. Gene-level models cannot reproduce this
|
| 351 |
experiment because individual gene names do not carry phrase-level semantics.
|
|
|
|
| 1 |
+
# Module Interpretability Experiments for BioMod
|
| 2 |
|
| 3 |
+
This directory contains analyses of BioMod module representations, covering
|
| 4 |
+
latent geometry, pathway-text alignment, module importance, and attention.
|
| 5 |
+
These workflows examine structural, semantic, and functional properties of the
|
| 6 |
+
learned modules. Use the
|
| 7 |
+
[BioMod model weights and matching configuration](../../README.md#biomod-model-weights)
|
| 8 |
+
when loading the pretrained backbone.
|
| 9 |
|
| 10 |
## Experiment Roadmap, Ordered by Priority
|
| 11 |
|
|
|
|
| 14 |
| 1 | `module_geometry.py` | Does the module latent space self-organize by pathway family, | Completed |
|
| 15 |
| 2 | `module_text_alignment/module_text_alignment.py` | Do module embeddings align with natural-language pathway descriptions, | Implemented |
|
| 16 |
| 3 | `module_importance/module_importance.py` | Does task-driven module importance match known pathway-cell-type or pathway-disease associations, | Implemented |
|
| 17 |
+
| 4 | `module_vs_gene.py` | Is BioMod better than GenePT/scGPT for pathway-level interpretability, | TODO |
|
| 18 |
| 6 | `attention/module_crosstalk.py` | Do module interactions match Reactome/KEGG pathway relationships, | Implemented |
|
| 19 |
|
| 20 |
Experiments 1 and 2 are the core differentiators because gene-level baselines
|
|
|
|
| 26 |
|
| 27 |
## Experiment 1: Intrinsic Geometry of Module Embeddings (`module_geometry.py`)
|
| 28 |
|
| 29 |
+
**Hypothesis:** the BioMod encoder is trained with a masked-module
|
| 30 |
reconstruction task, but it should still encode semantically related modules,
|
| 31 |
such as modules from the same pathway family or MSigDB hallmark biological
|
| 32 |
process, into nearby regions of latent space, even though family labels are
|
|
|
|
| 47 |
**Differentiation from gene-level baselines:** gene-level foundation models embed
|
| 48 |
single genes rather than pathway groups. To reproduce this experiment, they would
|
| 49 |
first have to manually aggregate gene embeddings by pathway, which compresses
|
| 50 |
+
gene-level signals and introduces a hand-designed step. BioMod learns this
|
| 51 |
aggregation end to end during pretraining.
|
| 52 |
|
| 53 |
## File Structure
|
|
|
|
| 346 |
|
| 347 |
## Experiment 2: Module Embedding to Pathway Text Alignment (`module_text_alignment/`)
|
| 348 |
|
| 349 |
+
**Hypothesis:** the structure of the BioMod module embedding space significantly
|
| 350 |
aligns with natural-language pathway semantics. Semantically similar pathways
|
| 351 |
are also encoded nearby by the model. Gene-level models cannot reproduce this
|
| 352 |
experiment because individual gene names do not carry phrase-level semantics.
|
BioMod/interpretability/attention/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# Experiment 6: Module Crosstalk Graph vs Pathway Interaction Database
|
| 2 |
|
| 3 |
-
This experiment tests whether module-module interactions learned by the
|
| 4 |
encoder are consistent with known pathway-pathway relationships.
|
| 5 |
|
| 6 |
## Methods
|
|
|
|
| 1 |
# Experiment 6: Module Crosstalk Graph vs Pathway Interaction Database
|
| 2 |
|
| 3 |
+
This experiment tests whether module-module interactions learned by the BioMod
|
| 4 |
encoder are consistent with known pathway-pathway relationships.
|
| 5 |
|
| 6 |
## Methods
|
BioMod/interpretability/module_text_alignment/DESIGN.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
## Overview
|
| 4 |
|
| 5 |
-
This experiment tests whether the structure of the
|
| 6 |
significantly aligns with the semantic space of natural-language pathway
|
| 7 |
descriptions. Gene-level models cannot reproduce this experiment because single
|
| 8 |
gene names do not carry phrase-level semantics.
|
|
|
|
| 2 |
|
| 3 |
## Overview
|
| 4 |
|
| 5 |
+
This experiment tests whether the structure of the BioMod module embedding space
|
| 6 |
significantly aligns with the semantic space of natural-language pathway
|
| 7 |
descriptions. Gene-level models cannot reproduce this experiment because single
|
| 8 |
gene names do not carry phrase-level semantics.
|
BioMod/zero_shot_batch/README.md
CHANGED
|
@@ -1,11 +1,12 @@
|
|
| 1 |
-
#
|
| 2 |
|
| 3 |
-
This directory contains
|
| 4 |
-
|
|
|
|
| 5 |
|
| 6 |
## Features
|
| 7 |
|
| 8 |
-
- **Task objective**: evaluate
|
| 9 |
- **Evaluation metrics**: compute ASW_label/batch metrics with the scib package.
|
| 10 |
- **Data format**: supports single-cell data in h5ad format.
|
| 11 |
|
|
@@ -14,7 +15,7 @@ on zero-shot batch correction.
|
|
| 14 |
### Basic Usage
|
| 15 |
|
| 16 |
```bash
|
| 17 |
-
python
|
| 18 |
--scfm_hyper_params_path <path_to_model_hyper_params.pkl> \
|
| 19 |
--scfm_ckpt_path <path_to_model_checkpoint.pth> \
|
| 20 |
--module_data_path <path_to_module_json_file.json> \
|
|
@@ -63,13 +64,13 @@ python performer_module_zero_shot.py \
|
|
| 63 |
|
| 64 |
| Argument | Description | Default |
|
| 65 |
| --- | --- | --- |
|
| 66 |
-
| `--scfm_hyper_params_path` | Path to the
|
| 67 |
-
| `--scfm_ckpt_path` | Path to the model
|
| 68 |
| `--module_data_path` | Path to one module JSON file | None |
|
| 69 |
| `--module_data_dir` | Directory containing multiple module JSON files | None |
|
| 70 |
| `--gene2vec_path` | Path to the Gene2Vec file | ../selected_gene2vec_27k.npy |
|
| 71 |
| `--gene_mapping_path` | CSV mapping gene symbols to Ensembl IDs | ../gene2ens.csv |
|
| 72 |
-
| `--scfm_genes_file` |
|
| 73 |
| `--data_path` | Input h5ad data file | ../zero-shot-batch/pancreas_scib.h5ad |
|
| 74 |
| `--output_key` | Key name for embeddings in `adata.obsm` | X_sclong_module |
|
| 75 |
| `--n_cells` | Number of cells to process; `all` means all cells | all |
|
|
@@ -91,10 +92,10 @@ After the run finishes, the following files are generated under `--output_dir`:
|
|
| 91 |
## Workflow
|
| 92 |
|
| 93 |
1. **Data loading**: load single-cell data in h5ad format.
|
| 94 |
-
2. **Gene mapping**: convert input gene symbols to Ensembl IDs and map them to the
|
| 95 |
3. **Model inference**:
|
| 96 |
-
- Reindex gene expression values to the
|
| 97 |
-
- Obtain encodings with the
|
| 98 |
- Extract cell embeddings by averaging `module_encodings`.
|
| 99 |
4. **Evaluation**: compute batch-correction metrics with scib.
|
| 100 |
|
|
@@ -119,7 +120,7 @@ The code extracts cell embeddings in the following priority order:
|
|
| 119 |
|
| 120 |
## Comparison with Original scLong
|
| 121 |
|
| 122 |
-
| Feature | scLong |
|
| 123 |
| --- | --- | --- |
|
| 124 |
| Model architecture | DualEncoder (mini + large) | Single Encoder |
|
| 125 |
| Input format | Flat gene expression | Module-based |
|
|
@@ -135,6 +136,9 @@ The code extracts cell embeddings in the following priority order:
|
|
| 135 |
|
| 136 |
## Example Output
|
| 137 |
|
|
|
|
|
|
|
|
|
|
| 138 |
```text
|
| 139 |
Using device: cuda
|
| 140 |
Loading hyper parameters from ./checkpoints/model_params.pkl
|
|
@@ -176,7 +180,8 @@ Zero-shot evaluation completed!
|
|
| 176 |
|
| 177 |
**Error**: `KeyError: 'module_encodings'`
|
| 178 |
|
| 179 |
-
**Fix**:
|
|
|
|
| 180 |
|
| 181 |
### Issue 4: Out of Memory
|
| 182 |
|
|
|
|
| 1 |
+
# BioMod Zero-Shot Embeddings and Batch-Effect Evaluation
|
| 2 |
|
| 3 |
+
This directory contains code for extracting BioMod cell embeddings and
|
| 4 |
+
evaluating batch effects without fine-tuning. Use the
|
| 5 |
+
[BioMod model weights and matching configuration](../../README.md#biomod-model-weights).
|
| 6 |
|
| 7 |
## Features
|
| 8 |
|
| 9 |
+
- **Task objective**: evaluate batch effects in cell embeddings extracted with the BioMod model weights.
|
| 10 |
- **Evaluation metrics**: compute ASW_label/batch metrics with the scib package.
|
| 11 |
- **Data format**: supports single-cell data in h5ad format.
|
| 12 |
|
|
|
|
| 15 |
### Basic Usage
|
| 16 |
|
| 17 |
```bash
|
| 18 |
+
python performer_module_zero_shot.py \
|
| 19 |
--scfm_hyper_params_path <path_to_model_hyper_params.pkl> \
|
| 20 |
--scfm_ckpt_path <path_to_model_checkpoint.pth> \
|
| 21 |
--module_data_path <path_to_module_json_file.json> \
|
|
|
|
| 64 |
|
| 65 |
| Argument | Description | Default |
|
| 66 |
| --- | --- | --- |
|
| 67 |
+
| `--scfm_hyper_params_path` | Path to the matching BioMod model configuration; required | - |
|
| 68 |
+
| `--scfm_ckpt_path` | Path to the BioMod model weights; required | - |
|
| 69 |
| `--module_data_path` | Path to one module JSON file | None |
|
| 70 |
| `--module_data_dir` | Directory containing multiple module JSON files | None |
|
| 71 |
| `--gene2vec_path` | Path to the Gene2Vec file | ../selected_gene2vec_27k.npy |
|
| 72 |
| `--gene_mapping_path` | CSV mapping gene symbols to Ensembl IDs | ../gene2ens.csv |
|
| 73 |
+
| `--scfm_genes_file` | BioMod gene list file | ../selected_genes_27k.txt |
|
| 74 |
| `--data_path` | Input h5ad data file | ../zero-shot-batch/pancreas_scib.h5ad |
|
| 75 |
| `--output_key` | Key name for embeddings in `adata.obsm` | X_sclong_module |
|
| 76 |
| `--n_cells` | Number of cells to process; `all` means all cells | all |
|
|
|
|
| 92 |
## Workflow
|
| 93 |
|
| 94 |
1. **Data loading**: load single-cell data in h5ad format.
|
| 95 |
+
2. **Gene mapping**: convert input gene symbols to Ensembl IDs and map them to the BioMod gene list.
|
| 96 |
3. **Model inference**:
|
| 97 |
+
- Reindex gene expression values to the BioMod gene order.
|
| 98 |
+
- Obtain encodings with the BioMod model.
|
| 99 |
- Extract cell embeddings by averaging `module_encodings`.
|
| 100 |
4. **Evaluation**: compute batch-correction metrics with scib.
|
| 101 |
|
|
|
|
| 120 |
|
| 121 |
## Comparison with Original scLong
|
| 122 |
|
| 123 |
+
| Feature | scLong | BioMod |
|
| 124 |
| --- | --- | --- |
|
| 125 |
| Model architecture | DualEncoder (mini + large) | Single Encoder |
|
| 126 |
| Input format | Flat gene expression | Module-based |
|
|
|
|
| 136 |
|
| 137 |
## Example Output
|
| 138 |
|
| 139 |
+
The following legacy log illustrates the output format; it does not document a
|
| 140 |
+
validated run with the bundled BioMod model weights.
|
| 141 |
+
|
| 142 |
```text
|
| 143 |
Using device: cuda
|
| 144 |
Loading hyper parameters from ./checkpoints/model_params.pkl
|
|
|
|
| 180 |
|
| 181 |
**Error**: `KeyError: 'module_encodings'`
|
| 182 |
|
| 183 |
+
**Fix**: use the BioMod model weights and their matching configuration, and
|
| 184 |
+
check that the model provides the required output.
|
| 185 |
|
| 186 |
### Issue 4: Out of Memory
|
| 187 |
|
README.md
CHANGED
|
@@ -18,7 +18,7 @@ tools.
|
|
| 18 |
BioMod is released by **CTC-Atlas**, the joint laboratory of **Raymind Biotechnology
|
| 19 |
Co., Ltd** and the **Shenzhen Research Institute of Big Data (SRIBD)**.
|
| 20 |
|
| 21 |
-
[Installation](#installation) · [
|
| 22 |
[Tasks and examples](#tasks-and-examples) · [Usage notes](#usage-notes) ·
|
| 23 |
[Repository layout](#repository-layout)
|
| 24 |
|
|
@@ -54,16 +54,16 @@ Pretraining and new-data reconstruction evaluation use CUDA/NCCL. The zero-shot
|
|
| 54 |
runner selects CUDA when available and otherwise uses CPU. Native Windows
|
| 55 |
execution has not been validated for the supplied environment.
|
| 56 |
|
| 57 |
-
##
|
| 58 |
|
| 59 |
-
|
| 60 |
-
configuration together; the configuration records the model
|
| 61 |
-
gene-module definitions.
|
| 62 |
|
| 63 |
| Asset | File |
|
| 64 |
| --- | --- |
|
| 65 |
-
|
|
| 66 |
-
|
|
| 67 |
| Model gene order | [selected_genes_27k.txt](model_dependencies/selected_genes_27k.txt) |
|
| 68 |
| Gene identifier mapping | [gene2ens.csv](model_dependencies/gene2ens.csv) |
|
| 69 |
| Gene2Vec embeddings | [selected_gene2vec_27k.npy](model_dependencies/selected_gene2vec_27k.npy) |
|
|
@@ -119,7 +119,7 @@ Before running a notebook, update its configuration cells:
|
|
| 119 |
| `BASE` or `ASSET_ROOT` | Project root |
|
| 120 |
| `PROJECT_ROOT` | Inner `BioMod/` source directory |
|
| 121 |
| `MODULE_MODEL` or `LEGACY_MODULE_MODEL` | `model_dependencies/` under the project root |
|
| 122 |
-
| Weight and configuration paths | The
|
| 123 |
| `MODULE_DATA_DIR` | `BioMod/module_data_all/` under the project root |
|
| 124 |
| Data and output paths | Your task inputs and a writable output directory |
|
| 125 |
|
|
@@ -286,7 +286,7 @@ BioMod/ # Project root: source
|
|
| 286 |
│ ├── selected_gene2vec_27k.npy
|
| 287 |
│ └── selected_genes_27k.txt
|
| 288 |
├── weight/ # Pretrained weights and matching model configurations
|
| 289 |
-
│ └── 0810model/ #
|
| 290 |
│ ├── mae_tost_70M_75mask_ec2.pkl
|
| 291 |
│ └── mae_tost_70M_75mask_ec2_epoch0.pth
|
| 292 |
├── .gitattributes
|
|
|
|
| 18 |
BioMod is released by **CTC-Atlas**, the joint laboratory of **Raymind Biotechnology
|
| 19 |
Co., Ltd** and the **Shenzhen Research Institute of Big Data (SRIBD)**.
|
| 20 |
|
| 21 |
+
[Installation](#installation) · [BioMod model weights](#biomod-model-weights) ·
|
| 22 |
[Tasks and examples](#tasks-and-examples) · [Usage notes](#usage-notes) ·
|
| 23 |
[Repository layout](#repository-layout)
|
| 24 |
|
|
|
|
| 54 |
runner selects CUDA when available and otherwise uses CPU. Native Windows
|
| 55 |
execution has not been validated for the supplied environment.
|
| 56 |
|
| 57 |
+
## BioMod model weights
|
| 58 |
|
| 59 |
+
This release includes the **BioMod model weights** and their matching
|
| 60 |
+
configuration. Use these files together; the configuration records the model
|
| 61 |
+
architecture and gene-module definitions.
|
| 62 |
|
| 63 |
| Asset | File |
|
| 64 |
| --- | --- |
|
| 65 |
+
| BioMod model weights | [mae_tost_70M_75mask_ec2_epoch0.pth](weight/0810model/mae_tost_70M_75mask_ec2_epoch0.pth) |
|
| 66 |
+
| Matching model configuration | [mae_tost_70M_75mask_ec2.pkl](weight/0810model/mae_tost_70M_75mask_ec2.pkl) |
|
| 67 |
| Model gene order | [selected_genes_27k.txt](model_dependencies/selected_genes_27k.txt) |
|
| 68 |
| Gene identifier mapping | [gene2ens.csv](model_dependencies/gene2ens.csv) |
|
| 69 |
| Gene2Vec embeddings | [selected_gene2vec_27k.npy](model_dependencies/selected_gene2vec_27k.npy) |
|
|
|
|
| 119 |
| `BASE` or `ASSET_ROOT` | Project root |
|
| 120 |
| `PROJECT_ROOT` | Inner `BioMod/` source directory |
|
| 121 |
| `MODULE_MODEL` or `LEGACY_MODULE_MODEL` | `model_dependencies/` under the project root |
|
| 122 |
+
| Weight and configuration paths | The BioMod model weights and matching configuration listed above |
|
| 123 |
| `MODULE_DATA_DIR` | `BioMod/module_data_all/` under the project root |
|
| 124 |
| Data and output paths | Your task inputs and a writable output directory |
|
| 125 |
|
|
|
|
| 286 |
│ ├── selected_gene2vec_27k.npy
|
| 287 |
│ └── selected_genes_27k.txt
|
| 288 |
├── weight/ # Pretrained weights and matching model configurations
|
| 289 |
+
│ └── 0810model/ # BioMod model weights and matching configuration
|
| 290 |
│ ├── mae_tost_70M_75mask_ec2.pkl
|
| 291 |
│ └── mae_tost_70M_75mask_ec2_epoch0.pth
|
| 292 |
├── .gitattributes
|