metadata
tags:
- spatial-transcriptomics
- single-cell
- celltype-classification
license: mit
CellST v3 — Spatial Transcriptomics
Fine-tuned CellFM 80M model for spatial transcriptomics analysis.
Architecture
- Encoder: 6 RetentionLayer blocks, 1536 dims, 48 heads (~80M params)
- Decoders: ValueDecoder, CellwiseDecoder, CelltypeDecoder
- Contrastive head: Spatial contrastive loss with distance-weighted InfoNCE
Training
- Base model: CellFM 80M (MindSpore to PyTorch conversion)
- Data: Mouse brain spatial transcriptomics (4 section splits, 215 training files)
- Hardware: 8x AMD MI325X GPUs
- Precision: bfloat16 (autocast)
- Optimizer: AdamW (pretrained LR=1e-4, new heads LR=1e-5)
- Epochs: 5
Checkpoints
| Epoch | Loss |
|---|---|
| 1 | 1.2386 |
| 2 | 1.1501 |
| 3 | 1.1283 |
| 4 | (saved) |
| 5 | (in progress) |