| tags: | |
| - spatial-transcriptomics | |
| - single-cell | |
| - celltype-classification | |
| license: mit | |
| # CellST v3 — Spatial Transcriptomics | |
| Fine-tuned CellFM 80M model for spatial transcriptomics analysis. | |
| ## Architecture | |
| - **Encoder**: 6 RetentionLayer blocks, 1536 dims, 48 heads (~80M params) | |
| - **Decoders**: ValueDecoder, CellwiseDecoder, CelltypeDecoder | |
| - **Contrastive head**: Spatial contrastive loss with distance-weighted InfoNCE | |
| ## Training | |
| - **Base model**: CellFM 80M (MindSpore to PyTorch conversion) | |
| - **Data**: Mouse brain spatial transcriptomics (4 section splits, 215 training files) | |
| - **Hardware**: 8x AMD MI325X GPUs | |
| - **Precision**: bfloat16 (autocast) | |
| - **Optimizer**: AdamW (pretrained LR=1e-4, new heads LR=1e-5) | |
| - **Epochs**: 5 | |
| ## Checkpoints | |
| | Epoch | Loss | | |
| |-------|------| | |
| | 1 | 1.2386 | | |
| | 2 | 1.1501 | | |
| | 3 | 1.1283 | | |
| | 4 | (saved) | | |
| | 5 | (in progress) | | |