Sync latest GeneMamba docs and next-token training updates
Browse files
README.md
CHANGED
|
@@ -26,7 +26,6 @@ A Hugging Face compatible implementation of GeneMamba, a foundational state-spac
|
|
| 26 |
- [Datasets](#datasets)
|
| 27 |
- [Usage Guide](#usage-guide)
|
| 28 |
- [Citation](#citation)
|
| 29 |
-
- [License](#license)
|
| 30 |
|
| 31 |
---
|
| 32 |
|
|
@@ -316,14 +315,14 @@ Task-Specific Heads:
|
|
| 316 |
|
| 317 |
### Key Design Choices
|
| 318 |
|
| 319 |
-
- **
|
| 320 |
-
- **Pooling Strategy**:
|
| 321 |
- **Regularization**: Dropout on classification head
|
| 322 |
- **Activation**: No explicit activation (Mamba uses internal gating)
|
| 323 |
|
| 324 |
---
|
| 325 |
|
| 326 |
-
## Usage Guide
|
| 327 |
|
| 328 |
### Loading Models
|
| 329 |
|
|
@@ -387,7 +386,7 @@ config = model.config
|
|
| 387 |
config.hidden_dropout_prob = 0.2
|
| 388 |
```
|
| 389 |
|
| 390 |
-
---
|
| 391 |
|
| 392 |
## Important Notes ⚠️
|
| 393 |
|
|
@@ -436,26 +435,20 @@ See the `examples/` directory for complete scripts:
|
|
| 436 |
|
| 437 |
- `1_extract_embeddings.py` - Extract cell embeddings
|
| 438 |
- `2_finetune_classification.py` - Cell type annotation
|
| 439 |
-
- `
|
| 440 |
-
|
| 441 |
-
Run any example:
|
| 442 |
-
|
| 443 |
-
```bash
|
| 444 |
-
python examples/01_extract_embeddings.py
|
| 445 |
-
```
|
| 446 |
|
| 447 |
---
|
| 448 |
|
| 449 |
## Citation
|
| 450 |
|
| 451 |
-
If you
|
| 452 |
|
| 453 |
```bibtex
|
| 454 |
-
@article{
|
| 455 |
-
title={GeneMamba:
|
| 456 |
-
author={
|
| 457 |
-
journal={
|
| 458 |
-
year={
|
| 459 |
}
|
| 460 |
```
|
| 461 |
|
|
@@ -515,30 +508,4 @@ GeneMamba_repo/
|
|
| 515 |
|
| 516 |
---
|
| 517 |
|
| 518 |
-
|
| 519 |
-
|
| 520 |
-
Contributions welcome! Please:
|
| 521 |
-
|
| 522 |
-
1. Fork the repository
|
| 523 |
-
2. Create a feature branch
|
| 524 |
-
3. Submit a pull request
|
| 525 |
-
|
| 526 |
-
---
|
| 527 |
-
|
| 528 |
-
## License
|
| 529 |
-
|
| 530 |
-
This project is licensed under the Apache 2.0 License - see [LICENSE](LICENSE) for details.
|
| 531 |
-
|
| 532 |
-
---
|
| 533 |
-
|
| 534 |
-
## Support
|
| 535 |
-
|
| 536 |
-
- 📖 **Documentation**: See `docs/` directory
|
| 537 |
-
- 🐛 **Issues**: Report on GitHub
|
| 538 |
-
- 💬 **Discussions**: Join our community forum
|
| 539 |
-
- 📧 **Email**: Support contact (to be added)
|
| 540 |
-
|
| 541 |
-
---
|
| 542 |
-
|
| 543 |
-
**Last Updated**: March 2026
|
| 544 |
-
**Maintained by**: GeneMamba Team
|
|
|
|
| 26 |
- [Datasets](#datasets)
|
| 27 |
- [Usage Guide](#usage-guide)
|
| 28 |
- [Citation](#citation)
|
|
|
|
| 29 |
|
| 30 |
---
|
| 31 |
|
|
|
|
| 315 |
|
| 316 |
### Key Design Choices
|
| 317 |
|
| 318 |
+
- **Bidirectional Mamba Block**: Bidirectional Mamba enables significant improvement in gene rank reconstruction task
|
| 319 |
+
- **Pooling Strategy**: Bidirectional Mamba with multiple aggregation modes (mean/sum/concat/gate)
|
| 320 |
- **Regularization**: Dropout on classification head
|
| 321 |
- **Activation**: No explicit activation (Mamba uses internal gating)
|
| 322 |
|
| 323 |
---
|
| 324 |
|
| 325 |
+
<!-- ## Usage Guide
|
| 326 |
|
| 327 |
### Loading Models
|
| 328 |
|
|
|
|
| 386 |
config.hidden_dropout_prob = 0.2
|
| 387 |
```
|
| 388 |
|
| 389 |
+
--- -->
|
| 390 |
|
| 391 |
## Important Notes ⚠️
|
| 392 |
|
|
|
|
| 435 |
|
| 436 |
- `1_extract_embeddings.py` - Extract cell embeddings
|
| 437 |
- `2_finetune_classification.py` - Cell type annotation
|
| 438 |
+
- `3_pretrain_from_scratch.py` - Train from scratch (next-token + optional resume)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 439 |
|
| 440 |
---
|
| 441 |
|
| 442 |
## Citation
|
| 443 |
|
| 444 |
+
If you find GeneMamba is useful in your research, please cite:
|
| 445 |
|
| 446 |
```bibtex
|
| 447 |
+
@article{qi2025genemamba,
|
| 448 |
+
title={GeneMamba: An Efficient and Effective Foundation Model on Single Cell Data},
|
| 449 |
+
author={Qi, Cong and Fang, Hanzhang and Jiang, Siqi and Song, Xun and Hu, Tianxing and Zhi, Wei},
|
| 450 |
+
journal={arXiv preprint arXiv:2504.16956},
|
| 451 |
+
year={2026}
|
| 452 |
}
|
| 453 |
```
|
| 454 |
|
|
|
|
| 508 |
|
| 509 |
---
|
| 510 |
|
| 511 |
+
**Last Updated**: March 2026
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|