|
|
--- |
|
|
license: mit |
|
|
language: |
|
|
- en |
|
|
--- |
|
|
|
|
|
# Melchior |
|
|
|
|
|
This model combines the strengths of selective state spaces (Mamba) and transformers for accurate RNA basecalling. It achieves SOTA accuracy while maintaining efficient inference speeds through its hybrid architecture, with further advantages described in the paper. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Model Type:** Hybrid Mamba-Transformer |
|
|
- **Task:** RNA Basecalling |
|
|
- **Parameters:** 134M |
|
|
- **Architecture:** isotropic 20-layer alternating Mamba and MHA backbone, with 8 attention heads |
|
|
- **Framework:** PyTorch |
|
|
|
|
|
## Training |
|
|
|
|
|
- Training dataset: Arabidopsis thaliana, Homo sapiens, Caenorhabditis elegans, Escherichia coli, and synthetic constructs from Epinano |
|
|
- Hardware used: One A100 |
|
|
- Training time: 9 days |
|
|
- Optimization: Adam, stochastic weight averaging, linear warmup followed by cosine decay |
|
|
|
|
|
See full code at [github.com/elonlit/Melchior](https://github.com/elonlit/Melchior). |
|
|
|
|
|
## Citation |
|
|
|
|
|
```bibtex |
|
|
@article{melchior2025, |
|
|
title={Melchior: A Hybrid Mamba-Transformer RNA Basecaller}, |
|
|
author={Litman, Elon}, |
|
|
journal={bioRxiv preprint bioRxiv:2025.01.11.632456}, |
|
|
year={2025} |
|
|
} |
|
|
``` |
|
|
|
|
|
## License |
|
|
|
|
|
MIT |
|
|
|
|
|
## Contact |
|
|
|
|
|
For any inquiries, contact [elonlit@stanford.edu](mailto:elonlit@stanford.edu). |
