ViroHyena
ViroHyena is a Hyena-based nucleotide language model pre-trained on the ViroBland (ViroBlend) corpus, a small (216 Mbp) mixed pretraining dataset with source-wise stratified sampling to balance human reference, multi-species genomes, and viral in-domain sequences.
Model Configurations
| Model | Params | d_model | Layers |
|---|---|---|---|
| ViroHyena-436K | 0.436M | 128 | 2 |
| ViroHyena-1.6M | 1.6M | 256 | 2 |
| ViroHyena-6.6M | 6.6M | 256 | 8 |
| ViroHyena-253M | 253M | 1024 | 20 |
- Downloads last month
- 1
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support