Orthrus (legacy code-and-checkpoints dump)

This repository is the original Orthrus code-and-checkpoints dump from the pre-print period. It is not loadable via AutoModel.from_pretrained — it ships a custom loader (load_model) and multiple raw .ckpt checkpoints under models/.

For the standardized HuggingFace AutoModel.from_pretrained(...) interface, use the repos below instead.

Use these instead

Repo	Tracks	Embed dim	Objective	Used in
antichronology/orthrus-4-track	4	512	contrastive	Nature Methods publication
antichronology/orthrus-6-track	6	512	contrastive	Nature Methods publication
antichronology/orthrus-small-6-track	6	256	contrastive	Nature Methods publication
antichronology/orthrus-mlm-6-track	6	512	contrastive + MLM	Nature Methods publication
quietflamingo/orthrus-base-4-track	4	256	contrastive	Pre-publication
quietflamingo/orthrus-large-4-track	4	512	contrastive	Pre-publication
quietflamingo/orthrus-large-6-track	6	512	contrastive	Pre-publication

Every model in the table exposes the same three inference methods:

from transformers import AutoModel

model = AutoModel.from_pretrained("antichronology/orthrus-4-track", trust_remote_code=True)

model.representation(x, lengths, channel_last=True)        # (B, D)     pooled
model.representation_unpooled(x, channel_last=True)        # (B, L, D)  per-position
model.predict_tokens(x, lengths, channel_last=True)        # (B, L, 4)  MLM logits, MLM repos only

See the README on any of the standardized repos for full setup instructions, GenomeKit-based 6-track input construction, and MLM scoring examples.

What's in this repo

The files here are preserved for reproducibility of pre-print era work and receive no further updates:

• model.py, gk_utils.py, colab_prototye.ipynb — the original loader and a Colab demo.
• models/orthrus_v0_4_track/, models/orthrus_v0_6_track/, models/orthrus_v0_small_4_track/, models/orthrus_v1_4_track/, models/orthrus_v1_6_track/, models/orthrus_v1_small_6_track/ — raw Lightning .ckpt checkpoints from earlier training runs, used by model.py::load_model(...).
• env.yml — the original conda environment.

If you specifically need the custom loader (for instance, to reproduce a number from the pre-print), the code matches what is in bowang-lab/Orthrus on GitHub.

Citation

@article{fradkinShi2026,
  title = {Orthrus: toward evolutionary and functional RNA foundation models},
  ISSN = {1548-7105},
  url = {http://dx.doi.org/10.1038/s41592-026-03064-3},
  DOI = {10.1038/s41592-026-03064-3},
  journal = {Nature Methods},
  publisher = {Springer Science and Business Media LLC},
  author = {Fradkin, Philip and Shi, Ruian "Ian" and Dalal, Taykhoom and Isaev, Keren and Frey, Brendan J. and Lee, Leo J. and Morris, Quaid and Wang, Bo},
  year = {2026},
  month = Apr
}

License

MIT