README.md

---
license: mit
language:
  - en
tags:
  - biology
  - rna
  - rna-language-model
  - rna-secondary-structure
  - shifu
library_name: pytorch
pipeline_tag: fill-mask
---

<!-- Copy this file into each HuggingFace repo as README.md and fill the LMR-nano/65M fields. -->

# LMR-nano  (SHIFU / LMR foundational backbone)

`LMR-nano` is a foundational RNA language-model backbone (65M parameters) from
**SHIFU: an integrated framework for deep learning of RNA secondary structure**
(Galvez and Vicens, Vicens Lab, University of Houston).

It is pretrained by masked language modeling on RNA sequence and serves as the
backbone for downstream RNA structure tasks. The three released backbones are:

| Model | Params | Context | Repo |
|---|---|---|---|
| LMR-v0 | 289M | 512 | GaboG7/LMR-v0 |
| LMR-G | 228M | 512 | GaboG7/LMR-G |
| LMR-nano | 65M | 512 | GaboG7/LMR-mini |
| LMR-Long | 290M | 4,096 | GaboG7/LMR-Long |

## Usage

```python
from huggingface_hub import snapshot_download
path = snapshot_download("GaboG7/LMR-mini")   # LMR-v0 / LMR-G / LMR-mini
# load with the LMR code: https://github.com/Vicens-Lab/LMR
```

Training code, configs, and the finetuning recipe are in the GitHub repo:
**https://github.com/Vicens-Lab/LMR**

## Training data

Pretrained on RNA sequence data; the downstream 2D structure benchmark
(**SHIFU-Corpus**, 254,123 sequences, leakage-audited, family-aware splits) is a
separate release.

## Intended use and limitations

A general-purpose RNA sequence backbone for finetuning (e.g. secondary-structure
prediction). It is a sequence model, not a standalone structure predictor. Inputs
cap at the context length in the config.

## License

MIT (code and weights). SHIFU-Corpus retains the licenses of its six source databases.

## Citation

This work is not yet published; a citation will be added when available.

```bibtex
@misc{lmr_shifu,
  title  = {SHIFU: an integrated framework for deep learning of RNA secondary structure},
  author = {Galvez, Gabriel and Vicens, Quentin},
  note   = {Manuscript in preparation. Citation to be updated.},
  year   = {TODO}
}
```