---
license: mit
library_name: pytorch-lightning
pipeline_tag: tabular-regression
tags:
- biology
- genomics
datasets:
- Genentech/enformer-data
---

# Enformer Model (Avsec et al. 2021)

## Model Description
This repository contains the weights for the Enformer model, a long-range transformer architecture designed to predict functional genomic tracks from genomic DNA sequences.

- **Architecture:** Convolutions followed by Transformer layers.
- **Input:** 196,608 bp of genomic DNA sequence.
- **Output Resolution:** 128 bp bins.
- **Source:** [Avsec, Ž. et al. Nature Methods (2021)](https://www.nature.com/articles/s41592-021-01252-x)


## Repository Content
The repository includes both full PyTorch Lightning checkpoints and raw state dictionaries for the human and mouse versions of the model. Note that the weights are derived from the publication but the model has been converted into the PyTorch Lightning format used by gReLU (https://github.com/Genentech/gReLU).

| File | Type | Description |
| :--- | :--- | :--- |
| `human.ckpt` | PyTorch Lightning | Full checkpoint including base model and human head. |
| `mouse.ckpt` | PyTorch Lightning | Full checkpoint including base model and mouse head. |
| `human_state_dict.h5` | HDF5 | Weights-only state dictionary for the human model. |
| `mouse_state_dict.h5` | HDF5 | Weights-only state dictionary for the mouse model. |
| `save_wandb_enformer_human.ipynb` | Jupyter Notebook | Code used to create `human.ckpt` |
| `save_wandb_enformer_mouse.ipynb` | Jupyter Notebook | Code used to create `mouse.ckpt` |

## Model Heads & Output Tracks
Both `.ckpt` files utilize the same core transformer trunk but differ in their species-specific output heads.

### Outputs

Human Head: 5,313 total tracks
Mouse Head: 1,643 total tracks

## Usage
The models are intended for use with the `grelu` library.

```python
from grelu.lightning import LightningModel
from huggingface_hub import hf_hub_download

# Download the desired checkpoint
ckpt_path = hf_hub_download(
    repo_id="Genentech/enformer-model", 
    filename="human.ckpt"
)

# Load the model
model = LightningModel.load_from_checkpoint(ckpt_path)
model.eval()
```