DeepSTARR

Convolutional neural network for predicting enhancer activity directly from DNA sequence.

Disclaimer

This is an UNOFFICIAL implementation of DeepSTARR predicts enhancer activity from DNA sequence and enables the de novo design of synthetic enhancers by Bernardo P. de Almeida, Franziska Reiter, et al.

The OFFICIAL repository of DeepSTARR is at bernardo-de-almeida/DeepSTARR.

The MultiMolecule team has confirmed that the provided model and checkpoints are producing the same intermediate representations as the original implementation.

The team releasing DeepSTARR did not write this model card for this model so this model card has been written by the MultiMolecule team.

Model Details

DeepSTARR is a convolutional neural network (CNN) trained to quantitatively predict enhancer activity from 249 bp DNA sequences. The model was trained on genome-wide STARR-seq data from Drosophila melanogaster S2 cells and predicts two regression outputs: developmental and housekeeping enhancer activity. The architecture consists of four convolutional blocks (Conv1D + BatchNorm + ReLU + MaxPool) followed by two fully-connected layers. Please refer to the Training Details section for more information on the training process.

Model Specification

• Architecture: 4 convolutional layers + 2 fully-connected layers
• Convolution filters: 256, 60, 60, 120
• Convolution kernel sizes: 7, 3, 5, 3
• Max-pool size: 2
• Fully-connected sizes: 256, 256
• Input length: 249 bp
• Number of labels: 2 (developmental and housekeeping enhancer activity, regression)

Num Conv Layers	Num FC Layers	Hidden Size	Num Parameters (M)	FLOPs (M)	MACs (M)	Max Num Tokens
4	2	256	0.62	21.03	10.26	249

Links

• Code: multimolecule.deepstarr
• Weights: multimolecule/deepstarr
• Paper: DeepSTARR predicts enhancer activity from DNA sequence and enables the de novo design of synthetic enhancers
• Developed by: Bernardo P. de Almeida, Franziska Reiter, Michaela Pagani, Alexander Stark
• Original Repository: bernardo-de-almeida/DeepSTARR

Usage

The model file depends on the multimolecule library. You can install it using pip:

pip install multimolecule

Direct Use

Enhancer Activity Prediction

You can use this model directly to predict the developmental and housekeeping enhancer activity of a 249 bp DNA sequence:

>>> import torch
>>> from multimolecule import DnaTokenizer, DeepStarrForSequencePrediction

>>> tokenizer = DnaTokenizer.from_pretrained("multimolecule/deepstarr")
>>> model = DeepStarrForSequencePrediction.from_pretrained("multimolecule/deepstarr")
>>> sequence = "ACGT" * 62 + "A"
>>> output = model(**tokenizer(sequence, return_tensors="pt"))

>>> output.logits.shape
torch.Size([1, 2])

Training Details

DeepSTARR was trained to predict quantitative enhancer activity from DNA sequence.

Training Data

DeepSTARR was trained on genome-wide UMI-STARR-seq data from Drosophila melanogaster S2 cells, measuring enhancer activity under two transcriptional programs: a developmental program (driven by a developmental core promoter) and a housekeeping program (driven by a housekeeping core promoter).

Each training example is a 249 bp genomic sequence with two continuous activity values (developmental and housekeeping, log2 enrichment over input). Chromosomes were split into training, validation, and test sets to avoid sequence leakage.

Training Procedure

Pre-training

The model was trained to minimize a mean-squared-error loss between predicted and measured enhancer activities.

• Optimizer: Adam
• Learning rate: 2e-3
• Loss: Mean Squared Error
• Input length: 249 bp
• Early stopping on validation loss

Citation

@article{deAlmeida2022deepstarr,
  author    = {de Almeida, Bernardo P. and Reiter, Franziska and Pagani, Michaela and Stark, Alexander},
  journal   = {Nature Genetics},
  month     = may,
  number    = 5,
  pages     = {613--624},
  publisher = {Springer Science and Business Media LLC},
  title     = {{DeepSTARR} predicts enhancer activity from {DNA} sequence and enables the de novo design of synthetic enhancers},
  volume    = 54,
  year      = 2022
}

The artifacts distributed in this repository are part of the MultiMolecule project. If you use MultiMolecule in your research, you must cite the MultiMolecule project as follows:

@software{chen_2024_12638419,
  author    = {Chen, Zhiyuan and Zhu, Sophia Y.},
  title     = {MultiMolecule},
  doi       = {10.5281/zenodo.12638419},
  publisher = {Zenodo},
  url       = {https://doi.org/10.5281/zenodo.12638419},
  year      = 2024,
  month     = may,
  day       = 4
}

Contact

Please use GitHub issues of MultiMolecule for any questions or comments on the model card.

Please contact the authors of the DeepSTARR paper for questions or comments on the paper/model.

License

This model implementation is licensed under the GNU Affero General Public License.

For additional terms and clarifications, please refer to our License FAQ.

SPDX-License-Identifier: AGPL-3.0-or-later