README.md

---
language: dna
tags:
  - Biology
  - DNA
license: agpl-3.0
library_name: multimolecule
---

# ChromBPNet

Bias-factorized, base-resolution convolutional neural network for predicting chromatin accessibility (ATAC-seq / DNase-seq) from DNA sequence.

## Disclaimer

This is an UNOFFICIAL implementation of [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) by Anusri Pampari et al.

> [!IMPORTANT]
> The reference publication is a **bioRxiv preprint**: DOI [10.1101/2024.12.25.630221](https://doi.org/10.1101/2024.12.25.630221).

The OFFICIAL repository of ChromBPNet is at [kundajelab/chrombpnet](https://github.com/kundajelab/chrombpnet).

> [!TIP]
> The MultiMolecule team has confirmed that the provided model and checkpoints are producing the same intermediate representations as the original implementation.

**The team releasing ChromBPNet did not write this model card for this model so this model card has been written by the MultiMolecule team.**

## Model Details

ChromBPNet is a convolutional neural network (CNN) trained to predict base-resolution chromatin accessibility (ATAC-seq or DNase-seq) from primary DNA sequence with explicit enzyme-bias correction. It builds on the BPNet architecture (a convolutional motif stem followed by a stack of dilated residual convolutions) and internally composes two BPNet-style sub-models:

- a **bias** sub-model that captures the Tn5/DNase enzyme cleavage bias on chromatin background;
- an **accessibility** sub-model that learns the bias-corrected accessibility signal.

The two sub-models are composed _internally_ by [`ChromBPNetModel`][multimolecule.models.ChromBPNetModel]: their per-position profile logits are added and their count logits are combined in log/exp space (`logsumexp`). The sub-model split is an implementation detail, not a user-facing API.

The composed prediction is a single base-resolution task whose output is factorized into two terminal branches:

- a **profile** branch predicting the _shape_ of the accessibility signal as per-position multinomial logits, trained with a multinomial negative log-likelihood;
- a **count** branch predicting the _total magnitude_ of the signal as a scalar (in log space), trained with mean-squared error.

The usable base-resolution prediction recombines the two branches as `softmax(profile_logits, positions) * expm1(count_logits)`, exposed via `ChromBPNetForProfilePrediction.postprocess`. Please refer to the [Training Details](#training-details) section for more information on the training process.

### Model Specification

| Input Length | Profile Length | Num Layers | Hidden Size | Bias Hidden Size | Num Parameters (M) |
| ------------ | -------------- | ---------- | ----------- | ---------------- | ------------------ |
| 2114         | 1000           | 9 + 5      | 512         | 128              | 5.5                |

The accessibility sub-model has 1 stem convolution + 8 dilated residual blocks (512 filters); the bias sub-model has 1 stem convolution + 4 dilated residual blocks (128 filters).

### Links

- **Code**: [multimolecule.chrombpnet](https://github.com/DLS5-Omics/multimolecule/tree/master/multimolecule/models/chrombpnet)
- **Weights**: [multimolecule/chrombpnet](https://huggingface.co/multimolecule/chrombpnet)
- **Data**: [RoboATAC ChromBPNet Models](https://zenodo.org/records/16295014)
- **Paper**: [ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants](https://doi.org/10.1101/2024.12.25.630221) (bioRxiv preprint)
- **Developed by**: Anusri Pampari, Anna Shcherbina, Evgeny Kvon, Michael Kosicki, Surag Nair, Soumya Kundu, Arwa S. Kathiria, Viviana I. Risca, Kristiina Simola, Melissa J. Funk, Eileen E. M. Furlong, Len A. Pennacchio, William J. Greenleaf, Anshul Kundaje
- **Original Repository**: [kundajelab/chrombpnet](https://github.com/kundajelab/chrombpnet)

## Usage

The model file depends on the [`multimolecule`](https://multimolecule.danling.org) library. You can install it using pip:

```bash
pip install multimolecule
```

### Direct Use

You can use this model directly to predict base-resolution chromatin accessibility of a DNA sequence:

```python
>>> from multimolecule import DnaTokenizer, ChromBPNetForProfilePrediction

>>> tokenizer = DnaTokenizer.from_pretrained("multimolecule/chrombpnet")
>>> model = ChromBPNetForProfilePrediction.from_pretrained("multimolecule/chrombpnet")
>>> output = model(**tokenizer(("ACGT" * 529)[:2114], return_tensors="pt"))

>>> output.keys()
odict_keys(['profile_logits', 'count_logits'])

>>> output["profile_logits"].shape
torch.Size([1, 1000, 1])

>>> output["count_logits"].shape
torch.Size([1, 1])

>>> track = model.postprocess(output)
>>> track.shape
torch.Size([1, 1000, 1])
```

The recombined `track` is the usable, bias-corrected base-resolution accessibility prediction.

## Training Details

ChromBPNet was trained to predict base-resolution chromatin accessibility profiles from ATAC-seq / DNase-seq with explicit enzyme-bias correction.

### Training Data

The representative converted checkpoint uses the HEK293T GFP-control ChromBPNet model from the [RoboATAC ChromBPNet Models](https://zenodo.org/records/16295014) release (an automated ATAC-seq dataset from the Kundaje/Greenleaf labs). The accessibility (`chrombpnet_nobias.h5`) and scaled bias (`bias_model_scaled.h5`) sub-models are converted and composed.

### Training Procedure

#### Training

The model was trained with a composite loss: a multinomial negative log-likelihood on the per-position profile shape plus a mean-squared-error regression on the log total counts.

- Bias sub-model: 1 motif convolution (128 filters, kernel 21, ReLU) + 4 dilated residual convolutions (128 filters, kernel 3, ReLU); trained on chromatin background and held fixed while training the accessibility sub-model
- Accessibility sub-model: 1 motif convolution (512 filters, kernel 21, ReLU) + 8 dilated residual convolutions (512 filters, kernel 3, dilations 2, 4, …, 256, ReLU)
- Profile head: per-task convolution (kernel 75) producing per-position logits
- Count head: per-task global average pooling + linear layer producing log-count scalars
- Composition: profile logits added; counts combined via `logsumexp`
- Optimizer: Adam

## Citation

```bibtex
@article{pampari2024chrombpnet,
  author    = {Pampari, Anusri and Shcherbina, Anna and Kvon, Evgeny and Kosicki, Michael and Nair, Surag and Kundu, Soumya and Kathiria, Arwa S. and Risca, Viviana I. and Simola, Kristiina and Funk, Melissa J. and Furlong, Eileen E. M. and Pennacchio, Len A. and Greenleaf, William J. and Kundaje, Anshul},
  title     = {ChromBPNet: bias factorized, base-resolution deep learning models of chromatin accessibility reveal cis-regulatory sequence syntax, transcription factor footprints and regulatory variants},
  journal   = {bioRxiv},
  year      = 2024,
  publisher = {Cold Spring Harbor Laboratory},
  doi       = {10.1101/2024.12.25.630221},
  note      = {Preprint}
}
```

> [!NOTE]
> The artifacts distributed in this repository are part of the MultiMolecule project.
> If you use MultiMolecule in your research, you must cite the MultiMolecule project as follows:

```bibtex
@software{chen_2024_12638419,
  author    = {Chen, Zhiyuan and Zhu, Sophia Y.},
  title     = {MultiMolecule},
  doi       = {10.5281/zenodo.12638419},
  publisher = {Zenodo},
  url       = {https://doi.org/10.5281/zenodo.12638419},
  year      = 2024,
  month     = may,
  day       = 4
}
```

## Contact

Please use GitHub issues of [MultiMolecule](https://github.com/DLS5-Omics/multimolecule/issues) for any questions or comments on the model card.

Please contact the authors of the [ChromBPNet paper](https://doi.org/10.1101/2024.12.25.630221) for questions or comments on the paper/model.

## License

This model implementation is licensed under the [GNU Affero General Public License](license.md).

For additional terms and clarifications, please refer to our [License FAQ](license-faq.md).

```spdx
SPDX-License-Identifier: AGPL-3.0-or-later
```