scBasset

Sequence-based convolutional neural network for modeling single-cell ATAC-seq chromatin accessibility from DNA sequence.

Disclaimer

This is an UNOFFICIAL implementation of scBasset: sequence-based modeling of single-cell ATAC-seq using convolutional neural networks by Han Yuan et al.

The OFFICIAL repository of scBasset is at calico/scBasset.

The MultiMolecule team has confirmed that the provided model and checkpoints are producing the same intermediate representations as the original implementation.

The team releasing scBasset did not write this model card for this model so this model card has been written by the MultiMolecule team.

Model Details

scBasset is a convolutional neural network (CNN) that predicts per-cell chromatin accessibility of a DNA peak sequence. The model consumes a fixed-length 1344 bp one-hot encoded DNA sequence and applies a pre-activation convolution stem, a reducing convolution tower, a pointwise convolution, and a dense bottleneck before a final cell-embedding layer that produces one accessibility logit per single cell.

scBasset uses a pre-activation block layout: each convolution block applies the activation (the sigmoid approximation of GELU, sigmoid(1.702 * x) * x) before the convolution, then batch normalization and max pooling. The dense bottleneck flattens the convolution output in Keras channels-last (length-major) order; this ordering is load-bearing and is reconciled in the MultiMolecule implementation.

The final cell-embedding (dense) layer of scBasset is dataset-specific: it has one row per single cell in the training atlas, so there is no dataset-independent foundation checkpoint. The shipped weights correspond to the Buenrostro2018 hematopoiesis tutorial dataset distributed by the scBasset authors, which has 2034 single cells (so num_labels = 2034). A different scBasset dataset would have a different number of cells and a differently sized cell-embedding layer.

The cell-embedding layer is exposed through the shared [SequencePredictionHead][multimolecule.SequencePredictionHead]; the per-cell accessibility task is modeled as a binary problem (problem_type="binary").

Model Specification

• Architecture: 1 stem convolution + 6 tower convolutions + 1 pointwise convolution + dense bottleneck
• Stem: 288 filters, kernel 17, max-pool 3
• Tower: 6 blocks, filters 288/323/363/407/456/512, kernel 5, max-pool 2
• Pointwise: 256 filters, kernel 1
• Bottleneck (hidden size): 32
• Input length: 1344 bp
• Number of labels: 2034 single cells (Buenrostro2018 hematopoiesis, dataset-specific)

Num Conv Layers	Hidden Size	Num Cells	Num Parameters (M)	FLOPs (G)	MACs (G)	Max Num Tokens
8	32	2034	4.59	0.95	0.47	1344

Links

• Code: multimolecule.scbasset
• Weights: multimolecule/scbasset
• Data: scBasset Buenrostro2018 tutorial data
• Paper: scBasset: sequence-based modeling of single-cell ATAC-seq using convolutional neural networks
• Developed by: Han Yuan, David R. Kelley
• Original Repository: calico/scBasset

Usage

The model file depends on the multimolecule library. You can install it using pip:

pip install multimolecule

Direct Use

Single-Cell Chromatin Accessibility Prediction

You can use this model directly to predict per-cell chromatin accessibility of a DNA peak sequence:

>>> from multimolecule import DnaTokenizer, ScBassetForSequencePrediction

>>> tokenizer = DnaTokenizer.from_pretrained("multimolecule/scbasset")
>>> model = ScBassetForSequencePrediction.from_pretrained("multimolecule/scbasset")
>>> input = tokenizer("ACGT" * 336, return_tensors="pt")
>>> output = model(**input)

>>> output.logits.shape
torch.Size([1, 2034])

Each of the 2034 logits is a per-cell accessibility score for the Buenrostro2018 hematopoiesis atlas.

Training Details

scBasset was trained to predict the per-cell chromatin accessibility of DNA peak sequences across a single-cell ATAC-seq atlas.

Training Data

The shipped converted checkpoint corresponds to the Buenrostro2018 hematopoiesis scBasset tutorial model (buen_model_sc.h5) distributed by the scBasset authors, trained on the Buenrostro et al. 2018 single-cell ATAC-seq hematopoiesis dataset (2034 single cells). Each 1344 bp peak is associated with a per-cell binary accessibility vector.

Training Procedure

Pre-training

The model was trained to minimize a per-cell binary cross-entropy loss, comparing its predicted per-cell accessibility probabilities (sigmoid of the cell-embedding logits) against the observed single-cell ATAC-seq accessibility labels.

• Optimizer: Adam
• Loss: Per-cell binary cross-entropy
• Regularization: Batch normalization and dropout
• Activation: Sigmoid approximation of GELU (sigmoid(1.702 * x) * x)

Citation

@article{yuan2022scbasset,
  author    = {Yuan, Han and Kelley, David R.},
  title     = {scBasset: sequence-based modeling of single-cell ATAC-seq using convolutional neural networks},
  journal   = {Nature Methods},
  volume    = 19,
  number    = 9,
  pages     = {1088--1096},
  year      = 2022,
  publisher = {Nature Publishing Group},
  doi       = {10.1038/s41592-022-01562-8}
}

The artifacts distributed in this repository are part of the MultiMolecule project. If you use MultiMolecule in your research, you must cite the MultiMolecule project as follows:

@software{chen_2024_12638419,
  author    = {Chen, Zhiyuan and Zhu, Sophia Y.},
  title     = {MultiMolecule},
  doi       = {10.5281/zenodo.12638419},
  publisher = {Zenodo},
  url       = {https://doi.org/10.5281/zenodo.12638419},
  year      = 2024,
  month     = may,
  day       = 4
}

Contact

Please use GitHub issues of MultiMolecule for any questions or comments on the model card.

Please contact the authors of the scBasset paper for questions or comments on the paper/model.

License

This model implementation is licensed under the GNU Affero General Public License.

For additional terms and clarifications, please refer to our License FAQ.

SPDX-License-Identifier: AGPL-3.0-or-later