Brieno
A self-supervised foundation model for raw nanopore signal.
Brieno converts raw ionic current into discrete tokens through residual vector quantization, then pretrains a transformer to predict masked tokens from their context. The pretrained transformer is the starting point for task-specific heads.
What is in this repository
Shared components, not runnable task models. Every Brieno model is built from a signal tokenizer and a token transformer; both are published here so that task models can reference them instead of each shipping its own copy.
| Component | Size | What it is |
|---|---|---|
dna_r9.4.1_tokenizer@v1.0 |
31 MB | RVQ tokenizer: 16,384 samples → 2,048 frames, 4 codebooks × 1,024 entries, quantised in a 64-d space |
dna_r9.4.1_token_tf_pretrained@v1.0 |
246 MB | Token transformer: 8 layers, 12 heads, d_model 768, max 2,048 frames |
The task models — basecalling, barcode demultiplexing, species identification and methylation calling — are being prepared and are not published yet. Until they are, this repository is useful for tokenizing signal and extracting representations, not for running a task end to end.
Using the components
pip install git+https://github.com/guangzhaocs/Brieno.git
Turn a corpus of signal chunks into tokens with the published tokenizer:
huggingface-cli download guangzhaocs/Brieno "dna_r9.4.1_tokenizer@v1.0/tokenizer.pt" \
--local-dir ./brieno-models
brieno train encode -m "./brieno-models/dna_r9.4.1_tokenizer@v1.0/tokenizer.pt" \
-i corpus/Ecoli.npy -o tokens/Ecoli.npy
Each checkpoint carries its own architecture alongside its weights, so there is no config to pass and no way for the two to disagree.
import torch
from brieno.data.encode import load_tokenizer
tokenizer, config = load_tokenizer("dna_r9.4.1_tokenizer@v1.0/tokenizer.pt")
signal = torch.randn(2, 1, 16384) # median/MAD-normalised current
_, tokens, _, _ = tokenizer(signal) # [num_quantizers, batch, 2048]
Signal must be normalised the same way it was during training — median/MAD over the
whole read. brieno train prep does this; see the
training guide.
How the components were trained
Both were trained on raw R9.4.1 DNA signal with no labels of any kind.
The tokenizer reconstructs the signal through the quantizer, minimising
L1(reconstruction, signal) + vq_loss. The codebooks are EMA-tracked, so the optimizer
sees only the encoder and decoder.
The transformer is pretrained with masked-token prediction on signal tokenized by that tokenizer. Spans of positions are hidden in every quantizer layer at once, and the model predicts them from context. Masking spans rather than isolated positions matters: neighbouring signal frames are highly correlated, so a single hidden position can be interpolated from its neighbours without learning anything.
The pretraining corpus spans 118 million reads across human, plant, fungal and bacterial genomes.
Model names
<analyte>_<pore>_<representation>_<sequence model>_<task>_<mode>@<version>
dna leaves room for RNA; r9.4.1 is the pore, and models do not transfer across
pores; token_tf says the model consumes discrete tokens with a transformer, so a
future continuous-input or Mamba variant gets its own name instead of silently
replacing this one. Classification models carry their class count (species10,
demux24) because they recognise a fixed label set and nothing else.
Model versions (@v1.0) and the brieno package version are independent: the weights
here are the ones behind the paper, while the software is still young. Each component
records the minimum package version that can load it.
Citation
@article{cheng2026brieno,
title = {Towards a foundation model for raw nanopore signal through discrete tokenization},
author = {Cheng, Guangzhao and Pitk{\"a}nen, Esa},
journal = {bioRxiv},
year = {2026},
doi = {10.64898/2026.11.05.764536},
url = {https://www.biorxiv.org/content/10.64898/2026.11.05.764536v1}
}
License
Apache 2.0.