The initial release described NTv3 as a single U-Net-style backbone pretrained on the OpenGenome2 corpus and then post-trained with a joint objective over functional tracks and annotation labels. It shipped three model sizes (8M, 100M, 650M), each in pretrained and post-trained form, plus context-length and architecture ablation variants.
The current release changes that structure: tracks post-training is its own stage, and annotation becomes a separate per-clade fine-tune rather than a head trained jointly. Numbers quoted in these materials therefore do not always match the current paper. Where the two differ, the current paper is authoritative.
All of the following are public on the Hub and unchanged. Sizes are the parameter counts as published.
| Model | Size | Pretrained | Post-trained | Usage |
|---|---|---|---|---|
| NTv3-8M | 8M params | โ | โ | Embeddings, light inference |
| NTv3-100M | 100M params | โ | โ | Embeddings, tracks, annotation |
| NTv3-650M | 650M params | โ | โ | Embeddings, tracks, annotation, best accuracy |
InstaDeepAI/NTv3_8M_pre_8kbInstaDeepAI/NTv3_100M_pre_8kbInstaDeepAI/NTv3_650M_pre_8kbInstaDeepAI/NTv3_100M_post_131kbInstaDeepAI/NTv3_650M_post_131kbInstaDeepAI/NTv3_5downsample_preInstaDeepAI/NTv3_5downsample_pre_8kbInstaDeepAI/NTv3_5downsample_postInstaDeepAI/NTv3_5downsample_post_131kbNTv3_100M_pos and NTv3_650M_pos. The correct identifiers end in _post. If a
script of yours 404s, that is the likely cause.
These notebooks were written and run against the initial-release checkpoints.
Pipeline notebooks:
Here is an example of how to load and use a pre-trained NTv3 model.
from transformers import AutoTokenizer, AutoModelForMaskedLM
model_name = "InstaDeepAI/NTv3_650M_pre"
model = AutoModelForMaskedLM.from_pretrained(model_name, trust_remote_code=True)
tok = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
batch = tok(["ATCGNATCG", "ACGT"], add_special_tokens=False, padding=True,
pad_to_multiple_of=128, return_tensors="pt")
out = model(**batch)
print(out.logits.shape) # (B, L, V = 11)
To pin a specific version rather than tracking the branch, pass a tag:
from_pretrained(model_name, revision="v1.0", trust_remote_code=True)
Here is an example of how to use the interpretation pipeline on the NTv3 post-trained model for multi-scale analysis of DNA sequences:
from transformers import pipeline
import torch
import matplotlib.pyplot as plt
model_name = "InstaDeepAI/NTv3_650M_post"
# Build interpretation pipeline
ntv3_interpret = pipeline(
"ntv3-interpret",
model=model_name,
trust_remote_code=True,
device=0 if torch.cuda.is_available() else -1,
)
# Run interpretation on a given genomic region with tracks, annotations, attention, and saliency
result = ntv3_interpret(
{"chrom": "chr11", "start": 5_253_561, "end": 5_286_329, "species": "human"},
output_attention=True,
output_saliency=True,
saliency_track_id="ENCSR000EFT", # K562 GATA1 ChIP-seq
plot=True, # plot predictions on tracks and annotations
tracks_to_plot={"K562 RNA-seq": "ENCSR056HPM", "K562 GATA1": "ENCSR000EFT"},
elements_to_plot=["exon", "promoter_Tissue_specific"],
)
# Access attention map results
result.plot_attention() # attention map (last layer)
plt.show()
# Access saliency scores results
result.plot_saliency(window_size=128)
plt.show()
Here is a quick example of how to use the post-trained NTv3 650M model to predict tracks for a human genomic window.
from transformers import pipeline
import torch
model_name = "InstaDeepAI/NTv3_650M_post"
ntv3_tracks = pipeline(
"ntv3-tracks",
model=model_name,
trust_remote_code=True,
device=0 if torch.cuda.is_available() else -1,
)
# Run track prediction
out = ntv3_tracks(
{
"chrom": "chr19",
"start": 6_700_000,
"end": 6_831_072,
"species": "human"
}
)
# Print output shapes
# 7k human tracks over 37.5 % center region of the input sequence
print("bigwig_tracks_logits:", tuple(out.bigwig_tracks_logits.shape))
# Location of 21 genomic elements over 37.5 % center region of the input sequence
print("bed_tracks_logits:", tuple(out.bed_tracks_logits.shape))
# Language model logits for whole sequence over vocabulary
print("language model logits:", tuple(out.mlm_logits.shape))Predictions can be plotted for a subset of tracks and genomic elements:
tracks_to_plot = {
"K562 RNA-seq": "ENCSR056HPM",
"K562 DNAse": "ENCSR921NMD",
"K562 H3k4me3": "ENCSR000DWD",
"K562 CTCF": "ENCSR000AKO",
"HepG2 RNA-seq": "ENCSR561FEE_P",
"HepG2 DNAse": "ENCSR000EJV",
"HepG2 H3k4me3": "ENCSR000AMP",
"HepG2 CTCF": "ENCSR000BIE",
}
elements_to_plot = ["protein_coding_gene", "exon", "intron", "splice_donor", "splice_acceptor"]
out = ntv3_tracks(
{"chrom": "chr19", "start": 6_700_000, "end": 6_831_072, "species": "human"},
plot=True,
tracks_to_plot=tracks_to_plot,
elements_to_plot=elements_to_plot,
)