Spaces:
Running
Running
| <div class="notice"> | |
| 🗄️ <strong>This page describes the initial NTv3 release, from 2025.</strong> Those repositories are retained unchanged for | |
| reproducibility and backward compatibility. Existing model IDs, files and interfaces continue to work exactly as they | |
| did, and nothing on this page is being withdrawn. If you are starting something new, prefer the current models on the | |
| <strong>Home</strong> tab; if you are reproducing published results or already have a working pipeline, stay here. | |
| </div> | |
| <div class="summary"> | |
| <h2>📖 About the initial release (2025)</h2> | |
| <p> | |
| The initial release described NTv3 as a single U-Net-style backbone pretrained on the OpenGenome2 corpus and then | |
| post-trained with a <em>joint</em> objective over functional tracks and annotation labels. It shipped three model sizes | |
| (8M, 100M, 650M), each in pretrained and post-trained form, plus context-length and architecture ablation variants. | |
| </p> | |
| <p> | |
| The current release changes that structure: tracks post-training is its own stage, and annotation becomes a separate | |
| per-clade fine-tune rather than a head trained jointly. Numbers quoted in these materials therefore do not always | |
| match the current paper. Where the two differ, the current paper is authoritative. | |
| </p> | |
| </div> | |
| <div class="paper-summary"> | |
| <img src="assets/paper_summary.png" alt="NTv3 initial-release overview figure" /> | |
| </div> | |
| <div class="grid"> | |
| <div class="card" style="grid-column: span 12;"> | |
| <h2>🤖 Initial-release checkpoints (2025)</h2> | |
| <p> | |
| All of the following are public on the Hub and unchanged. Sizes are the parameter counts as published. | |
| </p> | |
| <table> | |
| <thead> | |
| <tr> | |
| <th>Model</th> | |
| <th>Size</th> | |
| <th>Pretrained</th> | |
| <th>Post-trained</th> | |
| <th>Usage</th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr> | |
| <td><strong>NTv3-8M</strong></td> | |
| <td>8M params</td> | |
| <td><span class="checkmark">✅</span></td> | |
| <td>❌</td> | |
| <td>Embeddings, light inference</td> | |
| </tr> | |
| <tr> | |
| <td><strong>NTv3-100M</strong></td> | |
| <td>100M params</td> | |
| <td><span class="checkmark">✅</span></td> | |
| <td><span class="checkmark">✅</span></td> | |
| <td>Embeddings, tracks, annotation</td> | |
| </tr> | |
| <tr> | |
| <td><strong>NTv3-650M</strong></td> | |
| <td>650M params</td> | |
| <td><span class="checkmark">✅</span></td> | |
| <td><span class="checkmark">✅</span></td> | |
| <td>Embeddings, tracks, annotation, best accuracy</td> | |
| </tr> | |
| </tbody> | |
| </table> | |
| <h2 style="margin-top: 22px;">📦 Pretrained</h2> | |
| <ul> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_8M_pre" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_8M_pre</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_100M_pre" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_100M_pre</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_650M_pre" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_650M_pre</code></a></li> | |
| </ul> | |
| <h2 style="margin-top: 18px;">🎯 Post-trained</h2> | |
| <ul> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_100M_post" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_100M_post</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_650M_post" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_650M_post</code></a></li> | |
| </ul> | |
| <h2 style="margin-top: 18px;">📏 Context-length variants</h2> | |
| <ul> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_8M_pre_8kb" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_8M_pre_8kb</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_100M_pre_8kb" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_100M_pre_8kb</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_650M_pre_8kb" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_650M_pre_8kb</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_100M_post_131kb" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_100M_post_131kb</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_650M_post_131kb" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_650M_post_131kb</code></a></li> | |
| </ul> | |
| <h2 style="margin-top: 18px;">🔬 Architecture ablations (five-downsample variants)</h2> | |
| <ul> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_5downsample_pre" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_5downsample_pre</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_5downsample_pre_8kb" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_5downsample_pre_8kb</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_5downsample_post" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_5downsample_post</code></a></li> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_5downsample_post_131kb" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_5downsample_post_131kb</code></a></li> | |
| </ul> | |
| <h2 style="margin-top: 18px;">🧪 Generative</h2> | |
| <ul> | |
| <li><a href="https://huggingface.co/InstaDeepAI/NTv3_generative" target="_blank" rel="noopener noreferrer"><code>InstaDeepAI/NTv3_generative</code></a></li> | |
| </ul> | |
| <div class="notice"> | |
| <strong>Naming note.</strong> Some earlier materials referred to the post-trained checkpoints as | |
| <code>NTv3_100M_pos</code> and <code>NTv3_650M_pos</code>. The correct identifiers end in <code>_post</code>. If a | |
| script of yours 404s, that is the likely cause. | |
| </div> | |
| </div> | |
| <div class="card-stack"> | |
| <div class="card"> | |
| <h2>📓 Notebooks for the initial release (2025)</h2> | |
| <p>These notebooks were written and run against the initial-release checkpoints.</p> | |
| <ul> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_tutorials/00_quickstart_inference.ipynb" target="_blank" rel="noopener noreferrer">🚀 00 · Quickstart inference</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_tutorials/01_tracks_prediction.ipynb" target="_blank" rel="noopener noreferrer">📊 01 · Tracks prediction</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_tutorials/02_fine_tuning_pretrained_model_biwig.ipynb" target="_blank" rel="noopener noreferrer">🎯 02 · Fine-tune a pretrained model on bigwig tracks</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_tutorials/03_fine_tuning_posttrained_model_biwig.ipynb" target="_blank" rel="noopener noreferrer">🎯 03 · Fine-tune a post-trained model on bigwig tracks</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_tutorials/04_fine_tuning_pretrained_model_annotation.ipynb" target="_blank" rel="noopener noreferrer">🏷️ 04 · Fine-tune a pretrained model on annotations</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_tutorials/05_model_interpretation.ipynb" target="_blank" rel="noopener noreferrer">🔍 05 · Model interpretation</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_tutorials/06_NTv3_generative_training.ipynb" target="_blank" rel="noopener noreferrer">🧪 06 · Fine-tuning NTv3 into a diffusion model</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_tutorials/07_enhancer_generation.ipynb" target="_blank" rel="noopener noreferrer">🪰 07 · Generating enhancer sequences</a></li> | |
| </ul> | |
| <p style="margin-top: 12px;">Pipeline notebooks:</p> | |
| <ul> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_pipelines/01_functional_track_prediction.ipynb" target="_blank" rel="noopener noreferrer">🎯 01 · Generate bigwig predictions for given tracks</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/blob/initial-release/notebooks_pipelines/02_model_interpretation.ipynb" target="_blank" rel="noopener noreferrer">🔍 02 · Interpret a given genomic region</a></li> | |
| </ul> | |
| </div> | |
| <div class="card"> | |
| <h2>🔗 Initial-release links (2025)</h2> | |
| <ul> | |
| <li><a href="https://www.biorxiv.org/content/10.64898/2025.12.22.695963v1" target="_blank" rel="noopener noreferrer">📄 Initial preprint</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3/tree/initial-release" target="_blank" rel="noopener noreferrer">🏠 Original NTv3 Space (files at tag <code>initial-release</code>)</a></li> | |
| <li><a href="https://github.com/instadeepai/nucleotide-transformer/tree/ntv3-initial-release" target="_blank" rel="noopener noreferrer">💻 JAX model code (GitHub, tag <code>ntv3-initial-release</code>)</a></li> | |
| <li><a href="https://huggingface.co/collections/InstaDeepAI/nucleotide-transformer-v3" target="_blank" rel="noopener noreferrer">🎯 HF model collection (2025)</a></li> | |
| <li><a href="https://huggingface.co/datasets/InstaDeepAI/NTv3_benchmark_dataset" target="_blank" rel="noopener noreferrer">📚 NTv3 benchmark dataset</a></li> | |
| <li><a href="https://huggingface.co/datasets/InstaDeepAI/NTv3_enhancer_generation" target="_blank" rel="noopener noreferrer">📚 NTv3 enhancer generation dataset</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3_benchmark" target="_blank" rel="noopener noreferrer">🏆 NTv3 benchmark leaderboard (2025)</a></li> | |
| <li><a href="https://huggingface.co/spaces/InstaDeepAI/ntv3_tracks" target="_blank" rel="noopener noreferrer">🚀 Interactive tracks demo</a></li> | |
| </ul> | |
| </div> | |
| </div> | |
| <div class="card"> | |
| <h2>🤖 Load a pre-trained model</h2> | |
| <p>Here is an example of how to load and use a pre-trained NTv3 model.</p> | |
| <div class="code"><pre><code class="language-python">from transformers import AutoTokenizer, AutoModelForMaskedLM | |
| model_name = "InstaDeepAI/NTv3_650M_pre" | |
| model = AutoModelForMaskedLM.from_pretrained(model_name, trust_remote_code=True) | |
| tok = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) | |
| batch = tok(["ATCGNATCG", "ACGT"], add_special_tokens=False, padding=True, | |
| pad_to_multiple_of=128, return_tensors="pt") | |
| out = model(**batch) | |
| print(out.logits.shape) # (B, L, V = 11) | |
| </code></pre></div> | |
| <p> | |
| To pin a specific version rather than tracking the branch, pass a tag: | |
| <code>from_pretrained(model_name, revision="v1.0", trust_remote_code=True)</code> | |
| </p> | |
| </div> | |
| <div class="card"> | |
| <h2>🔍 Model interpretation</h2> | |
| <p> | |
| Here is an example of how to use the interpretation pipeline on the NTv3 post-trained model for multi-scale | |
| analysis of DNA sequences: | |
| </p> | |
| <div class="code"><pre><code class="language-python">from transformers import pipeline | |
| import torch | |
| import matplotlib.pyplot as plt | |
| model_name = "InstaDeepAI/NTv3_650M_post" | |
| # Build interpretation pipeline | |
| ntv3_interpret = pipeline( | |
| "ntv3-interpret", | |
| model=model_name, | |
| trust_remote_code=True, | |
| device=0 if torch.cuda.is_available() else -1, | |
| ) | |
| # Run interpretation on a given genomic region with tracks, annotations, attention, and saliency | |
| result = ntv3_interpret( | |
| {"chrom": "chr11", "start": 5_253_561, "end": 5_286_329, "species": "human"}, | |
| output_attention=True, | |
| output_saliency=True, | |
| saliency_track_id="ENCSR000EFT", # K562 GATA1 ChIP-seq | |
| plot=True, # plot predictions on tracks and annotations | |
| tracks_to_plot={"K562 RNA-seq": "ENCSR056HPM", "K562 GATA1": "ENCSR000EFT"}, | |
| elements_to_plot=["exon", "promoter_Tissue_specific"], | |
| ) | |
| # Access attention map results | |
| result.plot_attention() # attention map (last layer) | |
| plt.show() | |
| # Access saliency scores results | |
| result.plot_saliency(window_size=128) | |
| plt.show()</code></pre></div> | |
| <img src="assets/saliency_example.png" alt="Saliency example" style="max-width: 100%; margin-top: 20px;" /> | |
| </div> | |
| <div class="card" style="grid-column: span 12;"> | |
| <h2>💻 Use a post-trained model</h2> | |
| <p> | |
| Here is a quick example of how to use the post-trained NTv3 650M model to predict tracks for a human genomic | |
| window. | |
| </p> | |
| <div class="code"><pre><code class="language-python">from transformers import pipeline | |
| import torch | |
| model_name = "InstaDeepAI/NTv3_650M_post" | |
| ntv3_tracks = pipeline( | |
| "ntv3-tracks", | |
| model=model_name, | |
| trust_remote_code=True, | |
| device=0 if torch.cuda.is_available() else -1, | |
| ) | |
| # Run track prediction | |
| out = ntv3_tracks( | |
| { | |
| "chrom": "chr19", | |
| "start": 6_700_000, | |
| "end": 6_831_072, | |
| "species": "human" | |
| } | |
| ) | |
| # Print output shapes | |
| # 7k human tracks over 37.5 % center region of the input sequence | |
| print("bigwig_tracks_logits:", tuple(out.bigwig_tracks_logits.shape)) | |
| # Location of 21 genomic elements over 37.5 % center region of the input sequence | |
| print("bed_tracks_logits:", tuple(out.bed_tracks_logits.shape)) | |
| # Language model logits for whole sequence over vocabulary | |
| print("language model logits:", tuple(out.mlm_logits.shape))</code></pre></div> | |
| <p>Predictions can be plotted for a subset of tracks and genomic elements:</p> | |
| <div class="code"><pre><code class="language-python">tracks_to_plot = { | |
| "K562 RNA-seq": "ENCSR056HPM", | |
| "K562 DNAse": "ENCSR921NMD", | |
| "K562 H3k4me3": "ENCSR000DWD", | |
| "K562 CTCF": "ENCSR000AKO", | |
| "HepG2 RNA-seq": "ENCSR561FEE_P", | |
| "HepG2 DNAse": "ENCSR000EJV", | |
| "HepG2 H3k4me3": "ENCSR000AMP", | |
| "HepG2 CTCF": "ENCSR000BIE", | |
| } | |
| elements_to_plot = ["protein_coding_gene", "exon", "intron", "splice_donor", "splice_acceptor"] | |
| out = ntv3_tracks( | |
| {"chrom": "chr19", "start": 6_700_000, "end": 6_831_072, "species": "human"}, | |
| plot=True, | |
| tracks_to_plot=tracks_to_plot, | |
| elements_to_plot=elements_to_plot, | |
| )</code></pre></div> | |
| <img src="assets/output_tracks.png" alt="Predicted tracks over a human genomic window" style="max-width: 100%; margin-top: 20px;" /> | |
| </div> | |
| </div> | |