Explain the region interface and tighten the card

028385e verified 4 days ago

11.2 kB

---
library_name: transformers
tags:
  - genomics
  - dna
  - biology
  - ntv3
  - long-range
  - functional-genomics
license: other
license_name: instadeep-open-model-licence
license_link: https://huggingface.co/InstaDeepAI/NTv3_tracks/blob/main/MODEL-LICENSE.md
extra_gated_heading: Request access to the NTv3 models
extra_gated_description: >-
  One approval grants access to every gated NTv3 model. Requests are reviewed
  manually.
extra_gated_prompt: >-
  The NTv3 weights are released under the
  [InstaDeep Open Model Licence](https://huggingface.co/InstaDeepAI/NTv3_tracks/blob/main/MODEL-LICENSE.md) for non-commercial use only, and its terms
  also cover the models' outputs. For commercial use, contact legal@instadeep.com.
extra_gated_fields:
  Full name: text
  Affiliation: text
  Institutional email: text
  Organisation type:
    type: select
    options:
      - University
      - Public research institute
      - Non-profit
      - Hospital
      - Company
      - Other
  Country: country
  Intended use: text
  I will use NTv3 and its outputs for non-commercial purposes only, under the InstaDeep Open Model Licence: checkbox
extra_gated_button_content: Request access
---
# NTv3 Tracks

Predicts functional genomic tracks at single-base resolution, conditioned on species:
gene expression, histone marks, transcription-factor binding and chromatin
accessibility, from DNA sequence alone.

Built on the [`NTv3_pretrained`](https://huggingface.co/InstaDeepAI/NTv3_pretrained)
backbone and post-trained on 16,262 tracks across 15 species, then finetuned on a mix
of input lengths so it can be used from 16 kb to 1 Mb.

## Architecture

The NTv3 U-Net trunk (a convolutional encoder, a transformer tower at 1/128
resolution and a deconvolutional decoder back to base resolution) with prediction heads
on top. Species conditioning is applied through FiLM modulation inside the trunk and at
the head.

| | |
| --- | --- |
| Parameters | 711,256,680 |
| Resolution | 1 bp |
| Input length | 16,384 to 1,048,576 bp |
| Vocabulary | 16 tokens (soft-masked) |
| Species | 15 |
| Output channels | 7,362 |
| Precision | float32 |

## Usage

```python
import json
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModel, AutoTokenizer
repo = "InstaDeepAI/NTv3_tracks"
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
metadata = json.load(open(hf_hub_download(repo, "track_metadata.json")))
# The input length must be a multiple of 128; see "Input length" below.
sequence = "ACGT" * 4_096
batch = tokenizer([sequence], return_tensors="pt", add_special_tokens=False)
species = [torch.tensor([metadata["species_vocabulary"]["human"]])]
with torch.no_grad():
    out = model(**batch, conditions_tokens=species)
out.bigwig_tracks_logits.shape  # (1, 6144, 7362)
```

A 1 Mb forward pass needs a GPU with about 32 GB of memory or more; move the model
and the batch to it with `.to("cuda")`.

Outputs are reachable both as attributes (`out.bigwig_tracks_logits`) and as keys
(`out["bigwig_tracks_logits"]`). An `attention_mask` is accepted and ignored: the
model derives its own mask from the pad token.

`trust_remote_code=True` is required. The modelling code is **not** in this repository:
`config.json` resolves it from `InstaDeepAI/NTv3_pretrained`, which therefore has to be
reachable as well.

### Outputs

| Key | Shape | What it is |
| --- | --- | --- |
| `bigwig_tracks_logits` | `(batch, length × 0.375, 7362)` | Predicted track signal, one channel per track |
| `embedding` | `(batch, length, 1024)` | Decoder output, one vector per base |
| `logits` | `(batch, length, 16)` | Masked-token prediction |
| `species_logits` | `(batch, 21)` | Auxiliary head from training. **Not a usable species classifier**: it predicts the mask token on real sequence. |

## The region interface

For a genomic region rather than a bare sequence, the pipeline fetches the
window, conditions on the species and labels the output:

```python
from transformers import pipeline
predictor = pipeline("ntv3-tracks", model="InstaDeepAI/NTv3_tracks",
                     trust_remote_code=True)
out = predictor({"chrom": "chr11", "start": 5_219_320, "end": 5_235_704,
                 "species": "human"})
out.tracks.shape      # (6144, 7362)  positions x tracks
out.coordinates       # (5224440, 5230584)  what the predictions span
out.track("kai1")     # one track's signal, by id
```

The pipeline downloads the window, soft-masked, from the
[UCSC Genome Browser API](https://api.genome.ucsc.edu), so `start` and `end` are
0-based, end-exclusive coordinates on the species' assembly
(`metadata["species_to_assembly"]`). UCSC serves every assembly here except wheat
(`IWGSC`). For wheat, or for a sequence of your own, pass
`{"seq": ..., "species": ...}` instead.

`out.coordinates` is the span the predictions cover, the central 37.5% of the window
(see below). It is `None` when you pass `seq`.

## Reading the predictions

**The output does not cover the whole window.** It covers the **central 37.5%**. A 1 Mb
input yields predictions for the central 393,216 bp. The edges are consumed as context.
Tile your windows with overlap if you need continuous coverage.

**Channel `i` is track `i` of that species' list.** `track_metadata.json` holds
`bigwigs_per_file_assembly`, mapping each assembly to its ordered track ids:

```python
track_ids = metadata["bigwigs_per_file_assembly"]["hg38"]
signal = out["bigwig_tracks_logits"][0, :, 0]  # track_ids[0]
```

Species have different numbers of tracks and the head is padded to the widest one, so
only the first `len(track_ids)` channels are meaningful for a given species. The rest is
padding.

**The species id comes from the same file.** `metadata["species_vocabulary"]` maps each
species name to its id. Passing the wrong id gives you another species' tracks, silently.

**Track lists are keyed by assembly, not by species**, so
`metadata["species_to_assembly"]` gives the link:

```python
assembly = metadata["species_to_assembly"]["human"]        # "hg38"
track_ids = metadata["bigwigs_per_file_assembly"][assembly]
```

**The predictions are in the scaled space the model was trained on, not in signal
units.** Each track was divided by its genome-wide mean, RNA-seq tracks were then raised
to the power 0.75, and values above 10 were soft-clipped. The per-track means and the
RNA-seq flags ship in `track_metadata.json`, in the same order as the track ids, so the
predictions can be converted back to signal units:

```python
import numpy as np
def to_signal(predictions, assembly, track_ids):
    """Converts predictions for these tracks, (..., len(track_ids)), to signal units."""
    all_ids = metadata["bigwigs_per_file_assembly"][assembly]
    index = [all_ids.index(track_id) for track_id in track_ids]
    mean = np.array(metadata["track_means_per_file_assembly"][assembly])[index]
    rna_seq = np.array(metadata["rna_seq_per_file_assembly"][assembly])[index]
    x = np.where(predictions > 10, (predictions + 10) ** 2 / 40, predictions)
    x = np.where(rna_seq, x ** (1 / 0.75), x)
    return x * mean
```

## Input length

The model was finetuned from
[`NTv3_tracks_1mb`](https://huggingface.co/InstaDeepAI/NTv3_tracks_1mb) on inputs of
16, 32, 65, 131, 262 and 524 kb and 1 Mb. It matches that model at 1 Mb and stays
accurate on the shorter inputs, where the 1 Mb model does not. Accuracy is still
highest at 1 Mb, where the model sees the most context. Lengths between
the trained ones, and lengths outside 16 kb to 1 Mb, were not evaluated.

![Pearson correlation of predicted and measured tracks against input length, for
NTv3_tracks and NTv3_tracks_1mb](assets/context_length.png)

*Pearson correlation between predicted and measured tracks (log1p-transformed, 1 bp)
on held-out test regions of the 15 species, by input length. Right: the change per
species.*

## Input requirements

**Length must be a multiple of 128**, the total downsampling factor of the trunk.
Use one of the trained lengths above where you can.

**Pass soft-masked sequence as it is.** This model reads a 16-symbol alphabet: `A C G T
N` and their lowercase counterparts `a c g t n`. Lowercase marks repeat-masked bases,
which the model was trained on, so do not upper-case your input. `NTv3_pretrained` uses
the 11-symbol uppercase alphabet instead, so the two tokenizers are **not**
interchangeable.

Characters outside the vocabulary become `<unk>` and the sequence keeps its length.

## Intended use

Predicting functional track signal from sequence for the species the model covers;
comparing predicted signal between alleles or between designed sequences; and as a
backbone for fine-tuning on your own track data.

## Out of scope

Not a clinical or diagnostic tool. Predicted signal is a model output, not a
measurement.

Species outside the 15 it was trained on are out of distribution. The model still
returns predictions for them, but they have not been validated.

## Limitations

Predictions are conditioned on species and sequence only. There is no input for cell
type, tissue, developmental stage or treatment beyond what the track panel itself
encodes, so the model cannot be steered to a condition that is not already a track.

Accuracy varies substantially across assays, across species and across genomic regions.
The panel is not balanced: some species contribute many more tracks than others, and
performance follows that. Treat a single channel's output as weaker evidence than an
aggregate.

The 37.5% crop is fixed by training; it is not an inference setting.

## Related models

| Model | For |
| --- | --- |
| [`NTv3_tracks_1mb`](https://huggingface.co/InstaDeepAI/NTv3_tracks_1mb) | The 1 Mb specialist this model was finetuned from; the NTv3 paper's main results use it |
| [`NTv3_pretrained`](https://huggingface.co/InstaDeepAI/NTv3_pretrained) | Embeddings, masked-token prediction, fine-tuning |
| `NTv3_annot_{mammals,plants,insects}` | Ab initio gene annotation, one model per clade |

## Licence

Released under the [InstaDeep Open Model Licence](https://huggingface.co/InstaDeepAI/NTv3_tracks/blob/main/MODEL-LICENSE.md). The licence restricts **output** as
well as weights, so predicted tracks carry the same terms. Read it before building on
the model commercially.

## Citation

```bibtex
@article{ntv3,
  title   = {A multispecies foundation model for genome function prediction, gene annotation and sequence design},
  author  = {Boshar, Sam and Evans, Benjamin and Tang, Ziqi and Adel, Yanis and Bornachot, Yohan and Sidbon, Shawn and Lorbeer, Franziska K. and Picard, Armand and Rajesh, Chandana and Seitz, Evan and Emms, David and Mendoza-Revilla, Javier and Hernandez, Ariana and Al-Ani, Fatimah and Karch, Tristan and Schiff, Yair and Lopez, Marie and Laterre, Alexandre and Beguir, Karim and Koo, Peter and Kuleshov, Volodymyr and Stark, Alexander and de Almeida, Bernardo P. and Pierrot, Thomas},
  journal = {bioRxiv},
  year    = {2026},
  doi     = {10.64898/2025.12.22.695963},
  url     = {https://www.biorxiv.org/content/10.64898/2025.12.22.695963}
}
```

	---
	library_name: transformers
	tags:
	- genomics
	- dna
	- biology
	- ntv3
	- long-range
	- functional-genomics
	license: other
	license_name: instadeep-open-model-licence
	license_link: https://huggingface.co/InstaDeepAI/NTv3_tracks/blob/main/MODEL-LICENSE.md
	extra_gated_heading: Request access to the NTv3 models
	extra_gated_description: >-
	One approval grants access to every gated NTv3 model. Requests are reviewed
	manually.
	extra_gated_prompt: >-
	The NTv3 weights are released under the
	[InstaDeep Open Model Licence](https://huggingface.co/InstaDeepAI/NTv3_tracks/blob/main/MODEL-LICENSE.md) for non-commercial use only, and its terms
	also cover the models' outputs. For commercial use, contact legal@instadeep.com.
	extra_gated_fields:
	Full name: text
	Affiliation: text
	Institutional email: text
	Organisation type:
	type: select
	options:
	- University
	- Public research institute
	- Non-profit
	- Hospital
	- Company
	- Other
	Country: country
	Intended use: text
	I will use NTv3 and its outputs for non-commercial purposes only, under the InstaDeep Open Model Licence: checkbox
	extra_gated_button_content: Request access
	---

	# NTv3 Tracks

	Predicts functional genomic tracks at single-base resolution, conditioned on species:
	gene expression, histone marks, transcription-factor binding and chromatin
	accessibility, from DNA sequence alone.

	Built on the [`NTv3_pretrained`](https://huggingface.co/InstaDeepAI/NTv3_pretrained)
	backbone and post-trained on 16,262 tracks across 15 species, then finetuned on a mix
	of input lengths so it can be used from 16 kb to 1 Mb.

	## Architecture

	The NTv3 U-Net trunk (a convolutional encoder, a transformer tower at 1/128
	resolution and a deconvolutional decoder back to base resolution) with prediction heads
	on top. Species conditioning is applied through FiLM modulation inside the trunk and at
	the head.

	\| \| \|
	\| --- \| --- \|
	\| Parameters \| 711,256,680 \|
	\| Resolution \| 1 bp \|
	\| Input length \| 16,384 to 1,048,576 bp \|
	\| Vocabulary \| 16 tokens (soft-masked) \|
	\| Species \| 15 \|
	\| Output channels \| 7,362 \|
	\| Precision \| float32 \|

	## Usage

	```python
	import json

	import torch
	from huggingface_hub import hf_hub_download
	from transformers import AutoModel, AutoTokenizer

	repo = "InstaDeepAI/NTv3_tracks"
	model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()
	tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
	metadata = json.load(open(hf_hub_download(repo, "track_metadata.json")))

	# The input length must be a multiple of 128; see "Input length" below.
	sequence = "ACGT" * 4_096

	batch = tokenizer([sequence], return_tensors="pt", add_special_tokens=False)
	species = [torch.tensor([metadata["species_vocabulary"]["human"]])]

	with torch.no_grad():
	out = model(**batch, conditions_tokens=species)

	out.bigwig_tracks_logits.shape # (1, 6144, 7362)
	```

	A 1 Mb forward pass needs a GPU with about 32 GB of memory or more; move the model
	and the batch to it with `.to("cuda")`.

	Outputs are reachable both as attributes (`out.bigwig_tracks_logits`) and as keys
	(`out["bigwig_tracks_logits"]`). An `attention_mask` is accepted and ignored: the
	model derives its own mask from the pad token.

	`trust_remote_code=True` is required. The modelling code is not in this repository:
	`config.json` resolves it from `InstaDeepAI/NTv3_pretrained`, which therefore has to be
	reachable as well.

	### Outputs

	\| Key \| Shape \| What it is \|
	\| --- \| --- \| --- \|
	\| `bigwig_tracks_logits` \| `(batch, length × 0.375, 7362)` \| Predicted track signal, one channel per track \|
	\| `embedding` \| `(batch, length, 1024)` \| Decoder output, one vector per base \|
	\| `logits` \| `(batch, length, 16)` \| Masked-token prediction \|
	\| `species_logits` \| `(batch, 21)` \| Auxiliary head from training. Not a usable species classifier: it predicts the mask token on real sequence. \|

	## The region interface

	For a genomic region rather than a bare sequence, the pipeline fetches the
	window, conditions on the species and labels the output:

	```python
	from transformers import pipeline

	predictor = pipeline("ntv3-tracks", model="InstaDeepAI/NTv3_tracks",
	trust_remote_code=True)

	out = predictor({"chrom": "chr11", "start": 5_219_320, "end": 5_235_704,
	"species": "human"})

	out.tracks.shape # (6144, 7362) positions x tracks
	out.coordinates # (5224440, 5230584) what the predictions span
	out.track("kai1") # one track's signal, by id
	```

	The pipeline downloads the window, soft-masked, from the
	[UCSC Genome Browser API](https://api.genome.ucsc.edu), so `start` and `end` are
	0-based, end-exclusive coordinates on the species' assembly
	(`metadata["species_to_assembly"]`). UCSC serves every assembly here except wheat
	(`IWGSC`). For wheat, or for a sequence of your own, pass
	`{"seq": ..., "species": ...}` instead.

	`out.coordinates` is the span the predictions cover, the central 37.5% of the window
	(see below). It is `None` when you pass `seq`.

	## Reading the predictions

	The output does not cover the whole window. It covers the central 37.5%. A 1 Mb
	input yields predictions for the central 393,216 bp. The edges are consumed as context.
	Tile your windows with overlap if you need continuous coverage.

	Channel `i` is track `i` of that species' list. `track_metadata.json` holds
	`bigwigs_per_file_assembly`, mapping each assembly to its ordered track ids:

	```python
	track_ids = metadata["bigwigs_per_file_assembly"]["hg38"]
	signal = out["bigwig_tracks_logits"][0, :, 0] # track_ids[0]
	```

	Species have different numbers of tracks and the head is padded to the widest one, so
	only the first `len(track_ids)` channels are meaningful for a given species. The rest is
	padding.

	The species id comes from the same file. `metadata["species_vocabulary"]` maps each
	species name to its id. Passing the wrong id gives you another species' tracks, silently.

	Track lists are keyed by assembly, not by species, so
	`metadata["species_to_assembly"]` gives the link:

	```python
	assembly = metadata["species_to_assembly"]["human"] # "hg38"
	track_ids = metadata["bigwigs_per_file_assembly"][assembly]
	```

	**The predictions are in the scaled space the model was trained on, not in signal
	units.** Each track was divided by its genome-wide mean, RNA-seq tracks were then raised
	to the power 0.75, and values above 10 were soft-clipped. The per-track means and the
	RNA-seq flags ship in `track_metadata.json`, in the same order as the track ids, so the
	predictions can be converted back to signal units:

	```python
	import numpy as np

	def to_signal(predictions, assembly, track_ids):
	"""Converts predictions for these tracks, (..., len(track_ids)), to signal units."""
	all_ids = metadata["bigwigs_per_file_assembly"][assembly]
	index = [all_ids.index(track_id) for track_id in track_ids]
	mean = np.array(metadata["track_means_per_file_assembly"][assembly])[index]
	rna_seq = np.array(metadata["rna_seq_per_file_assembly"][assembly])[index]
	x = np.where(predictions > 10, (predictions + 10) ** 2 / 40, predictions)
	x = np.where(rna_seq, x ** (1 / 0.75), x)
	return x * mean
	```

	## Input length

	The model was finetuned from
	[`NTv3_tracks_1mb`](https://huggingface.co/InstaDeepAI/NTv3_tracks_1mb) on inputs of
	16, 32, 65, 131, 262 and 524 kb and 1 Mb. It matches that model at 1 Mb and stays
	accurate on the shorter inputs, where the 1 Mb model does not. Accuracy is still
	highest at 1 Mb, where the model sees the most context. Lengths between
	the trained ones, and lengths outside 16 kb to 1 Mb, were not evaluated.

	![Pearson correlation of predicted and measured tracks against input length, for
	NTv3_tracks and NTv3_tracks_1mb](assets/context_length.png)

	*Pearson correlation between predicted and measured tracks (log1p-transformed, 1 bp)
	on held-out test regions of the 15 species, by input length. Right: the change per
	species.*

	## Input requirements

	Length must be a multiple of 128, the total downsampling factor of the trunk.
	Use one of the trained lengths above where you can.

	Pass soft-masked sequence as it is. This model reads a 16-symbol alphabet: `A C G T
	N` and their lowercase counterparts `a c g t n`. Lowercase marks repeat-masked bases,
	which the model was trained on, so do not upper-case your input. `NTv3_pretrained` uses
	the 11-symbol uppercase alphabet instead, so the two tokenizers are not
	interchangeable.

	Characters outside the vocabulary become `<unk>` and the sequence keeps its length.

	## Intended use

	Predicting functional track signal from sequence for the species the model covers;
	comparing predicted signal between alleles or between designed sequences; and as a
	backbone for fine-tuning on your own track data.

	## Out of scope

	Not a clinical or diagnostic tool. Predicted signal is a model output, not a
	measurement.

	Species outside the 15 it was trained on are out of distribution. The model still
	returns predictions for them, but they have not been validated.

	## Limitations

	Predictions are conditioned on species and sequence only. There is no input for cell
	type, tissue, developmental stage or treatment beyond what the track panel itself
	encodes, so the model cannot be steered to a condition that is not already a track.

	Accuracy varies substantially across assays, across species and across genomic regions.
	The panel is not balanced: some species contribute many more tracks than others, and
	performance follows that. Treat a single channel's output as weaker evidence than an
	aggregate.

	The 37.5% crop is fixed by training; it is not an inference setting.

	## Related models

	\| Model \| For \|
	\| --- \| --- \|
	\| [`NTv3_tracks_1mb`](https://huggingface.co/InstaDeepAI/NTv3_tracks_1mb) \| The 1 Mb specialist this model was finetuned from; the NTv3 paper's main results use it \|
	\| [`NTv3_pretrained`](https://huggingface.co/InstaDeepAI/NTv3_pretrained) \| Embeddings, masked-token prediction, fine-tuning \|
	\| `NTv3_annot_{mammals,plants,insects}` \| Ab initio gene annotation, one model per clade \|

	## Licence

	Released under the [InstaDeep Open Model Licence](https://huggingface.co/InstaDeepAI/NTv3_tracks/blob/main/MODEL-LICENSE.md). The licence restricts output as
	well as weights, so predicted tracks carry the same terms. Read it before building on
	the model commercially.

	## Citation

	```bibtex
	@article{ntv3,
	title = {A multispecies foundation model for genome function prediction, gene annotation and sequence design},
	author = {Boshar, Sam and Evans, Benjamin and Tang, Ziqi and Adel, Yanis and Bornachot, Yohan and Sidbon, Shawn and Lorbeer, Franziska K. and Picard, Armand and Rajesh, Chandana and Seitz, Evan and Emms, David and Mendoza-Revilla, Javier and Hernandez, Ariana and Al-Ani, Fatimah and Karch, Tristan and Schiff, Yair and Lopez, Marie and Laterre, Alexandre and Beguir, Karim and Koo, Peter and Kuleshov, Volodymyr and Stark, Alexander and de Almeida, Bernardo P. and Pierrot, Thomas},
	journal = {bioRxiv},
	year = {2026},
	doi = {10.64898/2025.12.22.695963},
	url = {https://www.biorxiv.org/content/10.64898/2025.12.22.695963}
	}
	```