OneScience/evo2

Evo2

Model Introduction

Evo2 is a large-scale foundation model for genomic sequences. It can generate, complete, and predict functions for DNA/RNA sequences. The model supports pretraining on large-scale genomic datasets and can be rapidly adapted to downstream tasks such as regulatory element prediction and variant effect assessment, providing a unified sequence modeling tool for genomics research.

Model Description

Evo2 is built on a hybrid architecture that combines Hyena operators with convolutional gating, and supports context windows at the million-token scale. The model is pretrained on multi-species genomic data containing trillions of base pairs. It takes raw nucleotide sequences as input and supports autoregressive generation, sequence classification, and embedding extraction. Evo2 is compatible with multi-GPU distributed training and flexible data mixing, achieves leading performance on benchmarks such as GenBench, and is suitable for pretraining, fine-tuning, and downstream functional validation.

Applicable Scenarios

Scenario	Description
DNA/RNA sequence generation	Generate subsequent genomic sequence fragments from an input prompt
FASTA sequence prediction	Read FASTA input and output sequence prediction results
Local offline inference validation	Use the example inputs in this repository and a local checkpoint to verify the inference workflow end to end
Mini training workflow validation	Use data/data_mini/genome_data/ to validate the training scripts, data loading, and checkpoint workflow
Cluster training adaptation	Use scripts/train_slurm.py to run training in a Slurm/cluster environment
Data preprocessing	Generate the .bin/.idx data required by the Byte-Level tokenizer from FASTA/JSON data

Usage

1. Using OneCode

You can try intelligent one-click AI4S programming through the OneCode online environment:

Try intelligent one-click AI4S programming

2. Manual Installation and Usage

Hardware Requirements

• Running on a GPU or DCU is recommended.
• A CPU can be used for import checks and small-configuration connectivity validation, but full training and inference will be slow.
• DCU users need to install DTK in advance. DTK 25.04.2 or later is recommended, or the OneScience-recommended version that matches the current cluster.

Download the Model Package

modelscope download --model OneScience/evo2 --local_dir ./evo2
cd evo2

Install the Runtime Environment

DCU Environment

# Activate DTK and CONDA first
conda create -n onescience311 python=3.11 -y
conda activate onescience311
# uv installation is supported
pip install onescience[bio] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai

If the scripts need to access the OneScience source code, set:

export ONESCIENCE_ROOT=/path/to/onescience

When run from this directory, the scripts automatically add the project root and model/ to sys.path. You can also manually verify that the local package can be imported:

export PYTHONPATH=$(pwd)/model:${PYTHONPATH:-}
python -c "import evo2; print('evo2 import ok')"

3. Quick Start

Quick Checks

View the entry-point arguments:

python scripts/infer.py --help
python scripts/predict.py --help
python scripts/train.py --help
python scripts/train_slurm.py --help

4. Data and Weights

The example training data has been placed at:

data/data_mini/genome_data/

It contains:

• chr20.fa, chr21.fa, chr22.fa, and their compressed versions
• Merged FASTA: chr20_21_22.fa
• Preprocessed binary data for training, validation, and testing: preprocessed_data/

The dataset_prefix in config/genome_data_config.yaml is a relative path. During actual runs, it should be used together with:

--dataset-dir data/data_mini/genome_data

By default, the 7B NeMo checkpoint is placed inside the repository:

checkpoints/evo2_nemo_7b/

The inference and prediction entry points read this directory by default; training does not load a checkpoint by default. To fine-tune from existing weights or resume training, explicitly pass --ckpt-dir checkpoints/evo2_nemo_7b in the training command. You can also use EVO2_CKPT_DIR or --ckpt-dir to override it with another checkpoint path:

export EVO2_CKPT_DIR=/path/to/evo2_nemo_7b

The weight files are located under /path/to/evo2/checkpoints/evo2_nemo_7b/weights.

5. Prompt-Based Generation Inference

infer.py writes results to --output-file, but it does not automatically create the parent directory. Create the output directory before the first run:

mkdir -p outputs

Minimal example:

python scripts/infer.py \
  --prompt "ATGCGT" \
  --output-file outputs/evo2_generation.txt

Common parameters:

Parameter	Default	Description
--ckpt-dir	checkpoints/evo2_nemo_7b	NeMo checkpoint directory; can be overridden with EVO2_CKPT_DIR
--prompt	Built-in E. coli classification-label prompt	Input sequence or text prompt
--max-new-tokens	1024	Maximum number of tokens to generate
--temperature	1.0	Sampling temperature
--top-k	0	top-k sampling
--top-p	0.0	top-p sampling
--tensor-parallel-size	1	Tensor parallel size
--output-file	Empty	Write to a file when specified; otherwise print logs

6. FASTA Prediction

The directory includes an example FASTA:

data/predict_example.fa

Run prediction:

python scripts/predict.py \
  --fasta data/predict_example.fa \
  --output-dir outputs/predict

predict.py automatically creates --output-dir. If --fasta is not passed, it reads data/predict_example.fa by default; if --output-dir is not passed, it writes output to outputs/predict by default.

Common parameters:

Parameter	Default	Description
--fasta	data/predict_example.fa	Input FASTA file
--ckpt-dir	checkpoints/evo2_nemo_7b	NeMo checkpoint directory; can be overridden with EVO2_CKPT_DIR
--output-dir	outputs/predict	Prediction result directory
--batch-size	1	Prediction batch size
--model-size	7b	Model specification; available values are determined by NeMo HYENA_MODEL_OPTIONS
--output-log-prob-seqs	Disabled	Output sequence log probability

data/data_mini/genome_data/chr20.fa, chr21.fa, and chr22.fa are relatively large chromosome-level FASTA files. Direct prediction may consume substantial GPU memory. It is recommended to split them into shorter fragments first, or use a multi-GPU parallel configuration.

7. Training

The training entry point requires one of two options: pass -d/--dataset-config, or use --mock-data. For the built-in example data, config/genome_data_config.yaml and data/data_mini/genome_data are recommended.

Example for training the 1B architecture from scratch or validating the training workflow:

python scripts/train.py \
  -d config/genome_data_config.yaml \
  --dataset-dir data/data_mini/genome_data \
  --model-size 1b \
  --result-dir results_1b \
  --devices 8 \
  --num-nodes 1 \
  --seq-length 8192 \
  --micro-batch-size 2 \
  --lr 0.0001 \
  --warmup-steps 5 \
  --max-steps 1000 \
  --clip-grad 1 \
  --wd 0.01 \
  --activation-checkpoint-recompute-num-layers 1 \
  --val-check-interval 50 \
  --limit-val-batches 2

This example does not load a checkpoint and is suitable for validating the training workflow when 1B weights are unavailable. To fine-tune from a 1B NeMo checkpoint, additionally pass --ckpt-dir /path/to/evo2_nemo_1b.

Example for training the 7B long-context architecture from scratch or validating the training workflow:

python scripts/train.py \
  -d config/genome_data_config.yaml \
  --dataset-dir data/data_mini/genome_data \
  --model-size 7b_arc_longcontext \
  --result-dir results_7b \
  --devices 8 \
  --num-nodes 1 \
  --seq-length 1024 \
  --micro-batch-size 1 \
  --lr 0.0001 \
  --warmup-steps 5 \
  --max-steps 1000 \
  --clip-grad 1 \
  --wd 0.01 \
  --activation-checkpoint-recompute-num-layers 1 \
  --val-check-interval 50 \
  --limit-val-batches 2

This example does not load a checkpoint. If you need to fine-tune from an existing 7B NeMo checkpoint or resume training, additionally pass --ckpt-dir checkpoints/evo2_nemo_7b or another checkpoint path.

In a Slurm/cluster environment, replace the entry point with:

python scripts/train_slurm.py ...

Common training parameters:

The table below covers the main parameters used in the training examples above and in scripts/train_evo2_1b.sh and scripts/train_evo2_7b.sh. For more advanced parameters, run python scripts/train.py --help.

Parameter	Default	Description
-d, --dataset-config	Required, or use --mock-data	Training data configuration YAML
--mock-data	Disabled	Do not read real data; use mock data for connectivity testing. Mutually exclusive with -d/--dataset-config
--dataset-dir	EVO2_DATASET_DIR or data/data_mini/genome_data	Data root directory
--ckpt-dir	Empty	Initial checkpoint directory; pass explicitly when fine-tuning from existing weights or resuming training
--model-size	7b	Model specification. Common values include 1b, 7b, and 7b_arc_longcontext; use test for testing
--devices	1	Number of devices on a single node
--num-nodes	1	Number of nodes
--tensor-parallel-size	1	Tensor parallel size
--pipeline-model-parallel-size	1	Pipeline model parallel size
--context-parallel-size	1	Context parallel size
--sequence-parallel	Disabled	Enable sequence parallel, usually used together with tensor parallelism
--seq-length	8192	Training sequence length
--micro-batch-size	1	Micro batch size
--global-batch-size	Automatically inferred	Global batch size
--grad-acc-batches	1	Number of gradient accumulation batches, used to infer the global batch size
--lr	3e-4	Learning rate, explicitly set to 0.0001 in the examples
--min-lr	3e-5	Minimum learning rate of the cosine scheduler
--warmup-steps	2500	Number of warmup steps, explicitly set to 5 in the examples
--constant-steps	80000	Number of steps for which the learning rate stays constant
--max-steps	500000	Number of training steps
--wd	0.01	Optimizer weight decay
--clip-grad	1.0	Gradient clipping threshold
--workers	8	Number of DataLoader workers
--activation-checkpoint-recompute-num-layers	Empty	Override the number of activation recomputation layers; explicitly set to 1 in the examples
--val-check-interval	Empty	Validation and checkpoint check interval; explicitly set to 50 in the examples
--limit-val-batches	20	Maximum number of batches to run per validation; explicitly set to 2 in the examples
--result-dir	./results	Log and result directory
--experiment-name	evo2	Experiment name, which affects the log directory
--disable-checkpointing	Not passed by default; checkpoint callback enabled	Pass this parameter during connectivity tests to disable the checkpoint callback
--no-save-last-checkpoint	Save the last checkpoint by default	Disable saving the last checkpoint; often used together with --disable-checkpointing for quick tests
--save-top-k	5	Number of best checkpoints to save
--ckpt-async-save	Disabled	Enable asynchronous checkpoint saving

8. Data Preprocessing

The current example training data has already been preprocessed. To process new FASTA or JSON data, use:

bash scripts/tools/data_process/preprocess_data_fasta.sh
bash scripts/tools/data_process/preprocess_data_json.sh

Related configurations:

config/genome_preprocess_config.yaml
config/genome_data_config.yaml

Official OneScience Information

Platform	OneScience Main Repository	Skills Repository
Gitee	https://gitee.com/onescience-ai/onescience	https://gitee.com/onescience-ai/oneskills
GitHub	https://github.com/onescience-ai/OneScience	https://github.com/onescience-ai/oneskills

Citations and License

• Original Evo2 paper: Genome modelling and design across all domains of life with Evo 2.
• The original Evo2 code uses the Apache License 2.0. See LICENSE in this repository. Original license source: ArcInstitute/evo2 LICENSE.
• The Evo2 7B model weights are sourced from Hugging Face arcinstitute/evo2_7b. In this repository, checkpoints/evo2_nemo_7b is the converted NeMo checkpoint format. The original weight license is Apache-2.0.
• The example dataset data/data_mini/genome_data/ was downloaded and preprocessed from the hg38 chr20, chr21, and chr22 chromosome FASTA files provided by the UCSC Genome Browser. It is used only to validate the training and inference workflows. See the data download directory: UCSC hg38 chromosomes.
• Use of UCSC hg38 data must comply with the UCSC Genome Browser data usage terms. See: UCSC Terms of Use.
• If you use this repository, the model weights, or generated results in research, it is recommended to cite the original Evo2 paper, the model weight source, the UCSC hg38 data source, and relevant OneScience project information, and to add citations for downstream analysis tools or datasets according to the actual task.