|
|
--- |
|
|
license: mit |
|
|
--- |
|
|
|
|
|
**Sparse Autoencoders for *Evo 2*** — BatchTopK sparse autoencoders for Arc Institute's Evo 2 genomic foundation model. |
|
|
|
|
|
Evo 2 is a genomic foundation model capable of generalist prediction and design tasks across DNA, RNA, and proteins. It uses a frontier deep learning architecture to enable modeling of biological sequences at single-nucleotide resolution with near-linear scaling of compute and memory relative to context length. Evo 2 is trained with 40 billion parameters and 1 megabase context length on over 9 trillion nucleotides of diverse eukaryotic and prokaryotic genomes. |
|
|
|
|
|
This repository contains the layer 26 SAE mixed prokaryote/eukaryote SAE used in the Evo 2 paper. |
|
|
|
|
|
[More on Evo 2](https://arcinstitute.org/tools/evo) |
|
|
|
