lowdown-labs/fela-genomics

DISCLAIMER

This model is a research preview. Its training and evaluation data (Genomic Benchmarks, Apache-2.0, and the public GRCh38 reference assembly) is permissively licensed, but the released model itself is under the Lowdown Labs Lovely License 1.0. Lowdown Labs has put together this model in the interest of advancing public science.

FELA-DNA: a small, CPU runnable DNA sequence model

FELA-DNA reads raw DNA, the letters A, C, G, T, and labels it. For a biologist that means you can ask, right on your own laptop or even a Raspberry Pi, things like "is this region a promoter?", "is this stretch coding or not?", or "is this an enhancer?" No GPU, no cloud.

Two trained pieces ship here. One is a small ready to run classifier that calls a window coding or intergenomic. The other is a foundation encoder you fine tune for whatever task you have in mind. Both could run on a CPU system.

What goes in, what comes out

• Input: a DNA sequence as a string of A, C, G, T (and N for an unknown base). The model tokenizes one nucleotide per token (no k-mer vocabulary). A window is up to the variant's seq_len (256 for coding_clf, 1024 for backbone); shape after tokenizing is (batch, seq_len) of integer token ids.
• Output: a class label for the window. The class set depends on the task head you load, for example promoter vs not promoter, coding vs intergenomic, enhancer vs not enhancer, or human vs worm. The model returns one logit per class; take the argmax for the label, or the softmax for a probability.
• This enables on device, no cloud screening of regulatory and coding regions, and a per position score track over a long region (slide the window along a chromosome and keep the per window probability) that loads into a genome browser.

What ships (two variants, both run on CPU)

The repo ships two safetensors files, picked out by the variants map in config.json.

• coding_clf.safetensors (the default), 3.12M parameters. This is the ready to run classifier. It labels a window coding or intergenomic and reaches 0.918 test accuracy on the GenomicBenchmarks demo task. Load it and score a sequence, nothing to train.
• backbone.safetensors, 14.64M parameters. This is the foundation encoder. It has no task head of its own, it just turns DNA into embeddings, and you fine tune it on top to reproduce the benchmark table below. It was pretrained on the human reference genome by learning to fill in hidden bases. Under the hood it uses a Gated DeltaNet recall layer, which runs on CPU in plain PyTorch (matched exactly against the GPU reference), so the GPU kernel is a training time thing only.

Neither variant needs a GPU to run. A GPU only speeds up training for our practical purposes.

Why we built it this way

The sequence mixer is a Fourier Neural Operator, which is a learned global convolution done in the frequency domain. It has no all pairs attention matrix, so the memory needed to score a region does not grow with the square of the region length. In plain terms: a standard attention transformer scoring a long stretch of DNA has to build an N by N table that blows up the longer the input gets, while this model keeps a small, fixed working memory and streams the sequence through.

That is what lets it score a whole chromosome arm on a CPU box and run on hardware as small as a Raspberry Pi. We train with reverse complement augmentation and average predictions over a sequence and its reverse complement at test time, so the model is approximately strand symmetric, which is the right inductive bias for DNA.

Performance

Speed and footprint, measured on CPU (AMD EPYC 9555, batch size 1, median of 20 runs). Input is one 512 nucleotide window, shape (1, 512).

Pure FNO (fastest on CPU, the default on device model)

Format	Size on disk	Peak working RAM	Latency 1 core	Latency 4 core	Device class
fp32	51.96 MB	0.03 MB	177.301 ms	77.006 ms	Raspberry Pi Zero 2 W / Pi 4 class

Throughput is 5.64 windows per second on one core. int8 disk size is 14.86 MB; per core int8 latency was not separately measured.

GDN variant (more accurate on some tasks, also CPU, slower)

Format	Size on disk	Peak working RAM	Latency 1 core	Latency 4 core	Device class
fp32	58.57 MB	0.03 MB	185.0 ms	122.0 ms	Raspberry Pi class (CPU via pure torch recurrence)

The GDN variant is 14.64M parameters and runs on CPU at 5.41 windows per second on one core. Its CPU inference uses the pure torch gated delta rule recurrence and matches the fla reference to 3.0e-08. int8 disk size is 16.13 MB.

Long inputs (both variants): constant working memory

For long inputs the working RAM stays flat. With the pure FNO model, scoring a whole 200,000 bp region of GRCh38 chr21 took 5.0 s (about 40,000 bp/s) using 18 MB of RAM in a single streaming pass.

A standard softmax transformer attending over the same 200,000 bp would need an N by N attention matrix of about 1,280 GB, which does not fit in commodity RAM, so the memory advantage at that length is roughly 70,000x.

Accuracy

We evaluate upon top 1 test accuracy on Genomic Benchmarks (Gresova et al. 2023), single nucleotide input, with GRCh38 masked nucleotide pretraining, reverse complement augmentation, and test time reverse complement averaging. We hold a validation split out of the training set for model selection and never tune on the test set. HyenaDNA numbers are the published Genomic Benchmarks results (Nguyen et al. 2023). All values are measured on the official test folds (ours).

Benchmark	Metric	FELA-DNA GDN	HyenaDNA (published)
human nontata promoters	top-1 acc	96.8	96.6
demo coding vs intergenomic seqs	top-1 acc	93.9	91.3
demo human or worm	top-1 acc	96.8	96.6
human enhancers cohn	top-1 acc	73.6	74.2
human enhancers ensembl	top-1 acc	92.2	89.2
human ocr ensembl	top-1 acc	80.4	not reported

The GDN backbone beats HyenaDNA on four tasks (nontata promoters 96.8 vs 96.6, coding vs intergenomic 93.9 vs 91.3, human or worm 96.8 vs 96.6, and enhancers ensembl 92.2 vs 89.2; nontata and worm by thin ~0.2 point margins, coding and ensembl clearly), and is within about one point on the others.

Both backbones run on CPU, so again the choice is speed vs absolute accuracy. The honest limit is human enhancers cohn, where both of our backbones trail HyenaDNA (73.6 and 71.5 vs 74.2). With higher training and research budgets, we could probably close that gap.

How to run it

See quickstart/ for a minimal loader and the Hugging Face Space in space/ for an interactive playground. Both load through modeling.py:

cd quickstart && python run.py                     # coding_clf classifier (default)
cd quickstart && python run.py --variant backbone  # pretrained GDN encoder

For a production serving path, the variants run under the FELA server, a separate CPU native serving project (https://github.com/Lowdown-Labs/fela_server).

Loading with standard tooling

The repo ships config.json (a variants map with the architecture hyperparameters and metadata for each variant) and a self contained modeling.py with a load_model / from_pretrained entry. The weights ship as safetensors (not pickle): coding_clf.safetensors (the default classifier) and backbone.safetensors (the GDN foundation encoder). A few lines load a variant from a local directory or a Hugging Face repo:

from modeling import load_model, score_sequence
m = load_model("/path/to/weights_dir")                    # default variant: coding_clf
label, prob = score_sequence(m, "ACGTACGT...")

enc = load_model("/path/to/weights_dir", variant="backbone")  # the GDN foundation encoder
emb = enc.embed(ids)                                      # masked mean pooled 384 dimensional embedding

Formats

• fp32: reference and CPU.
• int8: on device deployment format (AVX512-VNNI on x86, NEON dot product on ARM). int8 disk size is 14.86 MB (pure FNO) and 16.13 MB (GDN).
• bf16: server inference; most commodity ARM CPUs lack native bf16, so it is not the on device format.

Training data

• Pretraining: GRCh38 (the Genome Reference Consortium human reference genome, build 38), masked nucleotide self supervised pretraining on 10 chromosomes, 1024 bp windows, with reverse complement augmentation. GRCh38 is a public reference assembly.
• Fine tuning and evaluation: Genomic Benchmarks (Gresova et al. 2023), six tasks (human nontata promoters, demo coding vs intergenomic seqs, demo human or worm, human enhancers cohn, human enhancers ensembl, human ocr ensembl). Genomic Benchmarks is released under the Apache 2.0 license.
• Reference baseline for comparison: HyenaDNA (Nguyen et al. 2023), published Genomic Benchmarks results. We did not retrain HyenaDNA.
• The GDN variant uses the GatedDeltaNet layer from the flash-linear-attention (fla) library; its Triton kernel is used for training, and CPU inference uses a pure torch recurrence that matches the fla reference.

Training data, splits and licensing

Fine tuning and evaluation dataset:

• Genomic Benchmarks (Gresova, Martinek, Cechak, Simecek, Alexiou. "Genomic Benchmarks: a collection of datasets for genomic sequence classification." BMC Genomic Data, 2023). Source: https://github.com/ML-Bioinfo-CEITEC/genomic_benchmarks. License: Apache-2.0 (verified in the repository LICENSE file). Commercial use verdict: permitted (Apache-2.0 is a permissive commercial friendly license, attribution required).
• Six tasks used: human_nontata_promoters, demo_coding_vs_intergenomic_seqs, demo_human_or_worm, human_enhancers_cohn, human_enhancers_ensembl, human_ocr_ensembl.

Pretraining data:

• GRCh38 (Genome Reference Consortium Human Build 38), a public reference assembly, no usage restriction for research use. Masked nucleotide self supervised pretraining on 10 chromosomes, 1024 bp windows, reverse complement augmentation.

Split method and sizes:

• Test set is the OFFICIAL Genomic Benchmarks test fold per task (never tuned on). No seeded resplit of the test data. Official test sizes: human_nontata_promoters 9034, demo_coding_vs_intergenomic_seqs 25000, demo_human_or_worm 25000, human_enhancers_cohn 6948, human_enhancers_ensembl 30970, human_ocr_ensembl 34952.
• Train/validation split: the official train fold is shuffled with --seed 0 (default) and the first --val_frac 0.1 (10%) is held out as validation for model selection; the remaining 90% is training. Example (human_nontata_promoters): train 24388, val 2709.
• Running python train.py --dataset <task> --smoke builds the split, asserts the official test size, and exits before training.

Intended use, limitations, and safety

What it is for: research use. Zero infrastructure scoring of regulatory elements, promoters, and coding regions on a laptop, a CPU box, or a Raspberry Pi, including long regions that do not fit a transformer's attention matrix. A per position score track for a genome browser, and simple variant effect scoring (score a reference window and an alternate allele window, take the difference in the task probability).

What it is not for: this is not a clinical or diagnostic tool. Predictions are statistical. Do not use FELA-DNA for any clinical, diagnostic, or safety critical decision without independent validation against established methods and appropriate review.

Privacy: the model runs on the device. DNA sequences do not have to leave your machine, which matters for sensitive genomic data.

Evaluated conditions and known failure modes:

• On human enhancers cohn, both backbones trail HyenaDNA. Short noisy enhancer windows are the weakest case for the global Fourier mixer.
• Pretraining used 10 GRCh38 chromosomes, not the whole genome, so coverage of rare context regions is partial.
• Evaluated on single nucleotide windows up to 512 bp for classification and on chr21 for the whole region streaming demo. Other species and very different sequence statistics were not evaluated.
• Latency and size were measured on an x86 server CPU; the Raspberry Pi claim is the size plus compute envelope, realized via the ONNX export.

How to cite

@misc{lowdownlabs_feladna,
  title  = {FELA-DNA: a small CPU runnable Fourier Neural Operator model for DNA classification},
  author = {Lowdown Labs},
  year   = {2026},
  note   = {Model card}
}

You must also cite the datasets, the reference benchmark, and the libraries used:

• GRCh38 reference genome (Genome Reference Consortium).
• Gresova, K., Martinek, V., Cechak, D., Simecek, P., Alexiou, P. (2023). Genomic Benchmarks: a collection of datasets for genomic sequence classification. BMC Genomic Data.
• Nguyen, E., Poli, M., Faizi, M., et al. (2023). HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution. NeurIPS.
• Li, Z., Kovachki, N., Azizzadenesheli, K., et al. (2021). Fourier Neural Operator for Parametric Partial Differential Equations. ICLR.
• flash-linear-attention (fla) library, for the GatedDeltaNet recall layer used in the GDN variant.
• PyTorch (Paszke et al. 2019).

Acknowledgements and references

• GRCh38: Genome Reference Consortium Human Build 38.
• Genomic Benchmarks: Gresova et al. (2023), BMC Genomic Data.
• HyenaDNA: Nguyen et al. (2023), NeurIPS.
• Fourier Neural Operator: Li et al. (2021), ICLR.
• flash-linear-attention (fla): GatedDeltaNet linear attention kernels.
• PyTorch: Paszke et al. (2019), NeurIPS.

Citations & licenses

This section gives the formal reference for every dataset and method the model actually uses, each with a direct hyperlink to the real license text (verified from source). The released model itself is under the Lowdown Labs Lovely License 1.0 (see License below), independent of the permissive training data licenses recorded here.

Datasets

Genomic Benchmarks (fine tuning + evaluation). Gresova, K., Martinek, V., Cechak, D., Simecek, P., Alexiou, P. (2023). "Genomic Benchmarks: a collection of datasets for genomic sequence classification." BMC Genomic Data 24, 25. DOI: https://doi.org/10.1186/s12863-023-01123-8. Code/data repository: https://github.com/ML-Bioinfo-CEITEC/genomic_benchmarks. License: Apache-2.0, verified from the repository LICENSE file (https://github.com/ML-Bioinfo-CEITEC/genomic_benchmarks/blob/main/LICENSE, "Apache License, Version 2.0"). Commercial use: permitted with attribution.

@article{gresova2023genomic,
  title   = {Genomic Benchmarks: a collection of datasets for genomic sequence classification},
  author  = {Gresova, Katarina and Martinek, Vlastimil and Cechak, David and Simecek, Petr and Alexiou, Panagiotis},
  journal = {BMC Genomic Data},
  volume  = {24},
  number  = {1},
  pages   = {25},
  year    = {2023},
  doi     = {10.1186/s12863-023-01123-8}
}

GRCh38 human reference assembly (self supervised pretraining). Genome Reference Consortium Human Build 38 (GRCh38). Assembly page: https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_000001405.26/. GRCh38 is a public reference assembly; the Genome Reference Consortium places no usage restriction on the reference for research use (https://www.ncbi.nlm.nih.gov/grc). Attribution to the Genome Reference Consortium is expected.

HyenaDNA (published comparison baseline only, not training data). Nguyen, E., Poli, M., Faizi, M., et al. (2023). "HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution." NeurIPS. arXiv:2306.15794 (https://arxiv.org/abs/2306.15794). We did not retrain HyenaDNA; the numbers quoted are the authors' published Genomic Benchmarks results.

Methods and code

Confirmed against modeling.py and config.json in this repo: the model is a Fourier Neural Operator sequence mixer, and the GDN variant adds a Gated DeltaNet recall layer (layer_pattern="SSSL", use_gdn=true) via the fla library at training time.

• PyTorch (the training and inference framework). Paszke, A., et al. (2019). "PyTorch: An Imperative Style, High-Performance Deep Learning Library." NeurIPS 32. https://papers.nips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.
• Fourier Neural Operator (the sequence mixer). Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A. (2021). "Fourier Neural Operator for Parametric Partial Differential Equations." ICLR. arXiv:2010.08895 (https://arxiv.org/abs/2010.08895).
• Gated DeltaNet (the recall layer in the GDN variant). Yang, S., Kautz, J., Hatamizadeh, A. (2024/2025). "Gated Delta Networks: Improving Mamba2 with Delta Rule." ICLR 2025. arXiv:2412.06464 (https://arxiv.org/abs/2412.06464). Builds on Gated Linear Attention: Yang, S., Wang, B., Shen, Y., Panda, R., Kim, Y. (2023). "Gated Linear Attention Transformers with Hardware-Efficient Training." arXiv:2312.06635 (https://arxiv.org/abs/2312.06635).
• flash-linear-attention (fla), the GatedDeltaNet Triton kernel used for training the GDN variant (CPU inference uses the pure torch recurrence in modeling.py instead). Repository: https://github.com/fla-org/flash-linear-attention (MIT license).

Model family

This is part of the FELA family from Lowdown Labs: one FNO architecture across many modalities, all CPU native and subquadratic. This repo is pushed as lowdown-labs/fela-genomics. The sibling repos (final Hugging Face names) are:

• lowdown-labs/fela-genomics (this repo): DNA sequence classification.
• lowdown-labs/fela-pdm: rotating machinery and turbofan health.
• lowdown-labs/fela-power-grid: probabilistic solar and wind power forecasting.
• lowdown-labs/fela-video: video moment retrieval and temporal grounding.
• lowdown-labs/fela-streaming-asr: streaming CPU speech recognition.

These are grouped under the FELA Collection on Hugging Face. The models are independently trained per modality and do not share weights, so none carries a base_model link.

License

Released under the Lowdown Labs Lovely License 1.0 (CC BY-NC 4.0 plus Hippocratic License 3.0). See LICENSE. For most LL models, a commercial license may be available; contact Lowdown Labs.