| --- |
| language: |
| - en |
| license: eupl-1.2 |
| tags: |
| - genomics |
| - dna |
| - model-inversion |
| - privacy |
| - pytorch |
| --- |
| |
| # DNA Embedding Inversion |
|
|
| Three final LCQD decoders from [How Private Are DNA Embeddings?](https://arxiv.org/abs/2603.06950), reconstructing DNA from mean-pooled foundation-model embeddings. |
|
|
| **[Code, embedding extraction and results](https://github.com/not-a-feature/DNA-Embedding-Inversion)** |
|
|
| | Model | Input dimensions | Extraction layer | |
| | --- | ---: | --- | |
| | `dnabert2` | 768 | Final hidden layer (12) | |
| | `evo2` | 4096 | `blocks.26.mlp.l3` (zero-based) | |
| | `ntv2` | 1024 | Final hidden layer (29) | |
|
|
| Each decoder covers 14 lengths from 10 to 100 nucleotides. These are the main-study checkpoints, trained on coordinate- and exact-sequence-disjoint hg38 data for 20 epochs, retaining the best validation checkpoint. |
|
|
| ## Quick start |
|
|
| Python 3.12; CPU or CUDA. Download this repository with Git LFS: |
|
|
| ```bash |
| git lfs install |
| git clone https://huggingface.co/not-a-feature/DNA_Embedding_Invesion |
| cd DNA_Embedding_Invesion |
| pip install -r requirements.txt |
| python infer.py --model ntv2 --embeddings embeddings.npy --output reconstructions.json |
| ``` |
|
|
| `embeddings.npy` must contain a finite float array shaped `[N, input_dimensions]` of **raw, unnormalized mean-pooled embeddings**, extracted using the matching model and recipe linked above. The script applies the bundled exact training normalization and returns a JSON list of DNA strings. Add `--device cuda` for GPU inference. |
|
|
| Python API: |
|
|
| ```python |
| import numpy as np |
| from infer import Decoder |
| |
| sequences = Decoder("ntv2").predict(np.load("embeddings.npy", allow_pickle=False)) |
| ``` |
|
|
| No original sequence length is required. Decoding stops at the first predicted EOS; predictions are not cropped to a requested length. Tokenizers are bundled, so inference works offline after installation and download. Foundation-model weights and embedding extraction are separate. |
|
|
| ## Validation and scope |
|
|
| CPU predictions matched the original evaluation code on 252 embeddings: three per length, per model, for hg38 and 1000 Genomes. CLI execution, EOS handling and invalid-input checks also passed. This verifies inference parity, not a new accuracy estimate; see `validation.json`. |
|
|
| Performance depends on sequence length and embedding source. Evaluation assumes reference orientation. These checkpoints are intended for research on genomic embedding privacy. |
|
|
| Weights and decoder code: **EUPL-1.2**. Bundled tokenizers originate from [DNABERT-2](https://huggingface.co/zhihan1996/DNABERT-2-117M) and [NTv2](https://huggingface.co/InstaDeepAI/nucleotide-transformer-v2-500m-multi-species); their upstream terms apply. Configurations, normalization provenance and SHA-256 checksums are included. |
|
|
