README.md

metadata

title: ColiFormer UI
emoji: 🧬
colorFrom: blue
colorTo: green
sdk: docker
pinned: false

ENCOT: A Transformer-Based Codon Optimization Model Balancing Multiple Objectives for Enhanced E. coli Gene Expression

Abstract

ENCOT is a transformer-based model for codon optimization of protein sequences in Escherichia coli. Built on top of CodonTransformer (a multi-species BigBird model trained on over 1 million DNA–protein pairs), ENCOT is fine-tuned specifically for E. coli codon preferences using 3,676 high-expression E. coli genes curated from NCBI.

ENCOT balances multiple objectives (CAI, GC content, tAI, RNA stability, and minimization of negative cis-regulatory elements) and uses an Augmented-Lagrangian Method (ALM) to enforce GC content control during training. Performance was evaluated on 37,053 native E. coli genes and 80 recombinant protein targets, demonstrating strong improvements in in silico expression metrics while maintaining biologically appropriate constraints.

Paper Reference

ENCOT: A Transformer-Based Codon Optimization Model Balancing Multiple Objectives for Enhanced E. coli Gene Expression

Saketh Baddam, Omar Emam, Abdelrahman Elfikky, Francesco Cavarretta, George Luka, Ibrahim Farag, Yasser Sanad

bioRxiv preprint (not peer-reviewed): https://doi.org/10.1101/2025.11.26.690826

What does “preprint and not peer-reviewed” mean? A preprint is a publicly available manuscript shared before formal journal peer review. It can be cited, but its claims have not yet been evaluated by journal referees.

Citation

If you use ENCOT in your research, please cite:

@article{encot2025,
  title{ENCOT: A Transformer-Based Codon Optimization Model Balancing Multiple Objectives for Enhanced E. coli Gene Expression},
  author={Baddam, Saketh and Emam, Omar and Elfikky, Abdelrahman and Cavarretta, Francesco and Luka, George and Farag, Ibrahim and Sanad, Yasser},
  journal={bioRxiv},
  year={2025},
  doi={10.1101/2025.11.26.690826},
  url={https://doi.org/10.1101/2025.11.26.690826},
  note={Preprint (not peer-reviewed)}
}

Quick Start

Optimize a protein sequence in just a few lines:

import torch
from transformers import AutoTokenizer
from CodonTransformer.CodonPrediction import load_model, predict_dna_sequence
from huggingface_hub import hf_hub_download

# Load model from Hugging Face
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
checkpoint_path = hf_hub_download(
    repo_id="saketh11/ColiFormer",
    filename="balanced_alm_finetune.ckpt",
    cache_dir="./hf_cache"
)
model = load_model(model_path=checkpoint_path, device=device)
tokenizer = AutoTokenizer.from_pretrained("adibvafa/CodonTransformer")

# Optimize a protein sequence
protein = "MALWMRLLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGERGFFYTPKTRREAEDLQVGQVELGG"
output = predict_dna_sequence(
    protein=protein,
    organism="Escherichia coli general",
    device=device,
    model=model,
    tokenizer=tokenizer,
    deterministic=True,
    match_protein=True
)

print(f"Optimized DNA: {output.predicted_dna}")

Or use the command-line interface:

python scripts/optimize_sequence.py \
    --input "MALWMRLLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGERGFFYTPKTRREAEDLQVGQVELGG" \
    --output optimized.fasta

Installation

Requirements

• Python >= 3.9
• CUDA-capable GPU (recommended for training, optional for inference)

Setup

1. Clone the repository:

git clone https://github.com/geno543/ENCOT.git
cd ENCOT

2. Create a virtual environment:

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt

The installation takes approximately 10-30 seconds depending on your system and existing packages.

Public Streamlit Demo (Anyone Can Try It)

If you want a public link so anyone can test ENCOT in a browser, deploy the app with either Streamlit Community Cloud or Hugging Face Spaces.

Option A: Streamlit Community Cloud (Fastest)

1. Push this repository to GitHub.
2. Go to https://share.streamlit.io and sign in.
3. Click New app and choose your repository.
4. Set Main file path to streamlit_app.py.
5. Use the repository requirements.txt for dependencies.
6. Deploy and share the generated public URL.

Option B: Hugging Face Spaces (Streamlit)

1. Create a new Space (SDK: Streamlit).
2. Upload this project (or connect the GitHub repo).
3. Ensure app file is streamlit_app.py.
4. Keep the repo public so anyone can access the Space URL.

Local check before deployment

streamlit run streamlit_app.py --server.port 8501

This uses the existing UI in streamlit_gui/app.py, including model loading from Hugging Face and optimization controls.

Data Preparation

Preparing E. coli Training Data

To prepare training data from raw E. coli gene sequences:

1. Place your data files in the data/ directory:

   • data/CAI.csv - CSV file with columns: gene_id, cai_score, dna_sequence
   • data/Database 3_4300 gene.csv - CSV file with high-CAI sequences (column: dna_sequence)

2. Run the preprocessing script:

python scripts/preprocess_data.py

This will:

• Validate and process DNA sequences
• Create data/ecoli_processed_genes.csv with validated sequences
• Generate data/finetune_set.json for training (high-CAI sequences)
• Generate data/test_set.json for evaluation (100 random sequences)

Custom paths:

python scripts/preprocess_data.py \
    --cai_csv data/my_cai_data.csv \
    --high_cai_csv data/my_high_cai_data.csv \
    --output_dir my_data \
    --test_size 200

Dataset Structure

The processed dataset includes:

• Training set: 4,300 high-CAI E. coli sequences (from Database 3_4300 gene.csv)
• Test set: 100 randomly sampled sequences (for evaluation)
• Reference sequences: 50,000+ E. coli genes for CAI/tAI calculation

The complete dataset is available at saketh11/ColiFormer-Data on Hugging Face.

Training

Quick Start Training

Train ENCOT with the default ALM configuration:

python scripts/train.py --config configs/train_ecoli_alm.yaml

Configuration Files

We provide three configuration files:

1. configs/train_ecoli_alm.yaml - Main training configuration with ALM GC control

   • 15 epochs, batch size 6, 4 GPUs
   • ALM enabled with GC target 52%
   • Curriculum learning: 3 warm-up epochs

2. configs/train_ecoli_quick.yaml - Quick sanity check

   • 1 epoch, batch size 2, CPU-only
   • Useful for testing your setup

3. configs/benchmark.yaml - Benchmark evaluation settings

Training Parameters

Key parameters in the config files:

• training.batch_size: Batch size (default: 6)
• training.max_epochs: Number of training epochs (default: 15)
• training.learning_rate: Learning rate (default: 5e-5)
• training.num_gpus: Number of GPUs (default: 4)
• alm.enabled: Enable ALM GC control (default: true)
• alm.gc_target: Target GC content (default: 0.52 for E. coli)
• alm.curriculum_epochs: Warm-up epochs before enforcing GC constraint (default: 3)

Override Config Values

You can override config values from the command line:

python scripts/train.py \
    --config configs/train_ecoli_alm.yaml \
    --num_gpus 2 \
    --batch_size 4 \
    --max_epochs 10

Training Output

Checkpoints are saved to the directory specified in checkpoint.checkpoint_dir:

• Model state dict: balanced_alm_finetune.ckpt
• Training logs: TensorBoard logs in the checkpoint directory

Monitor training progress:

tensorboard --logdir models/alm-enhanced-training

Inference / Sequence Optimization

Single Sequence Optimization

Optimize a single protein sequence:

python scripts/optimize_sequence.py \
    --input "MALWMRLLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGERGFFYTPKTRREAEDLQVGQVELGG" \
    --output optimized.fasta

Batch Processing

Process multiple sequences from a FASTA file:

python scripts/optimize_sequence.py \
    --input sequences.fasta \
    --output optimized.fasta \
    --batch

GC Content Constraints

Specify GC content bounds:

python scripts/optimize_sequence.py \
    --input protein.fasta \
    --output optimized.fasta \
    --gc-min 0.45 \
    --gc-max 0.55

Using Custom Checkpoint

python scripts/optimize_sequence.py \
    --input protein.fasta \
    --output optimized.fasta \
    --checkpoint models/my_model.ckpt

Python API

For programmatic use:

from CodonTransformer.CodonPrediction import load_model, predict_dna_sequence
from transformers import AutoTokenizer
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = load_model(model_path="models/alm-enhanced-training/balanced_alm_finetune.ckpt", device=device)
tokenizer = AutoTokenizer.from_pretrained("adibvafa/CodonTransformer")

output = predict_dna_sequence(
    protein="MALWMRLLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGERGFFYTPKTRREAEDLQVGQVELGG",
    organism="Escherichia coli general",
    device=device,
    model=model,
    tokenizer=tokenizer,
    deterministic=True,
    match_protein=True,
    use_constrained_search=True,
    gc_bounds=(0.45, 0.55),
    beam_size=20
)

print(f"Optimized DNA: {output.predicted_dna}")

Reproducing Paper Results

Benchmark Evaluation

To reproduce the benchmark results from the paper:

1. Prepare benchmark sequences:

   Place your benchmark sequences in an Excel file (see Benchmark 80 sequences.xlsx for format).

2. Run benchmark evaluation:

python scripts/run_benchmarks.py --config configs/benchmark.yaml

This will:

• Load the fine-tuned ENCOT model
• Optimize all sequences in the benchmark file
• Calculate metrics (CAI, tAI, GC content, CFD, negative cis-elements)
• Generate comparison plots and summary statistics
• Save results to benchmark_results/run_TIMESTAMP/

Expected Results

On the benchmark set of 80 sequences:

• CAI improvement: +6.2% vs base CodonTransformer
• tAI improvement: +8.6% vs base CodonTransformer
• GC content: Mean 52.1% (target: 52%)
• Runtime: ~1-3 seconds per sequence (GPU)

Custom Benchmark

python scripts/run_benchmarks.py \
    --excel_path my_benchmark.xlsx \
    --checkpoint_path models/my_model.ckpt \
    --output_dir my_results \
    --use_gpu

Model Architecture

Base Model

ENCOT is built on CodonTransformer, a BigBird transformer model:

• Architecture: BigBirdForMaskedLM (89.6M parameters)
• Pre-training: 1M+ DNA-protein pairs from 164 organisms
• Context length: 2048 tokens
• Attention: Block-sparse attention for efficiency

Fine-tuning

ENCOT is fine-tuned on E. coli-specific data:

• Training data: 4,300 high-CAI E. coli sequences
• Loss function: Masked Language Modeling (MLM) + GC constraint
• Optimizer: AdamW with CosineAnnealingWarmRestarts scheduler
• Learning rate: 5e-5 with 10% warmup

Augmented-Lagrangian Method (ALM)

The ALM approach enforces GC content constraints during training:

Objective function:

L = L_MLM + λ·(GC - μ) + (ρ/2)(GC - μ)²

Where:

• L_MLM: Masked language modeling loss
• λ: Lagrangian multiplier (updated adaptively)
• ρ: Penalty coefficient (self-tuning)
• GC: Mean GC content (sliding window of 50 codons)
• μ: Target GC content (0.52 for E. coli)

Key features:

• Curriculum learning: 3 warm-up epochs before enforcing GC constraint
• Adaptive penalty: Penalty coefficient increases if constraint violation doesn't improve
• Self-tuning: Lagrangian multiplier and penalty updated every 20 steps

This approach allows the model to learn codon preferences while maintaining precise GC content control, critical for synthesis and expression in E. coli.

Evaluation Metrics

ENCOT computes comprehensive metrics for optimized sequences:

• CAI (Codon Adaptation Index): Measures similarity to highly expressed genes (0-1, higher is better)
• tAI (tRNA Adaptation Index): Measures tRNA availability (0-1, higher is better)
• GC Content: Percentage of G+C nucleotides (target: 52% for E. coli)
• CFD (Codon Frequency Distribution): Similarity to reference codon frequencies
• Negative cis-elements: Count of problematic sequence motifs
• Homopolymer runs: Long repeats that can cause synthesis issues

Project Structure

encot/
├── configs/                    # YAML configuration files
│   ├── train_ecoli_alm.yaml   # Main training config
│   ├── train_ecoli_quick.yaml # Quick test config
│   └── benchmark.yaml         # Benchmark config
├── scripts/                    # Entry-point scripts
│   ├── preprocess_data.py     # Data preparation
│   ├── train.py               # Training wrapper
│   ├── optimize_sequence.py   # Sequence optimization
│   └── run_benchmarks.py      # Benchmark evaluation
├── CodonTransformer/          # Core module (custom, not PyPI)
│   ├── CodonPrediction.py     # Model loading & inference
│   ├── CodonEvaluation.py     # Metrics calculation
│   ├── CodonData.py           # Data preprocessing
│   └── ...
├── data/                       # Datasets
│   ├── finetune_set.json      # Training data
│   ├── test_set.json          # Test data
│   └── ecoli_processed_genes.csv  # Reference sequences
├── models/                     # Model checkpoints
├── notebooks/                  # Jupyter notebooks
├── tests/                      # Test suite
├── streamlit_gui/             # Streamlit web interface
├── finetune.py                # Training script (original)
├── benchmark_evaluation.py    # Evaluation script (original)
└── README.md                  # This file

Troubleshooting

Common Issues

1. CUDA out of memory:

• Reduce batch_size in config file
• Use gradient accumulation: increase accumulate_grad_batches

2. Model checkpoint not found:

• The script will auto-download from Hugging Face if local checkpoint missing
• Ensure you have internet connection for first run

3. Data preprocessing errors:

• Verify CSV files have correct column names
• Check that DNA sequences are valid (divisible by 3, proper start/stop codons)

4. Import errors:

• Ensure you've activated the virtual environment
• Run pip install -r requirements.txt again

Getting Help

• Issues: Open an issue on GitHub
• Questions: Check the documentation or contact the authors

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Acknowledgments

• CodonTransformer: Base model from adibvafa/CodonTransformer
• Hugging Face: Model hosting and distribution
• E. coli data: NCBI and Kazusa codon usage databases

Citation

If you use ENCOT in your research, please cite:

@article{encot2025,
  title={ENCOT: A Transformer-Based Codon Optimization Model Balancing Multiple Objectives for Enhanced E. coli Gene Expression},
  author={Baddam, Saketh and Emam, Omar and Elfikky, Abdelrahman and Cavarretta, Francesco and Luka, George and Farag, Ibrahim and Sanad, Yasser},
  journal={bioRxiv},
  year={2025},
  doi={10.1101/2025.11.26.690826},
  url={https://doi.org/10.1101/2025.11.26.690826},
  note={Preprint (not peer-reviewed)}
}

---

ENCOT - State-of-the-art codon optimization for E. coli expression systems.