Initial deploy: MAGI variant interpreter (gradio_app)

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+data/MANE_processed.csv filter=lfs diff=lfs merge=lfs -text
+data/Promoter_processed.csv filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,92 @@
+# Large data files
+hg38.fa
+hg38.fa.gz
+hg38.fa.fai
+*.fasta
+*.fa.gz
+*.fai
+
+# Model cache
+models/
+checkpoints/
+*.pt
+*.pth
+*.safetensors
+
+# Python cache
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+venv/
+env/
+ENV/
+env.bak/
+venv.bak/
+.venv/
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+.DS_Store
+
+# Jupyter
+.ipynb_checkpoints/
+*.ipynb
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+.tox/
+
+# Logs
+*.log
+logs/
+flagged/  # Gradio flagged data
+
+# Temporary files
+tmp/
+temp/
+*.tmp
+
+# Output files (results from app)
+outputs/
+results/
+*.csv.bak
+*.csv~
+last_variant_result.csv
+batch_results.csv
+last_variant_tracks.png
+*.png
+
+# System files
+.DS_Store
+Thumbs.db
+desktop.ini
+
+# HuggingFace Spaces
+.gradio/
+./data/hg38.fa
+./data/MANE_processed.csv

README.md

@@ -1,13 +1,190 @@
 ---
-title: MAGI
-emoji: 🐨
-colorFrom: indigo
-colorTo: red
+title: MAGI Variant Interpreter
+emoji: 🧬
+colorFrom: purple
+colorTo: blue
 sdk: gradio
-sdk_version: 6.15.2
-python_version: '3.13'
+sdk_version: 5.0.0
 app_file: app.py
 pinned: false
+license: mit
+hardware: zero-a10
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+## 🧬 MAGI Variant Interpreter
+
+### Variant impact scoring using MAGI and the NTv3 foundation model
+
+MAGI is a lightweight demo for reviewing variant-associated signals in a local genomic window. It uses the external NTv3 model for sequence predictions and adds MAGI scoring, annotation, ranking, and short rule-based summaries.
+
+MAGI was developed by Dan Ofer, Stav Zok, and Michal Linial. The NTv3 model was developed by InstaDeepAI and collaborators.
+
+## Features
+
+- **Single Variant Analysis**: Manual input of SNPs and indels with a compact summary
+- **Batch Processing**: Upload CSV files with up to 10 variants
+- **Multi-species support**: Human plus supported animals and plants via Ensembl sequence retrieval
+- **Gene Annotation**: Automatic annotation using MANE Select RefSeq transcripts
+- **Ranked Signal Review**:
+  - BED outputs from the NTv3 configuration
+  - Filtered BigWig context tracks
+  - Sequence-model metrics such as LLR, KL divergence, and embedding distances
+- **Region Track View**: Zoomable probability tracks for the top disrupted BED and BigWig outputs
+- **Rule-Based Signal Interpretation**: Short deterministic summary of the strongest ranked signals
+- **Impact Scoring**: Quantitative metrics for variant prioritization, including MAGI `Global_z_sum_log`
+
+## Usage
+
+### Single Variant
+
+1. Select species and chromosome.
+2. Enter a 1-based genomic position.
+  Human uses GRCh38/hg38 coordinates. Non-human species use the selected species' current Ensembl assembly, and chromosome names can be bare (`1`, `X`, `MT`) or `chr`-prefixed.
+3. Enter reference and alternate alleles (for example `C` → `T` for a SNP, `ATCT` → `A` for a deletion).
+4. Click **Predict Impact**.
+
+**Example:**
+
+- Chromosome: `chr17`  
+- Position: `7675088`  
+- Ref: `C`  
+- Alt: `T`  
+- (TP53 pathogenic missense variant)
+
+### Batch Upload
+
+Upload a CSV file with these columns:
+
+```csv
+chrom,pos,ref,alt
+chr17,7675088,C,T
+chr7,117559593,ATCT,A
+chr13,32332771,AGAGA,AGA
+```
+
+**Limit:** 10 variants per batch
+
+## Output Interpretation
+
+### Impact Scores
+
+- **MAGI `Global_z_sum_log`**: Burden score computed as `Σ log(1 + |z_j|)` across z-scored BED and BigWig delta tracks using bundled baseline statistics
+
+  - Higher values indicate broader or stronger deviation from the baseline set
+  - This is a ranking score, not a calibrated pathogenicity probability
+
+- **Impact_Score_BED**: Mean of top-3 largest absolute BED element deltas
+
+  - This score is also used for the simple high/moderate/low summary tier shown in the single-variant card
+  - Tier thresholds: `HIGH > 0.10`, `MODERATE > 0.05`, otherwise `LOW`
+
+- **LLR (Log-Likelihood Ratio)**: For SNPs only, log(P(ALT)/P(REF))
+
+  - Positive: Alternate allele more likely
+  - Negative: Reference allele more likely
+
+- **KL Divergence**: Measures how much the predicted token distribution changes around the variant
+
+  - Higher values indicate a larger local sequence-model shift
+
+### Rule-Based Signal Interpretation
+
+The single-variant view includes a short interpretation block that:
+
+- summarizes the most disrupted BED and BigWig signals in plain language
+- highlights whether the strongest evidence is coding-related, splice-related, promoter-related, or context-dependent
+- adds sequence-model context from LLR and KL divergence
+
+This panel is deterministic and uses the same ranked signals already displayed in the table and plots. It is a heuristic summary, not a calibrated pathogenicity assessment.
+
+The summary card also surfaces the top raw BED, BigWig, and MLM signals first, using a minimum absolute magnitude threshold of 0.03.
+
+**MAGI baseline note:** the app bundles baseline statistics under `data/magi_baseline_stats.csv`, so `Global_z_sum_log` is computed locally.
+
+### Functional Annotations
+
+**Region classes:**
+
+- `CODING`
+- `CODING, SPLICE`
+- `SPLICE`
+- `UTR_5` / `UTR_3`
+- `PROMOTER`
+- `INTRONIC`
+- `GENIC_OTHER`
+- `OTHER`
+
+**BigWig tracks:**
+
+- Histone modifications: H3K4me3, H3K27ac, H3K36me3, H3K27me3, etc.
+- Chromatin accessibility: ATAC-seq, DNase-seq
+- Gene expression or transcription-linked assays such as CAGE
+
+**Direction:**
+
+- **Gain of Function (GOF)**: Δ > 0 → predicted signal increased
+- **Loss of Function (LOF)**: Δ < 0 → predicted signal decreased
+
+### Region Classification
+
+Variants are automatically classified into:
+
+- `CODING`: Overlaps coding sequence
+- `CODING, SPLICE`: Coding region near splice junction
+- `SPLICE`: Intronic splice site (±2 bp from exon boundary)
+- `UTR_5` / `UTR_3`: 5' or 3' untranslated region
+- `PROMOTER`: Within 2 kb upstream of TSS
+- `INTRONIC`: Intronic (not splice site)
+- `GENIC_OTHER`: Within gene boundaries (other)
+- `OTHER`: Intergenic
+
+## Data Sources
+
+- **Model:** InstaDeepAI/NTv3_650M_post (HuggingFace)
+- **Annotation:** MANE Select v1.3 (RefSeq + Ensembl)
+- **BigWig Metadata:** ENCODE v3, GTEx, FANTOM5, GEO, CATLAS
+- **Reference Genome:** GRCh38/hg38 (local `hg38.fa` preferred; UCSC API used as fallback when local sequence is unavailable)
+
+## Species Support
+
+- **Human:** full app support, including BigWig context tracks and MANE transcript annotation
+- **Non-human animals and plants:** BED outputs and sequence-model metrics via Ensembl / Ensembl Plants sequence retrieval
+- **Important:** MAGI baseline z-scores are derived from human variants, so `Global_z_sum_log` is less directly comparable for non-human species
+
+## Runtime Behavior
+
+- A Hugging Face token may be required to access the gated NTv3 model weights.
+- No external LLM calls are made.
+- No secret-management flow is used by the Gradio app.
+- Ordinary external network access can still occur when downloading model assets from Hugging Face or when falling back to the UCSC sequence API.
+
+## Limitations
+
+1. Predictions are computational and require experimental validation
+2. The app uses a 32 kb local sequence window and does not model long-range chromatin effects
+3. No phasing information is used
+4. Human uses GRCh38/hg38; non-human coordinates must match the selected species assembly available through Ensembl
+5. Batch processing limited to 10 variants
+6. BigWig context tracks and MANE transcript annotation are currently human-only in this app
+7. Region Track View zoom is limited by the available NTv3 track-profile span for the current prediction
+
+## Citation
+
+If you use MAGI in your research, cite the MAGI manuscript. If you rely on the underlying foundation model, also cite NTv3.
+## Links
+
+- 📄 **Paper:** *MAGI: Mechanistic Consequences of Genetic Variants via Genomic Foundation Models* (Ofer, Zok & Linial — preprint forthcoming)
+- 💻 **GitHub:** https://github.com/ddofer/magi<!-- 
+## License
+
+MIT License - see LICENSE file for details -->
+
+## Acknowledgments
+
+- InstaDeepAI and collaborators for developing the NTv3 model
+- ENCODE, GTEx, FANTOM5 consortia for functional genomics data
+- NCBI RefSeq and Ensembl for transcript annotations
+
+---
+
+*Developed for research. Feedback and contributions welcome!*

SETUP_GUIDE.md

@@ -0,0 +1,195 @@
+# Setup and Usage Guide
+
+## Quick Start
+
+MAGI is a variant interpretation app that scores genomic variants using the NTv3 foundation model.
+
+Current runtime configuration:
+
+- Model: `InstaDeepAI/NTv3_650M_post`
+- Sequence window: `32768` bp (32 kb)
+- Batch limit: `10` variants
+- Reference genome: local `data/hg38.fa` when available for human, otherwise UCSC / Ensembl fallback
+
+## Hugging Face Spaces
+
+1. Create a new Space at [Hugging Face Spaces](https://huggingface.co/spaces).
+2. Select the `Gradio` SDK and `zero-a10` hardware.
+3. Upload the contents of this directory.
+   - **Important:** `data/hg38.fa` is ~915 MB. Do **not** commit it to the Space repo without Git LFS — instead rely on the UCSC API fallback (the app does this automatically when the file is absent).
+   - `data/MANE_processed.csv` (~235 MB) and `data/MANE_processed.parquet` (~24 MB) should be committed; the parquet is auto-generated from the CSV on first run.
+4. Set the `HF_TOKEN` Space secret so the app can access the gated `NTv3_650M_post` model weights.
+5. Start the Space.
+
+Notes:
+
+- The app downloads model weights from Hugging Face on first launch and caches them.
+- If `data/hg38.fa` is not present, sequence retrieval falls back to the UCSC API automatically — no extra configuration is needed.
+
+## Local Installation
+
+### Prerequisites
+
+- Python 3.8+
+- Enough disk space for the model cache and optional local genome
+- Internet access for model download and, if needed, UCSC fallback sequence retrieval
+- A Hugging Face account with access to `InstaDeepAI/NTv3_650M_post`
+
+### Install dependencies
+
+```bash
+cd ntv3_gradio_app
+pip install -r requirements.txt
+```
+
+### Optional: download the local genome
+
+Automatic:
+
+```bash
+python download_hg38.py
+```
+
+Manual:
+
+```bash
+wget -c https://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz
+gunzip hg38.fa.gz
+mv hg38.fa data/hg38.fa
+```
+
+If the local genome is absent, the app still runs by querying UCSC for sequence windows.
+
+### Configure Hugging Face access
+
+The active NTv3 model is gated. If model loading fails with an authentication error:
+
+1. Accept the model terms at [InstaDeepAI/NTv3_650M_post](https://huggingface.co/InstaDeepAI/NTv3_650M_post)
+2. Set `HF_TOKEN` in your environment
+
+Example:
+
+```bash
+export HF_TOKEN=your_token_here
+```
+
+### Validate installation
+
+```bash
+python test_installation.py
+```
+
+This checks:
+
+- Python dependencies
+- Required annotation and metadata files
+- Sequence access through the local genome or UCSC fallback
+- Model loading
+
+### Run the app
+
+```bash
+python app.py
+```
+
+Or:
+
+```bash
+gradio app.py
+```
+
+## Input Format
+
+### Single variant input
+
+- Human chromosome: `chr1`-`chr22`, `chrX`, `chrY`, `chrM`
+- Non-human chromosome: bare names such as `1`, `X`, `MT` or `chr`-prefixed names
+- Position: 1-based coordinate for the selected species assembly
+- `ref`: reference allele
+- `alt`: alternate allele
+
+Example:
+
+- Chromosome: `chr17`
+- Position: `7675088`
+- Ref: `C`
+- Alt: `T`
+
+### Batch CSV input
+
+Required columns:
+
+```csv
+chrom,pos,ref,alt
+chr17,7675088,C,T
+chr7,117559593,ATCT,A
+chr13,32340300,G,A
+```
+
+Batch runs are limited to 10 variants.
+
+## Output Overview
+
+The single-variant view includes:
+
+- Variant summary with gene, region class, and core metrics
+- Ranked BED and BigWig signals
+- Region track plot
+- Full BED table
+- Sequence-model metrics
+- Rule-based interpretation panel
+
+Important notes:
+
+- `Global_z_sum_log` is a MAGI ranking score, not a pathogenicity probability.
+- The simple high/moderate/low tier shown in the summary card is currently based on `Impact_Score_BED`.
+- The interpretation panel is heuristic and deterministic. It summarizes existing outputs; it does not add a new predictive model.
+
+## Troubleshooting
+
+### Model download or authentication failed
+
+- Confirm that you accepted the model terms for `InstaDeepAI/NTv3_650M_post`
+- Confirm that `HF_TOKEN` is set
+- Retry after verifying network access to Hugging Face
+
+### No local genome found
+
+- The app can run without `data/hg38.fa`
+- In that case, sequence windows are requested from the UCSC API
+- For faster local runs, download `hg38.fa` into `data/`
+
+### Slow inference
+
+- First launch is slower because weights may need to be downloaded
+- CPU inference is much slower than GPU inference
+- The 650M model requires more memory than smaller NTv3 variants
+
+### Batch upload rejected
+
+- Ensure the CSV contains `chrom`, `pos`, `ref`, and `alt`
+- Ensure the file has no more than 10 rows
+
+### Coordinates look wrong
+
+- Human predictions expect GRCh38/hg38 coordinates
+- Non-human predictions expect coordinates from the selected Ensembl assembly
+- Non-human chromosome names can be bare or `chr`-prefixed
+
+## Development Notes
+
+Source-of-truth files for runtime behavior:
+
+- `app.py`: UI, summaries, and batch handling
+- `inference.py`: model loading, context length, and sequence retrieval
+- `annotation.py`: region classes and annotation flags
+- `analysis.py`: ranking and MAGI score computation
+- `interpretation.py`: rule-based summary text
+
+## References
+
+- NTv3 model page: [InstaDeepAI/NTv3_650M_post](https://huggingface.co/InstaDeepAI/NTv3_650M_post)
+- NTv3 paper: [bioRxiv preprint](https://www.biorxiv.org/content/10.1101/2023.01.11.523679v2)
+- Gradio docs: [gradio.app/docs](https://www.gradio.app/docs)
+
+**Last updated:** March 2026

analysis.py

@@ -0,0 +1,747 @@
+#!/usr/bin/env python3
+"""
+Analysis and visualization module for the MAGI Gradio app.
+
+Adapted from elegant_solution.py for the app.
+
+Provides:
+- Impact score computation (top-K mean absolute delta)
+- Feature summarization with direction labels
+- Fingerprint-style plots for ranked signals
+- BigWig track description mapping
+
+Usage:
+    from analysis import compute_impact_scores, make_fingerprint_plot
+    df = compute_impact_scores(df)
+    fig = make_fingerprint_plot(row, bed_names, bw_names, metadata_df)
+"""
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import List, Tuple, Optional, Dict, Any
+
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+import matplotlib
+
+# Use non-interactive backend for server deployment
+matplotlib.use("Agg")
+
+
+BASE_DIR = Path(__file__).parent
+DATA_DIR = BASE_DIR / "data"
+MAGI_BASELINE_FILE = DATA_DIR / "magi_baseline_stats.csv"
+_MAGI_BASELINE_CACHE: Optional[Dict[str, Dict[str, pd.Series]]] = None
+
+MLM_SIGNAL_SPECS = [
+    ("LLR", "LLR", True),
+    ("MLM_logprob_delta", "Log-prob Δ", True),
+    ("MLM_Delta", "MLM Δ", True),
+    ("MLM_KL_mean", "KL mean", False),
+    ("MLM_KL_max", "KL max", False),
+    ("EMB_cosine_dist", "Embedding cosine dist", False),
+    ("EMB_l2_dist", "Embedding L2 dist", False),
+    ("EMB_max_pos_dist", "Embedding max-pos dist", False),
+    ("EMB_mean_pos_dist", "Embedding mean-pos dist", False),
+]
+
+
+# ============================================================================
+# DATA STRUCTURES
+# ============================================================================
+@dataclass
+class MechanisticSummary:
+    """Concise summary of top disrupted features for a variant."""
+
+    variant_id: str
+    gene_name: str
+    region_class: str
+    top_features: List[Tuple[str, float, str]]  # (name, delta, GOF/LOF)
+    impact_score_bed: float
+    llr: float
+    description: Optional[str] = None
+
+
+# ============================================================================
+# IMPACT SCORING
+# ============================================================================
+def identify_delta_columns(df: pd.DataFrame) -> Tuple[List[str], List[str]]:
+    """
+    Identify BED and BigWig delta columns in DataFrame.
+
+    Returns:
+        (bed_delta_cols, bw_delta_cols)
+    """
+    delta_bed_cols = [c for c in df.columns if c.startswith("D_BED_")]
+    delta_bw_cols = [c for c in df.columns if c.startswith("D_BW_")]
+    return delta_bed_cols, delta_bw_cols
+
+
+def compute_impact_scores(
+    df: pd.DataFrame,
+    bed_k: int = 3,
+    bw_k: int = 10,
+    magi_baseline: Optional[Dict[str, Dict[str, pd.Series]]] = None,
+) -> pd.DataFrame:
+    """
+    Compute impact scores from delta columns.
+
+    Impact_Score_BED: mean of top-K largest absolute BED deltas
+    Impact_Score_BW: mean of top-K largest absolute BigWig deltas
+
+    Args:
+        df: DataFrame with D_BED_* and D_BW_* columns
+        bed_k: Number of top BED deltas to average
+        bw_k: Number of top BigWig deltas to average
+
+    Returns:
+        DataFrame with added Impact_Score_BED, Impact_Score_BW, and
+        Global_z_sum_log columns
+    """
+    df = df.copy()
+    bed_cols, bw_cols = identify_delta_columns(df)
+
+    # BED impact — use argpartition for O(n) top-K selection instead of O(n log n) sort
+    if bed_cols:
+        bed_matrix = np.abs(df[bed_cols].to_numpy())
+        k_bed = min(bed_k, bed_matrix.shape[1])
+        if k_bed > 0:
+            # argpartition gives us indices that partition the array at position k
+            # We want the k largest elements, so we partition at (n - k) to get them on the right
+            indices = np.argpartition(bed_matrix, kth=-k_bed, axis=1)[:, -k_bed:]
+            top_abs_bed = np.take_along_axis(bed_matrix, indices, axis=1)
+            df["Impact_Score_BED"] = top_abs_bed.mean(axis=1)
+        else:
+            df["Impact_Score_BED"] = np.nan
+    else:
+        df["Impact_Score_BED"] = np.nan
+
+    # BigWig impact — use argpartition for O(n) top-K selection
+    if bw_cols:
+        bw_matrix = np.abs(df[bw_cols].to_numpy())
+        k_bw = min(bw_k, bw_matrix.shape[1])
+        if k_bw > 0:
+            indices = np.argpartition(bw_matrix, kth=-k_bw, axis=1)[:, -k_bw:]
+            top_abs_bw = np.take_along_axis(bw_matrix, indices, axis=1)
+            df["Impact_Score_BW"] = top_abs_bw.mean(axis=1)
+        else:
+            df["Impact_Score_BW"] = np.nan
+    else:
+        df["Impact_Score_BW"] = np.nan
+
+    baseline = magi_baseline if magi_baseline is not None else load_magi_baseline()
+    if baseline:
+        df["Global_z_sum_log"] = df.apply(
+            lambda row: compute_global_z_sum_log(row, baseline), axis=1
+        )
+    else:
+        df["Global_z_sum_log"] = np.nan
+
+    return df
+
+
+def _normalize_variant_type_key(
+    value: Optional[str],
+    row: Optional[pd.Series] = None,
+) -> str:
+    """Normalize variant-type labels for MAGI baseline lookup."""
+    text = (
+        str(value).strip().lower() if value is not None and str(value).strip() else ""
+    )
+    if text in {"snp", "snv", "single nucleotide variant"}:
+        return "snp"
+    if text in {"deletion", "del"} or ("deletion" in text and "insert" not in text):
+        return "deletion"
+    if text in {"insertion", "ins"} or "insertion" in text:
+        return "insertion"
+    if text in {"indel", "delins"} or "delins" in text:
+        return "indel"
+
+    if row is not None:
+        ref = str(row.get("ref", "") or "")
+        alt = str(row.get("alt", "") or "")
+        if len(ref) == 1 and len(alt) == 1:
+            return "snp"
+        if len(ref) > len(alt):
+            return "deletion"
+        if len(alt) > len(ref):
+            return "insertion"
+        indel_size = row.get("indel_size", np.nan)
+        if pd.notna(indel_size) and float(indel_size) != 0:
+            return "indel"
+
+    return "indel"
+
+
+def load_magi_baseline(
+    path: Path = MAGI_BASELINE_FILE,
+) -> Dict[str, Dict[str, pd.Series]]:
+    """Load cached benign baseline stats used for MAGI z-scoring."""
+    global _MAGI_BASELINE_CACHE
+
+    if _MAGI_BASELINE_CACHE is not None:
+        return _MAGI_BASELINE_CACHE
+    if not path.exists():
+        _MAGI_BASELINE_CACHE = {}
+        return _MAGI_BASELINE_CACHE
+
+    stats_df = pd.read_csv(path)
+    baseline: Dict[str, Dict[str, pd.Series]] = {}
+    for variant_type, subset in stats_df.groupby("variant_type"):
+        key = str(variant_type).strip().lower()
+        baseline[key] = {
+            "mean": pd.Series(
+                subset["mean"].astype(float).to_numpy(),
+                index=subset["delta_col"].astype(str),
+            ),
+            "std": pd.Series(
+                subset["std"].astype(float).to_numpy(),
+                index=subset["delta_col"].astype(str),
+            ),
+        }
+
+    _MAGI_BASELINE_CACHE = baseline
+    return baseline
+
+
+def compute_global_z_sum_log(
+    row: pd.Series,
+    baseline: Optional[Dict[str, Dict[str, pd.Series]]] = None,
+    variant_type: Optional[str] = None,
+) -> float:
+    """Compute MAGI `Global_z_sum_log` from BED and BigWig delta columns."""
+    baseline = baseline if baseline is not None else load_magi_baseline()
+    if not baseline:
+        return np.nan
+
+    variant_key = _normalize_variant_type_key(
+        variant_type if variant_type is not None else row.get("variant_type"),
+        row=row,
+    )
+    lookup_order = [variant_key]
+    if variant_key != "snp":
+        lookup_order.append("indel")
+
+    stats = None
+    for key in lookup_order:
+        stats = baseline.get(key)
+        if stats:
+            break
+    if not stats:
+        return np.nan
+
+    delta_cols = [
+        col
+        for col in row.index
+        if (col.startswith("D_BED_") or col.startswith("D_BW_"))
+        and not pd.isna(row[col])
+    ]
+    if not delta_cols:
+        return np.nan
+
+    mean_series = stats["mean"]
+    std_series = stats["std"]
+    present_cols = [
+        col
+        for col in delta_cols
+        if col in mean_series.index and col in std_series.index
+    ]
+    if not present_cols:
+        return np.nan
+
+    values = np.asarray([float(row[col]) for col in present_cols], dtype=np.float64)
+    mu = mean_series.reindex(present_cols).to_numpy(dtype=np.float64)
+    sigma = std_series.reindex(present_cols).to_numpy(dtype=np.float64)
+    sigma = np.where(np.isfinite(sigma) & (sigma >= 1e-8), sigma, 1.0)
+
+    z_scores = (values - mu) / sigma
+    z_scores[~np.isfinite(z_scores)] = np.nan
+    if not np.isfinite(z_scores).any():
+        return np.nan
+
+    return float(np.nansum(np.log1p(np.abs(z_scores))))
+
+
+def extract_top_summary_signals(
+    row: pd.Series,
+    ranked: List[Dict[str, Any]],
+    min_abs_threshold: float = 0.03,
+    max_per_source: int = 5,
+) -> Dict[str, List[Dict[str, Any]]]:
+    """Extract compact top BED, BigWig, and MLM signals for the summary card."""
+    summary = {"bed": [], "bigwig": [], "mlm": []}
+
+    for item in ranked:
+        if abs(float(item.get("delta", 0.0))) < min_abs_threshold:
+            continue
+        payload = {
+            "label": item.get("display_name", item.get("track_id", "")),
+            "track_id": item.get("track_id", ""),
+            "delta": float(item.get("delta", np.nan)),
+            "ref_val": item.get("ref_val", np.nan),
+            "alt_val": item.get("alt_val", np.nan),
+        }
+        if item.get("track_type") == "BED" and len(summary["bed"]) < max_per_source:
+            summary["bed"].append(payload)
+        if (
+            item.get("track_type") == "BigWig"
+            and len(summary["bigwig"]) < max_per_source
+        ):
+            summary["bigwig"].append(payload)
+        if (
+            len(summary["bed"]) >= max_per_source
+            and len(summary["bigwig"]) >= max_per_source
+        ):
+            break
+
+    mlm_items: List[Dict[str, Any]] = []
+    for column, label, signed in MLM_SIGNAL_SPECS:
+        if column not in row.index or pd.isna(row[column]):
+            continue
+        value = float(row[column])
+        magnitude = abs(value)
+        if magnitude < min_abs_threshold:
+            continue
+        mlm_items.append(
+            {
+                "label": label,
+                "column": column,
+                "value": value,
+                "magnitude": magnitude,
+                "signed": signed,
+            }
+        )
+    mlm_items.sort(key=lambda item: item["magnitude"], reverse=True)
+    summary["mlm"] = mlm_items[:max_per_source]
+    return summary
+
+
+def get_top_features(
+    row: pd.Series,
+    bed_cols: List[str],
+    bw_cols: Optional[List[str]] = None,
+    k: int = 10,
+) -> List[Tuple[str, float, str]]:
+    """
+    Extract top K disrupted features for a variant.
+
+    Args:
+        row: DataFrame row with delta columns
+        bed_cols: BED delta column names
+        bw_cols: BigWig delta column names (optional)
+        k: Number of features to return
+
+    Returns:
+        List of (feature_name, delta, direction) tuples
+        direction is 'GOF' (gain) or 'LOF' (loss)
+    """
+    deltas = {}
+
+    # Accumulate BED deltas
+    for col in bed_cols:
+        if col in row.index and not pd.isna(row[col]):
+            name = col.replace("D_BED_", "")
+            deltas[name] = float(row[col])
+
+    # Accumulate BigWig deltas (truncate long names)
+    if bw_cols:
+        for col in bw_cols:
+            if col in row.index and not pd.isna(row[col]):
+                name = col.replace("D_BW_", "")
+                if len(name) > 40:
+                    name = name[:37] + "..."
+                deltas[name] = float(row[col])
+
+    # Sort by absolute value
+    top_items = sorted(deltas.items(), key=lambda x: abs(x[1]), reverse=True)[:k]
+
+    # Add direction
+    result = []
+    for feat_name, delta_val in top_items:
+        direction = "GOF" if delta_val > 0 else "LOF"
+        result.append((feat_name, delta_val, direction))
+
+    return result
+
+
+def summarize_variant(
+    row: pd.Series, bed_cols: List[str], bw_cols: Optional[List[str]] = None, k: int = 5
+) -> MechanisticSummary:
+    """
+    Create a mechanistic summary for a variant.
+
+    Args:
+        row: DataFrame row
+        bed_cols: BED delta column names
+        bw_cols: BigWig delta column names
+        k: Number of top features
+
+    Returns:
+        MechanisticSummary object
+    """
+    top_features = get_top_features(row, bed_cols, bw_cols, k=k)
+
+    variant_id = f"{row.get('chrom', 'chr?')}:{row.get('pos', '?')} {row.get('ref', '?')}>{row.get('alt', '?')}"
+    gene_name = row.get("gene_name", "")
+    region_class = row.get("region_class", "OTHER")
+    impact_bed = row.get("Impact_Score_BED", np.nan)
+    llr = row.get("LLR", np.nan)
+
+    return MechanisticSummary(
+        variant_id=variant_id,
+        gene_name=gene_name,
+        region_class=region_class,
+        top_features=top_features,
+        impact_score_bed=impact_bed,
+        llr=llr,
+    )
+
+
+# ============================================================================
+# BIGWIG DESCRIPTION MAPPING
+# ============================================================================
+def get_top_bigwig_descriptions(
+    row: pd.Series, metadata_df: pd.DataFrame, k: int = 5
+) -> List[Tuple[str, float, str]]:
+    """
+    Get top BigWig track changes with human-readable descriptions.
+
+    Args:
+        row: DataFrame row with D_BW_* columns
+        metadata_df: Track metadata with columns [file_id, tissue, assay, experiment_target]
+        k: Number of tracks to return
+
+    Returns:
+        List of (description, delta, direction) tuples
+        description format: "{tissue} | {assay} | {target}"
+    """
+    bw_cols = [c for c in row.index if c.startswith("D_BW_")]
+
+    deltas = {}
+    for col in bw_cols:
+        if not pd.isna(row[col]):
+            track_id = col.replace("D_BW_", "")
+            delta = float(row[col])
+
+            # Look up metadata
+            meta = metadata_df[metadata_df["file_id"] == track_id]
+            if not meta.empty:
+                r = meta.iloc[0]
+                tissue = r.get("tissue", "")
+                assay = r.get("assay", "")
+                target = r.get("experiment_target", "")
+                desc = f"{tissue} | {assay}"
+                if pd.notna(target) and str(target).strip():
+                    desc += f" | {target}"
+            else:
+                desc = track_id
+
+            deltas[desc] = delta
+
+    # Sort by absolute value
+    top_items = sorted(deltas.items(), key=lambda x: abs(x[1]), reverse=True)[:k]
+
+    result = []
+    for desc, delta in top_items:
+        direction = "Gain" if delta > 0 else "Loss"
+        result.append((desc, delta, direction))
+
+    return result
+
+
+def _clean_part(s: str) -> str:
+    """Strip whitespace, trailing/leading commas, and collapse double commas."""
+    s = s.strip().strip(",").strip()
+    # collapse repeated commas with optional spaces
+    import re
+    s = re.sub(r",\s*,", ",", s)
+    return s
+
+
+_EMPTY_TARGETS = {"", "nan", "none", "n/a", "na", "null"}
+
+
+def _resolve_bw_name(
+    track_id: str,
+    metadata_df: Optional[pd.DataFrame] = None,
+    metadata_dict: Optional[Dict[str, Dict[str, str]]] = None,
+    max_len: int = 55,
+) -> str:
+    """Resolve BigWig track_id → human-readable display name."""
+    if metadata_dict:
+        meta = metadata_dict.get(track_id)
+        if meta:
+            parts = [
+                _clean_part(p)
+                for p in [
+                    meta.get("tissue", ""),
+                    meta.get("assay", ""),
+                    meta.get("target", ""),
+                ]
+                if _clean_part(p) and p.strip().lower() not in _EMPTY_TARGETS
+            ]
+            if parts:
+                name = " | ".join(parts)
+                return name[:max_len] if len(name) > max_len else name
+
+    if metadata_df is not None:
+        rows = metadata_df[metadata_df["file_id"] == track_id]
+        if not rows.empty:
+            r = rows.iloc[0]
+            parts = [
+                _clean_part(str(p))
+                for p in [
+                    r.get("tissue", ""),
+                    r.get("assay", ""),
+                    r.get("experiment_target", ""),
+                ]
+                if pd.notna(p) and _clean_part(str(p)) and str(p).strip().lower() not in _EMPTY_TARGETS
+            ]
+            if parts:
+                name = " | ".join(parts)
+                return name[:max_len] if len(name) > max_len else name
+
+    return track_id[:40]
+
+
+def rank_top_disrupted_tracks(
+    row: pd.Series,
+    bed_names: List[str],
+    bw_names: Optional[List[str]] = None,
+    metadata_df: Optional[pd.DataFrame] = None,
+    metadata_dict: Optional[Dict[str, Dict[str, str]]] = None,
+    top_k: Optional[int] = None,
+) -> List[Dict]:
+    """
+    Single-source ranking of the most disrupted tracks across BED + BigWig.
+
+    Returns an ordered list (by |delta| descending) of dicts:
+        {track_id, display_name, delta, track_type ("BED"/"BigWig"),
+         ref_val, alt_val}
+    """
+    items: List[Dict] = []
+
+    for name in bed_names:
+        d_col = f"D_BED_{name}"
+        r_col = f"REF_BED_{name}"
+        if d_col not in row.index or pd.isna(row[d_col]):
+            continue
+        delta = float(row[d_col])
+        ref_v = (
+            float(row[r_col])
+            if r_col in row.index and not pd.isna(row[r_col])
+            else np.nan
+        )
+        items.append(
+            {
+                "track_id": name,
+                "display_name": name,
+                "delta": delta,
+                "track_type": "BED",
+                "ref_val": ref_v,
+                "alt_val": ref_v + delta if not np.isnan(ref_v) else np.nan,
+            }
+        )
+
+    if bw_names:
+        for track_id in bw_names:
+            d_col = f"D_BW_{track_id}"
+            r_col = f"REF_BW_{track_id}"
+            if d_col not in row.index or pd.isna(row[d_col]):
+                continue
+            delta = float(row[d_col])
+            ref_v = (
+                float(row[r_col])
+                if r_col in row.index and not pd.isna(row[r_col])
+                else np.nan
+            )
+            items.append(
+                {
+                    "track_id": track_id,
+                    "display_name": _resolve_bw_name(
+                        track_id, metadata_df, metadata_dict
+                    ),
+                    "delta": delta,
+                    "track_type": "BigWig",
+                    "ref_val": ref_v,
+                    "alt_val": ref_v + delta if not np.isnan(ref_v) else np.nan,
+                }
+            )
+
+    items.sort(key=lambda x: abs(x["delta"]), reverse=True)
+    for idx, item in enumerate(items, start=1):
+        item["abs_delta"] = abs(item["delta"])
+        item["rank"] = idx
+
+    if top_k is None:
+        return items
+    return items[:top_k]
+
+
+def build_top_track_table(
+    ranked: List[Dict],
+    max_rows: int = 15,
+    min_rows_by_type: Optional[Dict[str, int]] = None,
+) -> pd.DataFrame:
+    """Build a single merged top-tracks table from the unified ranking."""
+    if not ranked:
+        return pd.DataFrame(
+            [
+                {
+                    "Track": "No disruptions detected",
+                    "Type": "",
+                    "REF": "",
+                    "ALT": "",
+                    "Δ": "",
+                    "Direction": "",
+                }
+            ]
+        )
+
+    min_rows_by_type = min_rows_by_type or {"BED": 4}
+    max_rows = max(max_rows, sum(min_rows_by_type.values()))
+
+    forced_keys = set()
+    for track_type, min_rows in min_rows_by_type.items():
+        count = 0
+        for item in ranked:
+            if item["track_type"] != track_type:
+                continue
+            forced_keys.add((item["track_type"], item["track_id"]))
+            count += 1
+            if count >= min_rows:
+                break
+
+    selected = []
+    selected_keys = set()
+    for item in ranked:
+        key = (item["track_type"], item["track_id"])
+        if key in forced_keys and key not in selected_keys:
+            selected.append(item)
+            selected_keys.add(key)
+
+    for item in ranked:
+        key = (item["track_type"], item["track_id"])
+        if key in selected_keys:
+            continue
+        if len(selected) >= max_rows:
+            break
+        selected.append(item)
+        selected_keys.add(key)
+
+    selected.sort(key=lambda item: item.get("rank", 10**9))
+
+    rows = []
+    for item in selected[:max_rows]:
+        ref_v = item["ref_val"]
+        alt_v = item["alt_val"]
+        rows.append(
+            {
+                "Track": item["display_name"],
+                "Type": item["track_type"],
+                "REF": f"{ref_v:.4f}" if not np.isnan(ref_v) else "N/A",
+                "ALT": f"{alt_v:.4f}" if not np.isnan(alt_v) else "N/A",
+                "Δ": f"{item['delta']:+.4f}",
+                "Direction": "Gain" if item["delta"] > 0 else "Loss",
+            }
+        )
+    return pd.DataFrame(rows)
+
+
+# ============================================================================
+# VISUALIZATION
+# ============================================================================
+def make_fingerprint_plot(
+    ranked: List[Dict],
+    top_k: int = 15,
+    figsize: Tuple[float, float] = (10, 6),
+) -> plt.Figure:
+    """
+    Horizontal bar chart of top disrupted features.
+
+    Args:
+        ranked: Pre-ranked list from rank_top_disrupted_tracks().
+        top_k: Number of features to show (capped by len(ranked)).
+        figsize: Figure size.
+    """
+    items = ranked[:top_k]
+
+    if not items:
+        fig, ax = plt.subplots(figsize=figsize)
+        ax.text(
+            0.5, 0.5, "No feature data available", ha="center", va="center", fontsize=14
+        )
+        ax.set_xlim(0, 1)
+        ax.set_ylim(0, 1)
+        ax.axis("off")
+        return fig
+
+    names = [it["display_name"] for it in items]
+    vals = [it["delta"] for it in items]
+    colors = ["#d73027" if v > 0 else "#4575b4" for v in vals]
+
+    fig, ax = plt.subplots(figsize=figsize)
+    y_pos = np.arange(len(names))
+    ax.barh(y_pos, vals, color=colors, alpha=0.8, edgecolor="black", linewidth=0.5)
+    ax.axvline(0, color="black", linewidth=1.5, linestyle="-", alpha=0.7)
+    ax.set_yticks(y_pos)
+    ax.set_yticklabels(names, fontsize=9)
+    ax.set_xlabel("Δ Probability (Alt − Ref)", fontsize=11, fontweight="bold")
+    ax.set_title(
+        "Top Disrupted Genomic Features", fontsize=13, fontweight="bold", pad=15
+    )
+    ax.grid(True, axis="x", alpha=0.3, linestyle="--")
+    ax.set_axisbelow(True)
+
+    from matplotlib.patches import Patch
+
+    legend_elements = [
+        Patch(facecolor="#d73027", alpha=0.8, label="Gain of Function"),
+        Patch(facecolor="#4575b4", alpha=0.8, label="Loss of Function"),
+    ]
+    ax.legend(handles=legend_elements, loc="lower right", frameon=True, fontsize=9)
+
+    plt.tight_layout()
+    return fig
+
+
+def format_summary_table(df: pd.DataFrame) -> pd.DataFrame:
+    """
+    Extract clinically relevant columns for display.
+
+    Args:
+        df: Full results DataFrame
+
+    Returns:
+        DataFrame with selected columns for display
+    """
+    display_cols = ["chrom", "pos", "ref", "alt"]
+
+    # Add annotation if available
+    if "gene_name" in df.columns:
+        display_cols.append("gene_name")
+    if "region_class" in df.columns:
+        display_cols.append("region_class")
+
+    # Add impact scores
+    if "Impact_Score_BED" in df.columns:
+        display_cols.append("Impact_Score_BED")
+    if "Impact_Score_BW" in df.columns:
+        display_cols.append("Impact_Score_BW")
+    if "Global_z_sum_log" in df.columns:
+        display_cols.append("Global_z_sum_log")
+
+    # Add MLM features
+    for col in ["LLR", "MLM_KL_mean", "MLM_KL_max"]:
+        if col in df.columns:
+            display_cols.append(col)
+
+    # Add indel size if present
+    if "indel_size" in df.columns:
+        display_cols.append("indel_size")
+
+    # Filter to available columns
+    available = [c for c in display_cols if c in df.columns]
+
+    return df[available].copy()

annotation.py

@@ -0,0 +1,429 @@
+#!/usr/bin/env python3
+"""
+MANE Genomic Annotation Module
+=======================================================
+Adapted from stav_analysis_v2.py for the Gradio app.
+
+Provides gene-level and structural annotation (exon, intron, UTR, splice, promoter)
+using the MANE Select transcript dataset (RefSeq).
+
+Usage:
+    from annotation import annotate_dataframe
+    df_annotated = annotate_dataframe(df)  # adds 30+ annotation columns
+"""
+from pathlib import Path
+from typing import Dict, Set, Tuple
+
+import pandas as pd
+import numpy as np
+
+# ============================================================================
+# CONFIGURATION
+# ============================================================================
+BASE_DIR = Path(__file__).parent
+DATA_DIR = BASE_DIR / "data"
+MANE_FILE = DATA_DIR / "MANE_processed.csv"
+MANE_PARQUET = DATA_DIR / "MANE_processed.parquet"
+PROMOTER_FILE = DATA_DIR / "Promoter_processed.csv"
+PROMOTER_PARQUET = DATA_DIR / "Promoter_processed.parquet"
+
+# Annotation columns (27 region flags + transcript sets)
+ANNOTATION_COLUMNS = [
+    'gene', 'mRNA', 'mRNA_promoter', 'mRNA_exon', 'coding_sequence',
+    'start_codon', 'stop_codon', 'five_prime_UTR', 'three_prime_UTR',
+    'mRNA_intron', 'mRNA_splice', 'lncRNA', 'lncRNA_promoter', 'lncRNA_exon',
+    'snRNA', 'snRNA_promoter', 'snRNA_exon', 'antisenseRNA',
+    'antisenseRNA_promoter', 'antisenseRNA_exon', 'telomeraseRNA',
+    'telomeraseRNA_promoter', 'telomeraseRNA_exon', 'RNaseMRPRNA',
+    'RNaseMRPRNA_promoter', 'RNaseMRPRNA_exon', 'snoRNA', 'snoRNA_promoter',
+    'snoRNA_exon', 'other'
+]
+
+RNA_TYPES = ['lncRNA', 'snRNA', 'antisenseRNA', 'telomeraseRNA',
+             'RNaseMRPRNA', 'snoRNA']
+
+# Global cache for MANE data
+_MANE_CACHE = {
+    "mane_by_chrom": None,
+    "promoter_by_chrom": None,
+    "mane_parent_idx": None,
+}
+
+
+# ============================================================================
+# HELPER FUNCTIONS
+# ============================================================================
+def collapse_region_class(region: str) -> str:
+    """
+    Collapse a comma-separated region annotation into a high-level class.
+    
+    Priority order:
+      CODING > SPLICE > UTR_5 > UTR_3 > PROMOTER > INTRONIC > GENIC_OTHER > OTHER
+    
+    Args:
+        region: Comma-separated string of annotation flags
+    
+    Returns:
+        High-level region class string
+    """
+    if not isinstance(region, str) or not region.strip():
+        return "OTHER"
+
+    parts = {r.strip() for r in region.split(",")}
+
+    if {"coding_sequence", "start_codon", "stop_codon"} & parts:
+        if "mRNA_splice" in parts:
+            return "CODING, SPLICE"
+        return "CODING"
+
+    if "mRNA_splice" in parts:
+        return "SPLICE"
+
+    if "five_prime_UTR" in parts:
+        return "UTR_5"
+
+    if "three_prime_UTR" in parts:
+        return "UTR_3"
+
+    if "mRNA_promoter" in parts:
+        return "PROMOTER"
+
+    if "mRNA_intron" in parts:
+        return "INTRONIC"
+
+    if parts & {"gene", "mRNA", "mRNA_exon"}:
+        return "GENIC_OTHER"
+
+    return "OTHER"
+
+
+def preprocess(mane_raw: pd.DataFrame, promoter_raw: pd.DataFrame) -> Tuple[Dict, Dict, Dict]:
+    """
+    Pre-cast types and build per-chromosome lookup structures.
+    
+    Returns:
+        (mane_by_chrom, promoter_by_chrom, mane_parent_idx)
+    """
+    mane = mane_raw.copy()
+    promoter = promoter_raw.copy()
+
+    # Cast integer columns
+    mane['Start'] = mane['Start'].astype(np.int64)
+    mane['End'] = mane['End'].astype(np.int64)
+    promoter['Promoter_Start'] = promoter['Promoter_Start'].astype(np.int64)
+    promoter['Promoter_End'] = promoter['Promoter_End'].astype(np.int64)
+
+    # Normalize chromosome naming
+    mane['chrom_key'] = mane['Chromosome']
+    promoter['chrom_key'] = promoter['Chromosome'].astype(str)
+
+    # Group by chromosome
+    mane_by_chrom = {k: g for k, g in mane.groupby('chrom_key')}
+    promoter_by_chrom = {k: g for k, g in promoter.groupby('chrom_key')}
+
+    # Build parent-indexed lookups
+    mane_parent_idx = {}
+    for chrom, df in mane_by_chrom.items():
+        parent_groups = {}
+        for parent_val, grp in df.groupby('Parent', sort=False):
+            parent_groups[parent_val] = grp
+        mane_parent_idx[chrom] = parent_groups
+
+    return mane_by_chrom, promoter_by_chrom, mane_parent_idx
+
+
+def annotate_variant(
+    chrom: str,
+    pos: int,
+    mane_by_chrom: Dict,
+    promoter_by_chrom: Dict,
+    mane_parent_idx: Dict
+) -> Tuple[dict, Set[str], Set[str]]:
+    """
+    Annotate a single variant at the given position.
+    
+    Args:
+        chrom: Chromosome (e.g., 'chr17')
+        pos: 1-based genomic position
+        mane_by_chrom: MANE data grouped by chromosome
+        promoter_by_chrom: Promoter data grouped by chromosome
+        mane_parent_idx: Parent-indexed MANE data
+    
+    Returns:
+        (annotation_dict, transcript_set, promoter_transcript_set)
+    """
+    result = {col: 0 for col in ANNOTATION_COLUMNS}
+    transcript_set = set()
+    promoter_transcript_set = set()
+
+    chrom_str = str(chrom)
+
+    # --- Overlap with MANE ---
+    mane_df = mane_by_chrom.get(chrom_str)
+    if mane_df is not None and len(mane_df) > 0:
+        mask = (mane_df['Start'].values <= pos) & (mane_df['End'].values >= pos)
+        annotation = mane_df[mask]
+    else:
+        annotation = pd.DataFrame()
+
+    # --- Overlap with Promoter ---
+    prom_df = promoter_by_chrom.get(chrom_str)
+    if prom_df is not None and len(prom_df) > 0:
+        mask = (prom_df['Promoter_Start'].values <= pos) & (prom_df['Promoter_End'].values >= pos)
+        annotation_promoter = prom_df[mask]
+    else:
+        annotation_promoter = pd.DataFrame()
+
+    if annotation.empty and annotation_promoter.empty:
+        result['other'] = 1
+        return result, transcript_set, promoter_transcript_set
+
+    types = set(annotation['Feature'].unique()) if not annotation.empty else set()
+    types_promoter = set(annotation_promoter['Feature'].unique()) if not annotation_promoter.empty else set()
+
+    # --- gene ---
+    if 'gene' in types:
+        result['gene'] = 1
+
+    # --- mRNA ---
+    if 'mRNA' in types:
+        result['mRNA'] = 1
+        tids = set(annotation.loc[annotation['Feature'] == 'mRNA', 'transcript_id'].dropna())
+        transcript_set.update(tids)
+
+        parent_idx = mane_parent_idx.get(chrom_str, {})
+
+        for tid in tids:
+            rna_key = f'rna-{tid}'
+
+            # Strand
+            id_match = annotation[annotation['ID'] == rna_key]
+            if id_match.empty:
+                continue
+            strand = id_match['Strand'].iloc[0]
+
+            # Exon/CDS overlapping this position
+            ann_exon = annotation[(annotation['Parent'] == rna_key) & (annotation['Feature'] == 'exon')]
+            ann_cds = annotation[(annotation['Parent'] == rna_key) & (annotation['Feature'] == 'CDS')]
+
+            # Full transcript exons/CDS
+            full_exon = parent_idx.get(rna_key)
+            if full_exon is not None:
+                tr_exon = full_exon[full_exon['Feature'] == 'exon']
+                tr_cds = full_exon[full_exon['Feature'] == 'CDS']
+            else:
+                tr_exon = pd.DataFrame()
+                tr_cds = pd.DataFrame()
+
+            if not ann_cds.empty and not ann_exon.empty:
+                # Exon + CDS
+                result['mRNA_exon'] = 1
+                result['coding_sequence'] = 1
+
+                if not tr_cds.empty:
+                    cds_starts = tr_cds['Start'].values
+                    cds_ends = tr_cds['End'].values
+                    if strand == '+':
+                        start_1 = cds_starts.min()
+                        if start_1 <= pos <= start_1 + 2:
+                            result['start_codon'] = 1
+                        stop_3 = cds_ends.max()
+                        if stop_3 - 2 <= pos <= stop_3:
+                            result['stop_codon'] = 1
+                    else:
+                        start_1 = cds_ends.max()
+                        if start_1 - 2 <= pos <= start_1:
+                            result['start_codon'] = 1
+                        stop_3 = cds_starts.min()
+                        if stop_3 <= pos <= stop_3 + 2:
+                            result['stop_codon'] = 1
+
+            elif ann_cds.empty and not ann_exon.empty:
+                # UTR
+                result['mRNA_exon'] = 1
+                if not tr_exon.empty and not tr_cds.empty:
+                    exon_starts = tr_exon['Start'].values
+                    exon_ends = tr_exon['End'].values
+                    cds_starts = tr_cds['Start'].values
+                    cds_ends = tr_cds['End'].values
+
+                    if strand == '+':
+                        five_start = exon_starts.min()
+                        five_end = cds_starts.min() - 1
+                        if five_start <= pos <= five_end:
+                            result['five_prime_UTR'] = 1
+                        three_start = cds_ends.max() + 1
+                        three_end = exon_ends.max()
+                        if three_start <= pos <= three_end:
+                            result['three_prime_UTR'] = 1
+                    else:
+                        five_start = exon_ends.max()
+                        five_end = cds_ends.max() + 1
+                        if five_end <= pos <= five_start:
+                            result['five_prime_UTR'] = 1
+                        three_start = cds_starts.min() - 1
+                        three_end = exon_starts.min()
+                        if three_end <= pos <= three_start:
+                            result['three_prime_UTR'] = 1
+
+            elif ann_cds.empty and ann_exon.empty:
+                # Intron
+                result['mRNA_intron'] = 1
+                if not tr_exon.empty:
+                    ex_starts = tr_exon['Start'].values
+                    ex_ends = tr_exon['End'].values
+                    splice_positions = np.concatenate([
+                        ex_starts - 1, ex_starts - 2,
+                        ex_ends + 1, ex_ends + 2
+                    ])
+                    if pos in splice_positions:
+                        result['mRNA_splice'] = 1
+
+    # --- mRNA promoter ---
+    if 'mRNA' in types_promoter:
+        result['mRNA_promoter'] = 1
+        tids = set(annotation_promoter.loc[
+            annotation_promoter['Feature'] == 'mRNA', 'transcript_id'
+        ].dropna())
+        promoter_transcript_set.update(tids)
+
+    # --- Other RNA types ---
+    for rna in RNA_TYPES:
+        if rna in types:
+            result[rna] = 1
+            tids = set(annotation.loc[annotation['Feature'] == rna, 'transcript_id'].dropna())
+            transcript_set.update(tids)
+            for tid in tids:
+                rna_key = f'rna-{tid}'
+                ann_exon = annotation[(annotation['Parent'] == rna_key) & (annotation['Feature'] == 'exon')]
+                if not ann_exon.empty:
+                    result[f'{rna}_exon'] = 1
+
+        if rna in types_promoter:
+            result[f'{rna}_promoter'] = 1
+            tids = set(annotation_promoter.loc[
+                annotation_promoter['Feature'] == rna, 'transcript_id'
+            ].dropna())
+            promoter_transcript_set.update(tids)
+
+    return result, transcript_set, promoter_transcript_set
+
+
+# ============================================================================
+# PUBLIC API
+# ============================================================================
+def _load_or_convert(csv_path: Path, parquet_path: Path) -> pd.DataFrame:
+    """Load from parquet if available, otherwise read CSV and cache as parquet."""
+    if parquet_path.exists():
+        return pd.read_parquet(parquet_path)
+    df = pd.read_csv(csv_path)
+    try:
+        df.to_parquet(parquet_path, index=False)
+        print(f"   Cached {parquet_path.name} for faster future loads")
+    except Exception as exc:
+        print(f"   ⚠️  Parquet cache write failed: {exc}")
+    return df
+
+
+def load_mane_data():
+    """Load and preprocess MANE and Promoter data. Caches globally."""
+    if _MANE_CACHE["mane_by_chrom"] is not None:
+        return  # Already loaded
+
+    print(f"📚 Loading MANE annotation data from {DATA_DIR}...")
+    mane_raw = _load_or_convert(MANE_FILE, MANE_PARQUET)
+    promoter_raw = _load_or_convert(PROMOTER_FILE, PROMOTER_PARQUET)
+
+    print(f"   MANE: {len(mane_raw):,} features")
+    print(f"   Promoter: {len(promoter_raw):,} features")
+
+    mane_by_chrom, promoter_by_chrom, mane_parent_idx = preprocess(mane_raw, promoter_raw)
+
+    _MANE_CACHE.update({
+        "mane_by_chrom": mane_by_chrom,
+        "promoter_by_chrom": promoter_by_chrom,
+        "mane_parent_idx": mane_parent_idx,
+    })
+
+    print(f"✅ MANE data loaded: {len(mane_by_chrom)} chromosomes indexed")
+
+
+def annotate_dataframe(df: pd.DataFrame) -> pd.DataFrame:
+    """
+    Add MANE annotations to a variants DataFrame.
+    
+    Args:
+        df: DataFrame with 'chrom' and 'pos' columns
+    
+    Returns:
+        DataFrame with added annotation columns:
+        - 30 binary flags (gene, mRNA, coding_sequence, start_codon, etc.)
+        - 'transcript_set', 'promoter_transcript_set' (sets of transcript IDs)
+        - 'region' (comma-separated list of active flags)
+        - 'region_class' (high-level category)
+        - 'gene_name' (from MANE, if available)
+    """
+    # Load MANE data if not already loaded
+    if _MANE_CACHE["mane_by_chrom"] is None:
+        load_mane_data()
+
+    mane_by_chrom = _MANE_CACHE["mane_by_chrom"]
+    promoter_by_chrom = _MANE_CACHE["promoter_by_chrom"]
+    mane_parent_idx = _MANE_CACHE["mane_parent_idx"]
+
+    # Validate input
+    if "chrom" not in df.columns or "pos" not in df.columns:
+        raise ValueError("DataFrame must have 'chrom' and 'pos' columns")
+
+    df = df.copy()
+    chroms = df['chrom'].values
+    positions = df['pos'].astype(np.int64).values
+
+    all_results = []
+    all_tsets = []
+    all_ptsets = []
+
+    for i in range(len(df)):
+        res, tset, ptset = annotate_variant(
+            chroms[i], positions[i],
+            mane_by_chrom, promoter_by_chrom, mane_parent_idx
+        )
+        all_results.append(res)
+        all_tsets.append(tset)
+        all_ptsets.append(ptset)
+
+    # Add annotation columns
+    ann_df = pd.DataFrame(all_results, index=df.index)
+    for col in ANNOTATION_COLUMNS:
+        df[col] = ann_df[col].values
+    df['transcript_set'] = all_tsets
+    df['promoter_transcript_set'] = all_ptsets
+
+    # Combine into region string
+    df['region'] = (
+        df[ANNOTATION_COLUMNS]
+        .apply(lambda r: ','.join(r.index[r == 1]), axis=1)
+    )
+
+    # Collapse to high-level class
+    df["region_class"] = df["region"].apply(collapse_region_class)
+
+    # Extract gene name from MANE (if available)
+    gene_names = []
+    for i in range(len(df)):
+        chrom_str = str(chroms[i])
+        pos = positions[i]
+        mane_df = mane_by_chrom.get(chrom_str)
+        gene_name = ""
+        if mane_df is not None:
+            mask = (mane_df['Start'].values <= pos) & (mane_df['End'].values >= pos)
+            overlaps = mane_df[mask]
+            if not overlaps.empty and 'gene' in overlaps.columns:
+                genes = overlaps['gene'].dropna().unique()
+                if len(genes) > 0:
+                    gene_name = genes[0]
+        gene_names.append(gene_name)
+    
+    df['gene_name'] = gene_names
+
+    return df

app.py

@@ -0,0 +1,1146 @@
+#!/usr/bin/env python3
+"""
+MAGI Variant Interpreter - Gradio Web Interface
+===============================================
+Interactive web app for scoring and summarizing the predicted effects of
+human genomic variants.
+
+The app uses the external NTv3 foundation model together with MAGI scoring,
+annotation, and interpretation layers.
+
+Usage:
+    python app.py
+
+For HuggingFace Spaces deployment with ZeroGPU:
+    Uses @spaces.GPU decorator for GPU bursting
+"""
+
+import os
+import warnings
+from pathlib import Path
+
+import torch
+import gradio as gr
+import pandas as pd
+import numpy as np
+import matplotlib.pyplot as plt
+
+# Import our modules
+from inference import (
+    CONTEXT_LEN,
+    predict_variants,
+    load_model_and_resources,
+    _MODEL_CACHE,
+    SUPPORTED_UI_SPECIES,
+)
+from annotation import annotate_dataframe, load_mane_data, _MANE_CACHE
+from analysis import (
+    rank_top_disrupted_tracks,
+    build_top_track_table,
+    compute_impact_scores,
+    extract_top_summary_signals,
+    summarize_variant,
+    get_top_bigwig_descriptions,
+    make_fingerprint_plot,
+    identify_delta_columns,
+    format_summary_table,
+)
+from tracks import generate_region_tracks_plot, get_track_view_bounds
+from interpretation import build_signal_interpretation
+
+# Suppress warnings for cleaner UI
+warnings.filterwarnings("ignore")
+
+# ============================================================================
+# CONFIGURATION
+# ============================================================================
+BASE_DIR = Path(__file__).parent
+DATA_DIR = BASE_DIR / "data"
+EXAMPLES_DIR = BASE_DIR / "examples"
+METADATA_FILE = DATA_DIR / "functional_tracks_metadata_human.csv"
+DEFAULT_ZOOM_BP = 128
+MIN_ZOOM_BP = 8
+INITIAL_ZOOM_MAX_BP = CONTEXT_LEN // 2
+ANNOTATION_FLAGS = [
+    "gene",
+    "mRNA",
+    "mRNA_promoter",
+    "mRNA_exon",
+    "coding_sequence",
+    "start_codon",
+    "stop_codon",
+    "five_prime_UTR",
+    "three_prime_UTR",
+    "mRNA_intron",
+    "mRNA_splice",
+    "lncRNA",
+    "lncRNA_promoter",
+    "lncRNA_exon",
+    "snRNA",
+    "snRNA_promoter",
+    "snRNA_exon",
+    "antisenseRNA",
+    "antisenseRNA_promoter",
+    "antisenseRNA_exon",
+    "telomeraseRNA",
+    "telomeraseRNA_promoter",
+    "telomeraseRNA_exon",
+    "RNaseMRPRNA",
+    "RNaseMRPRNA_promoter",
+    "RNaseMRPRNA_exon",
+    "snoRNA",
+    "snoRNA_promoter",
+    "snoRNA_exon",
+    "other",
+]
+
+
+def _export_figure_png(fig, file_name: str, dpi: int = 300):
+    """Persist a Matplotlib figure for figure-ready download."""
+    if fig is None:
+        return None
+
+    output_path = BASE_DIR / file_name
+    fig.savefig(output_path, dpi=dpi, bbox_inches="tight")
+    return str(output_path)
+
+
+def _is_human_species(species: str) -> bool:
+    return str(species).strip() == "human"
+
+
+def _species_label(species: str) -> str:
+    return SUPPORTED_UI_SPECIES.get(
+        str(species).strip(), str(species).replace("_", " ").title()
+    )
+
+
+def _apply_species_annotations(results_df: pd.DataFrame, species: str) -> pd.DataFrame:
+    if _is_human_species(species):
+        return annotate_dataframe(results_df)
+
+    annotated = results_df.copy()
+    for col in ANNOTATION_FLAGS:
+        annotated[col] = 0
+    annotated["transcript_set"] = [set() for _ in range(len(annotated))]
+    annotated["promoter_transcript_set"] = [set() for _ in range(len(annotated))]
+    annotated["region"] = "non_human_unavailable"
+    annotated["region_class"] = "NON_HUMAN"
+    annotated["gene_name"] = "N/A (non-human)"
+    return annotated
+
+
+def _build_zoom_slider_update(requested_zoom_bp):
+    """Update the zoom slider to the current variant's available track-view span."""
+    try:
+        requested_zoom_bp = max(MIN_ZOOM_BP, int(requested_zoom_bp))
+    except (TypeError, ValueError):
+        requested_zoom_bp = DEFAULT_ZOOM_BP
+
+    view_bounds = get_track_view_bounds()
+    max_radius = view_bounds.get("max_radius")
+    if max_radius is None:
+        return gr.update()
+
+    return gr.update(
+        value=min(requested_zoom_bp, max_radius),
+        maximum=max_radius,
+        info=f"Current available radius for this prediction: up to {max_radius:,} bp",
+    )
+
+
+def predict_single_variant_ui(species, chrom, pos, ref, alt, zoom_bp=DEFAULT_ZOOM_BP):
+    """UI wrapper: run prediction and prepare user-facing outputs."""
+    result = predict_single_variant(species, chrom, pos, ref, alt, zoom_bp=zoom_bp)
+    # Engine returns 9 values
+    (
+        summary_md,
+        interpretation_md,
+        top_table_df,
+        fingerprint_fig,
+        region_tracks_fig,
+        csv_path,
+        bed_table_df,
+        mlm_md,
+        ranked,
+    ) = result
+
+    tracks_png_path = None
+    try:
+        tracks_png_path = _export_figure_png(
+            region_tracks_fig, "last_variant_tracks.png"
+        )
+    except Exception as exc:
+        print(f"\u26a0\ufe0f  Track PNG export failed: {exc}")
+
+    zoom_update = gr.update()
+    if isinstance(ranked, list):
+        zoom_update = _build_zoom_slider_update(zoom_bp)
+
+    return (
+        summary_md,
+        interpretation_md,
+        top_table_df,
+        fingerprint_fig,
+        region_tracks_fig,
+        csv_path,
+        tracks_png_path,
+        bed_table_df,
+        mlm_md,
+        zoom_update,
+        ranked[:50]
+        if isinstance(ranked, list)
+        else ranked,  # → gr.State for zoom slider
+    )
+
+
+def _update_region_zoom(zoom_bp, ranked, track_filter="All"):
+    """Re-render region tracks plot when the zoom slider or filter changes."""
+    if not ranked:
+        return None, None
+    try:
+        filtered = ranked
+        if track_filter and track_filter != "All":
+            filtered = [r for r in ranked if r["track_type"] == track_filter]
+        if not filtered:
+            filtered = ranked
+        fig = generate_region_tracks_plot(
+            ranked_tracks=filtered,
+            max_ranked_tracks=10,
+            visible_radius_bp=max(int(zoom_bp), 8),
+        )
+        png_path = _export_figure_png(fig, "last_variant_tracks.png")
+        return fig, png_path
+    except Exception as exc:
+        print(f"\u26a0\ufe0f  Zoom re-render failed: {exc}")
+        return None, None
+
+
+def predict_single_variant_wrapper(
+    species, chrom, pos, ref, alt, zoom_bp=DEFAULT_ZOOM_BP
+):
+    """Wrapper with ZeroGPU decorator if available."""
+    return predict_single_variant_ui(species, chrom, pos, ref, alt, zoom_bp)
+
+
+# Check if running on HuggingFace Spaces with ZeroGPU
+try:
+    import spaces
+
+    HAS_SPACES = True
+    print("\u2705 Running with HuggingFace Spaces ZeroGPU support")
+except ImportError:
+    spaces = None
+    HAS_SPACES = False
+    print("\u26a0\ufe0f  Not running on HuggingFace Spaces - using local GPU/CPU")
+
+
+# Apply decorators if available
+if HAS_SPACES and spaces is not None:
+    predict_single_variant_wrapper = spaces.GPU(predict_single_variant_wrapper)
+
+# Device selection: prefer CUDA if available, fall back to CPU
+if os.environ.get("SPACES_GPU") or (HAS_SPACES and torch.cuda.is_available()):
+    DEVICE = "cuda"
+elif torch.cuda.is_available():
+    DEVICE = "cuda"
+else:
+    DEVICE = "cpu"
+print(f"⚙️  Using device: {DEVICE}")
+
+# Load metadata for BigWig track descriptions
+TRACK_METADATA = pd.read_csv(METADATA_FILE)
+
+# Pre-build metadata lookup dictionary for O(1) access
+TRACK_META_DICT = {
+    row["file_id"]: {
+        "tissue": str(row.get("tissue", "") or ""),
+        "assay": str(row.get("assay", "") or ""),
+        "target": str(row.get("experiment_target", "") or ""),
+    }
+    for _, row in TRACK_METADATA.iterrows()
+}
+
+
+# ============================================================================
+# STARTUP - LOAD MODEL AND ANNOTATION DATA
+# ============================================================================
+def initialize_app():
+    """Load model and annotation data at startup."""
+    print("🚀 Initializing MAGI Variant Interpreter...")
+    try:
+        load_model_and_resources(device=DEVICE)
+        load_mane_data()
+        print("✅ App ready!")
+        return True
+    except Exception as e:
+        print(f"❌ Model initialization failed: {e}")
+        import traceback
+
+        traceback.print_exc()
+        return False
+
+
+# Initialize on import
+_APP_READY = initialize_app()
+
+
+# ============================================================================
+# UTILITY HELPERS
+# ============================================================================
+def _fmt(val, fmt: str = ".4f", na: str = "N/A") -> str:
+    """Safely format a numeric value; return 'na' string for NaN/None."""
+    if val is None or (isinstance(val, float) and np.isnan(val)):
+        return na
+    try:
+        return format(float(val), fmt)
+    except (TypeError, ValueError):
+        return na
+
+
+def _render_track_signal_block(title: str, items) -> str:
+    """Render compact BED/BigWig bullets for the summary card."""
+    if not items:
+        return f"**{title}**\n- none above 3% absolute change"
+
+    lines = [f"**{title}**"]
+    for item in items:
+        lines.append(
+            "- "
+            f"{item['label']}: {item['delta']:+.1%} "
+            f"({_fmt(item['ref_val'])} → {_fmt(item['alt_val'])})"
+        )
+    return "\n".join(lines)
+
+
+def _render_mlm_signal_block(items) -> str:
+    """Render compact sequence-model bullets for the summary card."""
+    if not items:
+        return "**Top sequence-model signals**\n- none above 0.03 absolute magnitude"
+
+    lines = ["**Top sequence-model signals**"]
+    for item in items:
+        value_fmt = (
+            f"{item['value']:+.4f}"
+            if item.get("signed", False)
+            else f"{item['value']:.4f}"
+        )
+        lines.append(f"- {item['label']}: {value_fmt}")
+    return "\n".join(lines)
+
+
+def _build_top_signal_summary(signals, include_bigwig: bool = True) -> str:
+    """Build the top-signal markdown block for the summary card."""
+    blocks = [
+        "#### Top Signals",
+        _render_track_signal_block("Top BED signals", signals.get("bed", [])),
+        _render_mlm_signal_block(signals.get("mlm", [])),
+    ]
+    if include_bigwig:
+        blocks.insert(
+            2,
+            _render_track_signal_block("Top BigWig signals", signals.get("bigwig", [])),
+        )
+    return "\n\n".join(blocks)
+
+
+def _validate_allele(allele: str) -> bool:
+    """Return True if allele contains only valid nucleotide characters."""
+    return bool(allele) and all(c in "ACGTNacgtn" for c in allele.strip())
+
+
+# ============================================================================
+# SINGLE VARIANT PREDICTION
+# ============================================================================
+def predict_single_variant(
+    species: str,
+    chrom: str,
+    pos: int,
+    ref: str,
+    alt: str,
+    zoom_bp: int = DEFAULT_ZOOM_BP,
+):
+    """
+    Predict functional impact for a single variant.
+
+    Returns:
+        Tuple of (summary_md, interpretation_md, top_table_df,
+                ranked_tracks=ranked,
+                max_ranked_tracks=15,
+                  bed_table_df, mlm_md, ranked_tracks)
+    """
+    _none9 = (None,) * 9
+
+    try:
+        # Validate inputs
+        if not chrom or not ref or not alt:
+            return (
+                "❌ Error: Please provide all required fields (Chromosome, Position, Ref, Alt)",
+                *_none9[1:],
+            )
+
+        if pos <= 0:
+            return (
+                "❌ Error: Position must be a positive integer",
+                *_none9[1:],
+            )
+
+        ref_clean = ref.upper().strip()
+        alt_clean = alt.upper().strip()
+
+        if not _validate_allele(ref_clean):
+            return (
+                f"❌ Error: REF allele '{ref}' contains invalid characters. Use A/C/G/T/N only.",
+                *_none9[1:],
+            )
+        if not _validate_allele(alt_clean):
+            return (
+                f"❌ Error: ALT allele '{alt}' contains invalid characters. Use A/C/G/T/N only.",
+                *_none9[1:],
+            )
+
+        # Create input DataFrame
+        input_df = pd.DataFrame(
+            [
+                {
+                    "chrom": chrom,
+                    "pos": int(pos),
+                    "ref": ref_clean,
+                    "alt": alt_clean,
+                }
+            ]
+        )
+
+        # Run inference
+        species = str(species).strip()
+        print(f"🔬 Predicting {species} {chrom}:{pos} {ref_clean}>{alt_clean}...")
+        results_df = predict_variants(input_df, device=DEVICE, species=species)
+
+        # Annotate
+        results_df = _apply_species_annotations(results_df, species)
+
+        # Compute impact scores
+        results_df = compute_impact_scores(results_df)
+
+        # Extract data
+        row = results_df.iloc[0]
+
+        gene = row.get("gene_name", "Unknown")
+        region = row.get("region_class", "OTHER")
+        impact_bed = row.get("Impact_Score_BED", np.nan)
+        impact_bw = row.get("Impact_Score_BW", np.nan)
+        magi_score = row.get("Global_z_sum_log", np.nan)
+        llr = row.get("LLR", np.nan)
+        kl_mean = row.get("MLM_KL_mean", np.nan)
+        indel_size = row.get("indel_size", 0)
+        transcript_set = row.get("transcript_set", set())
+
+        variant_type = "Indel" if indel_size != 0 else "SNP"
+        indel_label = f" ({int(indel_size):+d} bp)" if indel_size != 0 else ""
+
+        if pd.isna(impact_bed):
+            impact_level = "**unknown**"
+        elif impact_bed > 0.1:
+            impact_level = "**high**"
+        elif impact_bed > 0.05:
+            impact_level = "**moderate**"
+        else:
+            impact_level = "**low**"
+
+        llr_note = " *(N/A for indels)*" if pd.isna(llr) else ""
+
+        transcript_str = ""
+        if transcript_set and len(transcript_set) > 0:
+            transcript_ids = ", ".join(sorted(list(transcript_set))[:3])
+            transcript_str = f"  \n**Transcripts:** `{transcript_ids}`"
+
+        context_start = max(1, pos - CONTEXT_LEN // 2)
+        context_end = pos + CONTEXT_LEN // 2
+
+        # Inline genomic context tags (replaces separate Annotation Detail accordion)
+        active_flags = [
+            flag
+            for flag in ANNOTATION_FLAGS
+            if flag in row.index and row.get(flag, 0) == 1
+        ]
+        if not _is_human_species(species):
+            context_tags = "Transcript-level human MANE annotation is not available for this species in the web app."
+        elif active_flags:
+            context_tags = " · ".join(f"`{f}`" for f in active_flags)
+        else:
+            context_tags = "Intergenic (no annotated features)"
+
+        # === 1. Unified ranking → fingerprint, table, region tracks ===
+        bed_names = _MODEL_CACHE.get("bed_names", [])
+        bw_names_selected = _MODEL_CACHE.get("selected_bw_indices", [])
+        bw_names_all = _MODEL_CACHE.get("bigwig_names", [])
+        bw_names_filtered = (
+            [bw_names_all[i] for i in bw_names_selected]
+            if _is_human_species(species) and bw_names_selected and bw_names_all
+            else None
+        )
+
+        ranked = rank_top_disrupted_tracks(
+            row,
+            bed_names,
+            bw_names_filtered,
+            metadata_df=TRACK_METADATA,
+            metadata_dict=TRACK_META_DICT,
+            top_k=None,
+        )
+
+        top_table_df = build_top_track_table(
+            ranked,
+            max_rows=50,
+            min_rows_by_type={"BED": 5},
+        )
+
+        fingerprint_fig = make_fingerprint_plot(ranked, top_k=15)
+
+        interpretation_md = build_signal_interpretation(row, ranked, variant_type)
+
+        summary_signals = extract_top_summary_signals(
+            row,
+            ranked,
+            min_abs_threshold=0.03,
+        )
+        top_signal_md = _build_top_signal_summary(
+            summary_signals,
+            include_bigwig=_is_human_species(species),
+        )
+
+        species_name = _species_label(species)
+        bigwig_value = (
+            _fmt(impact_bw)
+            if _is_human_species(species)
+            else "N/A (human-only context tracks)"
+        )
+        annotation_note = (
+            ""
+            if _is_human_species(species)
+            else "\n**Non-human note:** BigWig context tracks and MANE transcript annotation are not available in this app for non-human species."
+        )
+
+        summary_md = f"""
+### Variant Summary
+
+**Variant:** `{chrom}:{pos} {ref_clean}>{alt_clean}` ({variant_type}{indel_label})  
+**Species:** {species_name} (`{species}`)  
+**Gene:** {gene if gene else "Intergenic"}  
+**Region:** {region}{transcript_str}  
+**Genomic context:** {context_tags}
+
+{top_signal_md}
+
+| Metric | Value |
+|--------|-------|
+| MAGI score (`Global_z_sum_log`) | {_fmt(magi_score)} |
+| BED Impact Score | {_fmt(impact_bed)} |
+| BigWig Impact Score | {bigwig_value} |
+| LLR | {_fmt(llr)}{llr_note} |
+| KL Divergence (mean) | {_fmt(kl_mean)} |
+
+**Analysis window:** `{chrom}:{context_start:,}-{context_end:,}` ({context_end - context_start:,} bp)  
+**BED-based summary tier:** {impact_level}. This tier reflects `Impact_Score_BED` only.{annotation_note}
+"""
+
+        # === 2. BED Table (all 21 elements) ===
+        bed_table = []
+        for name in bed_names:
+            ref_col = f"REF_BED_{name}"
+            delta_col = f"D_BED_{name}"
+            if ref_col in row.index and delta_col in row.index:
+                ref_val = row[ref_col]
+                delta_val = row[delta_col]
+                alt_val = (
+                    ref_val + delta_val
+                    if not pd.isna(ref_val) and not pd.isna(delta_val)
+                    else np.nan
+                )
+                bed_table.append(
+                    {
+                        "Element": name,
+                        "REF": f"{ref_val:.4f}" if not pd.isna(ref_val) else "N/A",
+                        "ALT": f"{alt_val:.4f}" if not pd.isna(alt_val) else "N/A",
+                        "Delta": f"{delta_val:+.5f}"
+                        if not pd.isna(delta_val)
+                        else "N/A",
+                    }
+                )
+        bed_table_df = pd.DataFrame(bed_table)
+
+        # === 3. Sequence model metrics ===
+        kl_max = row.get("MLM_KL_max", np.nan)
+        logprob_ref = row.get("MLM_logprob_ref", np.nan)
+        logprob_alt = row.get("MLM_logprob_alt", np.nan)
+        ref_5mer = row.get("REF_5mer", "N/A") or "N/A"
+        alt_5mer = row.get("ALT_5mer", "N/A") or "N/A"
+
+        mlm_md = f"""
+These metrics summarize how the NTv3 model responds to the alternate sequence relative to the reference:
+
+- **Variant type:** {variant_type}{indel_label}  
+- **LLR (Log-Likelihood Ratio):** {_fmt(llr)}{"  *(SNPs only — N/A for indels)*" if pd.isna(llr) else ""}  
+- **KL Divergence (mean):** {_fmt(kl_mean)}  
+  Average KL(ALT \u2225\u2225 REF) across the evaluated span  
+- **KL Divergence (max):** {_fmt(kl_max)}  
+- **Log-prob REF:** {_fmt(logprob_ref, fmt=".4f")}  
+- **Log-prob ALT:** {_fmt(logprob_alt, fmt=".4f")}  
+- **Context (REF):** `{ref_5mer}`  
+- **Context (ALT):** `{alt_5mer}`  
+"""
+
+        if abs(indel_size) > 0:
+            emb_cosine = row.get("EMB_cosine_dist", np.nan)
+            emb_l2 = row.get("EMB_l2_dist", np.nan)
+            emb_max = row.get("EMB_max_pos_dist", np.nan)
+            emb_mean = row.get("EMB_mean_pos_dist", np.nan)
+            mlm_md += f"""
+**Embedding Distance Metrics (Indel-specific):**
+- Cosine Distance: {_fmt(emb_cosine)}
+- L2 Distance: {_fmt(emb_l2)}
+- Max per-position dist: {_fmt(emb_max)}
+- Mean per-position dist: {_fmt(emb_mean)}
+"""
+
+        # === 4. CSV Export ===
+        try:
+            result_csv_path = str(BASE_DIR / "last_variant_result.csv")
+            results_df.to_csv(result_csv_path, index=False)
+        except Exception as e:
+            print(f"⚠️  CSV export failed: {e}")
+            result_csv_path = None
+
+        # === 5. Region Tracks Plot (uses unified ranking) ===
+        try:
+            region_tracks_fig = generate_region_tracks_plot(
+                ranked_tracks=ranked,
+                max_ranked_tracks=10,
+                visible_radius_bp=max(int(zoom_bp), 8),
+            )
+        except Exception as e:
+            print(f"⚠️  Region tracks plot failed: {e}")
+            region_tracks_fig = None
+
+        return (
+            summary_md,
+            interpretation_md,
+            top_table_df,
+            fingerprint_fig,
+            region_tracks_fig,
+            result_csv_path,
+            bed_table_df,
+            mlm_md,
+            ranked,  # stashed for slider re-renders
+        )
+
+    except Exception as e:
+        import traceback
+
+        error_msg = f"❌ **Error during prediction:**\n\n```\n{str(e)}\n```\n\n**Traceback:**\n```\n{traceback.format_exc()}\n```"
+        return (error_msg, None, None, None, None, None, None, None, None)
+
+
+# ============================================================================
+# BATCH PREDICTION
+# ============================================================================
+def predict_batch_wrapper(species, file):
+    """Wrapper with ZeroGPU decorator if available."""
+    return predict_batch(species, file)
+
+
+# Apply batch decorator if available
+if HAS_SPACES and spaces is not None:
+    predict_batch_wrapper = spaces.GPU(predict_batch_wrapper)
+
+
+def predict_batch(species, file):
+    """
+    Predict functional impact for multiple variants from CSV.
+
+    Args:
+        file: Uploaded CSV file with columns [chrom, pos, ref, alt]
+
+    Returns:
+        Tuple of (preview_df, download_file_path)
+    """
+    try:
+        if file is None:
+            return (pd.DataFrame([{"Error": "No file uploaded"}]), None)
+
+        # Read CSV — handle both Gradio 3.x (file object) and 4.x (path string)
+        file_path = file if isinstance(file, str) else file.name
+        input_df = pd.read_csv(file_path)
+
+        # Validate columns
+        required = {"chrom", "pos", "ref", "alt"}
+        missing = required - set(input_df.columns)
+        if missing:
+            return (pd.DataFrame([{"Error": f"Missing columns: {missing}"}]), None)
+
+        # Limit batch size
+        if len(input_df) > 10:
+            return (
+                pd.DataFrame(
+                    [
+                        {
+                            "Error": f"Batch size limited to 10 variants (received {len(input_df)})"
+                        }
+                    ]
+                ),
+                None,
+            )
+
+        species = str(species).strip()
+        print(f"🔬 Processing batch of {len(input_df)} variants for {species}...")
+
+        # Run inference
+        results_df = predict_variants(input_df, device=DEVICE, species=species)
+
+        # Annotate
+        results_df = _apply_species_annotations(results_df, species)
+
+        # Compute impact scores
+        results_df = compute_impact_scores(results_df)
+
+        # Save full results
+        output_path = BASE_DIR / "batch_results.csv"
+        results_df.to_csv(output_path, index=False)
+
+        # Create preview (summary columns)
+        preview_df = format_summary_table(results_df)
+
+        print(f"✅ Batch processing complete: {len(results_df)} variants")
+
+        return (preview_df, str(output_path))
+
+    except Exception as e:
+        import traceback
+
+        error_msg = f"Error: {str(e)}\n\n{traceback.format_exc()}"
+        return (pd.DataFrame([{"Error": error_msg}]), None)
+
+
+# ============================================================================
+# GRADIO INTERFACE
+# ============================================================================
+def build_interface():
+    """Build the Gradio interface."""
+
+    # Custom CSS
+    custom_css = """
+    .variant-header {
+        background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
+        color: white;
+        padding: 2rem;
+        border-radius: 10px;
+        margin-bottom: 2rem;
+    }
+    .metric-card {
+        background: #f8f9fa;
+        padding: 1rem;
+        border-radius: 8px;
+        border-left: 4px solid #667eea;
+    }
+    """
+
+    # Build status line from cached model info (safe — loads at startup)
+    _seq_src = _MODEL_CACHE.get("sequence_source", "ucsc")
+    _bed_n = len(_MODEL_CACHE.get("bed_names", []) or [])
+    _bw_n = len(_MODEL_CACHE.get("selected_bw_indices", []) or [])
+    _species_choices = [
+        (f"{label} ({species})", species)
+        for species, label in SUPPORTED_UI_SPECIES.items()
+    ]
+
+    if _APP_READY:
+        _status_line = (
+            "🟢 **Model ready** — "
+            f"Device: `{DEVICE}` • "
+            f"Sequence source: `{_seq_src}` • "
+            f"Tracks: {_bed_n} BED + {_bw_n} BigWig"
+        )
+    else:
+        _status_line = "🔴 **Model failed to load** — Check configuration and logs"
+
+    # Gradio 6.x: create Blocks without theme/css, then set as attributes
+    app = gr.Blocks(title="MAGI Variant Interpreter")
+    app.theme = gr.themes.Soft()
+    app.css = custom_css
+
+    with app:
+        gr.Markdown(
+            """
+        # 🧬 MAGI Variant Interpreter
+        
+        **Variant interpretation demo built on the NTv3 foundation model**
+        
+        MAGI combines NTv3 sequence predictions with annotation, ranking, and rule-based summaries to help review variant-associated signals in a local genomic window.
+
+        The app reports changes across:
+        - **Regulatory elements** (promoters, enhancers, CTCF sites)
+        - **Structural features** (coding sequence, splice sites, UTRs)
+        - **Epigenetic marks** (histone modifications, chromatin accessibility)
+        - **Context-dependent tracks** (for example CAGE and chromatin assays)
+        
+        **Attribution:** MAGI was developed by Dan Ofer, Stav Zok, and Michal Linial. 
+        
+        **This web app** extends MAGI with multi-species support: 15+ animals and 6 plants can be scored via Ensembl REST APIs for on-the-fly sequence fetching. Human remains the primary supported species with full BigWig and MANE transcript annotation.
+        """,
+            elem_classes="variant-header",
+        )
+        gr.Markdown(_status_line)
+
+        with gr.Tabs():
+            # ===================================================================
+            # TAB 1: SINGLE VARIANT
+            # ===================================================================
+            with gr.Tab("🔬 Single Variant Analysis"):
+                gr.Markdown("""
+                ### Manual Variant Input
+                Enter a single variant for detailed scoring and interpretation.
+                Human defaults to **GRCh38/hg38**. For non-human species, coordinates should match the selected species assembly available through Ensembl.
+                """)
+
+                with gr.Row():
+                    with gr.Column(scale=1):
+                        species_input = gr.Dropdown(
+                            choices=_species_choices,
+                            label="Species",
+                            value="human",
+                        )
+                        chrom_input = gr.Textbox(
+                            label="Chromosome",
+                            value="chr17",
+                            max_lines=1,
+                        )
+                        gr.Markdown(
+                            "*Human: `chr1`–`chr22`, `chrX`, `chrY`, `chrM`. "
+                            "Non-human: bare names (e.g., `1`, `X`, `MT`) or with `chr` prefix. "
+                            "Coordinates should match your species' current Ensembl assembly.*"
+                        )
+                        pos_input = gr.Number(
+                            label="Position (1-based)",
+                            value=7675088,
+                            precision=0,
+                        )
+                    with gr.Column(scale=1):
+                        ref_input = gr.Textbox(
+                            label="Reference Allele",
+                            value="C",
+                            max_lines=1,
+                        )
+                        alt_input = gr.Textbox(
+                            label="Alternate Allele",
+                            value="T",
+                            max_lines=1,
+                        )
+
+                predict_btn = gr.Button(
+                    "🚀 Predict Impact", variant="primary", size="lg"
+                )
+
+                with gr.Accordion("📋 Example Variants", open=False):
+                    gr.Examples(
+                        examples=[
+                            [
+                                "human",
+                                "chr17",
+                                7675088,
+                                "C",
+                                "T",
+                            ],  # TP53 R175H — Pathogenic missense, most common cancer driver mutation
+                            [
+                                "human",
+                                "chr7",
+                                117559593,
+                                "ATCT",
+                                "A",
+                            ],  # CFTR F508del — Pathogenic 3-bp deletion, ~70% of cystic fibrosis alleles
+                            [
+                                "human",
+                                "chr13",
+                                32332771,
+                                "AGAGA",
+                                "AGA",
+                            ],  # BRCA2 c.5946delT — Pathogenic frameshift, hereditary breast/ovarian cancer
+                            [
+                                "human",
+                                "chr11",
+                                5227002,
+                                "T",
+                                "A",
+                            ],  # HBB E6V (rs334) — Pathogenic missense, causes sickle cell disease
+                            [
+                                "human",
+                                "chr17",
+                                43092418,
+                                "T",
+                                "C",
+                            ],  # BRCA1 c.3113A>G (rs16941) — Benign synonymous variant
+                        ],
+                        inputs=[
+                            species_input,
+                            chrom_input,
+                            pos_input,
+                            ref_input,
+                            alt_input,
+                        ],
+                        label="Click to load pre-filled examples",
+                    )
+
+                gr.Markdown("---")
+                gr.Markdown("### Results")
+
+                with gr.Row():
+                    with gr.Column(scale=1):
+                        summary_output = gr.Markdown()
+                    with gr.Column(scale=1):
+                        interpretation_output = gr.Markdown()
+
+                with gr.Row():
+                    with gr.Column(scale=2):
+                        top_table_output = gr.DataFrame(
+                            label="Top ranked tracks (BED + BigWig, ordered by |Δ|)",
+                            wrap=True,
+                            max_height=600,
+                        )
+                    with gr.Column(scale=3):
+                        fingerprint_output = gr.Plot(label="Top signals bar chart")
+
+                with gr.Accordion("📥 Download Results", open=False):
+                    download_single_output = gr.File(label="Download CSV")
+                    download_tracks_output = gr.File(
+                        label="Download Track PNG (current zoom/filter)"
+                    )
+
+                ranked_state = gr.State(value=None)
+
+                with gr.Accordion("🔬 Region Track View", open=True):
+                    gr.Markdown(
+                        "*Reference (dashed gray) vs alternate signal for the top disrupted tracks. "
+                        "Red shading = gain of function, blue shading = loss of function. "
+                        "The slider maximum updates to the available track span for the current prediction.*"
+                    )
+                    with gr.Row(equal_height=True):
+                        zoom_slider = gr.Slider(
+                            minimum=MIN_ZOOM_BP,
+                            maximum=INITIAL_ZOOM_MAX_BP,
+                            value=DEFAULT_ZOOM_BP,
+                            step=8,
+                            label="Zoom radius (bp around variant)",
+                            info="Updates after each prediction to the exact available track radius",
+                            scale=3,
+                        )
+                        track_filter_dd = gr.Dropdown(
+                            choices=["All", "BED", "BigWig"],
+                            value="All",
+                            label="Track type filter",
+                            scale=1,
+                        )
+                    region_tracks_output = gr.Plot(label="Region Track View")
+
+                with gr.Accordion("📊 All BED Elements", open=False):
+                    bed_table_output = gr.DataFrame(
+                        wrap=True,
+                    )
+
+                with gr.Accordion("📋 Sequence Model Metrics", open=True):
+                    mlm_output = gr.Markdown()
+
+                # Connect button
+                predict_btn.click(
+                    fn=predict_single_variant_wrapper,
+                    inputs=[
+                        species_input,
+                        chrom_input,
+                        pos_input,
+                        ref_input,
+                        alt_input,
+                        zoom_slider,
+                    ],
+                    outputs=[
+                        summary_output,
+                        interpretation_output,
+                        top_table_output,
+                        fingerprint_output,
+                        region_tracks_output,
+                        download_single_output,
+                        download_tracks_output,
+                        bed_table_output,
+                        mlm_output,
+                        zoom_slider,
+                        ranked_state,
+                    ],
+                )
+
+                # Zoom slider re-renders region tracks on release (not during drag)
+                zoom_slider.release(
+                    fn=_update_region_zoom,
+                    inputs=[zoom_slider, ranked_state, track_filter_dd],
+                    outputs=[region_tracks_output, download_tracks_output],
+                )
+                track_filter_dd.change(
+                    fn=_update_region_zoom,
+                    inputs=[zoom_slider, ranked_state, track_filter_dd],
+                    outputs=[region_tracks_output, download_tracks_output],
+                )
+
+            # ===================================================================
+            # TAB 2: BATCH UPLOAD
+            # ===================================================================
+            with gr.Tab("📤 Batch Analysis"):
+                gr.Markdown("""
+                ### Batch Variant Upload
+                Upload a CSV file with multiple variants for batch processing.
+                
+                **Required columns:** `chrom`, `pos`, `ref`, `alt`  
+                **Format:** 1-based coordinates matching the selected species assembly  
+                **Limit:** Maximum 10 variants per batch  
+                **Chromosome naming:** For non-human, bare names (e.g., `1`, `X`, `MT`) or `chr`-prefixed both work.
+                
+                **Example CSV format (human):**
+                ```
+                chrom,pos,ref,alt
+                chr17,7675088,C,T
+                chr7,117559593,ATCT,A
+                chr13,32340300,G,A
+                ```
+                
+                **Example CSV format (non-human):**
+                ```
+                chrom,pos,ref,alt
+                2,50000000,A,T
+                X,25000000,C,G
+                ```
+                """)
+
+                file_input = gr.File(
+                    label="Upload CSV File",
+                    file_types=[".csv"],
+                )
+
+                batch_species_input = gr.Dropdown(
+                    choices=_species_choices,
+                    label="Species",
+                    value="human",
+                )
+
+                batch_btn = gr.Button("🚀 Process Batch", variant="primary", size="lg")
+
+                gr.Markdown("---")
+                gr.Markdown("### Results Preview")
+
+                preview_output = gr.DataFrame(
+                    label="Summary Table (first 10 rows)",
+                    wrap=True,
+                )
+
+                download_output = gr.File(
+                    label="Download Full Results (CSV)",
+                )
+
+                # Connect button
+                batch_btn.click(
+                    fn=predict_batch_wrapper,
+                    inputs=[batch_species_input, file_input],
+                    outputs=[preview_output, download_output],
+                )
+
+            # ===================================================================
+            # TAB 3: DOCUMENTATION
+            # ===================================================================
+            with gr.Tab("📖 Documentation"):
+                gr.Markdown("""
+                ## About MAGI
+                
+                MAGI is a variant interpretation workflow that uses a Genomic foundation model (NTv3) for sequence scoring and adds:
+                - gene and region annotation from MANE Select transcripts
+                - ranking of the strongest BED and BigWig changes
+                - a compact rule-based text summary
+                - a baseline-derived MAGI score (`Global_z_sum_log`)
+
+                ### Model configuration
+                - Current model: `InstaDeepAI/NTv3_650M_post`
+                - Current sequence window: **32 kb** (`CONTEXT_LEN = 32768`)
+                - Region Track View zoom is capped by the available NTv3 track-profile span for the current prediction
+                - Sequence source:
+                  - **Human:** local `hg38.fa` when available, otherwise UCSC REST fallback
+                  - **Non-human animals:** Ensembl REST API
+                  - **Plants:** Ensembl Plants REST API (with Ensembl REST fallback)
+                
+                ### Output groups
+                
+                **BED outputs**
+                - Functional and structural elements supplied by the NTv3 model configuration
+                - Typically include coding, splice, promoter, UTR, exon, intron, and related annotations
+                
+                **BigWig outputs**
+                - A filtered subset of assay tracks such as histone marks, chromatin accessibility, and CAGE
+                - Used as contextual signals rather than direct mechanistic proof
+                - Currently reported for human only in this app
+                
+                **Sequence model metrics**
+                - `LLR` for SNPs
+                - `KL` divergence summaries
+                - log-probability differences
+                - embedding distance summaries for indels
+                
+                ### Gene Annotation
+                Variants are automatically annotated with:
+                - **Gene name** (from MANE Select RefSeq transcripts)
+                - **Region class:** CODING, SPLICE, UTR_5, UTR_3, PROMOTER, INTRONIC, GENIC_OTHER, OTHER
+                - **Annotation flags:** overlap with coding sequence, splice sites, promoters, UTRs, and related transcript features
+                - Non-human species skip MANE transcript annotation and report sequence- and BED-based outputs only
+                - All supported species (animals and plants) can be scored via BED elements and MLM sequence-model features
+                
+                ### Impact Scoring
+                - **MAGI score (`Global_z_sum_log`)**: baseline-derived burden score across BED and BigWig deltas
+                - **Impact_Score_BED**: mean absolute value of the top 3 BED deltas
+                - **Impact_Score_BW**: mean absolute value of the top 10 BigWig deltas
+                
+                Larger values indicate stronger deviation from the reference prediction. The summary tier shown in the Variant Summary is currently derived from `Impact_Score_BED` only.
+
+                Positive Δ indicates an increase in the predicted signal; negative Δ indicates a decrease.
+
+                ### Interpretation panel
+                The rule-based interpretation panel summarizes the strongest ranked BED, BigWig, and sequence-model signals already shown elsewhere in the app. It is a compact heuristic summary, not a calibrated pathogenicity assessment.
+                
+                ### Limitations
+                1. **Predictions are computational** and require experimental follow-up.
+                2. No phasing information is used.
+                3. **Coordinates must match the species assembly available through Ensembl:**
+                   - Human: GRCh38/hg38
+                   - Other animals: latest Ensembl assembly for that species
+                   - Plants: latest Ensembl Plants assembly for that species
+                4. BigWig context tracks are currently human-only in this app.
+                5. MANE transcript annotation is human-only; non-human predictions show BED-level and sequence-model features only.
+                6. Baseline z-scores (`Global_z_sum_log`, `magi_baseline_stats.csv`) are derived from human variants and may be less meaningful for non-human species. Use with caution.
+                
+                ### Citation
+                If you use MAGI in your research, please cite the MAGI manuscript. If you find this webserver useful, please mention it! If you rely on the underlying foundation model, we suggest cite NTv3.
+                
+                ```
+                Ofer, D., Zok, S., & Linial, M. (2026). MAGI: Mechanistic Interpretation of 
+                Genetic Variants Consequences via Genomic Foundation Models.
+                ```
+                
+                ### External model credit
+                **NTv3:** Dalla-Torre, H., et al. (2025). Nucleotide Transformer: building and evaluating robust foundation models for human genomics. Nature Methods .
+                
+                <!-- ### Contact & Source Code
+                - **Model:** [InstaDeepAI/NTv3_650M_post](https://huggingface.co/InstaDeepAI/NTv3_650M_post)
+                - **Paper:** [bioRxiv](https://www.biorxiv.org/content/10.1101/2023.01.11.523679v2)
+                - **GitHub:** [Source repository](https://github.com/instadeepai/nucleotide-transformer) -->
+                
+                ---
+""")
+
+    return app
+
+
+# Build app at module level for Gradio/Spaces auto-detection
+app = build_interface()
+
+if __name__ == "__main__":
+    app.launch()

data/MANE_processed.csv

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:676a3042889bdbb362a83e93df4f744857d0eae631d688caf273fc731bd865c5
+size 246259927

data/MANE_processed.parquet

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a11043f7c63b55a88cbb123e07427328fdd94c93668f33adaa085c8a61addb7
+size 24626551

data/Promoter_processed.csv

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c4bd66cea689648115d771c5e90e140f7ab6a6555f8faf119c2c37398645f7d
+size 11044412

data/Promoter_processed.parquet

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:305a725c6d281f36581d7bda96054f2d7bd710be8cf5540be2beb5996af02950
+size 4374214

download_hg38.py

@@ -0,0 +1,66 @@
+#!/usr/bin/env python3
+"""
+Download and extract hg38 reference genome
+==========================================
+Downloads hg38.fa.gz from UCSC and extracts it to data/hg38.fa
+Only runs if hg38.fa doesn't already exist (3GB download).
+
+Usage:
+    python download_hg38.py
+"""
+import os
+import urllib.request
+import gzip
+import shutil
+from pathlib import Path
+
+# Define paths relative to this script
+BASE_DIR = Path(__file__).parent
+DATA_DIR = BASE_DIR / "data"
+FASTA_PATH = DATA_DIR / "hg38.fa"
+GZ_PATH = DATA_DIR / "hg38.fa.gz"
+URL = "https://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz"
+
+
+def download_and_extract():
+    """Download and extract hg38 reference genome."""
+    DATA_DIR.mkdir(parents=True, exist_ok=True)
+
+    if FASTA_PATH.exists():
+        size_mb = FASTA_PATH.stat().st_size / (1024 * 1024)
+        print(f"✅ '{FASTA_PATH}' already exists ({size_mb:.0f} MB). Skipping download.")
+        return
+
+    print("⬇️  Downloading hg38.fa.gz from UCSC (~3 GB)...")
+    print(f"   URL: {URL}")
+    print("   This may take 10-30 minutes depending on your connection.\n")
+
+    # Download with progress reporting
+    def reporthook(block_num, block_size, total_size):
+        downloaded = block_num * block_size
+        if total_size > 0:
+            percent = min(100, downloaded * 100 / total_size)
+            mb_downloaded = downloaded / (1024 * 1024)
+            mb_total = total_size / (1024 * 1024)
+            print(
+                f"\r   Progress: {percent:.1f}% ({mb_downloaded:.0f}/{mb_total:.0f} MB)",
+                end="",
+            )
+
+    urllib.request.urlretrieve(URL, GZ_PATH, reporthook=reporthook)
+    print("\n")
+
+    print(f"📦 Extracting to {FASTA_PATH}...")
+    with gzip.open(GZ_PATH, "rb") as f_in:
+        with open(FASTA_PATH, "wb") as f_out:
+            shutil.copyfileobj(f_in, f_out)
+
+    print("🧹 Cleaning up compressed file...")
+    GZ_PATH.unlink()
+
+    size_mb = FASTA_PATH.stat().st_size / (1024 * 1024)
+    print(f"✅ Done! hg38.fa extracted ({size_mb:.0f} MB)\n")
+
+
+if __name__ == "__main__":
+    download_and_extract()

examples/example_variants.csv

@@ -0,0 +1,6 @@
+chrom,pos,ref,alt
+chr17,7675088,C,T
+chr7,117559593,ATCT,A
+chr13,32332771,AGAGA,AGA
+chr11,5227002,T,A
+chr17,43092418,T,C

inference.py

@@ -0,0 +1,895 @@
+#!/usr/bin/env python3
+"""
+NTv3 inference module for the MAGI Gradio app.
+
+Adapted from the top-level inference.py pipeline for web deployment.
+
+Key features:
+- Uses local hg38.fa via pyfaidx when available
+- Loads the configured NTv3 model once and caches it across requests
+- Returns BED/BigWig deltas together with sequence-model metrics
+
+Usage:
+    from inference import predict_variants
+    results_df = predict_variants(variants_df)
+"""
+
+import os
+import time
+import warnings
+from pathlib import Path
+from typing import Optional, Tuple, List, Any, Dict, cast
+from functools import lru_cache
+
+import numpy as np
+import pandas as pd
+import torch
+import torch.nn.functional as F
+import requests
+from pyfaidx import Fasta
+from transformers import AutoModel, AutoTokenizer
+
+# ============================================================================
+# CONFIGURATION
+# ============================================================================
+MODEL_NAME = "InstaDeepAI/NTv3_650M_post"
+CONTEXT_LEN = 32 * 1024  # 32 kb sequence window
+USE_BED = True
+USE_BIGWIGS = True
+USE_KL_DIVERGENCE = True
+USE_EMBEDDINGS = True  # Useful for indels
+MLM_WINDOW = 3  # ±3 positions for embedding window
+
+# Paths (relative to this file)
+BASE_DIR = Path(__file__).parent
+DATA_DIR = BASE_DIR / "data"
+GENOME_FILE = DATA_DIR / "hg38.fa"
+GENOME_GZ_FILE = DATA_DIR / "hg38.fa.gz"
+METADATA_FILE = DATA_DIR / "functional_tracks_metadata_human.csv"
+UCSC_SEQUENCE_URL = "https://api.genome.ucsc.edu/getData/sequence"
+ENSEMBL_SEQUENCE_URL = "https://rest.ensembl.org/sequence/region"
+ENSEMBL_PLANTS_SEQUENCE_URL = "https://rest.plants.ensembl.org/sequence/region"
+FORCE_UCSC = os.environ.get("NTV3_FORCE_UCSC", "0") == "1"
+
+SUPPORTED_UI_SPECIES: Dict[str, str] = {
+    "human": "Human",
+    "mouse": "Mouse",
+    "rattus_norvegicus": "Rat",
+    "canis_lupus_familiaris": "Dog",
+    "felis_catus": "Cat",
+    "gallus_gallus": "Chicken",
+    "danio_rerio": "Zebrafish",
+    "gorilla_gorilla": "Gorilla",
+    "macaca_nemestrina": "Pig-tailed macaque",
+    "bison_bison_bison": "Bison",
+    "chinchilla_lanigera": "Chinchilla",
+    "serinus_canaria": "Canary",
+    "salmo_trutta": "Brown trout",
+    "tetraodon_nigroviridis": "Green spotted puffer",
+    "amphiprion_ocellaris": "Clownfish",
+    "arabidopsis_thaliana": "Arabidopsis (Thale cress)",
+    "oryza_sativa": "Rice",
+    "glycine_max": "Soybean",
+    "gossypium_hirsutum": "Cotton",
+    "triticum_aestivum": "Wheat",
+    "zea_mays": "Maize",
+}
+
+SPECIES_TO_ENSEMBL: Dict[str, str] = {
+    "human": "homo_sapiens",
+    "mouse": "mus_musculus",
+    "rattus_norvegicus": "rattus_norvegicus",
+    "canis_lupus_familiaris": "canis_lupus_familiaris",
+    "felis_catus": "felis_catus",
+    "gallus_gallus": "gallus_gallus",
+    "danio_rerio": "danio_rerio",
+    "gorilla_gorilla": "gorilla_gorilla",
+    "macaca_nemestrina": "macaca_nemestrina",
+    "bison_bison_bison": "bison_bison_bison",
+    "chinchilla_lanigera": "chinchilla_lanigera",
+    "serinus_canaria": "serinus_canaria",
+    "salmo_trutta": "salmo_trutta",
+    "tetraodon_nigroviridis": "tetraodon_nigroviridis",
+    "amphiprion_ocellaris": "amphiprion_ocellaris",
+    "arabidopsis_thaliana": "arabidopsis_thaliana",
+    "oryza_sativa": "oryza_sativa",
+    "glycine_max": "glycine_max",
+    "gossypium_hirsutum": "gossypium_hirsutum",
+    "triticum_aestivum": "triticum_aestivum",
+    "zea_mays": "zea_mays",
+}
+
+PLANT_SPECIES = {
+    "arabidopsis_thaliana",
+    "oryza_sativa",
+    "glycine_max",
+    "gossypium_hirsutum",
+    "triticum_aestivum",
+    "zea_mays",
+}
+
+# Global cache for model and genome
+_MODEL_CACHE: Dict[str, Any] = {
+    "model": None,
+    "tokenizer": None,
+    "genome": None,
+    "bed_names": None,
+    "bigwig_names": None,
+    "selected_bw_indices": None,
+    "nuc_token_map": None,
+    "human_id": None,
+    "species_to_token_id": None,
+    "sequence_source": "ucsc",
+}
+
+# Cache for the most recent track profiles (used by tracks.py for plotting)
+_LAST_TRACK_PROFILES: Dict[str, Any] = {}
+
+
+# ============================================================================
+# HELPER FUNCTIONS
+# ============================================================================
+def _normalize_ucsc_chrom(chrom: str) -> str:
+    """Normalize chromosome string for UCSC API (expects chr-prefixed names)."""
+    chrom = str(chrom).strip()
+    if not chrom:
+        return chrom
+    return chrom if chrom.startswith("chr") else f"chr{chrom}"
+
+
+def _normalize_ensembl_chrom(chrom: str) -> str:
+    """Normalize chromosome string for Ensembl REST (expects no chr prefix)."""
+    clean = str(chrom).strip()
+    if clean.lower().startswith("chr"):
+        clean = clean[3:]
+    if clean.upper() == "M":
+        return "MT"
+    return clean
+
+
+@lru_cache(maxsize=2048)
+def _fetch_ucsc_window(chrom: str, start: int, end: int) -> Optional[str]:
+    """Fetch sequence window from UCSC API with in-session caching."""
+    try:
+        response = requests.get(
+            UCSC_SEQUENCE_URL,
+            params={
+                "genome": "hg38",
+                "chrom": chrom,
+                "start": int(start),
+                "end": int(end),
+            },
+            timeout=10,
+        )
+        response.raise_for_status()
+        payload = response.json()
+        dna = payload.get("dna", "")
+        if not dna:
+            return None
+        return str(dna).upper()
+    except Exception:
+        return None
+
+
+@lru_cache(maxsize=2048)
+def _fetch_ensembl_window(
+    species: str, chrom: str, start: int, end: int
+) -> Optional[str]:
+    """Fetch sequence window from Ensembl REST for non-human species.
+
+    For plant species, first tries Ensembl REST, then falls back to Ensembl Plants REST.
+    """
+    ensembl_species = SPECIES_TO_ENSEMBL.get(species)
+    if not ensembl_species:
+        return None
+
+    clean_chrom = _normalize_ensembl_chrom(chrom)
+    if not clean_chrom:
+        return None
+
+    start_1based = max(1, int(start) + 1)
+    end_1based = max(start_1based, int(end))
+
+    is_plant = species in PLANT_SPECIES
+    urls = []
+
+    if is_plant:
+        urls.append(
+            f"{ENSEMBL_PLANTS_SEQUENCE_URL}/{ensembl_species}/"
+            f"{clean_chrom}:{start_1based}..{end_1based}"
+        )
+        urls.append(
+            f"{ENSEMBL_SEQUENCE_URL}/{ensembl_species}/"
+            f"{clean_chrom}:{start_1based}..{end_1based}"
+        )
+    else:
+        urls.append(
+            f"{ENSEMBL_SEQUENCE_URL}/{ensembl_species}/"
+            f"{clean_chrom}:{start_1based}..{end_1based}"
+        )
+
+    for url in urls:
+        try:
+            response = requests.get(
+                url,
+                headers={"Content-Type": "text/plain", "Accept": "text/plain"},
+                timeout=15,
+            )
+            response.raise_for_status()
+            dna = response.text.strip()
+            if dna:
+                return dna.upper()
+        except Exception:
+            continue
+
+    return None
+
+
+def _fetch_sequence_from_local(
+    genome: Fasta, chrom: str, start: int, end: int
+) -> Optional[str]:
+    """Fetch sequence window from local pyfaidx genome."""
+    query_chrom = chrom
+    if query_chrom not in genome:
+        query_chrom = (
+            query_chrom if query_chrom.startswith("chr") else f"chr{query_chrom}"
+        )
+    if query_chrom not in genome:
+        query_chrom = (
+            query_chrom.replace("chr", "")
+            if "chr" in query_chrom
+            else f"chr{query_chrom}"
+        )
+    if query_chrom not in genome:
+        return None
+
+    try:
+        return str(genome[query_chrom][start:end]).upper()
+    except Exception:
+        return None
+
+
+def get_genomic_sequence(
+    genome: Optional[Fasta],
+    chrom: str,
+    pos: int,
+    ref: str,
+    alt: str,
+    context_len: int = 4096,
+    species: str = "human",
+) -> Tuple[Optional[str], Optional[str], Optional[int]]:
+    """
+    Extract Ref/Alt sequences centered at variant position.
+
+    Returns:
+        (ref_seq, alt_seq, variant_center_idx) or (None, None, None) on error
+    """
+    variant_idx = pos - 1  # Convert to 0-based
+    half = context_len // 2
+    start = max(0, variant_idx - half)
+    end = variant_idx + half
+
+    ref_seq = None
+    if species == "human":
+        if genome is not None:
+            ref_seq = _fetch_sequence_from_local(genome, str(chrom), start, end)
+
+        if ref_seq is None:
+            ucsc_chrom = _normalize_ucsc_chrom(str(chrom))
+            ref_seq = _fetch_ucsc_window(ucsc_chrom, start, end)
+            if ref_seq is None:
+                warnings.warn(
+                    f"Failed to fetch sequence for {chrom}:{pos} from local genome and UCSC API"
+                )
+                return None, None, None
+    else:
+        ref_seq = _fetch_ensembl_window(species, str(chrom), start, end)
+        if ref_seq is None:
+            warnings.warn(
+                f"Failed to fetch sequence for {species} {chrom}:{pos} from Ensembl REST API"
+            )
+            return None, None, None
+
+    if not ref_seq:
+        return None, None, None
+
+    center = variant_idx - start
+    center = max(0, min(center, len(ref_seq) - 1))
+
+    # Validate REF allele
+    ref_end = min(center + len(ref), len(ref_seq))
+    actual_ref = ref_seq[center:ref_end]
+    if actual_ref.upper() != ref.upper():
+        warnings.warn(
+            f"REF mismatch at {chrom}:{pos}: expected '{ref}' but genome has '{actual_ref}'"
+        )
+
+    # Build ALT sequence
+    alt_seq = ref_seq[:center] + alt + ref_seq[ref_end:]
+
+    # Crop to equal length
+    target_len = min(len(ref_seq), len(alt_seq))
+    center = min(center, target_len - 1)
+
+    return ref_seq[:target_len], alt_seq[:target_len], center
+
+
+def get_track_indices(
+    bigwig_names: List[str], metadata_file: Path
+) -> Tuple[List[int], List[str]]:
+    """Filter BigWig tracks to functional subset using metadata."""
+    if not metadata_file.exists():
+        print(f"⚠️  Metadata not found, using all {len(bigwig_names)} tracks")
+        return list(range(len(bigwig_names))), bigwig_names
+
+    metadata = pd.read_csv(metadata_file)
+
+    key_marks = {
+        "H3K4me3",
+        "H3K27ac",
+        "H3K36me3",
+        "H3K27me3",
+        "H3K9me3",
+        "H3K4me1",
+        "H3K9ac",
+    }
+    sel_idx, sel_names = [], []
+
+    for i, tid in enumerate(bigwig_names):
+        rows = metadata[metadata["file_id"] == tid]
+        if rows.empty:
+            continue
+        r = rows.iloc[0]
+        assay = str(r.get("assay", ""))
+        target = str(r.get("experiment_target", ""))
+        dataset = str(r.get("dataset", ""))
+
+        keep = (
+            (
+                "ChIP" in assay
+                and any(m in target for m in key_marks)
+                and dataset in ("encode_v3", "geo")
+            )
+            or (("ATAC" in assay or "DNase" in assay) and dataset == "encode_v3")
+            or dataset == "fantom5"
+            or "CAGE" in assay
+            or dataset == "gtex"
+        )
+        if keep:
+            sel_idx.append(i)
+            sel_names.append(tid)
+
+    print(f"📊 Selected {len(sel_idx)}/{len(bigwig_names)} BigWig tracks")
+    return sel_idx, sel_names
+
+
+def build_nuc_token_map(tokenizer):
+    """Pre-compute nucleotide -> token ID mapping for LLR calculation."""
+    return {
+        nuc: tokenizer(nuc, add_special_tokens=False)["input_ids"][0]
+        for nuc in "ACGTN"
+        if tokenizer(nuc, add_special_tokens=False)["input_ids"]
+    }
+
+
+def to_track_probabilities(track_values):
+    """
+    Convert NTv3 track logits to probabilities via sigmoid.
+
+    BED tracks: shape (B, L', 21, 2) → extract positive class [..., 1]
+    BigWig tracks: shape (B, L', N) → apply sigmoid directly
+    """
+    if track_values is None:
+        return None
+    if track_values.shape[-1] == 2:  # Binary classification (BED)
+        return torch.sigmoid(track_values[..., 1])
+    return torch.sigmoid(track_values)
+
+
+def compute_mlm_features(
+    out_ref,
+    out_alt,
+    ref_seq: str,
+    alt_seq: str,
+    idx: int,
+    variant_center: int,
+    ref_allele: str,
+    alt_allele: str,
+    nuc_token_map: dict,
+    use_kl: bool = True,
+    use_embeddings: bool = False,
+    window: int = 50,
+) -> dict:
+    """
+    Compute MLM language model features (LLR, KL divergence, log-probs, embeddings).
+
+    Returns dict with keys:
+        - LLR, MLM_Prior, MLM_Delta (SNPs only)
+        - MLM_KL_mean, MLM_KL_max
+        - MLM_logprob_ref, MLM_logprob_alt, MLM_logprob_delta
+        - REF_5mer, ALT_5mer
+        - EMB_* (if use_embeddings=True)
+    """
+    feat = {}
+    ref_logits = out_ref.logits[idx]
+    alt_logits = out_alt.logits[idx]
+    seq_len = min(ref_logits.shape[0], alt_logits.shape[0])
+
+    # Determine variant span (1 for SNP, max allele length for indel)
+    variant_span = max(1, len(ref_allele), len(alt_allele))
+    variant_center = int(max(0, min(variant_center, seq_len - 1)))
+
+    # KL window: cover exactly the variant span
+    kl_ws = variant_center
+    kl_we = min(seq_len, variant_center + variant_span)
+
+    # --- LLR (single-nucleotide substitutions only) ---
+    is_snp = len(ref_allele) == 1 and len(alt_allele) == 1
+    if is_snp and ref_allele in nuc_token_map and alt_allele in nuc_token_map:
+        probs = F.softmax(ref_logits[variant_center], dim=-1)
+        rp = float(probs[nuc_token_map[ref_allele]].cpu())
+        ap = float(probs[nuc_token_map[alt_allele]].cpu())
+        feat["LLR"] = np.log(ap / (rp + 1e-10) + 1e-10)
+        feat["MLM_Prior"] = rp
+        feat["MLM_Delta"] = ap - rp
+    else:
+        feat["LLR"] = feat["MLM_Prior"] = feat["MLM_Delta"] = np.nan
+
+    # --- Context k-mers ---
+    for pf, seq in [("REF", ref_seq), ("ALT", alt_seq)]:
+        if len(seq) >= variant_center + 3:
+            feat[f"{pf}_5mer"] = seq[max(0, variant_center - 2) : variant_center + 3]
+        else:
+            feat[f"{pf}_5mer"] = "NNNNN"
+
+    # --- KL divergence + log-prob ---
+    if use_kl:
+        if kl_we <= kl_ws:
+            feat["MLM_KL_mean"] = np.nan
+            feat["MLM_KL_max"] = np.nan
+            feat["MLM_logprob_ref"] = np.nan
+            feat["MLM_logprob_alt"] = np.nan
+            feat["MLM_logprob_delta"] = np.nan
+            return feat
+
+        rp_w = F.softmax(ref_logits[kl_ws:kl_we], dim=-1)
+        ap_w = F.softmax(alt_logits[kl_ws:kl_we], dim=-1)
+        kl = F.kl_div(rp_w.log(), ap_w, reduction="none", log_target=False).sum(-1)
+        feat["MLM_KL_mean"] = float(kl.mean().cpu())
+        feat["MLM_KL_max"] = float(kl.max().cpu())
+
+        # Log-probs
+        rlp = [
+            float(torch.log(rp_w[p - kl_ws, nuc_token_map[ref_seq[p]]] + 1e-10).cpu())
+            for p in range(kl_ws, kl_we)
+            if p < len(ref_seq) and ref_seq[p] in nuc_token_map
+        ]
+        alp = [
+            float(torch.log(ap_w[p - kl_ws, nuc_token_map[alt_seq[p]]] + 1e-10).cpu())
+            for p in range(kl_ws, kl_we)
+            if p < len(alt_seq) and alt_seq[p] in nuc_token_map
+        ]
+        feat["MLM_logprob_ref"] = np.mean(rlp) if rlp else np.nan
+        feat["MLM_logprob_alt"] = np.mean(alp) if alp else np.nan
+        feat["MLM_logprob_delta"] = feat["MLM_logprob_alt"] - feat["MLM_logprob_ref"]
+
+    # --- Embedding distances ---
+    if use_embeddings:
+        hr = getattr(out_ref, "last_hidden_state", None)
+        ha = getattr(out_alt, "last_hidden_state", None)
+        if hr is not None and ha is not None:
+            ws = max(0, variant_center - window)
+            we = min(seq_len, variant_center + window)
+            hr, ha = hr[idx, ws:we, :], ha[idx, ws:we, :]
+            hrm, ham = hr.mean(0), ha.mean(0)
+            feat["EMB_cosine_dist"] = float(
+                1.0 - F.cosine_similarity(hrm.unsqueeze(0), ham.unsqueeze(0)).cpu()
+            )
+            feat["EMB_l2_dist"] = float(torch.norm(hrm - ham, p=2).cpu())
+            per_pos = torch.norm(hr - ha, p=2, dim=-1)
+            feat["EMB_max_pos_dist"] = float(per_pos.max().cpu())
+            feat["EMB_mean_pos_dist"] = float(per_pos.mean().cpu())
+        else:
+            for k in (
+                "EMB_cosine_dist",
+                "EMB_l2_dist",
+                "EMB_max_pos_dist",
+                "EMB_mean_pos_dist",
+            ):
+                feat[k] = np.nan
+
+    return feat
+
+
+# ============================================================================
+# MODEL LOADING AND CACHING
+# ============================================================================
+def load_model_and_resources(device: str = "cuda"):
+    """
+    Load model, tokenizer, genome, and metadata once at startup.
+    Caches everything in global _MODEL_CACHE.
+
+    Handles:
+    - HF_TOKEN authentication for gated models
+    - bf16 mixed precision when supported by GPU
+    """
+    if _MODEL_CACHE["model"] is not None:
+        return  # Already loaded
+
+    # Get HF token from environment (needed for gated models like NTv3)
+    hf_token = os.environ.get("HF_TOKEN")
+    if not hf_token:
+        print("⚠️  HF_TOKEN not set; model loading may fail if the model is gated")
+
+    print(f"🧠 Loading NTv3 model '{MODEL_NAME}' on {device}...")
+
+    # Prepare bf16 kwargs if GPU supports bfloat16
+    bf16_kwargs = {}
+    if device == "cuda" and torch.cuda.is_bf16_supported():
+        print("💾 Enabling bf16 mixed precision to reduce memory usage")
+        bf16_kwargs = {
+            "stem_compute_dtype": "bfloat16",
+            "down_convolution_compute_dtype": "bfloat16",
+            "transformer_qkvo_compute_dtype": "bfloat16",
+            "transformer_ffn_compute_dtype": "bfloat16",
+            "up_convolution_compute_dtype": "bfloat16",
+            "modulation_compute_dtype": "bfloat16",
+        }
+
+    try:
+        model = (
+            AutoModel.from_pretrained(
+                MODEL_NAME, trust_remote_code=True, token=hf_token, **bf16_kwargs
+            )
+            .to(device)
+            .eval()
+        )
+        tokenizer = AutoTokenizer.from_pretrained(
+            MODEL_NAME, trust_remote_code=True, token=hf_token
+        )
+    except Exception as e:
+        error_msg = f"Failed to load model: {str(e)}"
+        if "gated" in str(e).lower() or "401" in str(e):
+            error_msg += "\n\nThe NTv3_650M_post model is gated. You need to:\n1. Accept the model terms at https://huggingface.co/InstaDeepAI/NTv3_650M_post\n2. Set HF_TOKEN as an environment variable with your HuggingFace API token"
+        raise RuntimeError(error_msg) from e
+
+    species_to_token_id = dict(getattr(model.config, "species_to_token_id", {}) or {})
+
+    # Extract human species ID
+    try:
+        human_id = model.encode_species(["human"]).item()
+    except (AttributeError, TypeError):
+        human_id = species_to_token_id.get("human", 6)
+
+    # Extract BED/BigWig names
+    bed_names = []
+    if USE_BED:
+        for attr in ("bed_elements_names", "bed_tracks", "bed_track_labels"):
+            if hasattr(model.config, attr):
+                bed_names = getattr(model.config, attr)
+                break
+    bw_names = []
+    if USE_BIGWIGS and hasattr(model.config, "bigwigs_per_species"):
+        bw_names = model.config.bigwigs_per_species.get("human", [])
+
+    # Filter BigWig tracks
+    selected_bw_indices, selected_bw_names = get_track_indices(bw_names, METADATA_FILE)
+
+    # Load genome optionally (for fast local lookups); otherwise UCSC fallback
+    genome = None
+    sequence_source = "ucsc"
+    if FORCE_UCSC:
+        print("⚠️  NTV3_FORCE_UCSC=1 set; using UCSC API for sequence retrieval")
+    elif GENOME_FILE.exists():
+        print(f"🧬 Loading local reference genome from {GENOME_FILE}...")
+        genome = Fasta(str(GENOME_FILE))
+        sequence_source = "local+ucsc-fallback"
+    elif GENOME_GZ_FILE.exists():
+        print(
+            f"⚠️  Found compressed genome at {GENOME_GZ_FILE}; using UCSC API (decompress to enable local fast path)"
+        )
+    else:
+        print("⚠️  No local hg38.fa found; using UCSC API for sequence retrieval")
+
+    # Build token map
+    nuc_token_map = build_nuc_token_map(tokenizer)
+
+    # Cache everything
+    _MODEL_CACHE.update(
+        {
+            "model": model,
+            "tokenizer": tokenizer,
+            "genome": genome,
+            "bed_names": bed_names,
+            "bigwig_names": bw_names,
+            "selected_bw_indices": selected_bw_indices,
+            "nuc_token_map": nuc_token_map,
+            "human_id": human_id,
+            "species_to_token_id": species_to_token_id,
+            "sequence_source": sequence_source,
+        }
+    )
+
+    print(
+        f"✅ Model loaded: {len(bed_names)} BED, {len(selected_bw_indices)} BigWig tracks | sequence source: {sequence_source}"
+    )
+
+
+# ============================================================================
+# INFERENCE
+# ============================================================================
+def predict_variants(
+    df: pd.DataFrame,
+    device: str = "cuda",
+    species: str = "human",
+    cache_profiles: bool = True,
+) -> pd.DataFrame:
+    """
+    Run NTv3 inference on variants DataFrame.
+
+    Args:
+        df: DataFrame with columns ['chrom', 'pos', 'ref', 'alt']
+        device: 'cuda' or 'cpu'
+
+    Returns:
+        DataFrame with original columns plus:
+        - D_BED_* (21 BED element deltas)
+        - REF_BED_* (21 BED element ref probabilities)
+        - D_BW_* (filtered BigWig deltas)
+        - REF_BW_* (filtered BigWig ref probabilities)
+        - LLR, MLM_Prior, MLM_Delta, MLM_KL_mean, MLM_KL_max
+        - MLM_logprob_ref, MLM_logprob_alt, MLM_logprob_delta
+        - REF_5mer, ALT_5mer
+        - EMB_* (if embeddings enabled)
+        - indel_size
+    """
+    # Load model if not already loaded
+    if _MODEL_CACHE["model"] is None:
+        load_model_and_resources(device)
+
+    model = _MODEL_CACHE.get("model")
+    tokenizer = _MODEL_CACHE.get("tokenizer")
+    genome = cast(Optional[Fasta], _MODEL_CACHE.get("genome"))
+    bed_names = cast(List[str], _MODEL_CACHE.get("bed_names") or [])
+    bigwig_names = cast(List[str], _MODEL_CACHE.get("bigwig_names") or [])
+    selected_bw_indices = cast(List[int], _MODEL_CACHE.get("selected_bw_indices") or [])
+    nuc_token_map = cast(dict, _MODEL_CACHE.get("nuc_token_map") or {})
+    species_to_token_id = cast(
+        Dict[str, int], _MODEL_CACHE.get("species_to_token_id") or {}
+    )
+
+    species = str(species).strip()
+    species_id = species_to_token_id.get(species)
+    active_bigwig_names = bigwig_names if species == "human" else []
+    active_bw_indices = selected_bw_indices if species == "human" else []
+
+    if model is None or tokenizer is None or species_id is None:
+        raise RuntimeError("Model resources are not initialized correctly")
+
+    # Validate input
+    required_cols = {"chrom", "pos", "ref", "alt"}
+    missing = required_cols - set(df.columns)
+    if missing:
+        raise ValueError(f"Missing required columns: {missing}")
+
+    # Clean data
+    df = df.copy()
+    df = df[df["ref"].notna() & df["alt"].notna()].reset_index(drop=True)
+
+    results = []
+
+    for idx, row in df.iterrows():
+        # Get sequences
+        ref_seq, alt_seq, vcenter = get_genomic_sequence(
+            genome,
+            row["chrom"],
+            row["pos"],
+            row["ref"],
+            row["alt"],
+            CONTEXT_LEN,
+            species=species,
+        )
+
+        if ref_seq is None or alt_seq is None or vcenter is None:
+            # Failed to fetch sequence - return NaN results
+            res = {c: row[c] for c in df.columns}
+            res["indel_size"] = len(str(row["alt"])) - len(str(row["ref"]))
+            for nm in bed_names:
+                res[f"REF_BED_{nm}"] = np.nan
+                res[f"D_BED_{nm}"] = np.nan
+            for gi in active_bw_indices:
+                res[f"REF_BW_{active_bigwig_names[gi]}"] = np.nan
+                res[f"D_BW_{active_bigwig_names[gi]}"] = np.nan
+            for k in (
+                "LLR",
+                "MLM_Prior",
+                "MLM_Delta",
+                "MLM_KL_mean",
+                "MLM_KL_max",
+                "MLM_logprob_ref",
+                "MLM_logprob_alt",
+                "MLM_logprob_delta",
+                "REF_5mer",
+                "ALT_5mer",
+            ):
+                res[k] = np.nan if k not in ("REF_5mer", "ALT_5mer") else "NNNNN"
+            results.append(res)
+            continue
+
+        # Tokenize
+        tok_kw = dict(
+            return_tensors="pt",
+            padding="max_length",
+            max_length=CONTEXT_LEN,
+            truncation=True,
+            add_special_tokens=False,
+            pad_to_multiple_of=128,
+        )
+        inp_r = tokenizer([ref_seq], **tok_kw).to(device)
+        inp_a = tokenizer([alt_seq], **tok_kw).to(device)
+
+        # Forward pass
+        with torch.no_grad():
+            sp = torch.tensor([species_id], device=device)
+            if USE_EMBEDDINGS:
+                try:
+                    out_r = model(**inp_r, species_ids=sp, output_hidden_states=True)
+                    out_a = model(**inp_a, species_ids=sp, output_hidden_states=True)
+                    for o in (out_r, out_a):
+                        if getattr(o, "last_hidden_state", None) is None and hasattr(
+                            o, "hidden_states"
+                        ):
+                            o.last_hidden_state = o.hidden_states[-1]
+                except TypeError:
+                    out_r = model(**inp_r, species_ids=sp)
+                    out_a = model(**inp_a, species_ids=sp)
+            else:
+                out_r = model(**inp_r, species_ids=sp)
+                out_a = model(**inp_a, species_ids=sp)
+
+        # Build result
+        res = {c: row[c] for c in df.columns}
+        ref_allele = str(row["ref"])
+        alt_allele = str(row["alt"])
+        res["indel_size"] = len(alt_allele) - len(ref_allele)
+        variant_span = max(1, len(ref_allele), len(alt_allele))
+        in_len = int(inp_r["input_ids"].shape[1])
+
+        # === BED tracks ===
+        bed_r = getattr(out_r, "bed_tracks_logits", None)
+        bed_a = getattr(out_a, "bed_tracks_logits", None)
+        if USE_BED and bed_r is not None and bed_a is not None:
+            bed_r_probs = to_track_probabilities(bed_r[0])
+            bed_a_probs = to_track_probabilities(bed_a[0])
+            track_len = int(bed_r_probs.shape[0])
+            track_start = max(0, (in_len - track_len) // 2)
+            bed_pos = vcenter - track_start
+            if 0 <= bed_pos < track_len:
+                be = min(bed_pos + variant_span, track_len)
+                br = bed_r_probs[bed_pos:be].mean(0).float().cpu().numpy()
+                ba = bed_a_probs[bed_pos:be].mean(0).float().cpu().numpy()
+                for j, nm in enumerate(bed_names):
+                    res[f"REF_BED_{nm}"] = float(br[j])
+                    res[f"D_BED_{nm}"] = float(ba[j] - br[j])
+            else:
+                for nm in bed_names:
+                    res[f"REF_BED_{nm}"] = np.nan
+                    res[f"D_BED_{nm}"] = np.nan
+        else:
+            for nm in bed_names:
+                res[f"REF_BED_{nm}"] = np.nan
+                res[f"D_BED_{nm}"] = np.nan
+
+        # === BigWig tracks ===
+        bw_r = getattr(out_r, "bigwig_tracks_logits", None)
+        bw_a = getattr(out_a, "bigwig_tracks_logits", None)
+        if species == "human" and USE_BIGWIGS and bw_r is not None and bw_a is not None:
+            bw_r_probs = to_track_probabilities(bw_r[0])
+            bw_a_probs = to_track_probabilities(bw_a[0])
+            track_len = int(bw_r_probs.shape[0])
+            track_start = max(0, (in_len - track_len) // 2)
+            bw_pos = vcenter - track_start
+            if 0 <= bw_pos < track_len:
+                bwe = min(bw_pos + variant_span, track_len)
+                bwr = bw_r_probs[bw_pos:bwe].mean(0).float().cpu().numpy()
+                bwa = bw_a_probs[bw_pos:bwe].mean(0).float().cpu().numpy()
+                for gi in active_bw_indices:
+                    res[f"REF_BW_{active_bigwig_names[gi]}"] = float(bwr[gi])
+                    res[f"D_BW_{active_bigwig_names[gi]}"] = float(bwa[gi] - bwr[gi])
+            else:
+                for gi in active_bw_indices:
+                    res[f"REF_BW_{active_bigwig_names[gi]}"] = np.nan
+                    res[f"D_BW_{active_bigwig_names[gi]}"] = np.nan
+        else:
+            for gi in active_bw_indices:
+                res[f"REF_BW_{active_bigwig_names[gi]}"] = np.nan
+                res[f"D_BW_{active_bigwig_names[gi]}"] = np.nan
+
+        # === MLM features ===
+        res.update(
+            compute_mlm_features(
+                out_r,
+                out_a,
+                ref_seq,
+                alt_seq,
+                0,
+                vcenter,
+                ref_allele,
+                alt_allele,
+                nuc_token_map,
+                use_kl=USE_KL_DIVERGENCE,
+                use_embeddings=USE_EMBEDDINGS,
+                window=MLM_WINDOW,
+            )
+        )
+
+        # === Cache full track profiles for plotting (single-variant mode only) ===
+        if cache_profiles:
+            _cache_track_profiles(
+                row,
+                vcenter,
+                in_len,
+                bed_names,
+                active_bigwig_names,
+                active_bw_indices,
+                bed_r,
+                bed_a,
+                bw_r,
+                bw_a,
+        )
+
+        results.append(res)
+
+    return pd.DataFrame(results)
+
+
+def _cache_track_profiles(
+    row,
+    vcenter,
+    in_len,
+    bed_names,
+    bigwig_names,
+    selected_bw_indices,
+    bed_r_logits,
+    bed_a_logits,
+    bw_r_logits,
+    bw_a_logits,
+):
+    """Cache the most recent variant's full track-length logit profiles for plotting."""
+    global _LAST_TRACK_PROFILES
+
+    profiles: Dict[str, Any] = {
+        "chrom": str(row["chrom"]),
+        "pos": int(row["pos"]),
+        "ref": str(row["ref"]),
+        "alt": str(row["alt"]),
+        "variant_center": vcenter,
+        "input_len": in_len,
+        "bed_names": list(bed_names),
+        "bigwig_names": list(bigwig_names),
+        "selected_bw_indices": list(selected_bw_indices),
+    }
+
+    # BED profiles: convert to probabilities and store as numpy (L, 21)
+    if bed_r_logits is not None and bed_a_logits is not None:
+        bed_ref_probs = to_track_probabilities(bed_r_logits[0]).float().cpu().numpy()
+        bed_alt_probs = to_track_probabilities(bed_a_logits[0]).float().cpu().numpy()
+        profiles["bed_ref"] = bed_ref_probs
+        profiles["bed_alt"] = bed_alt_probs
+        profiles["bed_track_len"] = bed_ref_probs.shape[0]
+        profiles["bed_track_start"] = max(0, (in_len - bed_ref_probs.shape[0]) // 2)
+    else:
+        profiles["bed_ref"] = profiles["bed_alt"] = None
+
+    # BigWig profiles: convert to probabilities and store as numpy (L, T)
+    if bw_r_logits is not None and bw_a_logits is not None:
+        bw_ref_probs = to_track_probabilities(bw_r_logits[0]).float().cpu().numpy()
+        bw_alt_probs = to_track_probabilities(bw_a_logits[0]).float().cpu().numpy()
+        profiles["bw_ref"] = bw_ref_probs
+        profiles["bw_alt"] = bw_alt_probs
+        profiles["bw_track_len"] = bw_ref_probs.shape[0]
+        profiles["bw_track_start"] = max(0, (in_len - bw_ref_probs.shape[0]) // 2)
+    else:
+        profiles["bw_ref"] = profiles["bw_alt"] = None
+
+    _LAST_TRACK_PROFILES = profiles

interpretation.py

@@ -0,0 +1,253 @@
+#!/usr/bin/env python3
+"""
+Rule-based signal interpretation for the MAGI Gradio app.
+
+This module is intentionally self-contained. It converts already-available
+single-variant outputs into a short interpretation panel without depending on
+notebook code or outer-folder runtime imports.
+"""
+
+from typing import Dict, List, Optional, Tuple
+
+import numpy as np
+import pandas as pd
+
+
+STRONG_DELTA = 0.20
+MODERATE_DELTA = 0.10
+WEAK_DELTA = 0.05
+
+LLR_STRONG_NEG = -3.0
+LLR_MODERATE_NEG = -1.5
+LLR_POSITIVE = 1.0
+
+KL_STRONG = 0.30
+KL_MODERATE = 0.15
+
+BED_FEATURE_LABELS: Dict[str, Tuple[str, str]] = {
+    "mRNA_splice": ("splice-site recognition", "splice disruption"),
+    "coding_sequence": ("coding sequence identity", "coding disruption"),
+    "mRNA_exon": ("exonic structure", "coding or exon-level disruption"),
+    "start_codon": ("translation initiation", "start-codon disruption"),
+    "stop_codon": ("translation termination", "stop-codon disruption"),
+    "mRNA_promoter": ("promoter activity", "promoter-regulatory disruption"),
+    "five_prime_UTR": ("5' UTR regulation", "UTR-level regulation change"),
+    "three_prime_UTR": ("3' UTR regulation", "UTR-level regulation change"),
+    "mRNA_intron": ("intronic transcript context", "intronic transcript disruption"),
+    "gene": ("genic context", "genic structural disruption"),
+    "other": ("annotated genomic context", "localized genomic disruption"),
+}
+
+
+def _is_missing(value) -> bool:
+    return value is None or (isinstance(value, float) and np.isnan(value))
+
+
+def _fmt_value(value, fmt: str = ".3f", na: str = "N/A") -> str:
+    if value is None:
+        return na
+    try:
+        if pd.isna(value):
+            return na
+    except TypeError:
+        pass
+    try:
+        return format(float(value), fmt)
+    except (TypeError, ValueError):
+        return na
+
+
+def _strength_from_delta(delta: float) -> str:
+    magnitude = abs(delta)
+    if magnitude >= STRONG_DELTA:
+        return "strong"
+    if magnitude >= MODERATE_DELTA:
+        return "moderate"
+    if magnitude >= WEAK_DELTA:
+        return "subtle"
+    return "weak"
+
+
+def _direction_word(delta: float) -> str:
+    return "gain" if delta > 0 else "loss"
+
+
+def _top_ranked_by_type(ranked: List[Dict], track_type: str, k: int = 2) -> List[Dict]:
+    return [item for item in ranked if item.get("track_type") == track_type][:k]
+
+
+def _humanize_bw_context(display_name: str) -> str:
+    parts = [part.strip() for part in str(display_name).split("|") if part.strip()]
+    if not parts:
+        return str(display_name)
+    if len(parts) == 1:
+        return parts[0]
+    if len(parts) == 2:
+        return f"{parts[0]} ({parts[1]})"
+    return f"{parts[0]} ({parts[1]}, {parts[2]})"
+
+
+def _bed_mechanism(track_id: str) -> Tuple[str, str]:
+    return BED_FEATURE_LABELS.get(
+        track_id,
+        (track_id.replace("_", " "), "localized structural disruption"),
+    )
+
+
+def _primary_mechanism(
+    ranked: List[Dict], row: pd.Series, variant_type: str
+) -> Tuple[str, str]:
+    bed_ranked = _top_ranked_by_type(ranked, "BED", k=5)
+    for item in bed_ranked:
+        if abs(float(item.get("delta", 0.0))) < WEAK_DELTA:
+            continue
+        label, mechanism = _bed_mechanism(str(item.get("track_id", "other")))
+        return mechanism, f"The strongest BED signal points to {label}."
+
+    bw_ranked = _top_ranked_by_type(ranked, "BigWig", k=3)
+    if bw_ranked and abs(float(bw_ranked[0].get("delta", 0.0))) >= MODERATE_DELTA:
+        context = _humanize_bw_context(str(bw_ranked[0].get("display_name", "track")))
+        return (
+            "context-specific regulatory change",
+            f"The strongest ranked context signal suggests a shift in {context}.",
+        )
+
+    llr = row.get("LLR", np.nan)
+    kl_mean = row.get("MLM_KL_mean", np.nan)
+    if not _is_missing(llr) and float(llr) <= LLR_MODERATE_NEG:
+        return (
+            "sequence-constraint signal",
+            "The sequence model ranks the alternate sequence as less likely even without a dominant BED or BigWig signal.",
+        )
+    if not _is_missing(kl_mean) and float(kl_mean) >= KL_MODERATE:
+        return (
+            "sequence-disruption signal",
+            "Token-level sequence distributions shift even though no single BED or BigWig track dominates.",
+        )
+
+    return (
+        "mixed or weak evidence",
+        f"No single {variant_type.lower()} mechanism dominates the current rule-based evidence.",
+    )
+
+
+def _bed_evidence_lines(ranked: List[Dict]) -> List[str]:
+    lines: List[str] = []
+    for item in _top_ranked_by_type(ranked, "BED", k=2):
+        delta = float(item.get("delta", 0.0))
+        label, _ = _bed_mechanism(str(item.get("track_id", "other")))
+        strength = _strength_from_delta(delta)
+        direction = _direction_word(delta)
+        ref_val = _fmt_value(item.get("ref_val"))
+        alt_val = _fmt_value(item.get("alt_val"))
+        lines.append(
+            f"BED `{item['track_id']}` shows a {strength} {direction} in {label} "
+            f"(REF {ref_val} → ALT {alt_val}, Δ={delta:+.3f})."
+        )
+    return lines
+
+
+def _bw_evidence_lines(ranked: List[Dict]) -> List[str]:
+    lines: List[str] = []
+    for item in _top_ranked_by_type(ranked, "BigWig", k=2):
+        delta = float(item.get("delta", 0.0))
+        strength = _strength_from_delta(delta)
+        direction = _direction_word(delta)
+        ref_val = _fmt_value(item.get("ref_val"))
+        alt_val = _fmt_value(item.get("alt_val"))
+        context = _humanize_bw_context(
+            str(item.get("display_name", item.get("track_id", "track")))
+        )
+        lines.append(
+            f"Context track `{context}` has a {strength} {direction} "
+            f"(REF {ref_val} → ALT {alt_val}, Δ={delta:+.3f})."
+        )
+    return lines
+
+
+def _sequence_evidence_lines(row: pd.Series, variant_type: str) -> List[str]:
+    lines: List[str] = []
+
+    llr = row.get("LLR", np.nan)
+    if not _is_missing(llr):
+        llr = float(llr)
+        if llr <= LLR_STRONG_NEG:
+            lines.append(
+                f"Sequence-model evidence is strong: LLR {llr:.3f} makes the alternate sequence much less plausible than reference."
+            )
+        elif llr <= LLR_MODERATE_NEG:
+            lines.append(
+                f"Sequence-model evidence is supportive: LLR {llr:.3f} penalizes the alternate sequence."
+            )
+        elif llr >= LLR_POSITIVE:
+            lines.append(
+                f"LLR {llr:.3f} does not penalize the alternate allele, so sequence-only support is limited."
+            )
+
+    kl_mean = row.get("MLM_KL_mean", np.nan)
+    if not _is_missing(kl_mean):
+        kl_mean = float(kl_mean)
+        if kl_mean >= KL_STRONG:
+            lines.append(
+                f"Mean KL {kl_mean:.3f} indicates a pronounced redistribution of token probabilities around the variant."
+            )
+        elif kl_mean >= KL_MODERATE:
+            lines.append(
+                f"Mean KL {kl_mean:.3f} indicates moderate local sequence perturbation."
+            )
+
+    if variant_type == "Indel":
+        emb_cosine = row.get("EMB_cosine_dist", np.nan)
+        emb_l2 = row.get("EMB_l2_dist", np.nan)
+        if not _is_missing(emb_cosine) or not _is_missing(emb_l2):
+            lines.append(
+                "Indel embedding distances are available as supportive context: "
+                f"cosine={_fmt_value(emb_cosine)}, L2={_fmt_value(emb_l2)}."
+            )
+
+    return lines
+
+
+def build_signal_interpretation(
+    row: pd.Series,
+    ranked: List[Dict],
+    variant_type: str,
+) -> str:
+    """Create a short deterministic interpretation panel."""
+    gene_name = row.get("gene_name", "Intergenic") or "Intergenic"
+    region_class = row.get("region_class", "OTHER")
+    if str(gene_name).startswith("N/A (non-human)") or str(region_class) == "NON_HUMAN":
+        context_anchor = "non-human BED + sequence outputs"
+    else:
+        context_anchor = f"{gene_name} / {region_class}"
+
+    if not ranked:
+        return (
+            "### Rule-Based Signal Interpretation\n\n"
+            "No ranked BED or BigWig signals available for rule-based interpretation.\n\n"
+            "**Note:** The MAGI score is computed separately from bundled baseline statistics and is shown in the Variant Summary rather than in this panel."
+        )
+
+    mechanism, rationale = _primary_mechanism(ranked, row, variant_type)
+    evidence_lines = []
+    evidence_lines.extend(_bed_evidence_lines(ranked))
+    evidence_lines.extend(_bw_evidence_lines(ranked))
+    evidence_lines.extend(_sequence_evidence_lines(row, variant_type))
+
+    if not evidence_lines:
+        evidence_lines.append(
+            "The current ranked outputs are weak, so this should be treated as a low-confidence summary only."
+        )
+
+    bullet_block = "\n".join(f"- {line}" for line in evidence_lines[:5])
+
+    return f"""
+### Rule-Based Signal Interpretation
+
+**Primary hypothesis:** {mechanism.capitalize()}  
+**Context anchor:** {context_anchor}  
+**Why this is suggested:** {rationale}
+
+**Top evidence**
+{bullet_block}
+""".strip()

requirements.txt

@@ -0,0 +1,9 @@
+torch>=2.0.0
+transformers>=4.55.0
+gradio>=4.0.0,<7.0.0
+pandas>=2.0.0
+numpy>=1.24.0
+matplotlib>=3.7.0
+pyfaidx>=0.7.0
+requests>=2.31.0
+spaces>=0.19.0

test_installation.py

@@ -0,0 +1,253 @@
+#!/usr/bin/env python3
+"""
+Quick validation script for the MAGI Gradio app.
+
+The local genome is optional. If `data/hg38.fa` is absent, the app can fall back
+to UCSC sequence retrieval.
+"""
+
+import sys
+import os
+import requests
+
+
+def check_dependencies():
+    """Check if all required packages are importable"""
+    print("🔍 Checking dependencies...")
+    required = [
+        ("torch", "PyTorch"),
+        ("transformers", "Transformers"),
+        ("gradio", "Gradio"),
+        ("pandas", "Pandas"),
+        ("numpy", "NumPy"),
+        ("matplotlib", "Matplotlib"),
+        ("pyfaidx", "pyfaidx"),
+    ]
+
+    missing = []
+    for module, name in required:
+        try:
+            __import__(module)
+            print(f"  ✓ {name}")
+        except ImportError:
+            print(f"  ✗ {name} (missing)")
+            missing.append(module)
+
+    if missing:
+        print(f"\n❌ Missing packages: {', '.join(missing)}")
+        print(f"   Install with: pip install {' '.join(missing)}")
+        return False
+    else:
+        print("\n✅ All dependencies installed\n")
+        return True
+
+
+def check_data_files():
+    """Check if required data files exist (local genome optional)."""
+    print("🔍 Checking data files...")
+
+    required_files = [
+        ("data/MANE_processed.csv", "MANE transcripts", 200_000_000),  # ~235 MB
+        ("data/Promoter_processed.csv", "Promoter annotations", 10_000_000),  # ~11 MB
+        (
+            "data/functional_tracks_metadata_human.csv",
+            "BigWig metadata",
+            500_000,
+        ),  # ~519 KB
+    ]
+
+    all_present = True
+    for file_path, description, min_size in required_files:
+        if os.path.exists(file_path):
+            size = os.path.getsize(file_path)
+            size_mb = size / 1_000_000
+            if size >= min_size:
+                print(f"  ✓ {description} ({size_mb:.1f} MB)")
+            else:
+                print(
+                    f"  ⚠ {description} exists but may be corrupted ({size_mb:.1f} MB < {min_size / 1_000_000:.1f} MB expected)"
+                )
+                all_present = False
+        else:
+            print(f"  ✗ {description} (missing: {file_path})")
+            all_present = False
+
+    local_fa = os.path.exists("data/hg38.fa")
+    local_fai = os.path.exists("data/hg38.fa.fai")
+    local_fa_gz = os.path.exists("data/hg38.fa.gz")
+
+    if local_fa:
+        size_mb = os.path.getsize("data/hg38.fa") / 1_000_000
+        print(f"  ✓ Local reference genome available ({size_mb:.1f} MB)")
+        if local_fai:
+            idx_mb = os.path.getsize("data/hg38.fa.fai") / 1_000_000
+            print(f"  ✓ Genome index available ({idx_mb:.2f} MB)")
+        else:
+            print(
+                "  ⚠ Local genome index missing (will auto-build on first local lookup)"
+            )
+    elif local_fa_gz:
+        gz_mb = os.path.getsize("data/hg38.fa.gz") / 1_000_000
+        print(
+            f"  ⚠ Compressed genome found ({gz_mb:.1f} MB); app will use UCSC API unless decompressed"
+        )
+    else:
+        print("  ⚠ Local genome not found; app will use UCSC API fallback")
+
+    if not all_present:
+        print("\n❌ Some data files missing or incomplete")
+        return False
+    else:
+        print("\n✅ Required data files present\n")
+        return True
+
+
+def test_sequence_access():
+    """Test sequence retrieval path: local hg38.fa if available, otherwise UCSC API."""
+    print("🔍 Testing sequence access...")
+
+    def _test_ucsc_api():
+        response = requests.get(
+            "https://api.genome.ucsc.edu/getData/sequence",
+            params={
+                "genome": "hg38",
+                "chrom": "chr17",
+                "start": 7675087,
+                "end": 7675088,
+            },
+            timeout=10,
+        )
+        response.raise_for_status()
+        dna = str(response.json().get("dna", "")).upper()
+        if dna and dna[0] in {"A", "C", "G", "T", "N"}:
+            print(f"  ✓ UCSC API reachable (chr17:7675088 -> {dna[0]})")
+            print("✅ Sequence access test passed\n")
+            return True
+        return False
+
+    if not os.path.exists("data/hg38.fa") or not os.path.exists("data/hg38.fa.fai"):
+        if os.path.exists("data/hg38.fa") and not os.path.exists("data/hg38.fa.fai"):
+            print(
+                "  ⚠ Local hg38.fa found but index missing; validating UCSC API path to avoid long index build"
+            )
+        try:
+            if _test_ucsc_api():
+                return True
+        except Exception as e:
+            print(f"  ✗ UCSC API test failed: {e}")
+            return False
+
+    try:
+        import pyfaidx
+
+        genome = pyfaidx.Fasta("data/hg38.fa")
+
+        # Test accessing chr17 (TP53 region), with the same fallback behavior as the app.
+        seq = None
+        for test_chrom in ("chr17", "17"):
+            if test_chrom in genome.keys():
+                seq = str(genome[test_chrom][7675087:7675088]).upper()
+                print(f"  ✓ Successfully accessed {test_chrom} locally (TP53 position: {seq})")
+                break
+
+        if seq is None:
+            print("  ⚠ Local genome could not serve chr17; testing UCSC fallback instead")
+            genome.close()
+            return _test_ucsc_api()
+
+        genome.close()
+        print("✅ Genome access test passed\n")
+        return True
+
+    except Exception as e:
+        print(f"  ✗ Error accessing genome: {e}")
+        return False
+
+
+def test_model_loading():
+    """Test that we can load the configured NTv3 model."""
+    print("🔍 Testing model loading (this may take 30-60 seconds)...")
+
+    try:
+        from transformers import AutoModel, AutoTokenizer
+
+        model_name = "InstaDeepAI/NTv3_650M_post"
+        hf_token = os.environ.get("HF_TOKEN")
+        print("  ⏳ Loading tokenizer...")
+        tokenizer = AutoTokenizer.from_pretrained(
+            model_name,
+            trust_remote_code=True,
+            token=hf_token,
+        )
+
+        print("  ⏳ Loading model (650M configuration)...")
+        model = AutoModel.from_pretrained(
+            model_name,
+            trust_remote_code=True,
+            token=hf_token,
+        )
+
+        print("  ✓ Model loaded successfully")
+        print(f"  ✓ Tokenizer vocab size: {tokenizer.vocab_size}")
+        print(
+            f"  ✓ Model parameters: {sum(p.numel() for p in model.parameters()) / 1e6:.1f}M"
+        )
+
+        print("✅ Model loading test passed\n")
+        return True
+
+    except Exception as e:
+        print(f"  ✗ Error loading model: {e}")
+        if "gated" in str(e).lower() or "401" in str(e):
+            print("  ⚠ Accept the model terms and set HF_TOKEN before retrying")
+        return False
+
+
+def main():
+    """Run all validation tests"""
+    print("\n" + "=" * 60)
+    print("  MAGI Gradio App - Installation Validation")
+    print("=" * 60 + "\n")
+
+    # Change to app directory if needed
+    if os.path.exists("ntv3_gradio_app"):
+        os.chdir("ntv3_gradio_app")
+        print("📁 Working directory: ntv3_gradio_app/\n")
+
+    # Run checks
+    results = []
+    results.append(("Dependencies", check_dependencies()))
+    results.append(("Data Files", check_data_files()))
+    results.append(("Sequence Access", test_sequence_access()))
+
+    # Model test is optional (downloads ~400MB if not cached)
+    if all(r[1] for r in results):
+        try:
+            results.append(("Model Loading", test_model_loading()))
+        except KeyboardInterrupt:
+            print("\n⚠ Model test skipped (user interrupt)")
+
+    # Summary
+    print("\n" + "=" * 60)
+    print("  Validation Summary")
+    print("=" * 60)
+    for name, passed in results:
+        status = "✅ PASS" if passed else "❌ FAIL"
+        print(f"  {name:20s} {status}")
+    print("=" * 60 + "\n")
+
+    if all(r[1] for r in results):
+        print("🎉 All tests passed! Ready to run the app:")
+        print("   python app.py")
+        print("   or")
+        print("   gradio app.py")
+        return 0
+    else:
+        print(
+            "⚠️  Some tests failed. Please fix the issues above before running the app."
+        )
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tracks.py

@@ -0,0 +1,455 @@
+#!/usr/bin/env python3
+"""
+Track Visualization Module
+===========================
+Generates region-level fill-between plots from cached NTv3 track profiles.
+
+Shows the continuous predicted probability of each genomic feature across the
+analysis window, highlighting the variant position and the most impacted tracks.
+Unified track selection is driven by the ranked list from analysis.py to
+ensure consistency with the fingerprint bar chart.
+
+Usage:
+    from tracks import generate_region_tracks_plot
+    fig = generate_region_tracks_plot(ranked_tracks=ranked, visible_radius_bp=1000)
+"""
+
+from typing import Optional, Dict, List
+
+import numpy as np
+import pandas as pd
+import matplotlib
+import matplotlib.pyplot as plt
+from matplotlib.patches import Patch
+
+matplotlib.use("Agg")
+
+# LOF/GOF palette — consistent with fingerprint bar chart
+_CLR_REF = "#555555"  # neutral gray for reference
+_CLR_GAIN = "#d73027"  # red — gain of function (ALT > REF)
+_CLR_LOSS = "#2166ac"  # blue — loss of function (ALT < REF)
+_CLR_VARIANT = "#333333"  # dark gray for variant position line
+_BED_BG = "#f7f7f7"  # subtle background for BED group
+_BW_BG = "#fffef5"  # subtle warm tint for BigWig group
+
+
+def get_track_view_bounds() -> Dict[str, Optional[int]]:
+    """Return the exact symmetric track-view bounds available for the current variant."""
+    from inference import _LAST_TRACK_PROFILES
+
+    profiles = _LAST_TRACK_PROFILES
+    if not profiles:
+        return {"max_radius": None, "window_start": None, "window_end": None}
+
+    pos = int(profiles["pos"])
+    vcenter = int(profiles["variant_center"])
+    candidate_radii: List[int] = []
+
+    for prefix in ("bed", "bw"):
+        if profiles.get(f"{prefix}_ref") is None:
+            continue
+        track_len = profiles.get(f"{prefix}_track_len")
+        track_start = profiles.get(f"{prefix}_track_start")
+        if track_len is None or track_start is None:
+            continue
+
+        left_radius = max(0, int(vcenter - int(track_start)))
+        right_radius = max(0, int(int(track_start) + int(track_len) - 1 - vcenter))
+        candidate_radii.append(min(left_radius, right_radius))
+
+    if not candidate_radii:
+        return {"max_radius": None, "window_start": None, "window_end": None}
+
+    max_radius = max(8, int(min(candidate_radii)))
+    return {
+        "max_radius": max_radius,
+        "window_start": pos - max_radius,
+        "window_end": pos + max_radius,
+    }
+
+
+def generate_region_tracks_plot(
+    ranked_tracks: Optional[List[Dict]] = None,
+    metadata_df: Optional[pd.DataFrame] = None,
+    metadata_dict: Optional[Dict[str, Dict[str, str]]] = None,
+    top_k_bed: int = 3,
+    top_k_bw: int = 3,
+    visible_radius_bp: int = 1000,
+    max_ranked_tracks: int = 10,
+    figsize_x: float = 14.0,
+    row_height: float = 1.6,
+) -> Optional[plt.Figure]:
+    """
+    Generate fill-between region view from cached track profiles.
+
+    If *ranked_tracks* is provided (from analysis.rank_top_disrupted_tracks),
+    those exact tracks are shown (ensuring consistency with fingerprint).
+    Otherwise falls back to auto-selecting top-N by point delta.
+
+    Args:
+        ranked_tracks: Pre-ranked list of dicts with keys {track_id, track_type, display_name, delta}.
+        metadata_df / metadata_dict: BigWig metadata (used only for fallback auto-select).
+        top_k_bed / top_k_bw: Fallback auto-select counts (ignored when ranked_tracks given).
+        visible_radius_bp: Half-width of visible window centred on variant (bp).
+        max_ranked_tracks: Maximum number of ranked tracks to render.
+        figsize_x: Figure width in inches.
+        row_height: Height per subplot row.
+    """
+    from inference import _LAST_TRACK_PROFILES
+
+    profiles = _LAST_TRACK_PROFILES
+    if not profiles:
+        return None
+
+    chrom = profiles["chrom"]
+    pos = profiles["pos"]
+    ref = profiles["ref"]
+    alt = profiles["alt"]
+    vcenter = profiles["variant_center"]
+    bed_names = profiles["bed_names"]
+    bigwig_names = profiles["bigwig_names"]
+    selected_bw = profiles["selected_bw_indices"]
+
+    # ── Build per-track rendering list ──────────────────────────────────
+    all_tracks: List[dict] = []
+
+    if ranked_tracks:
+        # Use unified ranking — pull continuous arrays from cache
+        for item in ranked_tracks[: max(1, int(max_ranked_tracks))]:
+            tid = item["track_id"]
+            ttype = item["track_type"]
+
+            if ttype == "BED":
+                if profiles.get("bed_ref") is None:
+                    continue
+                bed_ref = profiles["bed_ref"]
+                bed_alt = profiles["bed_alt"]
+                track_len = profiles["bed_track_len"]
+                track_start = profiles["bed_track_start"]
+                try:
+                    idx = bed_names.index(tid)
+                except ValueError:
+                    continue
+                bed_pos = vcenter - track_start
+                delta_at = (
+                    float(bed_alt[bed_pos, idx] - bed_ref[bed_pos, idx])
+                    if 0 <= bed_pos < track_len
+                    else 0.0
+                )
+                all_tracks.append(
+                    {
+                        "name": item["display_name"],
+                        "ref": bed_ref[:, idx],
+                        "alt": bed_alt[:, idx],
+                        "type": "BED",
+                        "track_start": track_start,
+                        "delta_at_variant": delta_at,
+                    }
+                )
+
+            elif ttype == "BigWig":
+                if profiles.get("bw_ref") is None:
+                    continue
+                bw_ref = profiles["bw_ref"]
+                bw_alt = profiles["bw_alt"]
+                track_len = profiles["bw_track_len"]
+                track_start = profiles["bw_track_start"]
+                try:
+                    global_idx = bigwig_names.index(tid)
+                except ValueError:
+                    continue
+                bw_pos = vcenter - track_start
+                delta_at = (
+                    float(bw_alt[bw_pos, global_idx] - bw_ref[bw_pos, global_idx])
+                    if 0 <= bw_pos < track_len
+                    else 0.0
+                )
+                all_tracks.append(
+                    {
+                        "name": item["display_name"],
+                        "ref": bw_ref[:, global_idx],
+                        "alt": bw_alt[:, global_idx],
+                        "type": "BigWig",
+                        "track_start": track_start,
+                        "delta_at_variant": delta_at,
+                    }
+                )
+    else:
+        # Fallback: auto-select by point delta (legacy behaviour)
+        all_tracks = _auto_select_tracks(
+            profiles, metadata_df, metadata_dict, top_k_bed, top_k_bw
+        )
+
+    if not all_tracks:
+        return None
+
+    # ── Separate BED and BigWig groups for visual banding ───────────────
+    bed_group = [t for t in all_tracks if t["type"] == "BED"]
+    bw_group = [t for t in all_tracks if t["type"] == "BigWig"]
+    ordered = bed_group + bw_group
+    n_tracks = len(ordered)
+
+    fig, axes = plt.subplots(
+        n_tracks,
+        1,
+        figsize=(figsize_x, row_height * n_tracks + 1.0),
+        sharex=True,
+        squeeze=False,
+    )
+    axes = axes.flatten()
+
+    genomic_origin = pos - vcenter  # genomic coord at token 0
+    n_bed = len(bed_group)
+    requested_radius_bp = max(int(visible_radius_bp), 8)
+    view_bounds = get_track_view_bounds()
+    max_radius_bp = view_bounds.get("max_radius")
+    effective_radius_bp = (
+        min(requested_radius_bp, max_radius_bp)
+        if max_radius_bp is not None
+        else requested_radius_bp
+    )
+    xleft = pos - effective_radius_bp
+    xright = pos + effective_radius_bp
+
+    for ax_idx, track_info in enumerate(ordered):
+        ax = axes[ax_idx]
+        ts = track_info["track_start"]
+        arr_len = len(track_info["ref"])
+        x_genomic = np.arange(ts, ts + arr_len) + genomic_origin
+
+        window_mask = (x_genomic >= xleft) & (x_genomic <= xright)
+        if not np.any(window_mask):
+            nearest_idx = int(np.argmin(np.abs(x_genomic - pos)))
+            lo = max(0, nearest_idx - 1)
+            hi = min(arr_len, nearest_idx + 2)
+            window_mask = np.zeros(arr_len, dtype=bool)
+            window_mask[lo:hi] = True
+
+        x_window = x_genomic[window_mask]
+        ref_y = track_info["ref"][window_mask]
+        alt_y = track_info["alt"][window_mask]
+
+        # Subtle group background
+        bg = _BED_BG if ax_idx < n_bed else _BW_BG
+        ax.set_facecolor(bg)
+
+        # ALT coloured by delta sign at variant (draw first, behind REF)
+        delta = track_info["delta_at_variant"]
+        alt_clr = _CLR_GAIN if delta > 0 else _CLR_LOSS
+        ax.plot(x_window, alt_y, color=alt_clr, linewidth=1.0, alpha=0.85, label="ALT")
+
+        # Directional delta fill: red where gain, blue where loss
+        ax.fill_between(
+            x_window,
+            ref_y,
+            alt_y,
+            where=(alt_y > ref_y),
+            color=_CLR_GAIN,
+            alpha=0.22,
+            interpolate=True,
+        )
+        ax.fill_between(
+            x_window,
+            ref_y,
+            alt_y,
+            where=(alt_y <= ref_y),
+            color=_CLR_LOSS,
+            alpha=0.22,
+            interpolate=True,
+        )
+
+        # REF as gray dashed line — drawn on top so it stays visible
+        ax.plot(x_window, ref_y, color=_CLR_REF, linewidth=1.1, alpha=0.7,
+                linestyle="--", dashes=(4, 2), label="REF")
+
+        # Variant position marker
+        ax.axvline(pos, color=_CLR_VARIANT, linewidth=1.2, linestyle="--", alpha=0.7)
+
+        # Track label
+        direction = "↑ Gain" if delta > 0 else "↓ Loss"
+        label_txt = f"{track_info['name']}  (Δ = {delta:+.4f}  {direction})"
+        ax.set_title(label_txt, fontsize=8.5, fontweight="bold", loc="left", pad=3)
+        ax.set_ylabel("P", fontsize=7, labelpad=1)
+        ax.tick_params(axis="both", labelsize=6.5)
+        ax.set_ylim(bottom=0)
+
+        # Minimal Tufte-style axes — only left spine + bottom on last
+        ax.spines["top"].set_visible(False)
+        ax.spines["right"].set_visible(False)
+        if ax_idx < n_tracks - 1:
+            ax.spines["bottom"].set_visible(False)
+            ax.tick_params(axis="x", length=0)
+
+    # Group separator line between BED and BigWig
+    if n_bed > 0 and len(bw_group) > 0:
+        # Use the axis position of the first BigWig row to draw a thin separator
+        sep_ax = axes[n_bed]
+        sep_ax.annotate(
+            "",
+            xy=(0, 1),
+            xycoords="axes fraction",
+            xytext=(1, 1),
+            textcoords="axes fraction",
+            arrowprops=dict(arrowstyle="-", color="#aaaaaa", lw=0.8),
+        )
+
+    # X-axis
+    axes[-1].set_xlabel(f"Genomic position ({chrom})", fontsize=9)
+    axes[-1].ticklabel_format(axis="x", style="plain", useOffset=False)
+    axes[-1].set_xlim(xleft, xright)
+    tick_positions = np.linspace(xleft, xright, num=5)
+    tick_positions = np.unique(np.rint(tick_positions).astype(int))
+    axes[-1].set_xticks(tick_positions)
+    axes[-1].set_xticklabels([f"{tick:,}" for tick in tick_positions], fontsize=6.5)
+
+    # Suptitle
+    fig.suptitle(
+        f"Region Track View — {chrom}:{pos:,} {ref}>{alt}",
+        fontsize=11,
+        fontweight="bold",
+        y=1.0,
+    )
+
+    # Legend
+    legend_elements = [
+        plt.Line2D([0], [0], color=_CLR_REF, linewidth=1.2, alpha=0.7,
+                   linestyle="--", dashes=(4, 2), label="REF"),
+        Patch(facecolor=_CLR_GAIN, alpha=0.3, label="Gain (ALT > REF)"),
+        Patch(facecolor=_CLR_LOSS, alpha=0.3, label="Loss (ALT < REF)"),
+        plt.Line2D(
+            [0], [0], color=_CLR_VARIANT, linewidth=1.2, linestyle="--", label="Variant"
+        ),
+    ]
+    fig.legend(
+        handles=legend_elements,
+        loc="upper right",
+        fontsize=7,
+        frameon=True,
+        ncol=4,
+        bbox_to_anchor=(0.99, 0.99),
+    )
+
+    window_note = (
+        f"Visible window: {chrom}:{xleft:,}-{xright:,} "
+        f"(radius {effective_radius_bp:,} bp)"
+    )
+    if effective_radius_bp != requested_radius_bp:
+        window_note += (
+            f" | requested {requested_radius_bp:,} bp, limited by available track signal"
+        )
+    fig.text(0.5, 0.016, window_note, ha="center", fontsize=7.5, color="#555555")
+
+    # Footnote: P = predicted probability
+    fig.text(
+        0.01, 0.002,
+        "P = predicted probability of genomic feature",
+        fontsize=7, color="#777777", style="italic",
+    )
+
+    plt.tight_layout(rect=(0, 0.03, 1, 0.96))
+    return fig
+
+
+# ── Fallback auto-select (preserves legacy behaviour) ──────────────────
+def _auto_select_tracks(profiles, metadata_df, metadata_dict, top_k_bed, top_k_bw):
+    """Select top tracks by point-delta when no pre-ranked list is given."""
+    tracks = []
+    vcenter = profiles["variant_center"]
+    bed_names = profiles["bed_names"]
+    bigwig_names = profiles["bigwig_names"]
+    selected_bw = profiles["selected_bw_indices"]
+
+    if profiles.get("bed_ref") is not None:
+        bed_ref = profiles["bed_ref"]
+        bed_alt = profiles["bed_alt"]
+        track_len = profiles["bed_track_len"]
+        track_start = profiles["bed_track_start"]
+        bed_pos = vcenter - track_start
+        if 0 <= bed_pos < track_len:
+            deltas = np.abs(bed_alt[bed_pos] - bed_ref[bed_pos])
+            top_idx = np.argsort(deltas)[-top_k_bed:][::-1]
+            for idx in top_idx:
+                name = bed_names[idx] if idx < len(bed_names) else f"BED_{idx}"
+                tracks.append(
+                    {
+                        "name": name,
+                        "ref": bed_ref[:, idx],
+                        "alt": bed_alt[:, idx],
+                        "type": "BED",
+                        "track_start": track_start,
+                        "delta_at_variant": float(
+                            bed_alt[bed_pos, idx] - bed_ref[bed_pos, idx]
+                        ),
+                    }
+                )
+
+    if profiles.get("bw_ref") is not None:
+        bw_ref = profiles["bw_ref"]
+        bw_alt = profiles["bw_alt"]
+        track_len = profiles["bw_track_len"]
+        track_start = profiles["bw_track_start"]
+        bw_pos = vcenter - track_start
+        if 0 <= bw_pos < track_len and selected_bw:
+            sel_ref = bw_ref[bw_pos, selected_bw]
+            sel_alt = bw_alt[bw_pos, selected_bw]
+            abs_d = np.abs(sel_alt - sel_ref)
+            top_local = np.argsort(abs_d)[-top_k_bw:][::-1]
+            for li in top_local:
+                gi = selected_bw[li]
+                tid = bigwig_names[gi] if gi < len(bigwig_names) else f"BW_{gi}"
+                display = _resolve_track_name(tid, metadata_df, metadata_dict)
+                tracks.append(
+                    {
+                        "name": display,
+                        "ref": bw_ref[:, gi],
+                        "alt": bw_alt[:, gi],
+                        "type": "BigWig",
+                        "track_start": track_start,
+                        "delta_at_variant": float(
+                            bw_alt[bw_pos, gi] - bw_ref[bw_pos, gi]
+                        ),
+                    }
+                )
+    return tracks
+
+
+def _resolve_track_name(
+    track_id: str,
+    metadata_df: Optional[pd.DataFrame] = None,
+    metadata_dict: Optional[Dict[str, Dict[str, str]]] = None,
+) -> str:
+    """Resolve a BigWig track ID to a human-readable name."""
+    if metadata_dict:
+        meta = metadata_dict.get(track_id)
+        if meta:
+            parts = [
+                p
+                for p in [
+                    meta.get("tissue", ""),
+                    meta.get("assay", ""),
+                    meta.get("target", ""),
+                ]
+                if p.strip() and p != "nan"
+            ]
+            if parts:
+                name = " | ".join(parts)
+                return name[:55] if len(name) > 55 else name
+
+    if metadata_df is not None:
+        rows = metadata_df[metadata_df["file_id"] == track_id]
+        if not rows.empty:
+            r = rows.iloc[0]
+            parts = [
+                str(p)
+                for p in [
+                    r.get("tissue", ""),
+                    r.get("assay", ""),
+                    r.get("experiment_target", ""),
+                ]
+                if pd.notna(p) and str(p).strip()
+            ]
+            if parts:
+                name = " | ".join(parts)
+                return name[:55] if len(name) > 55 else name
+
+    return track_id[:40]

validate_examples.py

@@ -0,0 +1,234 @@
+#!/usr/bin/env python3
+"""
+Comprehensive validation of example variants:
+1. Literature accuracy
+2. Example diversity and quality
+3. Track type coverage in results
+"""
+
+import sys
+
+sys.path.insert(0, ".")
+
+from app import predict_single_variant
+import pandas as pd
+from collections import defaultdict
+import json
+
+# Known variants from literature
+VARIANT_REFERENCES = {
+    ("chr17", 7675088, "C", "T"): {
+        "name": "TP53 R175H",
+        "disease": "Cancer (tumor suppressor)",
+        "frequency": "~6% of cancers",
+        "literature": "Highly common hotspot, eliminates DNA binding domain",
+        "impact_expected": "HIGH",
+        "regions_expected": ["coding", "regulatory"],
+    },
+    ("chr7", 117559593, "ATCT", "A"): {
+        "name": "CFTR F508del",
+        "disease": "Cystic Fibrosis",
+        "frequency": "~70% of CF patients",
+        "literature": "Most common CF mutation, causes protein misfolding",
+        "impact_expected": "HIGH",
+        "regions_expected": ["coding", "structural"],
+    },
+    ("chr13", 32332771, "AGAGA", "AGA"): {
+        "name": "BRCA2 frameshift",
+        "disease": "Hereditary breast/ovarian cancer",
+        "frequency": "Rare, pathogenic",
+        "literature": "BRCA2 frameshift deletion (c.5946delT), causes loss of function",
+        "impact_expected": "HIGH",
+        "regions_expected": ["coding", "frameshift"],
+    },
+    ("chr11", 5227002, "T", "A"): {
+        "name": "HBB E6V",
+        "disease": "Sickle cell disease",
+        "frequency": "Common in African populations",
+        "literature": "Missense mutation (rs334) causing hemoglobin S polymerization",
+        "impact_expected": "HIGH",
+        "regions_expected": ["coding", "regulatory"],
+    },
+    ("chr17", 43092418, "T", "C"): {
+        "name": "BRCA1 synonymous",
+        "disease": "Benign control variant",
+        "frequency": "Common",
+        "literature": "Synonymous variant (c.3113A>G, rs16941), expected benign",
+        "impact_expected": "LOW",
+        "regions_expected": ["coding"],
+    },
+}
+
+
+def categorize_track(feature_name, feature_type):
+    """Categorize track type (BED or BigWig, and specific kind)"""
+    if feature_type == "BED":
+        if any(
+            x in feature_name.lower() for x in ["splice", "exon", "intron", "codon"]
+        ):
+            return "bed_splicing"
+        elif any(x in feature_name.lower() for x in ["cds", "coding"]):
+            return "bed_coding"
+        elif any(x in feature_name.lower() for x in ["promoter", "enhancer"]):
+            return "bed_regulatory"
+        elif any(x in feature_name.lower() for x in ["utr", "5utr", "3utr"]):
+            return "bed_utr"
+        else:
+            return "bed_other"
+    elif feature_type == "BigWig":
+        if "Histone" in feature_name:
+            return "bw_histone"
+        elif "RNA" in feature_name or "CAGE" in feature_name:
+            return "bw_expression"
+        elif "DNase" in feature_name or "ATAC" in feature_name:
+            return "bw_accessibility"
+        elif "ChIP-seq" in feature_name:
+            return "bw_chipseq"
+        else:
+            return "bw_other"
+    return "unknown"
+
+
+def validate_example(chrom, pos, ref, alt):
+    """Validate a single example"""
+    key = (chrom, pos, ref, alt)
+    ref_data = VARIANT_REFERENCES.get(key)
+
+    if not ref_data:
+        return None
+
+    print("\n" + "=" * 80)
+    print(f"VARIANT: {ref_data['name']} ({chrom}:{pos} {ref}>{alt})")
+    print("=" * 80)
+
+    # Literature context
+    print("\n📚 LITERATURE CONTEXT:")
+    print(f"  Disease: {ref_data['disease']}")
+    print(f"  Frequency: {ref_data['frequency']}")
+    print(f"  Summary: {ref_data['literature']}")
+    print(f"  Expected Impact: {ref_data['impact_expected']}")
+
+    # Run prediction
+    print("\n🔬 Running prediction...")
+    result = predict_single_variant(chrom, pos, ref, alt)
+    (
+        summary_md,
+        interpretation_md,
+        top_table_df,
+        fp_fig,
+        rt_fig,
+        csv_path,
+        bed_df,
+        mlm_md,
+        ranked,
+    ) = result
+
+    # Extract impact score from summary
+    impact_from_summary = None
+    if "BED Impact Score" in summary_md:
+        import re
+
+        m = re.search(r"BED Impact Score\s*\|\s*([\d.]+)", summary_md)
+        if m:
+            impact_from_summary = float(m.group(1))
+
+    print("\n📊 RESULTS SUMMARY:")
+    gene_region = "N/A"
+    if "**Gene:**" in summary_md:
+        gene_part = summary_md.split("**Gene:**")[1]
+        gene_region = gene_part.split("\n")[0].strip()
+    print(f"  Gene/Region: {gene_region}")
+    print(f"  Top tracks: {len(top_table_df)} features")
+    print(f"  Total ranked: {len(ranked)} tracks")
+    print(f"  Interpretation panel present: {bool(interpretation_md)}")
+    print(
+        f"  Impact Score (BED): {impact_from_summary if impact_from_summary else 'N/A'}"
+    )
+
+    # Track diversity analysis
+    print("\n🎯 TRACK DIVERSITY ANALYSIS:")
+
+    track_categories = defaultdict(int)
+    sources = set()
+
+    # Analyze ranked tracks
+    for r in ranked:
+        feat = r["display_name"]
+        ftype = r["track_type"]
+        category = categorize_track(feat, ftype)
+        track_categories[category] += 1
+
+        if "|" in feat:  # BigWig with tissue info
+            tissue = feat.split("|")[0].strip()
+            sources.add(tissue)
+
+    print("\n  Track Categories:")
+    for cat, count in sorted(track_categories.items(), key=lambda x: -x[1]):
+        cat_display = cat.replace("bed_", "BED: ").replace("bw_", "BigWig: ")
+        print(f"    - {cat_display}: {count}")
+
+    print(f"\n  Tissue/Cell Sources Found: {len(sources)}")
+    if len(sources) > 0:
+        for source in sorted(list(sources))[:5]:  # Show first 5
+            print(f"    - {source}")
+        if len(sources) > 5:
+            print(f"    ... and {len(sources) - 5} more")
+
+    # Quality assessment
+    print("\n✓ VARIANT QUALITY ASSESSMENT:")
+
+    # Has diverse tracks
+    diversity_score = len(track_categories)
+    if diversity_score >= 3:
+        print(f"  ✓ Good track diversity ({diversity_score} categories)")
+    else:
+        print(f"  ⚠️  Limited track diversity ({diversity_score} categories)")
+
+    # Has gains and losses
+    gains = [r for r in ranked if r["delta"] > 0]
+    losses = [r for r in ranked if r["delta"] < 0]
+    if gains and losses:
+        print(f"  ✓ Shows both gains ({len(gains)}) and losses ({len(losses)})")
+    elif gains:
+        print("  ⚠️  Only shows gains, no losses")
+    elif losses:
+        print("  ⚠️  Only shows losses, no gains")
+    else:
+        print("  ✗ No gains or losses - may not be suitable example")
+
+    # Expected impact matches actual
+    has_high_impacts = any(abs(r["delta"]) >= 0.1 for r in ranked)
+
+    if ref_data["impact_expected"] == "HIGH" and has_high_impacts:
+        print("  ✓ HIGH impact expected and observed")
+    elif ref_data["impact_expected"] == "MODERATE" and not has_high_impacts:
+        print("  ✓ MODERATE impact expected and observed (no extreme deltas)")
+    else:
+        if ref_data["impact_expected"] == "HIGH":
+            print("  ⚠️  HIGH impact expected but weak observed")
+
+    # Relevant to disease
+    print(f"  ✓ Disease-relevant example ({ref_data['disease']})")
+
+    return True
+
+
+# Validate all examples
+examples = [
+    ("chr17", 7675088, "C", "T"),
+    ("chr7", 117559593, "ATCT", "A"),
+    ("chr13", 32332771, "AGAGA", "AGA"),
+    ("chr11", 5227002, "T", "A"),
+    ("chr17", 43092418, "T", "C"),
+]
+
+print("\n" + "=" * 80)
+print("COMPREHENSIVE EXAMPLE VALIDATION")
+print("=" * 80)
+
+for chrom, pos, ref, alt in examples:
+    validate_example(chrom, pos, ref, alt)
+
+print("\n" + "=" * 80)
+print("VALIDATION COMPLETE")
+print("=" * 80)