Genome off-target: make the existing engine actually shippable, and stop

telling users to go to CRISPOR

A scientist reviewing the tool concluded they had to leave for CRISPOR to get
off-target. The engine has had a full genome off-target search since Phase 2B
(dee/core/offtarget.py: lazy FASTA fetch, 8-nt PAM-proximal seed index, CFD
scoring, human/mouse/E. coli all ready) and it is wired end to end. Three
things hid it:

1. It could not finish. The search ran on EVERY candidate guide, before the
sort. Measured 2026-07-19: ~3.5s per mammalian query x 50 guides = ~3 min
of off-target per design, which is why it shipped default-off. Now only the
top-ranked guides are screened — the ones you would actually order —
verified 50 guides -> 10 queries, est. 172s -> 34s. The authoritative sort
is unaffected: genome hits never feed composite_score.
2. The UI said the opposite of the truth. The CRISPR hero claimed
"Off-target risk is not assessed", and three other places sent users to
CRISPOR unconditionally — including in runs where a real genome search had
just completed. All four now describe what actually ran.
3. The organism picker claimed a "~1 min" first run. Real measured build is
~165s for the human CDS index; corrected.

Also: /api/crispr/methods gains a genome_off entry documenting both real
scope limits (human/mouse are CODING SEQUENCE ONLY, so intronic/intergenic
sites are unseen; and only top-ranked guides are screened), and the response
now returns genome_searched_top_n so the UI states how many were checked
rather than implying all of them. A whole-genome tool is still the right call
for intergenic work — said plainly instead of either hiding it or overclaiming.

478 tests green.

dee/core/crispr.py

@@ -696,6 +696,12 @@ _INDEL_MMEJ_DECAY        = 10.0 # del-length-decay constant (Bae 2014)
 _INDEL_PLUS1_FRACTION    = 0.32 # baseline +1 templated insertion class size
 _INDEL_TOP_N             = 5    # how many outcomes to keep per guide
 
+# How many top-ranked guides get the (expensive) genome off-target query.
+# A mammalian query measures ~3.5 s, so this is the difference between a
+# ~35 s design and a ~3 min one. Public so the API can tell the user
+# exactly how many guides were screened rather than implying all of them.
+GENOME_OFFTARGET_TOP_N = 10
+
 
 def _predict_indels(
     spacer: str,
@@ -1114,14 +1120,32 @@ def find_guides(
     # Only runs when the caller named an organism AND the enzyme is
     # SpCas9 (the CFD matrix + indexing assume NGG-PAM 20-nt spacers).
     # The first call after a cold start pays the lazy index-build cost
-    # (~5 s for E. coli); subsequent calls hit the in-memory cache.
-    # Errors degrade silently — guides still rank, off-target fields
-    # stay at their defaults.
+    # (~5 s for E. coli, ~3 min for a mammalian CDS index); subsequent
+    # calls hit the in-memory cache. Errors degrade silently — guides
+    # still rank, off-target fields stay at their defaults.
+    #
+    # ONLY the top-ranked guides are searched. Measured 2026-07-19: a
+    # mammalian query costs ~3.5 s, so running all 50 candidates cost
+    # ~3 MINUTES per design and made the feature unshippable (it was
+    # left default-off as a result). Nobody orders guide #47 — the
+    # off-target question only matters for the handful you'd actually
+    # clone, so we spend the time there and leave the rest marked
+    # "not searched" (genome_organism stays "", which the UI already
+    # renders as "—", distinct from "searched, none found").
     if target_organism and enzyme == "cas9":
         try:
             from dee.core import offtarget as _ot
             if _ot.is_organism_ready(target_organism):
-                for g in guides:
+                # Rank-order a shallow copy purely to choose WHICH guides
+                # are worth the expensive query. The authoritative sort
+                # still happens below and is unaffected: genome hits never
+                # feed composite_score (that uses the input-only self-off).
+                if mode == "base_edit":
+                    _ranked = sorted(guides, key=lambda g: (-g.be_editability,
+                                                            -g.composite_score, g.position))
+                else:
+                    _ranked = sorted(guides, key=lambda g: (-g.composite_score, g.position))
+                for g in _ranked[:GENOME_OFFTARGET_TOP_N]:
                     hits = _ot.find_genomic_offtargets(
                         g.spacer, target_organism, max_results=10,
                     )

dee/core/crispr_methods.py

@@ -56,12 +56,37 @@ METHODS: Dict[str, Dict[str, Any]] = {
         "basis": "The exact 20×4 mismatch-penalty matrix from Doench 2016 "
                  "Supplementary Table 19 — the industry standard for "
                  "off-target ranking.",
-        "limits": "SCOPE WARNING: this searches only the sequence you pasted. "
-                  "It is NOT a whole-genome off-target scan, so a guide that is "
-                  "unique in your input can still cut elsewhere in the genome. "
-                  "Run top guides through a genome-wide tool before ordering.",
+        "limits": "SCOPE: this column searches only the sequence you pasted. "
+                  "For a real off-target check, pick an organism — the engine "
+                  "then screens your top guides against the genome and fills "
+                  "the “Genome off” column (see below).",
         "citations": ["Doench et al. 2016, Nat Biotechnol, Suppl. Table 19 (CFD)"],
     },
+    "genome_off": {
+        "label": "Genome off",
+        "what": "Real off-target search against the organism's genome — the "
+                "column that tells you whether a guide cuts somewhere it "
+                "shouldn't. Runs when you choose an organism.",
+        "formula": "Seed-indexed search (8 nt PAM-proximal seed, ≤1 seed and "
+                   "≤4 total mismatches), each candidate CFD-scored; hits "
+                   "above CFD 0.05 are kept and ranked.",
+        "basis": "Same Doench 2016 CFD matrix used for Self-off, applied "
+                 "across the indexed genome rather than just your input. Your "
+                 "guide never leaves the engine — only the public reference "
+                 "genome is downloaded, and it is cached and reused.",
+        "limits": "Two scope limits. (1) COVERAGE: E. coli is the complete "
+                  "genome, but human (GRCh38) and mouse (GRCm39) are indexed "
+                  "over CODING SEQUENCE ONLY — so off-targets in introns and "
+                  "intergenic DNA are not seen. If you need a whole-genome "
+                  "sweep, that still calls for a dedicated genome-wide tool. "
+                  "(2) DEPTH: only the top-ranked guides are screened (the "
+                  "ones you would realistically order), not every candidate "
+                  "in the table; the run tells you how many were checked.",
+        "citations": [
+            "Doench et al. 2016, Nat Biotechnol, Suppl. Table 19 (CFD)",
+            "Ensembl release 112 (GRCh38 / GRCm39 CDS); NCBI NC_000913.3 (E. coli K-12)",
+        ],
+    },
     "ko_score": {
         "label": "KO score",
         "what": "Probability the cut produces a true loss-of-function knockout.",
@@ -104,7 +129,8 @@ METHODS: Dict[str, Dict[str, Any]] = {
 }
 
 # Order the UI should present them in (matches the table's column order).
-METHOD_ORDER: List[str] = ["composite", "on_target", "self_off", "ko_score", "indels"]
+METHOD_ORDER: List[str] = ["composite", "on_target", "self_off", "genome_off",
+                           "ko_score", "indels"]
 
 
 def methods_payload() -> Dict[str, Any]:
@@ -114,8 +140,8 @@ def methods_payload() -> Dict[str, Any]:
         "methods": {k: dict(METHODS[k]) for k in METHOD_ORDER},
         "summary": (
             "Every score here is computed from published, sequence-based "
-            "methods — no black box. Two scope limits matter most: off-target "
-            "checking covers only the sequence you submit, and indel "
-            "predictions are not tuned to your cell type."
+            "methods — no black box. Two scope limits matter most: for human "
+            "and mouse the genome off-target search covers coding sequence "
+            "only, and indel predictions are not tuned to your cell type."
         ),
     }

dee/server.py

@@ -2065,6 +2065,14 @@ def create_app() -> Flask:
         # requested.
         from dee.core import offtarget as _ot_status
         genome_index_status = _ot_status.index_status(target_organism)
+        # Only the top-ranked guides get the (~3.5 s each) genome query, so
+        # say exactly how many were screened rather than letting the UI imply
+        # every guide was checked.
+        from dee.core.crispr import GENOME_OFFTARGET_TOP_N as _GENOME_TOP_N
+        genome_searched_top_n = (
+            min(_GENOME_TOP_N, len(guides))
+            if (genome_index_status == "ready" and enzyme == "cas9") else 0
+        )
         from dee.core import outcomes as _O
         guide_dicts = [
                 {
@@ -2134,6 +2142,7 @@ def create_app() -> Flask:
             "base_editor":         base_editor or (_be.DEFAULT_BASE_EDITOR if mode == "base_edit" else ""),
             "genome_organism":     target_organism,
             "genome_index_status": genome_index_status,
+            "genome_searched_top_n": genome_searched_top_n,
             "calibrated":          any("calibration" in g for g in guide_dicts),
             "guides":              guide_dicts,
         })

dee/static/app.js

@@ -5506,7 +5506,19 @@ if (_quitBtn) {
         if (g.flag_low_gc) watch.push('low GC (often less active)');
         if (g.flag_polyT) watch.push('a TTTT run (U6 may terminate early)');
         if (watch.length) p.push('Watch-outs: ' + watch.join(', ') + '.');
-        p.push('<em>Run your top guides through CRISPOR for whole-genome off-target before ordering.</em>');
+        // Off-target guidance, matched to what the engine ACTUALLY did for this
+        // guide. Previously this always told the user to go run CRISPOR — even
+        // when a real genome search had just run here, which is what made the
+        // tool feel like a stop on the way to somewhere else.
+        if (!g.genome_organism) {
+            p.push('<em>No genome off-target search was run — pick an organism above to screen your top guides against the genome.</em>');
+        } else if (g.genome_organism === 'ecoli') {
+            p.push('<em>Screened against the complete E. coli K-12 genome.</em>');
+        } else {
+            p.push('<em>Screened against ' + escapeHtml(g.genome_organism) +
+                   ' coding sequence. Intronic and intergenic off-targets are outside this index — ' +
+                   'use a whole-genome tool if your application needs them.</em>');
+        }
         // Phase 3 (M5): structure-view button when the cut maps to a residue
         // AND a human/mouse gene symbol is set (needed to resolve UniProt).
         // When the context is missing, show a hint telling the user exactly

dee/static/index.html

@@ -112,7 +112,7 @@
     <!-- ?v= query bumps invalidate browser + iframe asset caches when app.css /
          app.js change. Bump these numbers whenever you ship a frontend update —
          without them, users keep getting the stale file for up to a week. -->
-    <link rel="stylesheet" href="/static/app.css?v=20260718-methods" />
+    <link rel="stylesheet" href="/static/app.css?v=20260718-offtarget" />
     <link rel="icon" type="image/svg+xml" href="/static/favicon.svg?v=2" />
     <link rel="apple-touch-icon" href="/static/favicon.svg?v=2" />
     <!-- Mol* (PDBe) 3-D viewer is ~4.9 MB. We do NOT eager-load it on every
@@ -960,10 +960,12 @@
                       </p>
                       <p class="hero-meta crispr-disclaimer">
                           On-target scoring is heuristic (Doench-style sequence
-                          features). <strong>Off-target risk is not assessed</strong> &mdash;
-                          for therapeutic or in-vivo applications, run your top
-                          guides through <a href="http://crispor.tefor.net/" target="_blank" rel="noopener">CRISPOR</a>
-                          before ordering.
+                          features). <strong>Off-target is screened here</strong> &mdash;
+                          choose an organism and your top guides are searched
+                          against the genome and CFD-scored. Coverage is the
+                          complete genome for E.&nbsp;coli, and coding sequence
+                          for human and mouse; intronic and intergenic sites
+                          fall outside that index.
                       </p>
                   </section>
 
@@ -1090,8 +1092,8 @@
                               <select id="crisprOrganism" class="crispr-context-input">
                                   <option value="">None — skip genome search</option>
                                   <option value="ecoli">E. coli K-12 MG1655 · whole genome</option>
-                                  <option value="human">Homo sapiens (GRCh38) · coding regions · first run takes ~1 min</option>
-                                  <option value="mouse">Mus musculus (GRCm39) · coding regions · first run takes ~1 min</option>
+                                  <option value="human">Homo sapiens (GRCh38) · coding regions · index builds once, ~3 min</option>
+                                  <option value="mouse">Mus musculus (GRCm39) · coding regions · index builds once, ~3 min</option>
                               </select>
                           </div>
                           <div class="crispr-context-field">
@@ -1833,7 +1835,7 @@
                                   <li><strong>Knockout.</strong> Predicted indel spectrum, frameshift %, out-of-frame dominance, and a loss-of-function likelihood.</li>
                                   <li><strong>Cloning oligos.</strong> Ready-to-order sense/antisense pairs for the standard vectors (BbsI/BsmBI; Cas12a geometry), copied straight to a vendor order.</li>
                               </ul>
-                              <p class="docs-callout">Off-target search covers only the sequence you provide — it is <strong>not</strong> a whole-genome scan. Run your top guides through a genome-wide tool (e.g. CRISPOR) before ordering.</p>
+                              <p class="docs-callout">Off-target is screened against the genome when you choose an organism: your top-ranked guides are searched with a seed index and CFD-scored. Coverage is the <strong>complete genome</strong> for E.&nbsp;coli and <strong>coding sequence</strong> for human (GRCh38) and mouse (GRCm39) — intronic and intergenic off-targets sit outside that index, so a whole-genome tool is still the right call if your application depends on them. Leave the organism unset and only the sequence you pasted is checked.</p>
                           </section>
 
                           <section>
@@ -2304,7 +2306,7 @@
     <!-- Cloning reference data must load before app.js so the Designer
          can read VECTORS / ENZYMES / CLONING_METHODS / TAGS / LINKERS. -->
     <script src="/static/cloning_db.js?v=20260530-ui-polish" defer></script>
-    <script src="/static/app.js?v=20260718-methods" defer></script>
+    <script src="/static/app.js?v=20260718-offtarget" defer></script>
     <!-- Dwell-time heartbeat. Loads after auth.js so its /api/ping calls go
          through the JWT-attaching fetch wrapper (signed-in attribution). -->
     <script src="/static/telemetry.js?v=20260622-analytics" defer></script>

tests/test_crispr_methods.py

@@ -40,9 +40,12 @@ def test_on_target_does_not_claim_to_be_rule_set_2():
     assert "0.55" in m["limits"]        # the honest correlation range
 
 
-def test_self_off_declares_it_is_not_genome_wide():
+def test_self_off_declares_its_input_only_scope():
+    # Self-off is deliberately input-only; the genome-wide answer lives in the
+    # separate "Genome off" column, so this must say so rather than imply the
+    # tool has no off-target capability at all.
     m = cm.METHODS["self_off"]
-    assert "NOT a whole-genome" in m["limits"]
+    assert "only the sequence you pasted" in m["limits"]
 
 
 def test_indels_declare_heuristic_and_cell_type_agnostic():
@@ -64,3 +67,32 @@ def test_methods_route_is_public_and_shaped(client):
     assert body["order"] == cm.METHOD_ORDER
     assert set(body["methods"]) == set(cm.METHOD_ORDER)
     assert "off-target" in body["summary"]
+
+
+# --------------------------------------------------------------------------- #
+# Genome off-target: the column that decides whether "consolidation" is real
+# --------------------------------------------------------------------------- #
+def test_genome_off_is_documented_with_both_scope_limits():
+    m = cm.METHODS["genome_off"]
+    # Coverage limit: human/mouse are CDS-only, not whole genome.
+    assert "CODING SEQUENCE ONLY" in m["limits"]
+    assert "intergenic" in m["limits"]
+    # Depth limit: only top-ranked guides are screened.
+    assert "top-ranked" in m["limits"]
+    # Privacy: the guide never leaves the engine.
+    assert "never leaves" in m["basis"]
+
+
+def test_self_off_points_at_genome_search_not_an_external_tool():
+    # The old copy sent users to CRISPOR from here; it should now point at
+    # the engine's own genome column instead.
+    m = cm.METHODS["self_off"]
+    assert "Genome off" in m["limits"]
+    assert "CRISPOR" not in m["limits"]
+
+
+def test_genome_offtarget_top_n_is_bounded():
+    # The whole reason the feature was unshippable: a ~3.5s query per guide
+    # across all 50 candidates. Keep the screened set small and explicit.
+    from dee.core.crispr import GENOME_OFFTARGET_TOP_N
+    assert 1 <= GENOME_OFFTARGET_TOP_N <= 15