Tengo Gzirishvili Claude Sonnet 5 commited on
Commit
874f438
·
1 Parent(s): 0a2b916

Surface the active-learning surrogate as the Learn-phase "wow" moment

Browse files

Research finding: the field's actual bottleneck isn't tool routing, it's
getting smarter after every real measurement (ALDE, Nat. Commun.: 12%->93%
yield in 3 rounds via uncertainty-aware active learning) — and our biggest
funded competitor's headline feature is exactly "learns each time you
upload experimental data." We already had both technical pieces (the
per-user surrogate in active_learning.py, the cross-user aggregate +
scheduled rebuild in aggregate.py/server.py) — they were just invisible,
reduced to a one-line note nobody would notice. This makes them visible.

Backend:
- active_learning.Surrogate gains `components` (prior/learned/explore/
n_measured per mutation) and a new narrate() helper — a deterministic,
template-only explanation of why a mutation's score moved (or didn't).
Never fabricates: it's a pure decomposition of numbers fit_surrogate
already computes.
- _de_round2_library now also blends the cross-user global prior (parity
with round 1, previously round-2-only used the personal surrogate) and
assembles `pool_deltas` — every pool mutation's prior vs adjusted score
+ plain-language reason, sorted by the new ranking.
- JobState gains a structured `global_prior_info` field so round-1's
field-wide-prior blend (previously only ever visible in a `message`
string overwritten before the job finished) survives to /api/result.

Frontend:
- "What Turing learned": a re-rank panel showing the biggest movers with
before/after scores, a "you tested this" badge, and the reason text —
the visible "the model just got smarter" moment, staggered fade-in.
- Predicted-vs-measured calibration: client-side scatter (round-1
Predicted_Fitness_Score vs what you just logged) + Pearson r, with an
honest low-n caveat — trust-through-transparency instead of a claim.
- "Informed by pooled results from other labs (N substitution types)" on
round-1 results, once the aggregate has actually contributed — never
claims a lab count it can't back.

5 new backend tests (components/narrate) + 7 new tests (pool_deltas shape/
order, global_prior wiring on round1+round2, exercised through the real
top_percentile_pool/evolve/variants_to_dataframe pipeline, not mocked away).
414 tests green. Frontend verified in the preview browser end-to-end
(round 1 -> log outcomes -> round 2) in both themes; classic UI byte-for-
byte unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

dee/core/active_learning.py CHANGED
@@ -82,6 +82,13 @@ class Surrogate:
82
  n_effects: int # mutations that got a learned correction
83
  learned: bool # False ⇒ fell back to the prior
84
  note: str
 
 
 
 
 
 
 
85
 
86
  def adjust_pool(self, pool: list) -> list:
87
  """Return a copy of a `search.Mutation` pool with delta_ll replaced by
@@ -132,13 +139,21 @@ def fit_surrogate(
132
  # Exploration bonus: under-measured mutations get a larger nudge.
133
  unc = 1.0 / np.sqrt(1.0 + counts)
134
 
135
- # Not enough signal → honest fallback to the pure ΔLL prior.
 
 
136
  y_all = np.array([y for _, _, y in rows], dtype=float)
137
  if n < MIN_MEASUREMENTS or M == 0 or (n and np.std(y_all) < 1e-9):
 
 
 
 
 
138
  return Surrogate(
139
  adjusted=dict(prior), w_prior=1.0, n_train=n, n_effects=0, learned=False,
140
  note=(f"{n} measurement(s) logged — need ≥{MIN_MEASUREMENTS} with a spread of "
141
  "values to learn; round 2 uses the ESM-2 prior."),
 
142
  )
143
 
144
  # Standardize y (assay scale is arbitrary; ranking is scale-invariant).
@@ -163,14 +178,43 @@ def fit_surrogate(
163
  w_prior = float(w[1])
164
  beta = w[2:]
165
  # Acquisition per pool mutation: prior (re-weighted) + learned correction + explore.
166
- adjusted = {}
167
  for key in keys:
168
  c = index[key]
169
- adjusted[key] = w_prior * prior[key] + float(beta[c]) + kappa * float(unc[c])
 
 
 
 
 
170
  n_effects = int(np.sum(np.abs(beta) > 1e-6))
171
  return Surrogate(
172
  adjusted=adjusted, w_prior=w_prior, n_train=n, n_effects=n_effects, learned=True,
173
  note=(f"Learned from {n} measured variants (prior weight {w_prior:.2f}; "
174
  f"{n_effects} mutation effects corrected). Round 2 balances the "
175
  "learned model with exploration of under-tested positions."),
 
176
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
82
  n_effects: int # mutations that got a learned correction
83
  learned: bool # False ⇒ fell back to the prior
84
  note: str
85
+ # Per-mutation breakdown of `adjusted[key]` into its three additive terms
86
+ # (prior + learned + explore ≈ adjusted[key]) plus how many of the user's
87
+ # measured variants included this mutation. Exists purely so a caller can
88
+ # NARRATE round 2 ("recommended because your data corrected this position")
89
+ # and VISUALIZE the before/after re-rank — adjusted/adjust_pool are
90
+ # unaffected and existing callers/tests don't need to know this exists.
91
+ components: Dict[Tuple[int, str], Dict[str, float]]
92
 
93
  def adjust_pool(self, pool: list) -> list:
94
  """Return a copy of a `search.Mutation` pool with delta_ll replaced by
 
139
  # Exploration bonus: under-measured mutations get a larger nudge.
140
  unc = 1.0 / np.sqrt(1.0 + counts)
141
 
142
+ # Not enough signal → honest fallback to the pure ΔLL prior. Components
143
+ # still populate (prior-only, no correction, no explore) so a caller can
144
+ # render "not enough data yet" consistently rather than special-casing it.
145
  y_all = np.array([y for _, _, y in rows], dtype=float)
146
  if n < MIN_MEASUREMENTS or M == 0 or (n and np.std(y_all) < 1e-9):
147
+ fallback_components = {
148
+ key: {"prior": prior[key], "learned": 0.0, "explore": 0.0,
149
+ "n_measured": int(counts[index[key]])}
150
+ for key in keys
151
+ }
152
  return Surrogate(
153
  adjusted=dict(prior), w_prior=1.0, n_train=n, n_effects=0, learned=False,
154
  note=(f"{n} measurement(s) logged — need ≥{MIN_MEASUREMENTS} with a spread of "
155
  "values to learn; round 2 uses the ESM-2 prior."),
156
+ components=fallback_components,
157
  )
158
 
159
  # Standardize y (assay scale is arbitrary; ranking is scale-invariant).
 
178
  w_prior = float(w[1])
179
  beta = w[2:]
180
  # Acquisition per pool mutation: prior (re-weighted) + learned correction + explore.
181
+ adjusted, components = {}, {}
182
  for key in keys:
183
  c = index[key]
184
+ prior_term = w_prior * prior[key]
185
+ learned_term = float(beta[c])
186
+ explore_term = kappa * float(unc[c])
187
+ adjusted[key] = prior_term + learned_term + explore_term
188
+ components[key] = {"prior": prior_term, "learned": learned_term,
189
+ "explore": explore_term, "n_measured": int(counts[c])}
190
  n_effects = int(np.sum(np.abs(beta) > 1e-6))
191
  return Surrogate(
192
  adjusted=adjusted, w_prior=w_prior, n_train=n, n_effects=n_effects, learned=True,
193
  note=(f"Learned from {n} measured variants (prior weight {w_prior:.2f}; "
194
  f"{n_effects} mutation effects corrected). Round 2 balances the "
195
  "learned model with exploration of under-tested positions."),
196
+ components=components,
197
  )
198
+
199
+
200
+ # β magnitude below this is "no real correction" for narration purposes — a
201
+ # separate, looser threshold than n_effects' 1e-6 (that one's for counting
202
+ # whether ridge found ANY signal; this one's for whether it's worth a sentence).
203
+ _NARRATE_LEARNED_FLOOR = 0.05
204
+
205
+
206
+ def narrate(prior: float, learned: float, explore: float, n_measured: int) -> str:
207
+ """One short, honest sentence for why a mutation's round-2 score moved (or
208
+ didn't) — the same three components `fit_surrogate` computes, turned into
209
+ plain language. Deterministic and template-only: never invents a reason
210
+ beyond what the numbers actually show."""
211
+ if n_measured == 0:
212
+ return ("Not yet tested — kept close to the ESM-2 prior, nudged up for "
213
+ "exploration since your data hasn't covered it.")
214
+ if abs(learned) < _NARRATE_LEARNED_FLOOR:
215
+ return (f"Consistent with your {n_measured} measurement"
216
+ f"{'s' if n_measured != 1 else ''} so far — no correction needed.")
217
+ direction = "helped more than the ESM-2 prior expected" if learned > 0 else \
218
+ "underperformed what the ESM-2 prior expected"
219
+ return (f"Revised from {n_measured} of your measurement"
220
+ f"{'s' if n_measured != 1 else ''} — this substitution {direction}.")
dee/server.py CHANGED
@@ -221,6 +221,11 @@ class JobState:
221
  # library to the user when the pipeline finishes. None for anonymous
222
  # runs; the save-to-Storage path no-ops in that case.
223
  user_id: Optional[str] = None
 
 
 
 
 
224
 
225
  def elapsed(self) -> float:
226
  end = self.finished_at if self.finished_at else time.time()
@@ -1355,6 +1360,7 @@ def create_app() -> Flask:
1355
  "settings_used": job.settings_used,
1356
  "started_at": job.started_at,
1357
  "elapsed_seconds": job.elapsed(),
 
1358
  }
1359
  )
1360
 
@@ -3394,13 +3400,22 @@ def _compute_de_ll_maps(grouped_with_lib):
3394
  return maps
3395
 
3396
 
 
 
 
3397
  def _de_round2_library(wt_protein: str, settings: Dict[str, Any],
3398
  measurements: list) -> tuple[list, Dict[str, Any]]:
3399
- """Active-learning round 2 (Design→Build→Test→Learn). Score the WT, fit the
 
3400
  surrogate on the user's measured variants, re-rank the single-site pool by
3401
  the learned acquisition score, evolve, and assemble the same variant table
3402
- shape as a normal run. Returns (variant_rows, surrogate_info)."""
 
 
 
 
3403
  from dee.core import active_learning as _al
 
3404
  from dee.optimizer.search import Mutation as _Mut
3405
 
3406
  scorer = _scoring.get_scorer(
@@ -3411,18 +3426,58 @@ def _de_round2_library(wt_protein: str, settings: Dict[str, Any],
3411
  scores_df = _scoring.score_guarded(scorer, wt_protein)
3412
  pool_df = top_percentile_pool(scores_df, percentile=float(settings.get("percentile", 85.0)))
3413
 
3414
- # Fit the surrogate on the round-1 single-site pool + the user's results,
3415
- # then overwrite the pool's per-mutation score with the learned acquisition
3416
- # (additive ⇒ evolve() consumes it unchanged).
3417
  muts = [_Mut(int(r.position), str(r.wt_aa), str(r.mut_aa), float(r.delta_ll))
3418
  for r in pool_df.itertuples(index=False)]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3419
  surrogate = _al.fit_surrogate(muts, measurements)
 
3420
  adj_df = pool_df.copy()
3421
  adj_df["delta_ll"] = [
3422
  surrogate.adjusted.get((int(r.position), str(r.mut_aa)), float(r.delta_ll))
3423
  for r in pool_df.itertuples(index=False)
3424
  ]
3425
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3426
  variants = evolve(adj_df, SearchConfig(
3427
  k=int(settings.get("k", 30)),
3428
  max_mutations=int(settings.get("max_mutations", 5)),
@@ -3437,11 +3492,13 @@ def _de_round2_library(wt_protein: str, settings: Dict[str, Any],
3437
  forbidden_sites=DEFAULT_FORBIDDEN_SITES,
3438
  )
3439
  return df.to_dict(orient="records"), {
3440
- "learned": surrogate.learned,
3441
- "n_train": surrogate.n_train,
3442
- "w_prior": round(surrogate.w_prior, 3),
3443
- "n_effects": surrogate.n_effects,
3444
- "note": surrogate.note,
 
 
3445
  }
3446
 
3447
 
@@ -3515,6 +3572,12 @@ def _run_pipeline(
3515
  ]
3516
  job.message = (f"Filtered to {len(pool)} mutations; blended field-wide "
3517
  f"priors ({len(_gp.effects)} substitution types).")
 
 
 
 
 
 
3518
  except Exception: # noqa: BLE001
3519
  logger.exception("global-prior blend skipped")
3520
 
 
221
  # library to the user when the pipeline finishes. None for anonymous
222
  # runs; the save-to-Storage path no-ops in that case.
223
  user_id: Optional[str] = None
224
+ # Whether/how much the cross-user field-wide aggregate (dee.core.aggregate)
225
+ # contributed to this run's scores — was only ever recorded transiently in
226
+ # `message` (overwritten by the next status update, so gone by "done").
227
+ # Structured here so /api/result can surface it after the job finishes.
228
+ global_prior_info: Optional[Dict[str, Any]] = None
229
 
230
  def elapsed(self) -> float:
231
  end = self.finished_at if self.finished_at else time.time()
 
1360
  "settings_used": job.settings_used,
1361
  "started_at": job.started_at,
1362
  "elapsed_seconds": job.elapsed(),
1363
+ "global_prior": job.global_prior_info or {"applied": False, "substitution_types": 0},
1364
  }
1365
  )
1366
 
 
3400
  return maps
3401
 
3402
 
3403
+ _POOL_DELTAS_MAX = 40 # cap the "what changed" payload — biggest movers only
3404
+
3405
+
3406
  def _de_round2_library(wt_protein: str, settings: Dict[str, Any],
3407
  measurements: list) -> tuple[list, Dict[str, Any]]:
3408
+ """Active-learning round 2 (Design→Build→Test→Learn). Score the WT, blend
3409
+ the cross-user field-wide prior (parity with round 1), fit the personal
3410
  surrogate on the user's measured variants, re-rank the single-site pool by
3411
  the learned acquisition score, evolve, and assemble the same variant table
3412
+ shape as a normal run. Returns (variant_rows, surrogate_info) — the latter
3413
+ now also carries `pool_deltas` (per-mutation before/after + plain-language
3414
+ reason, sorted by round-2 score) and `global_prior` (whether the field-wide
3415
+ aggregate contributed), so the frontend can show a genuine before/after
3416
+ re-rank instead of a silent number change."""
3417
  from dee.core import active_learning as _al
3418
+ from dee.core import aggregate as _agg
3419
  from dee.optimizer.search import Mutation as _Mut
3420
 
3421
  scorer = _scoring.get_scorer(
 
3426
  scores_df = _scoring.score_guarded(scorer, wt_protein)
3427
  pool_df = top_percentile_pool(scores_df, percentile=float(settings.get("percentile", 85.0)))
3428
 
 
 
 
3429
  muts = [_Mut(int(r.position), str(r.wt_aa), str(r.mut_aa), float(r.delta_ll))
3430
  for r in pool_df.itertuples(index=False)]
3431
+
3432
+ # Same field-wide blend round 1 gets (soft nudge by substitution TYPE only,
3433
+ # no-op until a post-effective-date rebuild has populated the aggregate) —
3434
+ # applied BEFORE the personal surrogate so the two signals compose the same
3435
+ # way in both rounds: ESM-2 prior → +field-wide nudge → +your own data.
3436
+ global_prior_info = {"applied": False, "substitution_types": 0}
3437
+ try:
3438
+ _gp = _load_global_prior()
3439
+ if _gp and _gp.effects:
3440
+ muts = _agg.apply_global_prior(muts, _gp)
3441
+ global_prior_info = {"applied": True, "substitution_types": len(_gp.effects)}
3442
+ except Exception: # noqa: BLE001 — global-prior blend is a nice-to-have, never fatal
3443
+ logger.exception("global-prior blend skipped (round 2)")
3444
+
3445
+ # Fit the surrogate on the (possibly field-nudged) single-site pool + the
3446
+ # user's results, then overwrite the pool's per-mutation score with the
3447
+ # learned acquisition (additive ⇒ evolve() consumes it unchanged).
3448
  surrogate = _al.fit_surrogate(muts, measurements)
3449
+ baseline = {(m.position, m.mut_aa): float(m.delta_ll) for m in muts}
3450
  adj_df = pool_df.copy()
3451
  adj_df["delta_ll"] = [
3452
  surrogate.adjusted.get((int(r.position), str(r.mut_aa)), float(r.delta_ll))
3453
  for r in pool_df.itertuples(index=False)
3454
  ]
3455
 
3456
+ # "What Turing learned" — every pool mutation's prior vs adjusted score
3457
+ # plus a plain-language reason, sorted by the new score so the biggest
3458
+ # movers (and the ones round 2 now favors most) lead. Cheap: template
3459
+ # text only, no LLM call, and it's a straight decomposition of numbers
3460
+ # fit_surrogate already computed — nothing here can fabricate a reason
3461
+ # the data doesn't support.
3462
+ pool_deltas = []
3463
+ for m in muts:
3464
+ key = (m.position, m.mut_aa)
3465
+ prior_score = baseline.get(key, float(m.delta_ll))
3466
+ adjusted_score = surrogate.adjusted.get(key, prior_score)
3467
+ comp = surrogate.components.get(key, {"prior": prior_score, "learned": 0.0,
3468
+ "explore": 0.0, "n_measured": 0})
3469
+ pool_deltas.append({
3470
+ "label": f"{m.wt_aa}{m.position + 1}{m.mut_aa}",
3471
+ "prior_score": round(prior_score, 4),
3472
+ "adjusted_score": round(adjusted_score, 4),
3473
+ "delta": round(adjusted_score - prior_score, 4),
3474
+ "n_measured": int(comp.get("n_measured", 0)),
3475
+ "reason": _al.narrate(comp.get("prior", prior_score), comp.get("learned", 0.0),
3476
+ comp.get("explore", 0.0), int(comp.get("n_measured", 0))),
3477
+ })
3478
+ pool_deltas.sort(key=lambda d: d["adjusted_score"], reverse=True)
3479
+ pool_deltas = pool_deltas[:_POOL_DELTAS_MAX]
3480
+
3481
  variants = evolve(adj_df, SearchConfig(
3482
  k=int(settings.get("k", 30)),
3483
  max_mutations=int(settings.get("max_mutations", 5)),
 
3492
  forbidden_sites=DEFAULT_FORBIDDEN_SITES,
3493
  )
3494
  return df.to_dict(orient="records"), {
3495
+ "learned": surrogate.learned,
3496
+ "n_train": surrogate.n_train,
3497
+ "w_prior": round(surrogate.w_prior, 3),
3498
+ "n_effects": surrogate.n_effects,
3499
+ "note": surrogate.note,
3500
+ "pool_deltas": pool_deltas,
3501
+ "global_prior": global_prior_info,
3502
  }
3503
 
3504
 
 
3572
  ]
3573
  job.message = (f"Filtered to {len(pool)} mutations; blended field-wide "
3574
  f"priors ({len(_gp.effects)} substitution types).")
3575
+ # job.message gets overwritten by later status updates (searching,
3576
+ # done, ...), so by the time the job finishes this fact would
3577
+ # otherwise be lost — record it structurally so /api/result can
3578
+ # still tell the user "this was informed by other labs" after
3579
+ # the run is done, not just mid-flight in a progress string.
3580
+ job.global_prior_info = {"applied": True, "substitution_types": len(_gp.effects)}
3581
  except Exception: # noqa: BLE001
3582
  logger.exception("global-prior blend skipped")
3583
 
dee/static/app.css CHANGED
@@ -6361,6 +6361,58 @@ h3, h4 {
6361
  .round2-row { grid-template-columns: 50px 1fr 84px; gap: 8px; }
6362
  }
6363
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6364
  /* ── One-time policy-change notice (Privacy/Terms v2.0) ───────────────── */
6365
  /* Matches the .research-strip register so it reads as part of the chrome:
6366
  theme-aware surface, hairline rule, an inset accent stripe on the left. */
 
6361
  .round2-row { grid-template-columns: 50px 1fr 84px; gap: 8px; }
6362
  }
6363
 
6364
+ /* ── "What Turing learned" — round-2 re-rank + reasons ────────────────── */
6365
+ .learned-panel { margin: 14px 0 0; padding-top: 16px; border-top: 1px solid var(--line); }
6366
+ .learned-list { display: flex; flex-direction: column; gap: 1px; margin-top: 8px;
6367
+ background: var(--line); border: 1px solid var(--line); border-radius: var(--r-2); overflow: hidden; }
6368
+ .learned-row {
6369
+ display: grid; grid-template-columns: 22px 72px 128px auto 1fr;
6370
+ align-items: center; gap: 10px; padding: 9px 12px; background: var(--bg-card);
6371
+ /* Staggered entrance — the "watch it re-rank" moment. Each row's delay is
6372
+ driven by its own --i (set inline per row), so they cascade in order
6373
+ instead of all popping at once. Respects prefers-reduced-motion below. */
6374
+ animation: learnedRowIn 260ms var(--ease) both;
6375
+ animation-delay: calc(var(--i, 0) * 45ms);
6376
+ }
6377
+ @keyframes learnedRowIn {
6378
+ from { opacity: 0; transform: translateY(4px); }
6379
+ to { opacity: 1; transform: translateY(0); }
6380
+ }
6381
+ @media (prefers-reduced-motion: reduce) {
6382
+ .learned-row { animation: none; }
6383
+ }
6384
+ .learned-arrow { font-size: 15px; text-align: center; font-weight: 600; }
6385
+ .learned-up { color: var(--success, #047857); }
6386
+ .learned-down { color: var(--danger, #991B1B); }
6387
+ .learned-flat { color: var(--ink-faint); }
6388
+ .learned-label { font-family: var(--font-mono); font-size: 12.5px; color: var(--ink); }
6389
+ .learned-scores { font-family: var(--font-mono); font-size: 11.5px; color: var(--ink-faint);
6390
+ white-space: nowrap; }
6391
+ .learned-scores .ls-old { color: var(--ink-disabled); }
6392
+ .learned-scores .ls-new { color: var(--ink); font-weight: 600; }
6393
+ .learned-badge {
6394
+ font-family: var(--font-mono); font-size: 9px; letter-spacing: 0.08em; text-transform: uppercase;
6395
+ color: var(--brand-deep); background: var(--brand-50); border-radius: 999px; padding: 2px 8px;
6396
+ white-space: nowrap; justify-self: start;
6397
+ }
6398
+ .learned-reason { font-size: 12.5px; color: var(--ink-soft); line-height: 1.4; grid-column: 1 / -1;
6399
+ padding-left: 32px; margin-top: -2px; }
6400
+ @media (max-width: 640px) {
6401
+ .learned-row { grid-template-columns: 20px 1fr; row-gap: 4px; }
6402
+ .learned-scores, .learned-badge { grid-column: 2; }
6403
+ .learned-reason { padding-left: 0; grid-column: 1 / -1; }
6404
+ }
6405
+
6406
+ /* ── Predicted-vs-measured calibration ─────────────────────────────────── */
6407
+ .calib-panel { margin: 14px 0 0; padding-top: 16px; border-top: 1px solid var(--line); }
6408
+ .calib-body { margin-top: 8px; display: flex; flex-direction: column; align-items: flex-start; gap: 6px; }
6409
+ .calib-svg { width: 100%; max-width: 360px; height: auto; }
6410
+ .calib-axis { stroke: var(--line-strong); stroke-width: 1; }
6411
+ .calib-axislabel { font-family: var(--font-mono); font-size: 8px; letter-spacing: 0.06em;
6412
+ fill: var(--ink-faint); text-transform: uppercase; }
6413
+ .calib-point { fill: var(--ink); fill-opacity: 0.72; stroke: var(--bg-card); stroke-width: 1; }
6414
+ .calib-caption { font-size: 12px; margin: 0; }
6415
+
6416
  /* ── One-time policy-change notice (Privacy/Terms v2.0) ───────────────── */
6417
  /* Matches the .research-strip register so it reads as part of the chrome:
6418
  theme-aware surface, hairline rule, an inset accent stripe on the left. */
dee/static/app.js CHANGED
@@ -1297,6 +1297,149 @@ function _renderRound2Panel(data) {
1297
  panel.hidden = false;
1298
  }
1299
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1300
  function _collectRound2Measurements() {
1301
  const out = [];
1302
  document.querySelectorAll('#round2List .r2-val').forEach((inp) => {
@@ -1341,6 +1484,10 @@ function _round2Status(msg, kind) {
1341
  if (res.status === 403) { window.dispatchEvent(new Event('td:signin-required')); _round2Status('Sign in to use round 2.', 'warn'); return; }
1342
  if (!j.ok) { _round2Status(j.error || 'Round 2 failed.', 'warn'); return; }
1343
  renderResults(j); // renders the round-2 library + the "Round 2" banner
 
 
 
 
1344
  const card = document.getElementById('resultsCard');
1345
  if (card) card.scrollIntoView({ behavior: 'smooth', block: 'start' });
1346
  } catch (e) { _round2Status('Round 2 failed — please try again.', 'warn'); }
@@ -1381,8 +1528,17 @@ function renderResults(data) {
1381
  // and, for signed-in users, also persisted to Supabase Storage.
1382
  const evolvedCount = (data.variants || []).filter(v => v.Variant_ID !== 'WT').length;
1383
  const hasWt = (data.variants || []).some(v => v.Variant_ID === 'WT');
 
 
 
 
 
 
 
 
 
1384
  $('#resultSummary').innerHTML =
1385
- `<strong>${evolvedCount}</strong> variants${hasWt ? ' + wild type' : ''} of <strong>${escapeHtml(data.wt_identifier)}</strong> · ${data.wt_protein.length} aa · ready to download`;
1386
 
1387
  // Learning flywheel: remember this run + (re)build the round-2 panel.
1388
  state.lastRun = data;
@@ -1394,6 +1550,14 @@ function renderResults(data) {
1394
  _r2b.innerHTML = `<strong>Round 2.</strong> ${escapeHtml(data.surrogate.note || 'Designed from your logged bench results.')}`;
1395
  } else { _r2b.hidden = true; _r2b.innerHTML = ''; }
1396
  }
 
 
 
 
 
 
 
 
1397
 
1398
  renderStatsStrip(data);
1399
  renderMutationMap(data);
 
1297
  panel.hidden = false;
1298
  }
1299
 
1300
+ // "What Turing learned" — the round-2 before/after re-rank + plain-language
1301
+ // reasons, from surrogate.pool_deltas (dee.core.active_learning components +
1302
+ // narrate, assembled server-side in dee.server._de_round2_library). Only
1303
+ // shown when the surrogate actually learned something real; a fresh round 2
1304
+ // on too little data has pool_deltas but every delta is 0 (nothing to show).
1305
+ function _renderLearnedPanel(surrogate) {
1306
+ const panel = document.getElementById('learnedPanel');
1307
+ const list = document.getElementById('learnedList');
1308
+ if (!panel || !list) return;
1309
+ const deltas = (surrogate && surrogate.pool_deltas) || [];
1310
+ if (!surrogate || !surrogate.learned || !deltas.length) {
1311
+ panel.hidden = true; list.innerHTML = ''; return;
1312
+ }
1313
+ // Lead with what actually MOVED (biggest |delta|), not just the current
1314
+ // top score — that's the "the model just learned" moment, not a re-listing
1315
+ // of the same ranking.
1316
+ const movers = deltas.slice().sort((a, b) => Math.abs(b.delta) - Math.abs(a.delta)).slice(0, 12);
1317
+ list.innerHTML = movers.map((d, i) => {
1318
+ const flat = Math.abs(d.delta) < 0.01;
1319
+ const up = d.delta > 0;
1320
+ const cls = flat ? 'flat' : (up ? 'up' : 'down');
1321
+ const arrow = flat ? '&middot;' : (up ? '&uarr;' : '&darr;');
1322
+ const badge = d.n_measured > 0
1323
+ ? `<span class="learned-badge">you tested this</span>` : '';
1324
+ return `<div class="learned-row" style="--i:${i}">
1325
+ <span class="learned-arrow learned-${cls}">${arrow}</span>
1326
+ <span class="learned-label">${escapeHtml(d.label)}</span>
1327
+ <span class="learned-scores"><span class="ls-old">${d.prior_score.toFixed(2)}</span> &rarr; <span class="ls-new">${d.adjusted_score.toFixed(2)}</span></span>
1328
+ ${badge}
1329
+ <span class="learned-reason">${escapeHtml(d.reason)}</span>
1330
+ </div>`;
1331
+ }).join('');
1332
+ panel.hidden = false;
1333
+ }
1334
+
1335
+ function _pearsonR(xs, ys) {
1336
+ const n = xs.length;
1337
+ if (n < 2) return null;
1338
+ const mx = xs.reduce((a, b) => a + b, 0) / n;
1339
+ const my = ys.reduce((a, b) => a + b, 0) / n;
1340
+ let num = 0, dx2 = 0, dy2 = 0;
1341
+ for (let i = 0; i < n; i++) {
1342
+ const dx = xs[i] - mx, dy = ys[i] - my;
1343
+ num += dx * dy; dx2 += dx * dx; dy2 += dy * dy;
1344
+ }
1345
+ const denom = Math.sqrt(dx2 * dy2);
1346
+ return denom > 1e-9 ? num / denom : null;
1347
+ }
1348
+
1349
+ // Predicted-vs-measured calibration — entirely client-side. Matches round 1's
1350
+ // Predicted_Fitness_Score (already in the table, per variant) against the
1351
+ // values the user just typed into the round-2 panel, keyed by the exact same
1352
+ // Mutations_AA string both sides already share (see _renderRound2Panel's
1353
+ // data-mut attribute). No backend round-trip needed: this is the same
1354
+ // "predictor, not oracle" honesty the rest of the product already commits
1355
+ // to, just made visible instead of only textual.
1356
+ function _renderCalibration(round1Data, measurements) {
1357
+ const panel = document.getElementById('calibPanel');
1358
+ const body = document.getElementById('calibBody');
1359
+ if (!panel || !body) return;
1360
+ const variants = (round1Data && round1Data.variants) || [];
1361
+ const byMut = new Map();
1362
+ variants.forEach((v) => {
1363
+ if (v.Mutations_AA) byMut.set(v.Mutations_AA, Number(v.Predicted_Fitness_Score));
1364
+ });
1365
+ const points = [];
1366
+ (measurements || []).forEach((m) => {
1367
+ const pred = byMut.get(m.mutations);
1368
+ if (pred != null && isFinite(pred) && isFinite(m.measured_value)) {
1369
+ points.push({ x: pred, y: m.measured_value, label: m.mutations });
1370
+ }
1371
+ });
1372
+ if (points.length < 2) { panel.hidden = true; body.innerHTML = ''; return; }
1373
+
1374
+ const xs = points.map((p) => p.x), ys = points.map((p) => p.y);
1375
+ const r = _pearsonR(xs, ys);
1376
+
1377
+ const W = 320, H = 200, pad = { top: 14, right: 16, bottom: 30, left: 42 };
1378
+ const innerW = W - pad.left - pad.right, innerH = H - pad.top - pad.bottom;
1379
+ const xMin = Math.min(...xs), xMax = Math.max(...xs);
1380
+ const yMin = Math.min(...ys), yMax = Math.max(...ys);
1381
+ const xSpan = Math.max(1e-6, xMax - xMin), ySpan = Math.max(1e-6, yMax - yMin);
1382
+ const xScale = (x) => pad.left + (x - xMin) / xSpan * innerW;
1383
+ const yScale = (y) => pad.top + innerH - (y - yMin) / ySpan * innerH;
1384
+
1385
+ const svgNS = 'http://www.w3.org/2000/svg';
1386
+ const svg = document.createElementNS(svgNS, 'svg');
1387
+ svg.setAttribute('viewBox', `0 0 ${W} ${H}`);
1388
+ svg.setAttribute('class', 'calib-svg');
1389
+ svg.setAttribute('role', 'img');
1390
+ svg.setAttribute('aria-label', 'Scatter plot of ESM-2 predicted fitness versus your measured values');
1391
+
1392
+ const xAxis = document.createElementNS(svgNS, 'line');
1393
+ xAxis.setAttribute('x1', pad.left); xAxis.setAttribute('x2', pad.left + innerW);
1394
+ xAxis.setAttribute('y1', pad.top + innerH); xAxis.setAttribute('y2', pad.top + innerH);
1395
+ xAxis.setAttribute('class', 'calib-axis');
1396
+ svg.appendChild(xAxis);
1397
+ const yAxis = document.createElementNS(svgNS, 'line');
1398
+ yAxis.setAttribute('x1', pad.left); yAxis.setAttribute('x2', pad.left);
1399
+ yAxis.setAttribute('y1', pad.top); yAxis.setAttribute('y2', pad.top + innerH);
1400
+ yAxis.setAttribute('class', 'calib-axis');
1401
+ svg.appendChild(yAxis);
1402
+
1403
+ points.forEach((p) => {
1404
+ const c = document.createElementNS(svgNS, 'circle');
1405
+ c.setAttribute('cx', xScale(p.x)); c.setAttribute('cy', yScale(p.y));
1406
+ c.setAttribute('r', 4); c.setAttribute('class', 'calib-point');
1407
+ const title = document.createElementNS(svgNS, 'title');
1408
+ title.textContent = `${p.label}: predicted ${p.x.toFixed(2)}, measured ${p.y.toFixed(2)}`;
1409
+ c.appendChild(title);
1410
+ svg.appendChild(c);
1411
+ });
1412
+
1413
+ const xLabel = document.createElementNS(svgNS, 'text');
1414
+ xLabel.setAttribute('x', pad.left + innerW / 2); xLabel.setAttribute('y', H - 6);
1415
+ xLabel.setAttribute('text-anchor', 'middle'); xLabel.setAttribute('class', 'calib-axislabel');
1416
+ xLabel.textContent = 'ESM-2 predicted fitness (round 1)';
1417
+ svg.appendChild(xLabel);
1418
+ const yLabel = document.createElementNS(svgNS, 'text');
1419
+ yLabel.setAttribute('x', -(pad.top + innerH / 2)); yLabel.setAttribute('y', 12);
1420
+ yLabel.setAttribute('transform', 'rotate(-90)');
1421
+ yLabel.setAttribute('text-anchor', 'middle'); yLabel.setAttribute('class', 'calib-axislabel');
1422
+ yLabel.textContent = 'Your measured value';
1423
+ svg.appendChild(yLabel);
1424
+
1425
+ body.innerHTML = '';
1426
+ body.appendChild(svg);
1427
+
1428
+ const caption = document.createElement('p');
1429
+ caption.className = 'calib-caption muted';
1430
+ if (r == null) {
1431
+ caption.textContent = `${points.length} point${points.length === 1 ? '' : 's'} — not enough spread to compute a correlation.`;
1432
+ } else {
1433
+ const strength = Math.abs(r) > 0.6 ? 'strong' : Math.abs(r) > 0.3 ? 'moderate' : 'weak';
1434
+ const caveat = points.length < 6 ? ' — take this with a grain of salt below 6 points.' : '';
1435
+ caption.textContent =
1436
+ `r = ${r.toFixed(2)} (${strength} ${r >= 0 ? 'positive' : 'negative'} correlation) `
1437
+ + `across ${points.length} tested variant${points.length === 1 ? '' : 's'}${caveat}`;
1438
+ }
1439
+ body.appendChild(caption);
1440
+ panel.hidden = false;
1441
+ }
1442
+
1443
  function _collectRound2Measurements() {
1444
  const out = [];
1445
  document.querySelectorAll('#round2List .r2-val').forEach((inp) => {
 
1484
  if (res.status === 403) { window.dispatchEvent(new Event('td:signin-required')); _round2Status('Sign in to use round 2.', 'warn'); return; }
1485
  if (!j.ok) { _round2Status(j.error || 'Round 2 failed.', 'warn'); return; }
1486
  renderResults(j); // renders the round-2 library + the "Round 2" banner
1487
+ // `data` is still round 1 (captured above, before renderResults
1488
+ // reassigned state.lastRun) — exactly what calibration needs to
1489
+ // compare against what the user just typed in.
1490
+ _renderCalibration(data, meas);
1491
  const card = document.getElementById('resultsCard');
1492
  if (card) card.scrollIntoView({ behavior: 'smooth', block: 'start' });
1493
  } catch (e) { _round2Status('Round 2 failed — please try again.', 'warn'); }
 
1528
  // and, for signed-in users, also persisted to Supabase Storage.
1529
  const evolvedCount = (data.variants || []).filter(v => v.Variant_ID !== 'WT').length;
1530
  const hasWt = (data.variants || []).some(v => v.Variant_ID === 'WT');
1531
+ // Cross-user field-wide aggregate (dee.core.aggregate) blended into THIS
1532
+ // run's scores — inert/absent until enough labs have logged outcomes for
1533
+ // any substitution type, so this line only appears once it's honestly
1534
+ // true. Never claims a specific lab count, only the substitution-type
1535
+ // count actually behind it (what the k-anonymity-floored aggregate has).
1536
+ const gp = data.global_prior;
1537
+ const gpNote = (gp && gp.applied)
1538
+ ? ` · <span class="micro" title="A soft nudge by amino-acid substitution type, pooled de-identified across labs — never raw sequences or user data.">informed by pooled results from other labs (${gp.substitution_types} substitution type${gp.substitution_types === 1 ? '' : 's'})</span>`
1539
+ : '';
1540
  $('#resultSummary').innerHTML =
1541
+ `<strong>${evolvedCount}</strong> variants${hasWt ? ' + wild type' : ''} of <strong>${escapeHtml(data.wt_identifier)}</strong> · ${data.wt_protein.length} aa · ready to download${gpNote}`;
1542
 
1543
  // Learning flywheel: remember this run + (re)build the round-2 panel.
1544
  state.lastRun = data;
 
1550
  _r2b.innerHTML = `<strong>Round 2.</strong> ${escapeHtml(data.surrogate.note || 'Designed from your logged bench results.')}`;
1551
  } else { _r2b.hidden = true; _r2b.innerHTML = ''; }
1552
  }
1553
+ if (data.round === 2 && data.surrogate) {
1554
+ _renderLearnedPanel(data.surrogate);
1555
+ } else {
1556
+ // Fresh round-1 render — clear any leftover round-2 panels from a
1557
+ // PREVIOUS design in this same session so they don't show stale data.
1558
+ const lp = document.getElementById('learnedPanel'); if (lp) lp.hidden = true;
1559
+ const cp = document.getElementById('calibPanel'); if (cp) cp.hidden = true;
1560
+ }
1561
 
1562
  renderStatsStrip(data);
1563
  renderMutationMap(data);
dee/static/index.html CHANGED
@@ -112,7 +112,7 @@
112
  <!-- ?v= query bumps invalidate browser + iframe asset caches when app.css /
113
  app.js change. Bump these numbers whenever you ship a frontend update —
114
  without them, users keep getting the stale file for up to a week. -->
115
- <link rel="stylesheet" href="/static/app.css?v=20260714-bench-overflow-fix" />
116
  <link rel="icon" type="image/svg+xml" href="/static/favicon.svg?v=2" />
117
  <link rel="apple-touch-icon" href="/static/favicon.svg?v=2" />
118
  <!-- Mol* (PDBe) 3-D viewer is ~4.9 MB. We do NOT eager-load it on every
@@ -772,6 +772,23 @@
772
  <div class="run-meta" id="runMeta" hidden></div>
773
  <!-- Round-2 banner: set when a library was proposed from logged results. -->
774
  <div class="round2-banner" id="round2Banner" hidden></div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
775
 
776
  <!-- Mutation map (lollipop chart). Was carrying a chip-
777
  legend (Low / Med / High dots) that read as
@@ -2188,7 +2205,7 @@
2188
  <!-- Cloning reference data must load before app.js so the Designer
2189
  can read VECTORS / ENZYMES / CLONING_METHODS / TAGS / LINKERS. -->
2190
  <script src="/static/cloning_db.js?v=20260530-ui-polish" defer></script>
2191
- <script src="/static/app.js?v=20260714-consult-typewriter" defer></script>
2192
  <!-- Dwell-time heartbeat. Loads after auth.js so its /api/ping calls go
2193
  through the JWT-attaching fetch wrapper (signed-in attribution). -->
2194
  <script src="/static/telemetry.js?v=20260622-analytics" defer></script>
 
112
  <!-- ?v= query bumps invalidate browser + iframe asset caches when app.css /
113
  app.js change. Bump these numbers whenever you ship a frontend update —
114
  without them, users keep getting the stale file for up to a week. -->
115
+ <link rel="stylesheet" href="/static/app.css?v=20260714-learn-panel" />
116
  <link rel="icon" type="image/svg+xml" href="/static/favicon.svg?v=2" />
117
  <link rel="apple-touch-icon" href="/static/favicon.svg?v=2" />
118
  <!-- Mol* (PDBe) 3-D viewer is ~4.9 MB. We do NOT eager-load it on every
 
772
  <div class="run-meta" id="runMeta" hidden></div>
773
  <!-- Round-2 banner: set when a library was proposed from logged results. -->
774
  <div class="round2-banner" id="round2Banner" hidden></div>
775
+ <!-- "What Turing learned" — the round-2 re-rank + plain-language
776
+ reasons (dee.core.active_learning components/narrate, surfaced
777
+ via surrogate.pool_deltas). Only populated + shown when the
778
+ surrogate actually learned something (enough measurements +
779
+ real spread) — see _renderLearnedPanel in app.js. -->
780
+ <section class="learned-panel" id="learnedPanel" hidden>
781
+ <p class="card-kicker">&sect; What Turing learned</p>
782
+ <div class="learned-list" id="learnedList"></div>
783
+ </section>
784
+ <!-- Predicted-vs-measured calibration — client-side only, built
785
+ from the round-1 Predicted_Fitness_Score already in the table
786
+ matched against what the user just typed into the round-2
787
+ panel. See _renderCalibration in app.js. -->
788
+ <section class="calib-panel" id="calibPanel" hidden>
789
+ <p class="card-kicker">&sect; How well round 1 predicted your results</p>
790
+ <div class="calib-body" id="calibBody"></div>
791
+ </section>
792
 
793
  <!-- Mutation map (lollipop chart). Was carrying a chip-
794
  legend (Low / Med / High dots) that read as
 
2205
  <!-- Cloning reference data must load before app.js so the Designer
2206
  can read VECTORS / ENZYMES / CLONING_METHODS / TAGS / LINKERS. -->
2207
  <script src="/static/cloning_db.js?v=20260530-ui-polish" defer></script>
2208
+ <script src="/static/app.js?v=20260714-learn-panel" defer></script>
2209
  <!-- Dwell-time heartbeat. Loads after auth.js so its /api/ping calls go
2210
  through the JWT-attaching fetch wrapper (signed-in attribution). -->
2211
  <script src="/static/telemetry.js?v=20260622-analytics" defer></script>
tests/test_active_learning.py CHANGED
@@ -18,6 +18,7 @@ from dee.core.active_learning import (
18
  MIN_MEASUREMENTS,
19
  Surrogate,
20
  fit_surrogate,
 
21
  parse_label,
22
  parse_mutations,
23
  )
@@ -172,6 +173,56 @@ def test_adjust_pool_preserves_fields_and_swaps_score():
172
  assert pool[0].delta_ll == 1.0
173
 
174
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
175
  def test_unmeasured_mutation_falls_back_toward_prior():
176
  """A mutation never measured gets β≈0, so its adjusted score stays driven by
177
  the (re-weighted) prior plus only the exploration bonus — it isn't invented."""
 
18
  MIN_MEASUREMENTS,
19
  Surrogate,
20
  fit_surrogate,
21
+ narrate,
22
  parse_label,
23
  parse_mutations,
24
  )
 
173
  assert pool[0].delta_ll == 1.0
174
 
175
 
176
+ def test_components_populated_in_fallback():
177
+ """Below the measurement floor, components are still present (prior-only,
178
+ no correction, no explore) so a caller never has to special-case the
179
+ 'not learned yet' branch when rendering."""
180
+ pool = _pool({(0, "L"): 1.0, (1, "R"): 0.5})
181
+ meas = [([_label(0, "L")], 3.0)] # 1 < floor
182
+ s = fit_surrogate(pool, meas)
183
+ assert s.learned is False
184
+ assert s.components[(0, "L")] == {"prior": 1.0, "learned": 0.0, "explore": 0.0, "n_measured": 1}
185
+ assert s.components[(1, "R")] == {"prior": 0.5, "learned": 0.0, "explore": 0.0, "n_measured": 0}
186
+
187
+
188
+ def test_components_sum_to_adjusted_when_learned():
189
+ """prior + learned + explore must reconstruct adjusted[key] exactly —
190
+ components is a DECOMPOSITION of the same score, not a separate estimate."""
191
+ pool = _pool({(0, "L"): 1.0, (3, "R"): -0.5, (5, "K"): 0.2})
192
+ meas = [
193
+ ([_label(0, "L")], 9.0),
194
+ ([_label(3, "R")], 2.0),
195
+ ([_label(5, "K")], 5.0),
196
+ ([_label(0, "L"), _label(5, "K")], 8.0),
197
+ ([_label(3, "R"), _label(5, "K")], 4.0),
198
+ ]
199
+ s = fit_surrogate(pool, meas)
200
+ assert s.learned is True
201
+ for key, comp in s.components.items():
202
+ total = comp["prior"] + comp["learned"] + comp["explore"]
203
+ assert total == pytest.approx(s.adjusted[key])
204
+ assert comp["n_measured"] >= 0
205
+
206
+
207
+ def test_narrate_unmeasured():
208
+ text = narrate(prior=1.0, learned=0.0, explore=0.3, n_measured=0)
209
+ assert "not yet tested" in text.lower()
210
+
211
+
212
+ def test_narrate_consistent_with_small_correction():
213
+ text = narrate(prior=1.0, learned=0.01, explore=0.05, n_measured=3)
214
+ assert "consistent" in text.lower()
215
+ assert "3" in text
216
+
217
+
218
+ def test_narrate_revised_up_and_down():
219
+ up = narrate(prior=1.0, learned=0.5, explore=0.0, n_measured=4)
220
+ down = narrate(prior=1.0, learned=-0.5, explore=0.0, n_measured=4)
221
+ assert "helped more" in up.lower()
222
+ assert "underperformed" in down.lower()
223
+ assert up != down
224
+
225
+
226
  def test_unmeasured_mutation_falls_back_toward_prior():
227
  """A mutation never measured gets β≈0, so its adjusted score stays driven by
228
  the (re-weighted) prior plus only the exploration bonus — it isn't invented."""
tests/test_de_round2_learn_features.py ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tests for the Learn-phase "wow" additions to DE round 2
2
+ (dee.server._de_round2_library): pool_deltas (the before/after re-rank +
3
+ plain-language reasons) and global_prior (cross-user aggregate surfacing).
4
+
5
+ Exercises the real top_percentile_pool / evolve / variants_to_dataframe /
6
+ active_learning / aggregate pipeline on a tiny synthetic pool — only the ESM-2
7
+ scorer itself and the stored cross-user aggregate are mocked, so this proves
8
+ the new wiring (not just the isolated math already covered by
9
+ test_active_learning.py / test_aggregate.py)."""
10
+ import pandas as pd
11
+ import pytest
12
+
13
+ from dee import server
14
+ from dee.core.aggregate import GlobalPrior
15
+
16
+ _SETTINGS = {
17
+ "model": "small", "host": "e_coli", "percentile": 85.0, "k": 5,
18
+ "min_mutations": 1, "max_mutations": 2, "restarts": 1, "steps": 50, "seed": 1,
19
+ }
20
+
21
+
22
+ def _fake_scores_df(n=5):
23
+ # wt_aa is always 'A' -> labels are A1G, A2G, ... A5G.
24
+ return pd.DataFrame({
25
+ "position": list(range(n)), "wt_aa": ["A"] * n, "mut_aa": ["G"] * n,
26
+ "delta_ll": [float(i) - 2 for i in range(n)], # -2, -1, 0, 1, 2
27
+ })
28
+
29
+
30
+ def _label(i):
31
+ return f"A{i + 1}G"
32
+
33
+
34
+ @pytest.fixture(autouse=True)
35
+ def _mock_scorer(monkeypatch):
36
+ monkeypatch.setattr(server._scoring, "get_scorer", lambda *a, **kw: "the-scorer")
37
+ monkeypatch.setattr(server._scoring, "score_guarded",
38
+ lambda scorer, protein: _fake_scores_df())
39
+ monkeypatch.setattr(server, "top_percentile_pool", lambda df, percentile: df)
40
+
41
+
42
+ def test_pool_deltas_shape_sorted_and_reasoned_when_learned(monkeypatch):
43
+ monkeypatch.setattr(server, "_load_global_prior",
44
+ lambda: GlobalPrior(effects={}, n_users={}, n_obs={}))
45
+ # Enough varied measurements to clear MIN_MEASUREMENTS with real spread.
46
+ measurements = [
47
+ ([_label(0)], 1.0), ([_label(1)], 3.0), ([_label(2)], 5.0),
48
+ ([_label(3)], 7.0), ([_label(4)], 9.0),
49
+ ]
50
+ rows, info = server._de_round2_library("A" * 5, _SETTINGS, measurements)
51
+
52
+ assert info["learned"] is True
53
+ deltas = info["pool_deltas"]
54
+ assert 1 <= len(deltas) <= server._POOL_DELTAS_MAX
55
+ # Sorted descending by adjusted_score.
56
+ scores = [d["adjusted_score"] for d in deltas]
57
+ assert scores == sorted(scores, reverse=True)
58
+ for d in deltas:
59
+ assert set(d.keys()) == {"label", "prior_score", "adjusted_score",
60
+ "delta", "n_measured", "reason"}
61
+ assert isinstance(d["reason"], str) and d["reason"]
62
+ assert d["delta"] == pytest.approx(d["adjusted_score"] - d["prior_score"])
63
+ # Every measured mutation should show n_measured >= 1.
64
+ measured_labels = {_label(i) for i in range(5)}
65
+ assert all(d["n_measured"] >= 1 for d in deltas if d["label"] in measured_labels)
66
+ assert rows # a real variant table came back
67
+
68
+
69
+ def test_pool_deltas_all_zero_delta_when_not_enough_signal(monkeypatch):
70
+ monkeypatch.setattr(server, "_load_global_prior",
71
+ lambda: GlobalPrior(effects={}, n_users={}, n_obs={}))
72
+ # Only 1 measurement — below MIN_MEASUREMENTS -> honest fallback, no change.
73
+ rows, info = server._de_round2_library("A" * 5, _SETTINGS, [([_label(0)], 3.0)])
74
+ assert info["learned"] is False
75
+ assert all(d["delta"] == 0.0 for d in info["pool_deltas"])
76
+ assert rows
77
+
78
+
79
+ def test_global_prior_absent_reports_not_applied(monkeypatch):
80
+ monkeypatch.setattr(server, "_load_global_prior",
81
+ lambda: GlobalPrior(effects={}, n_users={}, n_obs={}))
82
+ _, info = server._de_round2_library("A" * 5, _SETTINGS, [([_label(0)], 3.0)])
83
+ assert info["global_prior"] == {"applied": False, "substitution_types": 0}
84
+
85
+
86
+ def test_global_prior_present_blends_and_reports_applied(monkeypatch):
87
+ # A field-wide prior that says A>G substitutions tend to be strongly
88
+ # favorable — should nudge prior_score upward vs the no-prior case, and
89
+ # be honestly reported (substitution_types == 1, the one key present).
90
+ gp = GlobalPrior(effects={("A", "G"): 2.0}, n_users={("A", "G"): 5}, n_obs={("A", "G"): 12})
91
+ monkeypatch.setattr(server, "_load_global_prior", lambda: gp)
92
+ _, info = server._de_round2_library("A" * 5, _SETTINGS, [([_label(0)], 3.0)])
93
+ assert info["global_prior"] == {"applied": True, "substitution_types": 1}
94
+ # Not enough measurements to learn, but the global-prior nudge still shows
95
+ # up in prior_score (baseline moved even though round 2 fell back).
96
+ unpatched_gp = GlobalPrior(effects={}, n_users={}, n_obs={})
97
+ monkeypatch.setattr(server, "_load_global_prior", lambda: unpatched_gp)
98
+ _, info_no_gp = server._de_round2_library("A" * 5, _SETTINGS, [([_label(0)], 3.0)])
99
+ by_label = {d["label"]: d["prior_score"] for d in info["pool_deltas"]}
100
+ by_label_no_gp = {d["label"]: d["prior_score"] for d in info_no_gp["pool_deltas"]}
101
+ assert by_label[_label(0)] > by_label_no_gp[_label(0)]
102
+
103
+
104
+ def test_round2_route_exposes_pool_deltas_and_global_prior(monkeypatch):
105
+ import types
106
+
107
+ app = server.create_app()
108
+ app.config.update(TESTING=True)
109
+ client = app.test_client()
110
+ monkeypatch.setattr(server._auth, "get_auth",
111
+ lambda: types.SimpleNamespace(anonymous=False, user_id="u1",
112
+ email="x@y.z", plan="free"))
113
+ monkeypatch.setattr(server._auth, "cleanup_expired_de_outcomes_async", lambda uid: None)
114
+ monkeypatch.setattr(server, "_load_global_prior",
115
+ lambda: GlobalPrior(effects={}, n_users={}, n_obs={}))
116
+
117
+ r = client.post("/api/de/round2", json={
118
+ "wt_protein": "A" * 5,
119
+ "measurements": [
120
+ {"mutations": _label(0), "measured_value": 1.0},
121
+ {"mutations": _label(1), "measured_value": 3.0},
122
+ {"mutations": _label(2), "measured_value": 5.0},
123
+ {"mutations": _label(3), "measured_value": 7.0},
124
+ ],
125
+ "settings": _SETTINGS,
126
+ })
127
+ assert r.status_code == 200
128
+ body = r.get_json()
129
+ assert body["ok"] is True
130
+ assert body["round"] == 2
131
+ assert "pool_deltas" in body["surrogate"]
132
+ assert "global_prior" in body["surrogate"]
133
+
134
+
135
+ def test_result_route_defaults_global_prior_when_never_set():
136
+ app = server.create_app()
137
+ app.config.update(TESTING=True)
138
+ client = app.test_client()
139
+ job = server.JobState(job_id="j1", status="done", wt_identifier="WT",
140
+ wt_protein="ACDEFG", variants=[])
141
+ with server._JOBS_LOCK:
142
+ server._JOBS["j1"] = job
143
+ r = client.get("/api/result/j1")
144
+ assert r.status_code == 200
145
+ assert r.get_json()["global_prior"] == {"applied": False, "substitution_types": 0}
146
+
147
+
148
+ def test_result_route_surfaces_global_prior_when_set():
149
+ app = server.create_app()
150
+ app.config.update(TESTING=True)
151
+ client = app.test_client()
152
+ job = server.JobState(job_id="j2", status="done", wt_identifier="WT",
153
+ wt_protein="ACDEFG", variants=[],
154
+ global_prior_info={"applied": True, "substitution_types": 7})
155
+ with server._JOBS_LOCK:
156
+ server._JOBS["j2"] = job
157
+ r = client.get("/api/result/j2")
158
+ assert r.status_code == 200
159
+ assert r.get_json()["global_prior"] == {"applied": True, "substitution_types": 7}