Miyu Horiuchi Claude Opus 4.7 (1M context) commited on
Commit
4570b64
·
1 Parent(s): d23315e

Streamlit UI — catalog-first design with NCBI name resolution

Browse files

app.py: three-tab UI for browsing predictions and testing the model.

🦠 Catalog tab (the primary view):
- Toggle to filter to genuinely never-cultured candidates (1,294 of 5,000)
— distinguishes from "absent from BacDive but cultured elsewhere"
(e.g. Mycobacterium clinical isolates).
- Quick-filter pills: thermophiles, psychrophiles, anaerobes, halotolerant.
- Compact table with predicted top medium + 4 phenotypes inline.
- Sorted by top-medium confidence so highest-conviction candidates float.
- CSV export of filtered set.

🔬 Test tab (verify the model):
- Three preset sanity-check buttons (E. coli, B. subtilis, T. thermophilus)
that show predicted-vs-published with ✅/⚠️ marks at 80% interval level.
- Smart search: type an organism name (e.g. "Thermus thermophilus") and
hit NCBI E-utilities (JSON REST via requests, not Biopython Entrez to
bypass an SSL chain issue on this machine). Sorted by assembly level so
Complete Genome surfaces first.
- Accessions (GCA_/GCF_ prefix) bypass the search and run directly.
- FASTA upload as the third path.

📊 About tab:
- One-line verdict + per-target plain-English interpretation
("MAE 3°C means you'd pick the right incubator shelf").
- Methodology, full eval reports in expanders.

Sidebar (collapsed by default):
- Tool overview, training-set sizes, methodology, how to read predictions.

Bug fix: ProgressColumn(format="%.0f%%") formats raw value, not percent —
displayed 99% as "1%". Fixed by scaling to 0-100 before display.

pyproject.toml: added [project.optional-dependencies] ui = [streamlit, altair].

Run: uv run --extra ui streamlit run app.py

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (3) hide show
  1. app.py +643 -0
  2. pyproject.toml +4 -0
  3. uv.lock +0 -0
app.py ADDED
@@ -0,0 +1,643 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Streamlit UI for microbe-model.
2
+
3
+ Catalog-first design: the primary view is the 5,000 never-cultured candidates,
4
+ each annotated with a recommended medium. Secondary tabs let you verify the model
5
+ on known organisms or run on a custom genome.
6
+
7
+ Run:
8
+ uv run --extra ui streamlit run app.py
9
+ """
10
+ from __future__ import annotations
11
+
12
+ import json
13
+ import sys
14
+ from pathlib import Path
15
+
16
+ import pandas as pd
17
+ import streamlit as st
18
+
19
+ ROOT = Path(__file__).resolve().parent
20
+ sys.path.insert(0, str(ROOT / "scripts"))
21
+
22
+ import os # noqa: E402
23
+
24
+ import requests # noqa: E402
25
+
26
+ from microbe_model import config # noqa: E402
27
+ from microbe_model.train.media_recommender import load_models # noqa: E402
28
+ from recommend import ( # noqa: E402
29
+ _format_recipe_summary,
30
+ _load_genome_features,
31
+ _predict_phenotypes,
32
+ )
33
+
34
+ EUTILS_BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils"
35
+
36
+
37
+ def _is_accession(s: str) -> bool:
38
+ s = s.strip().upper()
39
+ return s.startswith(("GCA_", "GCF_"))
40
+
41
+
42
+ @st.cache_data(ttl=3600, show_spinner=False)
43
+ def search_ncbi_assembly(name: str, retmax: int = 10) -> list[dict]:
44
+ """Look up assemblies for an organism name via NCBI E-utilities (JSON REST)."""
45
+ if not name.strip():
46
+ return []
47
+ api_key = os.environ.get("NCBI_API_KEY")
48
+ common_params = {"api_key": api_key} if api_key else {}
49
+ try:
50
+ r = requests.get(
51
+ f"{EUTILS_BASE}/esearch.fcgi",
52
+ params={
53
+ "db": "assembly",
54
+ "term": f"{name}[Organism] AND latest[filter]",
55
+ "retmode": "json",
56
+ "retmax": retmax,
57
+ **common_params,
58
+ },
59
+ timeout=20,
60
+ )
61
+ r.raise_for_status()
62
+ ids = r.json().get("esearchresult", {}).get("idlist", [])
63
+ if not ids:
64
+ return []
65
+ r = requests.get(
66
+ f"{EUTILS_BASE}/esummary.fcgi",
67
+ params={
68
+ "db": "assembly",
69
+ "id": ",".join(ids),
70
+ "retmode": "json",
71
+ **common_params,
72
+ },
73
+ timeout=20,
74
+ )
75
+ r.raise_for_status()
76
+ result = r.json().get("result", {})
77
+ except requests.RequestException as e:
78
+ st.error(f"NCBI search failed: {e}")
79
+ return []
80
+ out = []
81
+ for uid in result.get("uids", []):
82
+ doc = result.get(uid, {})
83
+ out.append({
84
+ "accession": str(doc.get("assemblyaccession", "")),
85
+ "organism": str(doc.get("organism", "")),
86
+ "level": str(doc.get("assemblystatus", "")),
87
+ "size_mb": float(doc.get("total_length") or doc.get("assemblylength") or 0) / 1e6,
88
+ "submitter": str(doc.get("submitterorganization", "")),
89
+ })
90
+ # Prefer complete genomes first
91
+ level_rank = {"Complete Genome": 0, "Chromosome": 1, "Scaffold": 2, "Contig": 3}
92
+ out.sort(key=lambda r: level_rank.get(r["level"], 99))
93
+ return out
94
+
95
+ st.set_page_config(
96
+ page_title="microbe-model — what to grow it in",
97
+ page_icon="🦠",
98
+ layout="wide",
99
+ initial_sidebar_state="collapsed",
100
+ )
101
+
102
+
103
+ # ──────────────────────────────────────────────────────────────────────
104
+ # Cached loaders
105
+ # ──────────────────────────────────────────────────────────────────────
106
+ @st.cache_data
107
+ def load_results():
108
+ p = config.ARTIFACTS / "baseline_results.json"
109
+ if not p.exists():
110
+ return {}
111
+ data = json.loads(p.read_text())
112
+ data.pop("__meta__", None)
113
+ return data
114
+
115
+
116
+ @st.cache_resource
117
+ def load_recommender():
118
+ return load_models(config.ROOT / "models" / "recommender")
119
+
120
+
121
+ @st.cache_data
122
+ def load_uncultured() -> pd.DataFrame:
123
+ return pd.read_parquet(config.ARTIFACTS / "uncultured_predictions.parquet")
124
+
125
+
126
+ @st.cache_data
127
+ def load_media_meta() -> pd.DataFrame:
128
+ return pd.read_parquet(config.DATA / "media_metadata.parquet")
129
+
130
+
131
+ @st.cache_data
132
+ def load_recipes() -> pd.DataFrame:
133
+ return pd.read_parquet(config.DATA / "media_recipes.parquet")
134
+
135
+
136
+ # ──────────────────────────────────────────────────────────────────────
137
+ # Known organisms for sanity-check (predicted vs published)
138
+ # ──────────────────────────────────────────────────────────────────────
139
+ SANITY_ORGANISMS = [
140
+ {
141
+ "accession": "GCF_000005845.2",
142
+ "name": "Escherichia coli K-12 MG1655",
143
+ "known": {
144
+ "optimal_temperature_c": 37.0,
145
+ "optimal_ph": 7.0,
146
+ "oxygen_requirement": "facultative anaerobe",
147
+ "salt_tolerance_pct": 1.0,
148
+ "medium": "LB (Luria-Bertani)",
149
+ },
150
+ },
151
+ {
152
+ "accession": "GCF_000009045.1",
153
+ "name": "Bacillus subtilis 168",
154
+ "known": {
155
+ "optimal_temperature_c": 30.0,
156
+ "optimal_ph": 7.0,
157
+ "oxygen_requirement": "facultative anaerobe",
158
+ "salt_tolerance_pct": 2.0,
159
+ "medium": "LB or Nutrient Broth",
160
+ },
161
+ },
162
+ {
163
+ "accession": "GCF_000091545.1",
164
+ "name": "Thermus thermophilus HB8",
165
+ "known": {
166
+ "optimal_temperature_c": 70.0,
167
+ "optimal_ph": 7.5,
168
+ "oxygen_requirement": "aerobe",
169
+ "salt_tolerance_pct": 0.5,
170
+ "medium": "DSMZ 74 Castenholz TYE",
171
+ },
172
+ },
173
+ ]
174
+
175
+
176
+ def _phylum_from_taxonomy(tax: str | None) -> str:
177
+ if not isinstance(tax, str):
178
+ return "(unknown)"
179
+ for part in tax.split(";"):
180
+ part = part.strip()
181
+ if part.startswith("p__"):
182
+ return part[3:] or "(unclassified)"
183
+ return "(unknown)"
184
+
185
+
186
+ def _run_inference(target: str):
187
+ """Resolve a genome (accession or path), predict phenotypes + media. Returns dict."""
188
+ feats, acc, n_contigs = _load_genome_features(target)
189
+ feats_series = pd.Series(feats)
190
+ phenotypes = _predict_phenotypes(feats_series)
191
+
192
+ models, feature_cols = load_recommender()
193
+ media_meta = load_media_meta()
194
+ recipes = load_recipes()
195
+ name_by_id = dict(
196
+ zip(media_meta["medium_id"].astype(str), media_meta["name"], strict=True)
197
+ )
198
+ X_pred = feats_series[feature_cols].to_frame().T
199
+ recs = []
200
+ for medium_id, model in models.items():
201
+ proba = float(model.predict_proba(X_pred)[0, 1])
202
+ recs.append({
203
+ "medium_id": medium_id,
204
+ "name": name_by_id.get(medium_id, "(unknown)"),
205
+ "confidence": proba,
206
+ "recipe": _format_recipe_summary(medium_id, recipes),
207
+ })
208
+ recs.sort(key=lambda r: r["confidence"], reverse=True)
209
+ return {
210
+ "accession": acc,
211
+ "n_contigs": n_contigs,
212
+ "n_cds": int(feats["n_predicted_cds"]),
213
+ "gc": float(feats["gc_content"]),
214
+ "phenotypes": phenotypes,
215
+ "media": recs,
216
+ }
217
+
218
+
219
+ # ──────────────────────────────────────────────────────────────────────
220
+ # Header
221
+ # ──────────────────────────────────────────────────────────────────────
222
+ st.title("🦠 microbe-model")
223
+ st.markdown("##### Pick a never-cultured microbe and see what to grow it in")
224
+ st.caption(
225
+ "5,000 microbes from GTDB that have never been grown in a lab — each one "
226
+ "scored against 24 standard DSMZ media. Browse, filter, pick something to try. "
227
+ "All predictions come from a model trained on 17,000 BacDive strains "
228
+ "(5-fold cross-validation by family)."
229
+ )
230
+
231
+
232
+ tab_catalog, tab_test, tab_about = st.tabs(
233
+ ["🦠 Catalog of uncultured microbes", "🔬 Test on a known genome", "📊 How accurate is it?"]
234
+ )
235
+
236
+
237
+ # ──────────────────────────────────────────────────────────────────────
238
+ # Tab 1 — Catalog (the main product)
239
+ # ──────────────────────────────────────────────────────────────────────
240
+ with tab_catalog:
241
+ unc_all = load_uncultured().copy()
242
+ unc_all["phylum"] = unc_all["gtdb_taxonomy"].map(_phylum_from_taxonomy)
243
+ unc_all["is_genuinely_uncultured"] = (
244
+ unc_all["ncbi_organism_name"].fillna("").str.lower().str.startswith("uncultured")
245
+ )
246
+ n_genuine = int(unc_all["is_genuinely_uncultured"].sum())
247
+
248
+ only_uncultured = st.toggle(
249
+ f"Only show genuinely never-cultured microbes ({n_genuine:,} of {len(unc_all):,})",
250
+ value=True,
251
+ help="GTDB lists 5,000 candidates that aren't in BacDive, but many — like "
252
+ "Mycobacterium clinical isolates — have actually been cultured (just not by "
253
+ "BacDive). When this toggle is on, we restrict to genomes whose NCBI "
254
+ "organism name explicitly starts with 'uncultured'. These are the ones "
255
+ "with no published cultivation conditions, where this tool is genuinely useful.",
256
+ )
257
+ unc = unc_all[unc_all["is_genuinely_uncultured"]] if only_uncultured else unc_all
258
+ unc = unc.copy()
259
+
260
+ # Quick-filter pills
261
+ st.markdown("**What kind of microbe are you looking for?**")
262
+ pcols = st.columns(5)
263
+ if "catalog_preset" not in st.session_state:
264
+ st.session_state["catalog_preset"] = "all"
265
+ if pcols[0].button("All 5,000", use_container_width=True):
266
+ st.session_state["catalog_preset"] = "all"
267
+ if pcols[1].button("🔥 Thermophiles (>55°C)", use_container_width=True):
268
+ st.session_state["catalog_preset"] = "thermo"
269
+ if pcols[2].button("❄️ Psychrophiles (<15°C)", use_container_width=True):
270
+ st.session_state["catalog_preset"] = "psychro"
271
+ if pcols[3].button("🚫 Anaerobes", use_container_width=True):
272
+ st.session_state["catalog_preset"] = "anaerobe"
273
+ if pcols[4].button("🧂 Halotolerant (>3% salt)", use_container_width=True):
274
+ st.session_state["catalog_preset"] = "halo"
275
+
276
+ preset = st.session_state["catalog_preset"]
277
+
278
+ mask = pd.Series(True, index=unc.index)
279
+ if preset == "thermo":
280
+ mask &= unc["pred_optimal_temperature_c"] > 55
281
+ elif preset == "psychro":
282
+ mask &= unc["pred_optimal_temperature_c"] < 15
283
+ elif preset == "anaerobe":
284
+ mask &= unc["pred_oxygen_requirement"].isin(
285
+ ["anaerobe", "obligate anaerobe", "facultative anaerobe"]
286
+ )
287
+ elif preset == "halo":
288
+ mask &= unc["pred_salt_tolerance_pct"] > 3
289
+
290
+ with st.expander("⚙️ More filters", expanded=False):
291
+ c1, c2 = st.columns(2)
292
+ with c1:
293
+ phyla = sorted(unc["phylum"].dropna().unique().tolist())
294
+ sel_phyla = st.multiselect("Phylum", phyla, default=[])
295
+ min_completeness = st.slider("Min CheckM completeness (%)", 50, 100, 90)
296
+ with c2:
297
+ search = st.text_input("Search organism name", "")
298
+
299
+ if sel_phyla:
300
+ mask &= unc["phylum"].isin(sel_phyla)
301
+ mask &= unc["checkm_completeness"] >= min_completeness
302
+ if search:
303
+ mask &= unc["ncbi_organism_name"].fillna("").str.contains(
304
+ search, case=False, na=False
305
+ )
306
+
307
+ filtered = unc.loc[mask].copy()
308
+ if "top1_confidence" in filtered.columns:
309
+ filtered = filtered.sort_values("top1_confidence", ascending=False)
310
+
311
+ st.markdown(f"**{len(filtered):,}** of {len(unc):,} candidates match.")
312
+
313
+ # Compact, focused table — what to grow it in is the headline column
314
+ display = filtered[[
315
+ "genome_accession",
316
+ "ncbi_organism_name",
317
+ "phylum",
318
+ "top1_medium_name",
319
+ "top1_confidence",
320
+ "pred_optimal_temperature_c",
321
+ "pred_optimal_ph",
322
+ "pred_oxygen_requirement",
323
+ "pred_salt_tolerance_pct",
324
+ "checkm_completeness",
325
+ ]].copy()
326
+ # ProgressColumn displays the raw value, so scale 0-1 → 0-100 for percent rendering
327
+ display["top1_confidence"] = (display["top1_confidence"] * 100).round(1)
328
+
329
+ st.dataframe(
330
+ display,
331
+ hide_index=True,
332
+ use_container_width=True,
333
+ height=520,
334
+ column_config={
335
+ "genome_accession": "Accession",
336
+ "ncbi_organism_name": "Organism",
337
+ "phylum": "Phylum",
338
+ "top1_medium_name": st.column_config.TextColumn("👉 Try this medium", width="medium"),
339
+ "top1_confidence": st.column_config.ProgressColumn(
340
+ "Confidence", min_value=0, max_value=100, format="%.1f%%",
341
+ help="Probability the trained classifier assigns to this medium. "
342
+ "Higher = the model is more sure. Uncultured strains close to "
343
+ "well-studied groups (e.g. uncultured Mycobacterium → MIDDLEBROOK) "
344
+ "score very high; phylogenetically isolated genomes score lower.",
345
+ ),
346
+ "pred_optimal_temperature_c": st.column_config.NumberColumn("T (°C)", format="%.0f"),
347
+ "pred_optimal_ph": st.column_config.NumberColumn("pH", format="%.1f"),
348
+ "pred_oxygen_requirement": "O₂",
349
+ "pred_salt_tolerance_pct": st.column_config.NumberColumn("Salt %", format="%.1f"),
350
+ "checkm_completeness": st.column_config.NumberColumn("CheckM %", format="%.0f"),
351
+ },
352
+ )
353
+
354
+ st.caption(
355
+ "📋 **How to read this**: the model predicts each row's optimal growth conditions, "
356
+ "then ranks 24 DSMZ media by predicted growth probability. The "
357
+ "*'👉 Try this medium'* column is its top pick. Confidence is the classifier's "
358
+ "predicted probability. Click a row's accession and paste it in **🔬 Test on a known "
359
+ "genome** to see the full ranked list with recipes and uncertainty intervals."
360
+ )
361
+
362
+ csv = filtered.to_csv(index=False).encode()
363
+ st.download_button(
364
+ "⬇️ Download filtered set as CSV",
365
+ csv,
366
+ file_name="microbe_model_uncultured_candidates.csv",
367
+ mime="text/csv",
368
+ )
369
+
370
+
371
+ # ──────────────────────────────────────────────────────────────────────
372
+ # Tab 2 — Test on a genome (sanity check + custom)
373
+ # ───────────────────────────────────────────────────────���──────────────
374
+ with tab_test:
375
+ st.markdown("### Verify the model on organisms with known biology")
376
+ st.caption(
377
+ "These three organisms have well-published growth conditions. Click "
378
+ "to predict — if the predictions match the literature, the model is working."
379
+ )
380
+
381
+ scols = st.columns(len(SANITY_ORGANISMS))
382
+ for col, org in zip(scols, SANITY_ORGANISMS, strict=True):
383
+ with col:
384
+ with st.container(border=True):
385
+ st.markdown(f"**{org['name']}**")
386
+ k = org["known"]
387
+ st.caption(
388
+ f"Known: {k['optimal_temperature_c']:.0f}°C, "
389
+ f"pH {k['optimal_ph']:.1f}, "
390
+ f"{k['oxygen_requirement']}, "
391
+ f"~{k['salt_tolerance_pct']}% salt → *{k['medium']}*"
392
+ )
393
+ if st.button(f"Predict {org['name'].split()[0]}", key=f"sanity_{org['accession']}", use_container_width=True):
394
+ st.session_state["test_target"] = org["accession"]
395
+ st.session_state["test_known"] = org["known"]
396
+ st.session_state["test_run"] = True
397
+
398
+ st.markdown("---")
399
+ st.markdown("### Or test on any organism")
400
+ st.caption(
401
+ "Type an organism name (e.g. *Thermus thermophilus*) or paste an "
402
+ "NCBI assembly accession. We'll resolve names through NCBI Assembly automatically."
403
+ )
404
+
405
+ with st.form("name_or_accession_form"):
406
+ query = st.text_input(
407
+ "Organism name or NCBI accession",
408
+ value=st.session_state.get("test_target", ""),
409
+ placeholder='e.g. "Thermus thermophilus" or GCF_000005845.2',
410
+ )
411
+ uploaded = st.file_uploader(
412
+ "…or upload a FASTA file directly",
413
+ type=["fna", "fa", "fasta", "gz"],
414
+ )
415
+ top_k = st.slider("Number of media to show", 3, 15, 5)
416
+ submit = st.form_submit_button("🔎 Search / 🚀 Run", type="primary", use_container_width=True)
417
+
418
+ auto = st.session_state.pop("test_run", False)
419
+ known = st.session_state.pop("test_known", None)
420
+
421
+ target = None
422
+ if uploaded is not None:
423
+ tmp = ROOT / "data" / "_uploaded" / uploaded.name
424
+ tmp.parent.mkdir(parents=True, exist_ok=True)
425
+ tmp.write_bytes(uploaded.getbuffer())
426
+ target = str(tmp)
427
+ elif submit and query.strip() and _is_accession(query):
428
+ target = query.strip()
429
+ elif submit and query.strip():
430
+ # It's a name — do NCBI search and let the user pick
431
+ with st.spinner(f"Searching NCBI Assembly for '{query.strip()}'…"):
432
+ hits = search_ncbi_assembly(query.strip(), retmax=10)
433
+ if not hits:
434
+ st.warning(
435
+ f"No NCBI Assembly hits for '{query.strip()}'. Try a different "
436
+ "spelling, a broader name (e.g. genus only), or paste an accession directly."
437
+ )
438
+ else:
439
+ st.session_state["ncbi_hits"] = hits
440
+ elif auto:
441
+ target = st.session_state.get("test_target") or None
442
+
443
+ # If we have NCBI hits from a previous form submission, render the picker
444
+ hits = st.session_state.get("ncbi_hits", [])
445
+ if hits and target is None:
446
+ st.markdown(f"**{len(hits)} NCBI matches** — pick one to predict:")
447
+ labels = [
448
+ f"**{h['accession']}** — {h['organism']} · {h['level']} · "
449
+ f"{h['size_mb']:.2f} Mb · {h['submitter'] or '—'}"
450
+ for h in hits
451
+ ]
452
+ choice = st.radio(
453
+ "Select assembly",
454
+ options=list(range(len(hits))),
455
+ format_func=lambda i: labels[i],
456
+ label_visibility="collapsed",
457
+ )
458
+ if st.button("🚀 Run on selected assembly", type="primary"):
459
+ target = hits[choice]["accession"]
460
+ st.session_state.pop("ncbi_hits", None)
461
+
462
+ if not target and submit and not query.strip() and uploaded is None:
463
+ st.error("Provide a name, accession, or FASTA file.")
464
+
465
+ if target:
466
+ with st.spinner(f"Resolving and running on `{target}`…"):
467
+ try:
468
+ result = _run_inference(target)
469
+ except SystemExit as e:
470
+ st.error(str(e))
471
+ st.stop()
472
+
473
+ with st.container(border=True):
474
+ m1, m2, m3, m4 = st.columns(4)
475
+ m1.metric("Genome", result["accession"])
476
+ m2.metric("Contigs", result["n_contigs"])
477
+ m3.metric("Predicted CDS", f"{result['n_cds']:,}")
478
+ m4.metric("GC content", f"{result['gc']:.1%}")
479
+
480
+ # If we have known values, show predicted-vs-known table
481
+ if known:
482
+ st.markdown("### Predicted vs published")
483
+ rows = []
484
+ p = result["phenotypes"]
485
+ pt = p.get("optimal_temperature_c", {})
486
+ if pt:
487
+ in_range = (pt.get("low_80", 0) <= known["optimal_temperature_c"] <= pt.get("high_80", 1e9))
488
+ rows.append({
489
+ "Property": "Optimal temperature",
490
+ "Predicted": f"{pt['prediction']:.1f}°C ({pt.get('low_80', 0):.1f}–{pt.get('high_80', 0):.1f})",
491
+ "Published": f"{known['optimal_temperature_c']:.0f}°C",
492
+ "Check": "✅ within 80% interval" if in_range else "⚠️ outside 80% interval",
493
+ })
494
+ pph = p.get("optimal_ph", {})
495
+ if pph:
496
+ in_range = (pph.get("low_80", 0) <= known["optimal_ph"] <= pph.get("high_80", 1e9))
497
+ rows.append({
498
+ "Property": "Optimal pH",
499
+ "Predicted": f"{pph['prediction']:.2f} ({pph.get('low_80', 0):.2f}–{pph.get('high_80', 0):.2f})",
500
+ "Published": f"{known['optimal_ph']:.1f}",
501
+ "Check": "✅ within 80% interval" if in_range else "⚠️ outside 80% interval",
502
+ })
503
+ pox = p.get("oxygen_requirement", {})
504
+ if pox:
505
+ match = (pox["prediction"] == known["oxygen_requirement"])
506
+ rows.append({
507
+ "Property": "Oxygen requirement",
508
+ "Predicted": f"{pox['prediction']} ({pox['confidence']:.0%})",
509
+ "Published": known["oxygen_requirement"],
510
+ "Check": "✅ match" if match else "⚠️ mismatch",
511
+ })
512
+ ps = p.get("salt_tolerance_pct", {})
513
+ if ps:
514
+ in_range = (ps.get("low_80", 0) <= known["salt_tolerance_pct"] <= ps.get("high_80", 1e9))
515
+ rows.append({
516
+ "Property": "Salt tolerance",
517
+ "Predicted": f"{ps['prediction']:.2f}% ({ps.get('low_80', 0):.2f}–{ps.get('high_80', 0):.2f})",
518
+ "Published": f"~{known['salt_tolerance_pct']:.1f}%",
519
+ "Check": "✅ within 80% interval" if in_range else "⚠️ outside 80% interval",
520
+ })
521
+ if result["media"]:
522
+ top1 = result["media"][0]
523
+ rows.append({
524
+ "Property": "Top medium",
525
+ "Predicted": f"{top1['name']} ({top1['confidence']:.0%})",
526
+ "Published": known["medium"],
527
+ "Check": "—",
528
+ })
529
+ st.dataframe(pd.DataFrame(rows), hide_index=True, use_container_width=True)
530
+ st.caption(
531
+ "✅ in the Check column means predicted and published agree at the "
532
+ "model's stated 80% confidence level. ⚠️ means the model disagrees "
533
+ "with the literature for this organism."
534
+ )
535
+ else:
536
+ st.markdown("### Predicted growth conditions")
537
+ pcols = st.columns(4)
538
+ phen = result["phenotypes"]
539
+ for col, (key, label, unit) in zip(
540
+ pcols,
541
+ [
542
+ ("optimal_temperature_c", "Temperature", "°C"),
543
+ ("optimal_ph", "pH", ""),
544
+ ("oxygen_requirement", "Oxygen", ""),
545
+ ("salt_tolerance_pct", "Salt tolerance", "%"),
546
+ ],
547
+ strict=True,
548
+ ):
549
+ info = phen.get(key)
550
+ with col:
551
+ if info is None:
552
+ st.metric(label, "—")
553
+ elif info["task"] == "regression":
554
+ st.metric(label, f"{info['prediction']:.2f}{unit}")
555
+ low = info.get("low_80")
556
+ high = info.get("high_80")
557
+ if low is not None and high is not None:
558
+ st.caption(f"80% CI: {low:.2f}–{high:.2f}{unit}")
559
+ else:
560
+ st.metric(label, info["prediction"])
561
+ st.caption(f"Confidence: {info['confidence']:.0%}")
562
+
563
+ st.markdown(f"### Top {top_k} recommended media")
564
+ for i, r in enumerate(result["media"][:top_k], 1):
565
+ with st.container(border=True):
566
+ a, b = st.columns([3, 1])
567
+ with a:
568
+ st.markdown(f"**{i}. DSMZ Medium {r['medium_id']} — {r['name']}**")
569
+ if r["recipe"]:
570
+ st.caption(f"Top compounds: {r['recipe']}")
571
+ with b:
572
+ st.progress(min(max(r["confidence"], 0), 1), text=f"{r['confidence']:.0%}")
573
+
574
+
575
+ # ──────────────────────────────────────────────────────────────────────
576
+ # Tab 3 — About / accuracy
577
+ # ──────────────────────────────────────────────────────────────────────
578
+ with tab_about:
579
+ st.markdown("### How accurate is the model?")
580
+ st.info(
581
+ "Cross-validation by family (the model never sees the same family in train + test). "
582
+ "Numbers below are mean across 5 folds."
583
+ )
584
+ results = load_results()
585
+ if not results:
586
+ st.warning("No results found.")
587
+ else:
588
+ cards = [
589
+ ("Temperature", f"MAE = {results.get('optimal_temperature_c', {}).get('mean_metric', 0):.2f}°C",
590
+ "n = 17,007 strains. Useful: most labs incubate in 5°C steps (25/30/37/45). "
591
+ "An MAE of ~3°C means you'd usually pick the right shelf."),
592
+ ("pH", f"MAE = {results.get('optimal_ph', {}).get('mean_metric', 0):.2f}",
593
+ "n = 4,652 strains. Marginal. Distinguishes ‘acidic’ vs ‘neutral’ vs ‘alkaline’ "
594
+ "but not finer than that."),
595
+ ("Oxygen", f"F1 = {results.get('oxygen_requirement', {}).get('mean_metric', 0):.2f}",
596
+ "n = 10,426 strains, 9 imbalanced classes. Weak — frequent confusion between "
597
+ "aerobe ↔ aerotolerant. Use predictions as a coarse hint, not a definitive answer."),
598
+ ("Salt tolerance", f"MAE = {results.get('salt_tolerance_pct', {}).get('mean_metric', 0):.2f}%",
599
+ "n = 4,793 strains. Decent. Distinguishes freshwater (<1%) from marine (~2.5%) "
600
+ "from halotolerant (>5%)."),
601
+ ]
602
+ ccols = st.columns(4)
603
+ for col, (label, metric, note) in zip(ccols, cards, strict=True):
604
+ with col:
605
+ with st.container(border=True):
606
+ st.markdown(f"**{label}**")
607
+ st.markdown(f"##### {metric}")
608
+ st.caption(note)
609
+
610
+ st.markdown("### Methodology")
611
+ st.markdown(
612
+ """
613
+ - **Training set:** 17,047 bacterial strains from BacDive with at least one phenotype label
614
+ and a public NCBI genome.
615
+ - **Features:** 353 genome statistics — GC content, codon usage, tetranucleotide
616
+ frequencies, amino-acid composition, gene density, predicted-CDS stats.
617
+ - **Model:** XGBoost (one model per phenotype target). Quantile regression at α = 0.1, 0.5, 0.9
618
+ for regression targets to produce 80% prediction intervals.
619
+ - **Media recommender:** 24 binary classifiers, one per DSMZ medium, trained on
620
+ strain↔medium links from MediaDive.
621
+ - **Cross-validation:** 5-fold **GroupKFold by family** — folds split at the family
622
+ level so the model never sees the same family in train and test. This is the
623
+ honest test for whether a prediction generalizes to a phylogenetically novel genome.
624
+ - **Uncultured candidates:** 5,000 GTDB MAGs that are not in BacDive, scored against
625
+ the trained models. Genomes filtered to ≥50% CheckM completeness.
626
+
627
+ We also tested ESM-2 (8M-parameter protein language model) embeddings as
628
+ features, alone and combined with v1. Results: ESM-2 alone underperforms v1;
629
+ combining wins everywhere but only meaningfully on oxygen requirement (+5% F1).
630
+ The remaining error is likely a data ceiling — BacDive only records *successful*
631
+ cultivations, never failures.
632
+ """
633
+ )
634
+
635
+ eval_path = config.ARTIFACTS / "eval_report.md"
636
+ if eval_path.exists():
637
+ with st.expander("📄 Full v1 eval report (per-family error, correlations)"):
638
+ st.markdown(eval_path.read_text())
639
+
640
+ cmp_path = config.ARTIFACTS / "v1_vs_v2_comparison.md"
641
+ if cmp_path.exists():
642
+ with st.expander("📄 v1 vs v2 ESM-2 comparison"):
643
+ st.markdown(cmp_path.read_text())
pyproject.toml CHANGED
@@ -27,6 +27,10 @@ embeddings = [
27
  "transformers>=4.40",
28
  "accelerate>=0.30",
29
  ]
 
 
 
 
30
 
31
  [build-system]
32
  requires = ["hatchling"]
 
27
  "transformers>=4.40",
28
  "accelerate>=0.30",
29
  ]
30
+ ui = [
31
+ "streamlit>=1.30",
32
+ "altair>=5",
33
+ ]
34
 
35
  [build-system]
36
  requires = ["hatchling"]
uv.lock CHANGED
The diff for this file is too large to render. See raw diff