Dimitris commited on
Commit
df889fb
Β·
1 Parent(s): ba4739e

docs: add revised plan grounded in current repo state

Browse files

Maps what's done (extraction MVP + UI) to the remaining work and prizes:
interpretation + cross-marker reasoning, offline llama.cpp transition,
fine-tune on Modal, agent trace, eval, and submission artifacts.

Files changed (1) hide show
  1. PLAN.md +111 -0
PLAN.md ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Blood Test Explainer ("Pulse") β€” Plan, revised against current repo
2
+
3
+ > Grounded in the actual code as of 2026-06-09. Two developers, parallel.
4
+ > Target prizes: **OpenAI + OpenBMB + Modal** + all badges.
5
+ > ⚠️ Confirm the real submission deadline first β€” teams are already final-submitting.
6
+
7
+ ---
8
+
9
+ ## 1. Where we are (DONE β€” solid foundation)
10
+
11
+ - **Polished Gradio app** (`app.py`): light clinical theme, animated "formation" hero,
12
+ loading/empty states, report cards with status pills, tabs (Report / Values / JSON / Raw),
13
+ responsive. This is genuinely good and is a real head start on the "premium document" wow.
14
+ - **Extraction works** (`src/openbmb_client.py`, `src/document_processing.py`): OpenBMB
15
+ **MiniCPM-V-4.6** (vision) via OpenAI-compatible API. PDF→images (PyMuPDF), images, and
16
+ text files supported; base64 image payloads; robust JSON-repair parsing.
17
+ - **Schema:** `{marker, value, unit, reference_range, status, source_text, confidence}` + notes.
18
+ - **Env/secret handling** ready (`.env` local, Space secret). Codex is contributing.
19
+ - Extraction prompt is deliberately **extraction-only** ("do not diagnose/interpret").
20
+
21
+ ## 2. Blunt gap analysis β€” what's missing and which prize it unblocks
22
+
23
+ The current app is a beautiful **extractor that calls an external API**. As-is it does not win.
24
+
25
+ | Missing piece | Why it matters | Unblocks |
26
+ |---|---|---|
27
+ | **Interpretation + cross-marker reasoning** | Right now MiniCPM looks like OCR/plumbing. The model must *reason* (e.g. "ALT+AST+GGT all high β†’ liver-enzyme pattern") to be visibly central. | **OpenBMB** "central model" + req #5 |
28
+ | **Cited knowledge base (~40 markers)** | Reliable reference ranges + "what it measures" + "questions for your doctor", grounded not hallucinated. | medical credibility |
29
+ | **Run the model LOCALLY (llama.cpp)** | The current external API call **fails off-grid / "fully offline."** | **off-grid badge** + offline requirement |
30
+ | **Fine-tune + GGUF on Modal** | No fine-tune yet. Needed for the badge + the OpenBMB before/after story. | **fine-tune + quantization badges, Modal prize** |
31
+ | **Agent trace** | Single API call now; needs a visible multi-step pipeline. | req #5 |
32
+ | **Eval harness + before/after** | No metrics yet. | **OpenBMB** proof |
33
+ | **Traces dataset + model card + multi-repo** | Top teams (compliment-forest) do this. | competitiveness |
34
+ | **Video (local run) + social + README lines** | Required submission artifacts. | **general pool** |
35
+
36
+ ## 3. The biggest technical fork: getting OFF the external API
37
+
38
+ Off-grid + "fully offline" require the model to **run inside the Space**, no external calls.
39
+ The README already flags this ("API-backed extractor is temporary"). Options:
40
+
41
+ - **Recommended (hybrid):** keep MiniCPM-V for extraction but run it **locally in the Space**
42
+ (MiniCPM-V GGUF via llama.cpp multimodal, or via transformers on **ZeroGPU**), and **fine-tune
43
+ a small MiniCPM *text* model** for the **interpretation + cross-marker reasoning** layer (easier
44
+ to fine-tune, this is the "model is the star" part, runs offline via llama.cpp). Earns off-grid
45
+ + fine-tune + quantization together.
46
+ - **Simpler fallback:** drop vision; do **local OCR/text extraction** (PyMuPDF text + tesseract)
47
+ β†’ one fine-tuned MiniCPM text model does structuring **and** interpretation. Fully offline,
48
+ easiest fine-tune; weaker on scans/photos.
49
+ - **Decide in Phase B.** Main technical risk = running MiniCPM-V locally; the fallback de-risks it.
50
+
51
+ ## 4. Parallel plan from here (2 devs, shared stack, contract-first)
52
+
53
+ **Shared contract to agree first (extend the existing dataclass):** add `Interpretation`
54
+ (`marker, plain, meaning, questions[], citation`) and a `cross_marker: list[str]` + `summary`
55
+ to the result object. Both tracks build against it; Track B can use a fixture until Track A ships.
56
+
57
+ **Module ownership (avoid collisions):**
58
+ - **Track A (model/data):** `src/openbmb_client.py` (+ interpretation), new `src/kb/`,
59
+ `src/reasoning/`, `train/`, `eval/`. Owns: extraction-offline, fine-tune, KB, reasoning, eval.
60
+ - **Track B (product/UI):** `app.py`, report rendering, **interpretation cards**, **cross-marker
61
+ insight section**, **doctor-questions**, **agent-trace panel**, `.html` download, sample report, deploy.
62
+ - **Co-owned:** the result dataclass (the contract).
63
+
64
+ ### Phase A β€” DONE βœ“ (extraction MVP + UI shell)
65
+
66
+ ### Phase B β€” Interpretation + KB + reasoning (the win-maker)
67
+ - **Track A:** build the **cited KB** (~40 markers: CBC, metabolic, lipid, thyroid, key vitamins);
68
+ add an **interpretation pass** (grounded in KB + the user's value) and the **cross-marker
69
+ reasoning** (W1). Decide the offline path (Β§3).
70
+ - **Track B:** extend the report to render interpretation per marker + a **cross-marker insights**
71
+ block + **"questions for your doctor"**; add the **agent-trace** panel (Ingest β†’ extract β†’
72
+ normalize β†’ KB lookup β†’ reason β†’ render) streaming via generator `yield`.
73
+ - **Milestone:** real report β†’ extracted + explained + reasoned document.
74
+
75
+ ### Phase C β€” Offline + fine-tune (the badges)
76
+ - **Track A:** synthetic data (Claude) for extraction/interpretation; **LoRA fine-tune on Modal**
77
+ β†’ merge β†’ **GGUF Q4_K_M**; wire local **llama.cpp** inference; **before/after chart**.
78
+ - **Track B:** deploy to the **org Space** with the local model; verify **zero external calls**;
79
+ graceful failure; downloadable `.html`; bundle a **sample report**.
80
+ - **Milestone:** runs fully offline on a stranger's report; fine-tune metrics in hand.
81
+
82
+ ### Phase D β€” Robustness + artifacts
83
+ - International units (mg/dL↔mmol/L), messy formats, unknown markers.
84
+ - Publish **agent traces as a HF dataset** + **model card with GGUF/eval metrics**; multi-repo.
85
+
86
+ ### Phase E β€” Submission
87
+ - Record **2-min video locally** (so "nothing left my laptop" is literally true); **social post**;
88
+ README with every eligibility line; flip repo public if needed; final eval numbers. Freeze + submit.
89
+
90
+ ## 5. The two weaknesses we engineer around
91
+ - **W1 β€” model must be the star:** cross-marker reasoning is a first-class component, not an add-on.
92
+ - **W2 β€” medical credibility:** every fact from a **cited KB**; "questions for your doctor," never
93
+ diagnosis (the current prompt's restraint is the right instinct β€” keep it).
94
+ - Privacy is only honest run locally β†’ record the **video locally**; the Space ships a sample report.
95
+
96
+ ## 6. Submission checklist
97
+ - [ ] Gradio app, HF Space under the **org**.
98
+ - [ ] All models <32B; the model <4B; **fine-tuned MiniCPM declared central**.
99
+ - [ ] **Runs fully offline** (no external API β€” the current OpenBMB call must be replaced) under **llama.cpp** (GGUF).
100
+ - [ ] **Demo video** (local) + **social post** linked in README.
101
+ - [ ] **Public repo** + dense **Codex-attributed commits**.
102
+ - [ ] README states: MiniCPM central, **Modal for fine-tuning**, off-grid, fine-tuned, quantized.
103
+ - [ ] Before/after chart + **traces dataset** + **model card** published.
104
+ - [ ] Space runs without our hardware (sample report bundled).
105
+
106
+ ## 7. Prize map (one line each)
107
+ - **OpenAI $10K** β€” build via Codex (already a contributor); public repo.
108
+ - **OpenBMB $10K** β€” MiniCPM central (extraction **+ reasoning**); before/after chart.
109
+ - **Modal $20K credits** β€” LoRA fine-tune (+ eval/data-gen) on Modal.
110
+ - **General pool** β€” Gradio Space + premium document + video + post.
111
+ - **Badges** β€” off-grid (local model, no API), fine-tune (LoRA), quantization (GGUF Q4_K_M).