Spaces:
Running on Zero
Running on Zero
Remove dead chat code and obsolete planning docs.
Browse filesDrop unwired results chat, hosted API extractor, experimental medreason script, and stale markdown. Trim interpretation rendering to patterns_html only and make model path tests cache-independent.
Co-authored-by: Codex <chatgpt-codex-connector[bot]@users.noreply.github.com>
- DEPLOYMENT_LOG.md +0 -149
- PLAN.md +0 -116
- eval/data/real/README.md +3 -3
- src/extraction/auto.py +0 -4
- src/extraction/factory.py +0 -1
- src/extraction/llamacpp_gpu.py +0 -49
- src/extraction/text_generation.py +0 -63
- src/interpretation_render.py +2 -93
- src/openbmb_client.py +0 -120
- src/pipeline_trace.py +1 -75
- src/results_chat.py +0 -156
- tests/test_model_paths.py +10 -1
- tests/test_pipeline_trace.py +1 -12
- tests/test_results_chat.py +0 -60
- train/modal_eval.py +2 -2
- train/modal_medreason.py +0 -121
DEPLOYMENT_LOG.md
DELETED
|
@@ -1,149 +0,0 @@
|
|
| 1 |
-
# Deployment Log
|
| 2 |
-
|
| 3 |
-
## 2026-06-13 — Opt-in llama.cpp vision + doc refresh
|
| 4 |
-
|
| 5 |
-
Decision: keep **Transformers** as the default extraction backend; make llama.cpp an explicit,
|
| 6 |
-
opt-in lane controlled by environment variables.
|
| 7 |
-
|
| 8 |
-
Why:
|
| 9 |
-
- PDF/image uploads always go through the vision document pipeline in `src/document_processing.py`.
|
| 10 |
-
- The previous docs described `auto` falling back to CPU llama.cpp, but `AutoExtractor` now always
|
| 11 |
-
selects Transformers.
|
| 12 |
-
- The hackathon **Llama Champion** badge still needs a GGUF path through `llama-cpp-python`.
|
| 13 |
-
- Fine-tuned deployment may swap in a GGUF repo without changing the Gradio app.
|
| 14 |
-
|
| 15 |
-
How to enable llama.cpp vision:
|
| 16 |
-
|
| 17 |
-
```bash
|
| 18 |
-
EXTRACTOR_BACKEND=llamacpp-gpu
|
| 19 |
-
LLAMACPP_VISION=1
|
| 20 |
-
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 21 |
-
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 22 |
-
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
| 23 |
-
```
|
| 24 |
-
|
| 25 |
-
Default production Space variables remain:
|
| 26 |
-
|
| 27 |
-
```bash
|
| 28 |
-
EXTRACTOR_BACKEND=transformers
|
| 29 |
-
ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
|
| 30 |
-
```
|
| 31 |
-
|
| 32 |
-
Docs updated: `README.md`, `RUNBOOK.md`, `DEPLOY.md`.
|
| 33 |
-
|
| 34 |
-
## 2026-06-13 — Route ZeroGPU to Transformers, CPU to llama.cpp
|
| 35 |
-
|
| 36 |
-
> **Superseded** by the opt-in llama.cpp lane above. `auto` no longer auto-selects llama.cpp on
|
| 37 |
-
> CPU; use `EXTRACTOR_BACKEND=llamacpp-gpu` explicitly when the GGUF lane is needed.
|
| 38 |
-
|
| 39 |
-
Decision: keep `EXTRACTOR_BACKEND=auto`, but make ZeroGPU select the official OpenBMB
|
| 40 |
-
Transformers backend instead of relying on app-level CUDA visibility.
|
| 41 |
-
|
| 42 |
-
Why:
|
| 43 |
-
- On ZeroGPU, CUDA is allocated only inside a `@spaces.GPU` worker, so
|
| 44 |
-
`torch.cuda.is_available()` can be false in normal Gradio app code.
|
| 45 |
-
- The app was therefore selecting the CPU llama.cpp fallback even while the Space hardware was
|
| 46 |
-
configured as ZeroGPU.
|
| 47 |
-
- The intended runtime behavior is now explicit: `ACCELERATOR=zero-a10g`, `ZERO_GPU=TRUE`, or
|
| 48 |
-
visible CUDA selects Transformers; CPU-only runtime selects llama.cpp.
|
| 49 |
-
|
| 50 |
-
Space variables:
|
| 51 |
-
|
| 52 |
-
```bash
|
| 53 |
-
EXTRACTOR_BACKEND=auto
|
| 54 |
-
ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
|
| 55 |
-
```
|
| 56 |
-
|
| 57 |
-
CPU fallback after a Transformers failure is now opt-in with:
|
| 58 |
-
|
| 59 |
-
```bash
|
| 60 |
-
AUTO_FALLBACK_TO_LLAMACPP=1
|
| 61 |
-
```
|
| 62 |
-
|
| 63 |
-
Follow-up fix: MiniCPM-V 4.6 declares `model_type: minicpmv4_6` and its official model card
|
| 64 |
-
requires `transformers[torch]>=5.7.0`. The Space was still pinned to `transformers==4.57.3`, which
|
| 65 |
-
caused the ZeroGPU worker to fail with:
|
| 66 |
-
|
| 67 |
-
```text
|
| 68 |
-
Transformers does not recognize this architecture
|
| 69 |
-
```
|
| 70 |
-
|
| 71 |
-
The runtime now pins:
|
| 72 |
-
|
| 73 |
-
```text
|
| 74 |
-
transformers[torch]==5.7.0
|
| 75 |
-
```
|
| 76 |
-
|
| 77 |
-
## 2026-06-10 — Switch from Docker Space to Gradio ZeroGPU
|
| 78 |
-
|
| 79 |
-
Decision: use **Gradio ZeroGPU** as the active Hugging Face Space architecture.
|
| 80 |
-
|
| 81 |
-
Why:
|
| 82 |
-
|
| 83 |
-
- The Docker Space build failed on free CPU hardware with `OOMKilled`.
|
| 84 |
-
- Hugging Face ZeroGPU is available only for Gradio SDK Spaces, not Docker Spaces.
|
| 85 |
-
- The project needs a free dynamic GPU path for demoability.
|
| 86 |
-
|
| 87 |
-
What changed:
|
| 88 |
-
|
| 89 |
-
- `README.md` metadata changed from `sdk: docker` to `sdk: gradio`.
|
| 90 |
-
- `DEPLOY.md` and `RUNBOOK.md` now describe Gradio + ZeroGPU + Transformers as the active path.
|
| 91 |
-
- `src/extraction/zerogpu_transformers.py` adds the official OpenBMB MiniCPM-V Transformers backend.
|
| 92 |
-
- `src/extraction/factory.py` initially resolved `auto` to the ZeroGPU Transformers backend.
|
| 93 |
-
- Docker-only files (`Dockerfile`, `start.sh`, `.dockerignore`) were removed from the active deployment.
|
| 94 |
-
|
| 95 |
-
Current model:
|
| 96 |
-
|
| 97 |
-
```bash
|
| 98 |
-
ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
|
| 99 |
-
```
|
| 100 |
-
|
| 101 |
-
Future fine-tuned model:
|
| 102 |
-
|
| 103 |
-
Only change this variable:
|
| 104 |
-
|
| 105 |
-
```bash
|
| 106 |
-
ZEROGPU_MODEL_ID=<owner>/<fine-tuned-minicpm-v-model>
|
| 107 |
-
```
|
| 108 |
-
|
| 109 |
-
Do not reintroduce Docker or `llama-server` while the project is targeting ZeroGPU. Do not commit model files to the Space repository.
|
| 110 |
-
|
| 111 |
-
## 2026-06-11 — Add llama.cpp badge path on ZeroGPU
|
| 112 |
-
|
| 113 |
-
Decision: keep the Space as **Gradio ZeroGPU**, but target the hackathon llama.cpp badge with the
|
| 114 |
-
`auto` / `llamacpp-gpu` backend.
|
| 115 |
-
|
| 116 |
-
Why:
|
| 117 |
-
|
| 118 |
-
- ZeroGPU requires Gradio SDK, so Docker is still not the right deployment surface.
|
| 119 |
-
- The llama.cpp badge can still be targeted from a Gradio Space if inference runs through
|
| 120 |
-
`llama-cpp-python` over GGUF inside `@spaces.GPU`.
|
| 121 |
-
- The official OpenBMB GGUF repo stays outside the Space git repo and is downloaded through the
|
| 122 |
-
Hugging Face cache at runtime.
|
| 123 |
-
|
| 124 |
-
Current badge-target defaults:
|
| 125 |
-
|
| 126 |
-
```bash
|
| 127 |
-
EXTRACTOR_BACKEND=auto
|
| 128 |
-
LLAMACPP_GGUF_REPO=openbmb/MiniCPM-V-4.6-gguf
|
| 129 |
-
LLAMACPP_MODEL_FILE=MiniCPM-V-4_6-Q4_K_M.gguf
|
| 130 |
-
LLAMACPP_MMPROJ_FILE=mmproj-model-f16.gguf
|
| 131 |
-
```
|
| 132 |
-
|
| 133 |
-
Fallback if `llama-cpp-python` is incompatible with MiniCPM-V 4.6 on ZeroGPU:
|
| 134 |
-
|
| 135 |
-
```bash
|
| 136 |
-
EXTRACTOR_BACKEND=zerogpu
|
| 137 |
-
ZEROGPU_MODEL_ID=openbmb/MiniCPM-V-4.6
|
| 138 |
-
```
|
| 139 |
-
|
| 140 |
-
Future fine-tuned model:
|
| 141 |
-
|
| 142 |
-
Only change the `LLAMACPP_*` variables to point at the fine-tuned GGUF repo/files.
|
| 143 |
-
|
| 144 |
-
The app now also surfaces backend load errors directly in the UI and falls back to the
|
| 145 |
-
Transformers ZeroGPU backend when the llama.cpp GGUF load path fails, so a runtime mismatch does
|
| 146 |
-
not collapse the whole extraction flow.
|
| 147 |
-
|
| 148 |
-
We also upgraded the llama-cpp-python binding to a MiniCPM-V 4.6-capable build, which is the
|
| 149 |
-
actual fix for the earlier `Failed to load model from file` failure.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
PLAN.md
DELETED
|
@@ -1,116 +0,0 @@
|
|
| 1 |
-
# Blood Test Explainer ("Pulse") — Plan, revised against current repo
|
| 2 |
-
|
| 3 |
-
> Grounded in the actual code as of 2026-06-09. Two developers, parallel.
|
| 4 |
-
> Target prizes: **OpenAI + OpenBMB + Modal** + all badges.
|
| 5 |
-
> ⚠️ Confirm the real submission deadline first — teams are already final-submitting.
|
| 6 |
-
|
| 7 |
-
> 2026-06-10 deployment update: the active HF Space path is now **Gradio ZeroGPU + official
|
| 8 |
-
> OpenBMB Transformers model**. The Docker/llama.cpp serving plan was replaced because ZeroGPU is
|
| 9 |
-
> Gradio-only and the Docker build was OOM-killed on free CPU hardware. For the fine-tuned model,
|
| 10 |
-
> keep this ZeroGPU architecture and replace only `ZEROGPU_MODEL_ID`.
|
| 11 |
-
|
| 12 |
-
---
|
| 13 |
-
|
| 14 |
-
## 1. Where we are (DONE — solid foundation)
|
| 15 |
-
|
| 16 |
-
- **Polished Gradio app** (`app.py`): light clinical theme, animated "formation" hero,
|
| 17 |
-
loading/empty states, report cards with status pills, tabs (Report / Values / JSON / Raw),
|
| 18 |
-
responsive. This is genuinely good and is a real head start on the "premium document" wow.
|
| 19 |
-
- **Extraction works** (`src/openbmb_client.py`, `src/document_processing.py`): OpenBMB
|
| 20 |
-
**MiniCPM-V-4.6** (vision) via OpenAI-compatible API. PDF→images (PyMuPDF), images, and
|
| 21 |
-
text files supported; base64 image payloads; robust JSON-repair parsing.
|
| 22 |
-
- **Schema:** `{marker, value, unit, reference_range, status, source_text, confidence}` + notes.
|
| 23 |
-
- **Env/secret handling** ready (`.env` local, Space secret). Codex is contributing.
|
| 24 |
-
- Extraction prompt is deliberately **extraction-only** ("do not diagnose/interpret").
|
| 25 |
-
|
| 26 |
-
## 2. Blunt gap analysis — what's missing and which prize it unblocks
|
| 27 |
-
|
| 28 |
-
The current app is a beautiful **extractor that calls an external API**. As-is it does not win.
|
| 29 |
-
|
| 30 |
-
| Missing piece | Why it matters | Unblocks |
|
| 31 |
-
|---|---|---|
|
| 32 |
-
| **Interpretation + cross-marker reasoning** | Right now MiniCPM looks like OCR/plumbing. The model must *reason* (e.g. "ALT+AST+GGT all high → liver-enzyme pattern") to be visibly central. | **OpenBMB** "central model" + req #5 |
|
| 33 |
-
| **Cited knowledge base (~40 markers)** | Reliable reference ranges + "what it measures" + "questions for your doctor", grounded not hallucinated. | medical credibility |
|
| 34 |
-
| **Run the model LOCALLY (llama.cpp)** | The current external API call **fails off-grid / "fully offline."** | **off-grid badge** + offline requirement |
|
| 35 |
-
| **Fine-tune + GGUF on Modal** | No fine-tune yet. Needed for the badge + the OpenBMB before/after story. | **fine-tune + quantization badges, Modal prize** |
|
| 36 |
-
| **Agent trace** | Single API call now; needs a visible multi-step pipeline. | req #5 |
|
| 37 |
-
| **Eval harness + before/after** | No metrics yet. | **OpenBMB** proof |
|
| 38 |
-
| **Traces dataset + model card + multi-repo** | Top teams (compliment-forest) do this. | competitiveness |
|
| 39 |
-
| **Video (local run) + social + README lines** | Required submission artifacts. | **general pool** |
|
| 40 |
-
|
| 41 |
-
## 3. The biggest technical fork: getting OFF the external API
|
| 42 |
-
|
| 43 |
-
Off-grid + "fully offline" require the model to **run inside the Space**, no external calls.
|
| 44 |
-
The README already flags this ("API-backed extractor is temporary"). Options:
|
| 45 |
-
|
| 46 |
-
- **Recommended (hybrid):** keep MiniCPM-V for extraction but run it **locally in the Space**
|
| 47 |
-
(MiniCPM-V GGUF via llama.cpp multimodal, or via transformers on **ZeroGPU**), and **fine-tune
|
| 48 |
-
a small MiniCPM *text* model** for the **interpretation + cross-marker reasoning** layer (easier
|
| 49 |
-
to fine-tune, this is the "model is the star" part, runs offline via llama.cpp). Earns off-grid
|
| 50 |
-
+ fine-tune + quantization together.
|
| 51 |
-
- **Simpler fallback:** drop vision; do **local OCR/text extraction** (PyMuPDF text + tesseract)
|
| 52 |
-
→ one fine-tuned MiniCPM text model does structuring **and** interpretation. Fully offline,
|
| 53 |
-
easiest fine-tune; weaker on scans/photos.
|
| 54 |
-
- **Decide in Phase B.** Main technical risk = running MiniCPM-V locally; the fallback de-risks it.
|
| 55 |
-
|
| 56 |
-
## 4. Parallel plan from here (2 devs, shared stack, contract-first)
|
| 57 |
-
|
| 58 |
-
**Shared contract to agree first (extend the existing dataclass):** add `Interpretation`
|
| 59 |
-
(`marker, plain, meaning, questions[], citation`) and a `cross_marker: list[str]` + `summary`
|
| 60 |
-
to the result object. Both tracks build against it; Track B can use a fixture until Track A ships.
|
| 61 |
-
|
| 62 |
-
**Module ownership (avoid collisions):**
|
| 63 |
-
- **Track A (model/data):** `src/openbmb_client.py` (+ interpretation), new `src/kb/`,
|
| 64 |
-
`src/reasoning/`, `train/`, `eval/`. Owns: extraction-offline, fine-tune, KB, reasoning, eval.
|
| 65 |
-
- **Track B (product/UI):** `app.py`, report rendering, **interpretation cards**, **cross-marker
|
| 66 |
-
insight section**, **doctor-questions**, **agent-trace panel**, `.html` download, sample report, deploy.
|
| 67 |
-
- **Co-owned:** the result dataclass (the contract).
|
| 68 |
-
|
| 69 |
-
### Phase A — DONE ✓ (extraction MVP + UI shell)
|
| 70 |
-
|
| 71 |
-
### Phase B — Interpretation + KB + reasoning (the win-maker)
|
| 72 |
-
- **Track A:** build the **cited KB** (~40 markers: CBC, metabolic, lipid, thyroid, key vitamins);
|
| 73 |
-
add an **interpretation pass** (grounded in KB + the user's value) and the **cross-marker
|
| 74 |
-
reasoning** (W1). Decide the offline path (��3).
|
| 75 |
-
- **Track B:** extend the report to render interpretation per marker + a **cross-marker insights**
|
| 76 |
-
block + **"questions for your doctor"**; add the **agent-trace** panel (Ingest → extract →
|
| 77 |
-
normalize → KB lookup → reason → render) streaming via generator `yield`.
|
| 78 |
-
- **Milestone:** real report → extracted + explained + reasoned document.
|
| 79 |
-
|
| 80 |
-
### Phase C — Offline + fine-tune (the badges)
|
| 81 |
-
- **Track A:** synthetic data (Claude) for extraction/interpretation; **LoRA fine-tune on Modal**
|
| 82 |
-
→ merge → **GGUF Q4_K_M**; wire local **llama.cpp** inference; **before/after chart**.
|
| 83 |
-
- **Track B:** deploy to the **org Space** with the local model; verify **zero external calls**;
|
| 84 |
-
graceful failure; downloadable `.html`; bundle a **sample report**.
|
| 85 |
-
- **Milestone:** runs fully offline on a stranger's report; fine-tune metrics in hand.
|
| 86 |
-
|
| 87 |
-
### Phase D — Robustness + artifacts
|
| 88 |
-
- International units (mg/dL↔mmol/L), messy formats, unknown markers.
|
| 89 |
-
- Publish **agent traces as a HF dataset** + **model card with GGUF/eval metrics**; multi-repo.
|
| 90 |
-
|
| 91 |
-
### Phase E — Submission
|
| 92 |
-
- Record **2-min video locally** (so "nothing left my laptop" is literally true); **social post**;
|
| 93 |
-
README with every eligibility line; flip repo public if needed; final eval numbers. Freeze + submit.
|
| 94 |
-
|
| 95 |
-
## 5. The two weaknesses we engineer around
|
| 96 |
-
- **W1 — model must be the star:** cross-marker reasoning is a first-class component, not an add-on.
|
| 97 |
-
- **W2 — medical credibility:** every fact from a **cited KB**; "questions for your doctor," never
|
| 98 |
-
diagnosis (the current prompt's restraint is the right instinct — keep it).
|
| 99 |
-
- Privacy is only honest run locally → record the **video locally**; the Space ships a sample report.
|
| 100 |
-
|
| 101 |
-
## 6. Submission checklist
|
| 102 |
-
- [ ] Gradio app, HF Space under the **org**.
|
| 103 |
-
- [ ] All models <32B; the model <4B; **fine-tuned MiniCPM declared central**.
|
| 104 |
-
- [ ] **Runs fully offline** (no external API — the current OpenBMB call must be replaced) under **llama.cpp** (GGUF).
|
| 105 |
-
- [ ] **Demo video** (local) + **social post** linked in README.
|
| 106 |
-
- [ ] **Public repo** + dense **Codex-attributed commits**.
|
| 107 |
-
- [ ] README states: MiniCPM central, **Modal for fine-tuning**, off-grid, fine-tuned, quantized.
|
| 108 |
-
- [ ] Before/after chart + **traces dataset** + **model card** published.
|
| 109 |
-
- [ ] Space runs without our hardware (sample report bundled).
|
| 110 |
-
|
| 111 |
-
## 7. Prize map (one line each)
|
| 112 |
-
- **OpenAI $10K** — build via Codex (already a contributor); public repo.
|
| 113 |
-
- **OpenBMB $10K** — MiniCPM central (extraction **+ reasoning**); before/after chart.
|
| 114 |
-
- **Modal $20K credits** — LoRA fine-tune (+ eval/data-gen) on Modal.
|
| 115 |
-
- **General pool** — Gradio Space + premium document + video + post.
|
| 116 |
-
- **Badges** — off-grid (local model, no API), fine-tune (LoRA), quantization (GGUF Q4_K_M).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
eval/data/real/README.md
CHANGED
|
@@ -33,14 +33,14 @@ The extractor reads PDFs directly (it renders pages to images), so `image` point
|
|
| 33 |
|
| 34 |
```bash
|
| 35 |
# 1) draft labels with the current extractor, then hand-correct into gold
|
| 36 |
-
EXTRACTOR_BACKEND=
|
| 37 |
-
--labels eval/data/real/labels.jsonl --run
|
| 38 |
```
|
| 39 |
To add a report: run the extractor on it, copy the predicted `tests` into a new `labels.jsonl`
|
| 40 |
row, and correct any mistakes against the PDF. Faster and more accurate than typing from scratch.
|
| 41 |
|
| 42 |
## Run the eval
|
| 43 |
```bash
|
| 44 |
-
python eval/run_eval.py --labels eval/data/real/labels.jsonl --run
|
| 45 |
```
|
| 46 |
Use it twice (base vs fine-tuned GGUF) for the before/after.
|
|
|
|
| 33 |
|
| 34 |
```bash
|
| 35 |
# 1) draft labels with the current extractor, then hand-correct into gold
|
| 36 |
+
EXTRACTOR_BACKEND=transformers python eval/run_eval.py \
|
| 37 |
+
--labels eval/data/real/labels.jsonl --run
|
| 38 |
```
|
| 39 |
To add a report: run the extractor on it, copy the predicted `tests` into a new `labels.jsonl`
|
| 40 |
row, and correct any mistakes against the PDF. Faster and more accurate than typing from scratch.
|
| 41 |
|
| 42 |
## Run the eval
|
| 43 |
```bash
|
| 44 |
+
python eval/run_eval.py --labels eval/data/real/labels.jsonl --run
|
| 45 |
```
|
| 46 |
Use it twice (base vs fine-tuned GGUF) for the before/after.
|
src/extraction/auto.py
CHANGED
|
@@ -31,7 +31,3 @@ class AutoExtractor:
|
|
| 31 |
self._selected = ZeroGPUTransformersExtractor(model_id=self.model_id)
|
| 32 |
return self._selected
|
| 33 |
|
| 34 |
-
|
| 35 |
-
def runtime_target() -> str:
|
| 36 |
-
"""Local app always runs through Transformers."""
|
| 37 |
-
return "transformers"
|
|
|
|
| 31 |
self._selected = ZeroGPUTransformersExtractor(model_id=self.model_id)
|
| 32 |
return self._selected
|
| 33 |
|
|
|
|
|
|
|
|
|
|
|
|
src/extraction/factory.py
CHANGED
|
@@ -21,7 +21,6 @@ from src.extraction.auto import AutoExtractor
|
|
| 21 |
from src.extraction.llamacpp_gpu import LlamaCppGPUExtractor
|
| 22 |
from src.extraction.local_minicpmv import LocalMiniCPMVExtractor
|
| 23 |
from src.extraction.local_server import LocalServerExtractor
|
| 24 |
-
from src.extraction.zerogpu_transformers import ZeroGPUTransformersExtractor
|
| 25 |
|
| 26 |
_DEFAULT_BACKEND = "transformers"
|
| 27 |
_DISABLED_BACKENDS = {"api", "openbmb", "hosted"}
|
|
|
|
| 21 |
from src.extraction.llamacpp_gpu import LlamaCppGPUExtractor
|
| 22 |
from src.extraction.local_minicpmv import LocalMiniCPMVExtractor
|
| 23 |
from src.extraction.local_server import LocalServerExtractor
|
|
|
|
| 24 |
|
| 25 |
_DEFAULT_BACKEND = "transformers"
|
| 26 |
_DISABLED_BACKENDS = {"api", "openbmb", "hosted"}
|
src/extraction/llamacpp_gpu.py
CHANGED
|
@@ -248,55 +248,6 @@ def _run_llamacpp_generation(
|
|
| 248 |
raise _raise_generation_error(exc, vision=False) from exc
|
| 249 |
|
| 250 |
|
| 251 |
-
@spaces.GPU(duration=120)
|
| 252 |
-
def _run_llamacpp_chat(
|
| 253 |
-
messages: list[dict[str, str]],
|
| 254 |
-
repo: str,
|
| 255 |
-
model_file: str,
|
| 256 |
-
max_tokens: int,
|
| 257 |
-
n_ctx: int,
|
| 258 |
-
n_gpu_layers: int,
|
| 259 |
-
*,
|
| 260 |
-
vision_enabled: bool = False,
|
| 261 |
-
mmproj_file: str = DEFAULT_MMPROJ_FILE,
|
| 262 |
-
chat_handler: str = DEFAULT_CHAT_HANDLER,
|
| 263 |
-
) -> str:
|
| 264 |
-
try:
|
| 265 |
-
model_path = download_hf_file(repo, model_file)
|
| 266 |
-
if vision_enabled:
|
| 267 |
-
mmproj_path = download_hf_file(repo, mmproj_file)
|
| 268 |
-
except Exception as exc:
|
| 269 |
-
raise RuntimeError(
|
| 270 |
-
"llama.cpp download failed while preparing the GGUF model: "
|
| 271 |
-
f"{type(exc).__name__}: {exc}"
|
| 272 |
-
) from exc
|
| 273 |
-
|
| 274 |
-
try:
|
| 275 |
-
if vision_enabled:
|
| 276 |
-
llm = load_vision_llama(model_path, mmproj_path, n_ctx, n_gpu_layers, chat_handler)
|
| 277 |
-
else:
|
| 278 |
-
llm = _load_text(model_path, n_ctx, n_gpu_layers)
|
| 279 |
-
except Exception as exc:
|
| 280 |
-
label = "vision GGUF model for chat" if vision_enabled else "text-only GGUF model for chat"
|
| 281 |
-
raise RuntimeError(
|
| 282 |
-
f"The llama.cpp backend could not load the {label}. "
|
| 283 |
-
f"Inner error: {type(exc).__name__}: {exc}"
|
| 284 |
-
) from exc
|
| 285 |
-
|
| 286 |
-
try:
|
| 287 |
-
response = llm.create_chat_completion(
|
| 288 |
-
messages=messages,
|
| 289 |
-
temperature=0.2,
|
| 290 |
-
max_tokens=max_tokens,
|
| 291 |
-
)
|
| 292 |
-
return str(response["choices"][0]["message"].get("content") or "").strip()
|
| 293 |
-
except Exception as exc:
|
| 294 |
-
raise RuntimeError(
|
| 295 |
-
"llama.cpp chat generation failed. "
|
| 296 |
-
f"Inner error: {type(exc).__name__}: {exc}"
|
| 297 |
-
) from exc
|
| 298 |
-
|
| 299 |
-
|
| 300 |
def _compose_prompt(parts: list[dict[str, Any]]) -> str:
|
| 301 |
text_parts: list[str] = [EXTRACTION_PROMPT]
|
| 302 |
image_count = 0
|
|
|
|
| 248 |
raise _raise_generation_error(exc, vision=False) from exc
|
| 249 |
|
| 250 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 251 |
def _compose_prompt(parts: list[dict[str, Any]]) -> str:
|
| 252 |
text_parts: list[str] = [EXTRACTION_PROMPT]
|
| 253 |
image_count = 0
|
src/extraction/text_generation.py
DELETED
|
@@ -1,63 +0,0 @@
|
|
| 1 |
-
"""Shared text-generation helpers for chat (mirrors EXTRACTOR_BACKEND)."""
|
| 2 |
-
|
| 3 |
-
from __future__ import annotations
|
| 4 |
-
|
| 5 |
-
import os
|
| 6 |
-
|
| 7 |
-
from src.extraction.llamacpp_gpu import DEFAULT_GGUF_REPO, DEFAULT_MODEL_FILE
|
| 8 |
-
from src.extraction.llamacpp_vision import DEFAULT_CHAT_HANDLER, DEFAULT_MMPROJ_FILE, llamacpp_vision_enabled
|
| 9 |
-
from src.local_env import load_local_env
|
| 10 |
-
|
| 11 |
-
load_local_env()
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
def generate_text_chat(messages: list[dict[str, str]], max_tokens: int | None = None) -> str:
|
| 15 |
-
"""Run a text-only chat completion using the configured extraction backend family."""
|
| 16 |
-
backend = os.getenv("EXTRACTOR_BACKEND", "transformers").strip().lower()
|
| 17 |
-
token_limit = max_tokens or int(os.getenv("CHAT_MAX_TOKENS", "1024"))
|
| 18 |
-
|
| 19 |
-
if backend in {"api", "openbmb", "hosted"}:
|
| 20 |
-
raise RuntimeError(
|
| 21 |
-
"Chat via the hosted OpenBMB API is disabled. Use EXTRACTOR_BACKEND=transformers."
|
| 22 |
-
)
|
| 23 |
-
if backend in {"auto", "zerogpu", "zero-gpu", "transformers"}:
|
| 24 |
-
return _transformers_chat(messages, token_limit)
|
| 25 |
-
if backend in {"llamacpp-gpu", "gpu-llamacpp", "llama-champion", "llamacpp"}:
|
| 26 |
-
return _llamacpp_chat(messages, token_limit)
|
| 27 |
-
|
| 28 |
-
raise RuntimeError(f"Chat is not configured for EXTRACTOR_BACKEND={backend!r}.")
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
def _transformers_chat(messages: list[dict[str, str]], max_tokens: int) -> str:
|
| 32 |
-
from src.extraction.zerogpu_transformers import _run_zerogpu_generation
|
| 33 |
-
from src.model_paths import resolve_transformers_model_source
|
| 34 |
-
|
| 35 |
-
model_source = resolve_transformers_model_source(os.getenv("ZEROGPU_MODEL_ID"))
|
| 36 |
-
downsample_mode = (os.getenv("ZEROGPU_DOWNSAMPLE_MODE") or "16x").strip()
|
| 37 |
-
structured = [{"role": m["role"], "content": m["content"]} for m in messages]
|
| 38 |
-
return _run_zerogpu_generation(
|
| 39 |
-
messages=structured,
|
| 40 |
-
model_source=model_source,
|
| 41 |
-
max_new_tokens=max_tokens,
|
| 42 |
-
downsample_mode=downsample_mode,
|
| 43 |
-
)
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
def _llamacpp_chat(messages: list[dict[str, str]], max_tokens: int) -> str:
|
| 47 |
-
from src.extraction.llamacpp_gpu import _run_llamacpp_chat
|
| 48 |
-
|
| 49 |
-
repo = os.getenv("LLAMACPP_GGUF_REPO", DEFAULT_GGUF_REPO).strip()
|
| 50 |
-
model_file = os.getenv("LLAMACPP_MODEL_FILE", DEFAULT_MODEL_FILE).strip()
|
| 51 |
-
n_ctx = int(os.getenv("LLAMACPP_N_CTX", "8192"))
|
| 52 |
-
n_gpu_layers = int(os.getenv("LLAMACPP_N_GPU_LAYERS", "0"))
|
| 53 |
-
return _run_llamacpp_chat(
|
| 54 |
-
messages=messages,
|
| 55 |
-
repo=repo,
|
| 56 |
-
model_file=model_file,
|
| 57 |
-
max_tokens=max_tokens,
|
| 58 |
-
n_ctx=n_ctx,
|
| 59 |
-
n_gpu_layers=n_gpu_layers,
|
| 60 |
-
vision_enabled=llamacpp_vision_enabled(),
|
| 61 |
-
mmproj_file=os.getenv("LLAMACPP_MMPROJ_FILE", DEFAULT_MMPROJ_FILE).strip(),
|
| 62 |
-
chat_handler=os.getenv("LLAMACPP_CHAT_HANDLER", DEFAULT_CHAT_HANDLER).strip(),
|
| 63 |
-
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
src/interpretation_render.py
CHANGED
|
@@ -1,28 +1,10 @@
|
|
| 1 |
-
"""Render
|
| 2 |
-
|
| 3 |
-
Pure presentation — every fact comes from src/interpretation.py (which comes from the KB), nothing
|
| 4 |
-
is invented here. Framework-agnostic so the app can drop `interpretation_html(tests)` into a
|
| 5 |
-
`gr.HTML` (styled cards) or `interpretation_markdown(tests)` into a `gr.Markdown`.
|
| 6 |
-
"""
|
| 7 |
|
| 8 |
from __future__ import annotations
|
| 9 |
|
| 10 |
import html
|
| 11 |
|
| 12 |
-
from src.interpretation import
|
| 13 |
-
|
| 14 |
-
_STATUS_LABEL = {"low": "Low", "high": "High"}
|
| 15 |
-
_STATUS_COLOR = {"low": "#b06b00", "high": "#b22222"}
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
def interpretation_markdown(tests: list[dict]) -> str:
|
| 19 |
-
"""tests (extracted) -> markdown for a gr.Markdown component."""
|
| 20 |
-
return render_markdown(build_interpretation(tests))
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
def interpretation_html(tests: list[dict]) -> str:
|
| 24 |
-
"""tests (extracted) -> styled HTML cards for a gr.HTML component."""
|
| 25 |
-
return render_html(build_interpretation(tests))
|
| 26 |
|
| 27 |
|
| 28 |
def patterns_html(tests: list[dict]) -> str:
|
|
@@ -55,76 +37,3 @@ def patterns_html(tests: list[dict]) -> str:
|
|
| 55 |
f'<div style="color:#9ca3af;font-size:11px;margin-top:4px;">{esc(interp.disclaimer)}</div>'
|
| 56 |
"</div>"
|
| 57 |
)
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
def render_markdown(interp: Interpretation) -> str:
|
| 61 |
-
out = ["### What your results may mean", "_Educational information, not a diagnosis._", ""]
|
| 62 |
-
if not interp.has_findings:
|
| 63 |
-
out.append(f"All {interp.normal_count} recognized markers are within their reference ranges.")
|
| 64 |
-
out += ["", f"> {interp.disclaimer}"]
|
| 65 |
-
return "\n".join(out)
|
| 66 |
-
|
| 67 |
-
for c in interp.flagged:
|
| 68 |
-
unit = f" {c.unit}" if c.unit else ""
|
| 69 |
-
status = _STATUS_LABEL.get(c.status, c.status)
|
| 70 |
-
out.append(f"**{c.marker} — {c.value}{unit} ({status}, ref {c.reference_range})**")
|
| 71 |
-
if c.note:
|
| 72 |
-
out.append(c.note)
|
| 73 |
-
if c.questions:
|
| 74 |
-
out.append("Questions for your doctor: " + " ".join(f"_{q}_" for q in c.questions))
|
| 75 |
-
out.append("")
|
| 76 |
-
|
| 77 |
-
if interp.patterns:
|
| 78 |
-
out.append("#### Patterns across markers")
|
| 79 |
-
out += [f"- **{p.name}** — {p.note}" for p in interp.patterns]
|
| 80 |
-
out.append("")
|
| 81 |
-
|
| 82 |
-
out.append(f"{interp.normal_count} other recognized markers were within range.")
|
| 83 |
-
out += ["", f"> {interp.disclaimer}"]
|
| 84 |
-
return "\n".join(out)
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
def render_html(interp: Interpretation) -> str:
|
| 88 |
-
def esc(value: object) -> str:
|
| 89 |
-
return html.escape(str(value))
|
| 90 |
-
|
| 91 |
-
parts = ['<div style="font-family:system-ui,-apple-system,sans-serif;max-width:760px;">']
|
| 92 |
-
parts.append('<h3 style="margin:0 0 2px;">What your results may mean</h3>')
|
| 93 |
-
parts.append('<div style="color:#6b7280;font-size:13px;margin-bottom:14px;">'
|
| 94 |
-
'Educational information, not a diagnosis.</div>')
|
| 95 |
-
|
| 96 |
-
if not interp.has_findings:
|
| 97 |
-
parts.append(f'<div style="padding:12px;border-radius:10px;background:#f0fdf4;color:#166534;">'
|
| 98 |
-
f'All {interp.normal_count} recognized markers are within their reference ranges.</div>')
|
| 99 |
-
else:
|
| 100 |
-
for c in interp.flagged:
|
| 101 |
-
color = _STATUS_COLOR.get(c.status, "#374151")
|
| 102 |
-
unit = f" {esc(c.unit)}" if c.unit else ""
|
| 103 |
-
status = _STATUS_LABEL.get(c.status, esc(c.status))
|
| 104 |
-
parts.append(
|
| 105 |
-
f'<div style="border:1px solid #e5e7eb;border-left:4px solid {color};border-radius:10px;'
|
| 106 |
-
f'padding:12px 14px;margin-bottom:10px;">'
|
| 107 |
-
f'<div style="font-weight:600;">{esc(c.marker)} '
|
| 108 |
-
f'<span style="color:{color};">{esc(c.value)}{unit} ({status})</span> '
|
| 109 |
-
f'<span style="color:#9ca3af;font-weight:400;font-size:12px;">ref {esc(c.reference_range)}</span></div>'
|
| 110 |
-
)
|
| 111 |
-
if c.note:
|
| 112 |
-
parts.append(f'<div style="color:#374151;margin-top:4px;font-size:14px;">{esc(c.note)}</div>')
|
| 113 |
-
if c.questions:
|
| 114 |
-
items = "".join(f"<li>{esc(q)}</li>" for q in c.questions)
|
| 115 |
-
parts.append('<div style="margin-top:6px;font-size:13px;color:#6b7280;">'
|
| 116 |
-
f'Questions for your doctor:<ul style="margin:4px 0 0;padding-left:18px;">{items}</ul></div>')
|
| 117 |
-
parts.append("</div>")
|
| 118 |
-
|
| 119 |
-
if interp.patterns:
|
| 120 |
-
parts.append('<h4 style="margin:14px 0 6px;">Patterns across markers</h4>')
|
| 121 |
-
for p in interp.patterns:
|
| 122 |
-
parts.append('<div style="background:#eff6ff;border-radius:8px;padding:10px 12px;margin-bottom:8px;'
|
| 123 |
-
f'font-size:14px;"><b>{esc(p.name)}</b> — {esc(p.note)}</div>')
|
| 124 |
-
parts.append(f'<div style="color:#6b7280;font-size:13px;margin-top:6px;">'
|
| 125 |
-
f'{interp.normal_count} other recognized markers were within range.</div>')
|
| 126 |
-
|
| 127 |
-
parts.append(f'<div style="margin-top:14px;padding-top:10px;border-top:1px solid #eee;'
|
| 128 |
-
f'color:#9ca3af;font-size:12px;">{esc(interp.disclaimer)}</div>')
|
| 129 |
-
parts.append("</div>")
|
| 130 |
-
return "".join(parts)
|
|
|
|
| 1 |
+
"""Render cross-marker interpretation patterns for the health report UI."""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
from __future__ import annotations
|
| 4 |
|
| 5 |
import html
|
| 6 |
|
| 7 |
+
from src.interpretation import build_interpretation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
|
| 9 |
|
| 10 |
def patterns_html(tests: list[dict]) -> str:
|
|
|
|
| 37 |
f'<div style="color:#9ca3af;font-size:11px;margin-top:4px;">{esc(interp.disclaimer)}</div>'
|
| 38 |
"</div>"
|
| 39 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
src/openbmb_client.py
CHANGED
|
@@ -1,26 +1,17 @@
|
|
| 1 |
from __future__ import annotations
|
| 2 |
|
| 3 |
import json
|
| 4 |
-
import os
|
| 5 |
import re
|
| 6 |
-
import time
|
| 7 |
from dataclasses import dataclass, field
|
| 8 |
from typing import Any
|
| 9 |
|
| 10 |
-
import requests
|
| 11 |
from json_repair import loads as repair_json_loads
|
| 12 |
-
from requests import HTTPError
|
| 13 |
|
| 14 |
-
from src.document_processing import document_intake_metadata, document_to_payload_parts
|
| 15 |
from src.local_env import load_local_env
|
| 16 |
|
| 17 |
|
| 18 |
load_local_env()
|
| 19 |
|
| 20 |
-
DEFAULT_API_URL = "http://35.203.155.71:8003/v1/chat/completions"
|
| 21 |
-
DEFAULT_MODEL = "MiniCPM-V-4.6"
|
| 22 |
-
|
| 23 |
-
|
| 24 |
EXTRACTION_PROMPT = """
|
| 25 |
You are extracting laboratory test results from a medical document.
|
| 26 |
|
|
@@ -68,89 +59,6 @@ class ExtractionResult:
|
|
| 68 |
patient: dict[str, Any] = field(default_factory=dict)
|
| 69 |
|
| 70 |
|
| 71 |
-
class OpenBMBExtractor:
|
| 72 |
-
def __init__(
|
| 73 |
-
self,
|
| 74 |
-
api_url: str | None = None,
|
| 75 |
-
model: str | None = None,
|
| 76 |
-
api_key: str | None = None,
|
| 77 |
-
timeout_seconds: int = 90,
|
| 78 |
-
) -> None:
|
| 79 |
-
self.api_url = (api_url or os.getenv("OPENBMB_API_URL") or DEFAULT_API_URL).strip()
|
| 80 |
-
self.model = (model or os.getenv("OPENBMB_MODEL") or DEFAULT_MODEL).strip()
|
| 81 |
-
self.api_key = _normalize_api_key(api_key or os.getenv("OPENBMB_API_KEY"))
|
| 82 |
-
self.timeout_seconds = timeout_seconds
|
| 83 |
-
|
| 84 |
-
@property
|
| 85 |
-
def is_configured(self) -> bool:
|
| 86 |
-
return bool(self.api_key)
|
| 87 |
-
|
| 88 |
-
def extract(self, file_path: str, max_pages: int | None = None) -> ExtractionResult:
|
| 89 |
-
if not self.api_key:
|
| 90 |
-
raise RuntimeError(
|
| 91 |
-
"OpenBMB API key is not configured. Set OPENBMB_API_KEY locally or add it as a Hugging Face Space secret."
|
| 92 |
-
)
|
| 93 |
-
|
| 94 |
-
document_parts = document_to_payload_parts(file_path, max_pages=max_pages)
|
| 95 |
-
payload = {
|
| 96 |
-
"model": self.model,
|
| 97 |
-
"messages": [
|
| 98 |
-
{
|
| 99 |
-
"role": "user",
|
| 100 |
-
"content": [
|
| 101 |
-
{"type": "text", "text": EXTRACTION_PROMPT},
|
| 102 |
-
*document_parts,
|
| 103 |
-
],
|
| 104 |
-
}
|
| 105 |
-
],
|
| 106 |
-
"temperature": 0,
|
| 107 |
-
"max_tokens": 2048,
|
| 108 |
-
}
|
| 109 |
-
|
| 110 |
-
started = time.perf_counter()
|
| 111 |
-
response = requests.post(
|
| 112 |
-
self.api_url,
|
| 113 |
-
headers={
|
| 114 |
-
"Authorization": f"Bearer {self.api_key}",
|
| 115 |
-
"Content-Type": "application/json",
|
| 116 |
-
},
|
| 117 |
-
json=payload,
|
| 118 |
-
timeout=self.timeout_seconds,
|
| 119 |
-
)
|
| 120 |
-
duration_ms = int((time.perf_counter() - started) * 1000)
|
| 121 |
-
try:
|
| 122 |
-
response.raise_for_status()
|
| 123 |
-
except HTTPError as error:
|
| 124 |
-
if response.status_code == 401:
|
| 125 |
-
raise RuntimeError(
|
| 126 |
-
"OpenBMB rejected the API key with 401 Unauthorized. Check that the token is exact, active, and belongs to this endpoint."
|
| 127 |
-
) from error
|
| 128 |
-
raise
|
| 129 |
-
|
| 130 |
-
raw_response = _extract_message_content(response.json())
|
| 131 |
-
parsed = _parse_json_response(raw_response)
|
| 132 |
-
|
| 133 |
-
return ExtractionResult(
|
| 134 |
-
patient=_normalize_patient(parsed.get("patient", {})),
|
| 135 |
-
tests=_normalize_tests(parsed.get("tests", [])),
|
| 136 |
-
notes=_normalize_notes(parsed.get("notes", [])),
|
| 137 |
-
raw_response=raw_response,
|
| 138 |
-
request_summary={
|
| 139 |
-
"backend": "api",
|
| 140 |
-
"api_url": self.api_url,
|
| 141 |
-
"model": self.model,
|
| 142 |
-
"document_parts": len(document_parts),
|
| 143 |
-
"pages": max_pages or "auto",
|
| 144 |
-
"extraction_prompt": EXTRACTION_PROMPT,
|
| 145 |
-
"user_message_preview": summarize_document_parts(document_parts),
|
| 146 |
-
**document_intake_metadata(file_path, document_parts),
|
| 147 |
-
"http_status": response.status_code,
|
| 148 |
-
"return_code": 0,
|
| 149 |
-
"duration_ms": duration_ms,
|
| 150 |
-
},
|
| 151 |
-
)
|
| 152 |
-
|
| 153 |
-
|
| 154 |
def summarize_document_parts(parts: list[dict[str, Any]]) -> dict[str, int]:
|
| 155 |
"""Lightweight payload stats for pipeline traces (no base64 blobs)."""
|
| 156 |
image_count = 0
|
|
@@ -163,34 +71,6 @@ def summarize_document_parts(parts: list[dict[str, Any]]) -> dict[str, int]:
|
|
| 163 |
return {"image_count": image_count, "text_characters": text_characters}
|
| 164 |
|
| 165 |
|
| 166 |
-
def _extract_message_content(payload: dict[str, Any]) -> str:
|
| 167 |
-
try:
|
| 168 |
-
message = payload["choices"][0]["message"]
|
| 169 |
-
except (KeyError, IndexError, TypeError) as error:
|
| 170 |
-
raise ValueError("OpenBMB response did not include choices[0].message.") from error
|
| 171 |
-
|
| 172 |
-
content = message.get("content", "")
|
| 173 |
-
if isinstance(content, str):
|
| 174 |
-
return content.strip()
|
| 175 |
-
|
| 176 |
-
if isinstance(content, list):
|
| 177 |
-
text_parts = [part.get("text", "") for part in content if isinstance(part, dict)]
|
| 178 |
-
return "\n".join(text_parts).strip()
|
| 179 |
-
|
| 180 |
-
raise ValueError("OpenBMB response message content was not text.")
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
def _normalize_api_key(value: str | None) -> str | None:
|
| 184 |
-
if not value:
|
| 185 |
-
return None
|
| 186 |
-
|
| 187 |
-
cleaned = value.strip()
|
| 188 |
-
if cleaned.lower().startswith("bearer "):
|
| 189 |
-
cleaned = cleaned[7:].strip()
|
| 190 |
-
|
| 191 |
-
return cleaned or None
|
| 192 |
-
|
| 193 |
-
|
| 194 |
def _parse_json_response(text: str) -> dict[str, Any]:
|
| 195 |
cleaned = _strip_think(_strip_code_fence(text))
|
| 196 |
parsed = _loads_model_json(cleaned)
|
|
|
|
| 1 |
from __future__ import annotations
|
| 2 |
|
| 3 |
import json
|
|
|
|
| 4 |
import re
|
|
|
|
| 5 |
from dataclasses import dataclass, field
|
| 6 |
from typing import Any
|
| 7 |
|
|
|
|
| 8 |
from json_repair import loads as repair_json_loads
|
|
|
|
| 9 |
|
|
|
|
| 10 |
from src.local_env import load_local_env
|
| 11 |
|
| 12 |
|
| 13 |
load_local_env()
|
| 14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
EXTRACTION_PROMPT = """
|
| 16 |
You are extracting laboratory test results from a medical document.
|
| 17 |
|
|
|
|
| 59 |
patient: dict[str, Any] = field(default_factory=dict)
|
| 60 |
|
| 61 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
def summarize_document_parts(parts: list[dict[str, Any]]) -> dict[str, int]:
|
| 63 |
"""Lightweight payload stats for pipeline traces (no base64 blobs)."""
|
| 64 |
image_count = 0
|
|
|
|
| 71 |
return {"image_count": image_count, "text_characters": text_characters}
|
| 72 |
|
| 73 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
def _parse_json_response(text: str) -> dict[str, Any]:
|
| 75 |
cleaned = _strip_think(_strip_code_fence(text))
|
| 76 |
parsed = _loads_model_json(cleaned)
|
src/pipeline_trace.py
CHANGED
|
@@ -4,7 +4,7 @@ from __future__ import annotations
|
|
| 4 |
|
| 5 |
import html
|
| 6 |
import json
|
| 7 |
-
from dataclasses import
|
| 8 |
from pathlib import Path
|
| 9 |
from typing import Any
|
| 10 |
|
|
@@ -615,77 +615,3 @@ def error_trace_html(message: str) -> str:
|
|
| 615 |
body = step_to_html(failed_step, interactive=False)
|
| 616 |
return _trace_block(body, interactive=False)
|
| 617 |
|
| 618 |
-
|
| 619 |
-
def step_to_markdown(step: PipelineStep) -> str:
|
| 620 |
-
parts = [f"**{step.title}**", step.summary]
|
| 621 |
-
if step.prompt:
|
| 622 |
-
parts.append(
|
| 623 |
-
f"<details><summary>Full prompt</summary>\n\n```\n{step.prompt}\n```\n</details>"
|
| 624 |
-
)
|
| 625 |
-
if step.input_preview:
|
| 626 |
-
parts.append(
|
| 627 |
-
f"<details><summary>Input preview</summary>\n\n```\n{step.input_preview}\n```\n</details>"
|
| 628 |
-
)
|
| 629 |
-
if step.output_preview:
|
| 630 |
-
parts.append(
|
| 631 |
-
f"<details><summary>Output preview</summary>\n\n```\n{step.output_preview}\n```\n</details>"
|
| 632 |
-
)
|
| 633 |
-
return "\n\n".join(parts)
|
| 634 |
-
|
| 635 |
-
|
| 636 |
-
def trace_to_chat_messages(steps: list[PipelineStep]) -> list[dict[str, str]]:
|
| 637 |
-
intro = (
|
| 638 |
-
"**Analysis pipeline complete.** Below are the agent steps that processed your document."
|
| 639 |
-
)
|
| 640 |
-
messages = [{"role": "assistant", "content": intro}]
|
| 641 |
-
for step in steps:
|
| 642 |
-
messages.append({"role": "assistant", "content": step_to_markdown(step)})
|
| 643 |
-
return messages
|
| 644 |
-
|
| 645 |
-
|
| 646 |
-
def serialize_steps(steps: list[PipelineStep]) -> list[dict[str, Any]]:
|
| 647 |
-
return [asdict(step) for step in steps]
|
| 648 |
-
|
| 649 |
-
|
| 650 |
-
def interpretation_to_dict(interpretation: Interpretation) -> dict[str, Any]:
|
| 651 |
-
return {
|
| 652 |
-
"flagged": [
|
| 653 |
-
{
|
| 654 |
-
"marker": item.marker,
|
| 655 |
-
"value": item.value,
|
| 656 |
-
"unit": item.unit,
|
| 657 |
-
"status": item.status,
|
| 658 |
-
"reference_range": item.reference_range,
|
| 659 |
-
"note": item.note,
|
| 660 |
-
"questions": list(item.questions),
|
| 661 |
-
}
|
| 662 |
-
for item in interpretation.flagged
|
| 663 |
-
],
|
| 664 |
-
"normal_count": interpretation.normal_count,
|
| 665 |
-
"patterns": [{"name": p.name, "note": p.note} for p in interpretation.patterns],
|
| 666 |
-
"disclaimer": interpretation.disclaimer,
|
| 667 |
-
}
|
| 668 |
-
|
| 669 |
-
|
| 670 |
-
def extraction_to_dict(extraction: ExtractionResult) -> dict[str, Any]:
|
| 671 |
-
return {
|
| 672 |
-
"patient": extraction.patient,
|
| 673 |
-
"tests": extraction.tests,
|
| 674 |
-
"notes": extraction.notes,
|
| 675 |
-
"raw_response": extraction.raw_response,
|
| 676 |
-
"request_summary": extraction.request_summary,
|
| 677 |
-
}
|
| 678 |
-
|
| 679 |
-
|
| 680 |
-
def build_session_state(
|
| 681 |
-
extraction: ExtractionResult,
|
| 682 |
-
health_report: dict[str, Any],
|
| 683 |
-
steps: list[PipelineStep],
|
| 684 |
-
) -> dict[str, Any]:
|
| 685 |
-
interpretation = build_interpretation(extraction.tests)
|
| 686 |
-
return {
|
| 687 |
-
"extraction": extraction_to_dict(extraction),
|
| 688 |
-
"health_report": health_report,
|
| 689 |
-
"interpretation": interpretation_to_dict(interpretation),
|
| 690 |
-
"trace_steps": serialize_steps(steps),
|
| 691 |
-
}
|
|
|
|
| 4 |
|
| 5 |
import html
|
| 6 |
import json
|
| 7 |
+
from dataclasses import dataclass, field
|
| 8 |
from pathlib import Path
|
| 9 |
from typing import Any
|
| 10 |
|
|
|
|
| 615 |
body = step_to_html(failed_step, interactive=False)
|
| 616 |
return _trace_block(body, interactive=False)
|
| 617 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
src/results_chat.py
DELETED
|
@@ -1,156 +0,0 @@
|
|
| 1 |
-
"""Context-aware chat about uploaded blood test results."""
|
| 2 |
-
|
| 3 |
-
from __future__ import annotations
|
| 4 |
-
|
| 5 |
-
import json
|
| 6 |
-
from typing import Any
|
| 7 |
-
|
| 8 |
-
from src.extraction.text_generation import generate_text_chat
|
| 9 |
-
|
| 10 |
-
CHAT_SYSTEM_PROMPT = """
|
| 11 |
-
You are an educational assistant helping a patient understand blood test results.
|
| 12 |
-
|
| 13 |
-
Rules:
|
| 14 |
-
- Use ONLY the patient context, extracted lab values, knowledge-graph enrichment, and grounded
|
| 15 |
-
interpretation notes provided below.
|
| 16 |
-
- Do not diagnose, prescribe, or invent medical facts not present in the context.
|
| 17 |
-
- Use plain language and say when a clinician should interpret a result in person.
|
| 18 |
-
- If the user asks about something not in the context, say you do not have that information.
|
| 19 |
-
- Keep answers concise unless the user asks for detail.
|
| 20 |
-
""".strip()
|
| 21 |
-
|
| 22 |
-
_MAX_CONTEXT_CHARS = 8000
|
| 23 |
-
_MAX_HISTORY_TURNS = 6
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
class ResultsChatAssistant:
|
| 27 |
-
def reply(
|
| 28 |
-
self,
|
| 29 |
-
user_message: str,
|
| 30 |
-
chat_history: list[dict[str, str]] | None,
|
| 31 |
-
session: dict[str, Any] | None,
|
| 32 |
-
) -> str:
|
| 33 |
-
message = (user_message or "").strip()
|
| 34 |
-
if not message:
|
| 35 |
-
return "Please enter a question about your blood test results."
|
| 36 |
-
|
| 37 |
-
if not session or not session.get("health_report"):
|
| 38 |
-
return "Upload and analyze a lab report first, then I can answer questions about your results."
|
| 39 |
-
|
| 40 |
-
context = build_chat_context(session)
|
| 41 |
-
messages = _build_messages(context, chat_history or [], message)
|
| 42 |
-
try:
|
| 43 |
-
return generate_text_chat(messages)
|
| 44 |
-
except Exception as exc:
|
| 45 |
-
return (
|
| 46 |
-
"I couldn't generate a chat reply with the current model backend. "
|
| 47 |
-
f"Details: {exc}"
|
| 48 |
-
)
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
def build_chat_context(session: dict[str, Any]) -> str:
|
| 52 |
-
health_report = session.get("health_report") or {}
|
| 53 |
-
extraction = session.get("extraction") or {}
|
| 54 |
-
interpretation = session.get("interpretation") or {}
|
| 55 |
-
|
| 56 |
-
patient = health_report.get("patient") or extraction.get("patient") or {}
|
| 57 |
-
markers = health_report.get("markers") or []
|
| 58 |
-
summary = health_report.get("summary") or {}
|
| 59 |
-
|
| 60 |
-
lines: list[str] = [
|
| 61 |
-
"=== Patient context ===",
|
| 62 |
-
json.dumps(
|
| 63 |
-
{
|
| 64 |
-
"age": patient.get("age"),
|
| 65 |
-
"age_years": patient.get("age_years"),
|
| 66 |
-
"age_group": patient.get("age_group"),
|
| 67 |
-
"sex": patient.get("sex"),
|
| 68 |
-
},
|
| 69 |
-
indent=2,
|
| 70 |
-
),
|
| 71 |
-
"",
|
| 72 |
-
"=== Report summary ===",
|
| 73 |
-
json.dumps(summary, indent=2),
|
| 74 |
-
"",
|
| 75 |
-
"=== Extracted markers ===",
|
| 76 |
-
]
|
| 77 |
-
|
| 78 |
-
for marker in markers[:40]:
|
| 79 |
-
lines.append(
|
| 80 |
-
json.dumps(
|
| 81 |
-
{
|
| 82 |
-
"name": marker.get("display_name") or marker.get("raw_name"),
|
| 83 |
-
"value": marker.get("value"),
|
| 84 |
-
"unit": marker.get("unit"),
|
| 85 |
-
"status": marker.get("status"),
|
| 86 |
-
"lab_reference_range": marker.get("lab_reference_range"),
|
| 87 |
-
"comparison_basis": (marker.get("comparison") or {}).get("basis"),
|
| 88 |
-
"kg_description": ((marker.get("knowledge") or {}).get("description")),
|
| 89 |
-
"kg_importance": ((marker.get("knowledge") or {}).get("why_important")),
|
| 90 |
-
},
|
| 91 |
-
ensure_ascii=False,
|
| 92 |
-
)
|
| 93 |
-
)
|
| 94 |
-
|
| 95 |
-
if interpretation.get("flagged"):
|
| 96 |
-
lines.extend(["", "=== Flagged markers (KB-grounded) ==="])
|
| 97 |
-
for item in interpretation["flagged"]:
|
| 98 |
-
lines.append(json.dumps(item, ensure_ascii=False))
|
| 99 |
-
|
| 100 |
-
if interpretation.get("patterns"):
|
| 101 |
-
lines.extend(["", "=== Cross-marker patterns ==="])
|
| 102 |
-
for item in interpretation["patterns"]:
|
| 103 |
-
lines.append(json.dumps(item, ensure_ascii=False))
|
| 104 |
-
|
| 105 |
-
if extraction.get("notes"):
|
| 106 |
-
lines.extend(["", "=== Extraction notes ===", json.dumps(extraction["notes"], indent=2)])
|
| 107 |
-
|
| 108 |
-
lines.extend(["", "=== Disclaimer ===", interpretation.get("disclaimer", "")])
|
| 109 |
-
|
| 110 |
-
context = "\n".join(lines)
|
| 111 |
-
if len(context) <= _MAX_CONTEXT_CHARS:
|
| 112 |
-
return context
|
| 113 |
-
return context[: _MAX_CONTEXT_CHARS - 3].rstrip() + "..."
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
def _build_messages(
|
| 117 |
-
context: str,
|
| 118 |
-
chat_history: list[dict[str, str]],
|
| 119 |
-
user_message: str,
|
| 120 |
-
) -> list[dict[str, str]]:
|
| 121 |
-
messages: list[dict[str, str]] = [
|
| 122 |
-
{"role": "system", "content": CHAT_SYSTEM_PROMPT},
|
| 123 |
-
{"role": "user", "content": f"Blood test context:\n\n{context}"},
|
| 124 |
-
{
|
| 125 |
-
"role": "assistant",
|
| 126 |
-
"content": "I have your blood test context. Ask me anything about these results.",
|
| 127 |
-
},
|
| 128 |
-
]
|
| 129 |
-
|
| 130 |
-
recent = _recent_chat_turns(chat_history)
|
| 131 |
-
messages.extend(recent)
|
| 132 |
-
messages.append({"role": "user", "content": user_message})
|
| 133 |
-
return messages
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
def _recent_chat_turns(chat_history: list[dict[str, str]]) -> list[dict[str, str]]:
|
| 137 |
-
"""Keep only user follow-up turns, skipping pipeline trace assistant messages."""
|
| 138 |
-
turns: list[dict[str, str]] = []
|
| 139 |
-
for item in chat_history:
|
| 140 |
-
role = item.get("role")
|
| 141 |
-
content = str(item.get("content") or "").strip()
|
| 142 |
-
if not content or role not in {"user", "assistant"}:
|
| 143 |
-
continue
|
| 144 |
-
if role == "assistant" and content.startswith("**Step "):
|
| 145 |
-
continue
|
| 146 |
-
if role == "assistant" and content.startswith("**Analysis pipeline complete."):
|
| 147 |
-
continue
|
| 148 |
-
if role == "assistant" and content.startswith("**Pipeline running"):
|
| 149 |
-
continue
|
| 150 |
-
if role == "assistant" and content.startswith("Upload a lab report"):
|
| 151 |
-
continue
|
| 152 |
-
turns.append({"role": role, "content": content})
|
| 153 |
-
|
| 154 |
-
if len(turns) > _MAX_HISTORY_TURNS * 2:
|
| 155 |
-
turns = turns[-(_MAX_HISTORY_TURNS * 2) :]
|
| 156 |
-
return turns
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
tests/test_model_paths.py
CHANGED
|
@@ -7,8 +7,17 @@ sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
|
| 7 |
from src.model_paths import is_transformers_model_dir, resolve_transformers_model_source
|
| 8 |
|
| 9 |
|
| 10 |
-
def test_resolve_uses_hub_download_when_no_local_weights():
|
| 11 |
with tempfile.TemporaryDirectory() as tmp:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
source = resolve_transformers_model_source("openbmb/MiniCPM-V-4.6")
|
| 13 |
assert source.local_files_only is False
|
| 14 |
assert source.origin == "hub-download"
|
|
|
|
| 7 |
from src.model_paths import is_transformers_model_dir, resolve_transformers_model_source
|
| 8 |
|
| 9 |
|
| 10 |
+
def test_resolve_uses_hub_download_when_no_local_weights(monkeypatch):
|
| 11 |
with tempfile.TemporaryDirectory() as tmp:
|
| 12 |
+
empty_models = Path(tmp) / "models"
|
| 13 |
+
empty_models.mkdir()
|
| 14 |
+
monkeypatch.setenv("BTE_MODELS_DIR", str(empty_models))
|
| 15 |
+
monkeypatch.setenv("HF_HOME", str(Path(tmp) / "hf"))
|
| 16 |
+
monkeypatch.setattr(
|
| 17 |
+
"src.model_paths.latest_complete_snapshot",
|
| 18 |
+
lambda repo_id, hub_cache: None,
|
| 19 |
+
)
|
| 20 |
+
|
| 21 |
source = resolve_transformers_model_source("openbmb/MiniCPM-V-4.6")
|
| 22 |
assert source.local_files_only is False
|
| 23 |
assert source.origin == "hub-download"
|
tests/test_pipeline_trace.py
CHANGED
|
@@ -4,7 +4,7 @@ from pathlib import Path
|
|
| 4 |
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
| 5 |
|
| 6 |
from src.openbmb_client import EXTRACTION_PROMPT, ExtractionResult
|
| 7 |
-
from src.pipeline_trace import build_pipeline_trace, empty_trace_html, trace_hover_js, trace_to_html
|
| 8 |
from src.report_pipeline import build_health_report
|
| 9 |
|
| 10 |
|
|
@@ -119,19 +119,8 @@ def test_trace_hover_js_registers_listeners_once():
|
|
| 119 |
assert "<script" not in js.lower()
|
| 120 |
|
| 121 |
|
| 122 |
-
def test_trace_to_chat_messages_shape():
|
| 123 |
-
extraction = _sample_extraction()
|
| 124 |
-
report = build_health_report(extraction)
|
| 125 |
-
steps = build_pipeline_trace(extraction, report)
|
| 126 |
-
messages = trace_to_chat_messages(steps)
|
| 127 |
-
assert messages[0]["role"] == "assistant"
|
| 128 |
-
assert all(msg["role"] == "assistant" for msg in messages)
|
| 129 |
-
assert len(messages) == len(steps) + 1
|
| 130 |
-
|
| 131 |
-
|
| 132 |
if __name__ == "__main__":
|
| 133 |
test_build_pipeline_trace_has_five_steps()
|
| 134 |
test_extraction_step_includes_full_prompt()
|
| 135 |
test_trace_to_html_collapsible_steps()
|
| 136 |
-
test_trace_to_chat_messages_shape()
|
| 137 |
print("test_pipeline_trace: ok")
|
|
|
|
| 4 |
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
| 5 |
|
| 6 |
from src.openbmb_client import EXTRACTION_PROMPT, ExtractionResult
|
| 7 |
+
from src.pipeline_trace import build_pipeline_trace, empty_trace_html, trace_hover_js, trace_to_html
|
| 8 |
from src.report_pipeline import build_health_report
|
| 9 |
|
| 10 |
|
|
|
|
| 119 |
assert "<script" not in js.lower()
|
| 120 |
|
| 121 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 122 |
if __name__ == "__main__":
|
| 123 |
test_build_pipeline_trace_has_five_steps()
|
| 124 |
test_extraction_step_includes_full_prompt()
|
| 125 |
test_trace_to_html_collapsible_steps()
|
|
|
|
| 126 |
print("test_pipeline_trace: ok")
|
tests/test_results_chat.py
DELETED
|
@@ -1,60 +0,0 @@
|
|
| 1 |
-
import sys
|
| 2 |
-
from pathlib import Path
|
| 3 |
-
from unittest.mock import patch
|
| 4 |
-
|
| 5 |
-
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
| 6 |
-
|
| 7 |
-
from src.openbmb_client import ExtractionResult
|
| 8 |
-
from src.pipeline_trace import build_pipeline_trace, build_session_state
|
| 9 |
-
from src.report_pipeline import build_health_report
|
| 10 |
-
from src.results_chat import ResultsChatAssistant, build_chat_context
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
def _session() -> dict:
|
| 14 |
-
extraction = ExtractionResult(
|
| 15 |
-
patient={"age_years": 42, "sex": "female"},
|
| 16 |
-
tests=[
|
| 17 |
-
{
|
| 18 |
-
"marker": "Hemoglobin",
|
| 19 |
-
"value": "11.2",
|
| 20 |
-
"unit": "g/dL",
|
| 21 |
-
"reference_range": "12.0-16.0",
|
| 22 |
-
"status": "low",
|
| 23 |
-
"confidence": 0.9,
|
| 24 |
-
}
|
| 25 |
-
],
|
| 26 |
-
notes=[],
|
| 27 |
-
raw_response="{}",
|
| 28 |
-
request_summary={"backend": "test", "document_parts": 1},
|
| 29 |
-
)
|
| 30 |
-
report = build_health_report(extraction)
|
| 31 |
-
steps = build_pipeline_trace(extraction, report)
|
| 32 |
-
return build_session_state(extraction, report, steps)
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
def test_build_chat_context_includes_patient_and_markers():
|
| 36 |
-
context = build_chat_context(_session())
|
| 37 |
-
assert "female" in context
|
| 38 |
-
assert "Hemoglobin" in context
|
| 39 |
-
assert "Report summary" in context
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
def test_reply_requires_session():
|
| 43 |
-
assistant = ResultsChatAssistant()
|
| 44 |
-
reply = assistant.reply("What is low hemoglobin?", [], {})
|
| 45 |
-
assert "Upload and analyze" in reply
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
def test_reply_uses_llm_when_session_present():
|
| 49 |
-
assistant = ResultsChatAssistant()
|
| 50 |
-
session = _session()
|
| 51 |
-
with patch("src.results_chat.generate_text_chat", return_value="Educational reply."):
|
| 52 |
-
reply = assistant.reply("Explain my hemoglobin.", [], session)
|
| 53 |
-
assert reply == "Educational reply."
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
if __name__ == "__main__":
|
| 57 |
-
test_build_chat_context_includes_patient_and_markers()
|
| 58 |
-
test_reply_requires_session()
|
| 59 |
-
test_reply_uses_llm_when_session_present()
|
| 60 |
-
print("test_results_chat: ok")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
train/modal_eval.py
CHANGED
|
@@ -7,7 +7,7 @@ runs directly on the Modal GPU).
|
|
| 7 |
|
| 8 |
modal run train/modal_eval.py::compare --finetuned-id dimitriskalligaridis/blood-test-minicpmv-4_6
|
| 9 |
|
| 10 |
-
Writes eval/before_after.json locally
|
| 11 |
"""
|
| 12 |
|
| 13 |
from __future__ import annotations
|
|
@@ -117,7 +117,7 @@ def compare(
|
|
| 117 |
|
| 118 |
out = Path("eval/before_after.json")
|
| 119 |
out.write_text(json.dumps({"base": base, "finetuned": fine}, indent=2), encoding="utf-8")
|
| 120 |
-
print(f"\n wrote {out}
|
| 121 |
|
| 122 |
|
| 123 |
@app.function(
|
|
|
|
| 7 |
|
| 8 |
modal run train/modal_eval.py::compare --finetuned-id dimitriskalligaridis/blood-test-minicpmv-4_6
|
| 9 |
|
| 10 |
+
Writes eval/before_after.json locally with the base vs fine-tuned metrics.
|
| 11 |
"""
|
| 12 |
|
| 13 |
from __future__ import annotations
|
|
|
|
| 117 |
|
| 118 |
out = Path("eval/before_after.json")
|
| 119 |
out.write_text(json.dumps({"base": base, "finetuned": fine}, indent=2), encoding="utf-8")
|
| 120 |
+
print(f"\n wrote {out}\n")
|
| 121 |
|
| 122 |
|
| 123 |
@app.function(
|
train/modal_medreason.py
DELETED
|
@@ -1,121 +0,0 @@
|
|
| 1 |
-
"""Medical-reasoning LoRA fine-tune (Track 1) — earns Well-Tuned WITHOUT touching extraction.
|
| 2 |
-
|
| 3 |
-
Roman's idea, done as LoRA (not full FT, which would catastrophically forget the vision/extraction
|
| 4 |
-
ability). We freeze the vision encoder and LoRA the LLM only, on the general medical-reasoning
|
| 5 |
-
dataset FreedomIntelligence/medical-o1-reasoning-SFT (TEXT, no images). The result is used as the
|
| 6 |
-
*interpretation phraser* that speaks the KB-grounded facts fluently — extraction stays on base.
|
| 7 |
-
|
| 8 |
-
A held-out slice is used for eval (reasoning loss). Gate A also re-runs the extraction eval on the
|
| 9 |
-
merged model to confirm extraction did not regress.
|
| 10 |
-
|
| 11 |
-
modal run train/modal_medreason.py::main --n 100 --epochs 1 # cheap smoke test first
|
| 12 |
-
modal run --detach train/modal_medreason.py::main # full run (n=4000)
|
| 13 |
-
modal run train/modal_finetune.py::merge --adapter-dir /adapters/medreason-lora --repo-id <owner>/<name>-medreason
|
| 14 |
-
modal run train/modal_eval.py::compare --finetuned-id <owner>/<name>-medreason # Gate A: extraction unharmed?
|
| 15 |
-
"""
|
| 16 |
-
|
| 17 |
-
from __future__ import annotations
|
| 18 |
-
|
| 19 |
-
import modal
|
| 20 |
-
|
| 21 |
-
MODEL_ID = "openbmb/MiniCPM-V-4.6"
|
| 22 |
-
|
| 23 |
-
app = modal.App("blood-test-medreason")
|
| 24 |
-
|
| 25 |
-
image = (
|
| 26 |
-
modal.Image.debian_slim(python_version="3.11")
|
| 27 |
-
.apt_install("git")
|
| 28 |
-
.pip_install(
|
| 29 |
-
# Pin the exact ms-swift that recognizes MiniCPM-V 4.6 (the extraction run used this);
|
| 30 |
-
# an unpinned `datasets` previously dragged ms-swift down to a version that didn't. ms-swift
|
| 31 |
-
# brings a compatible `datasets`, so we don't add it ourselves.
|
| 32 |
-
"torch",
|
| 33 |
-
"transformers>=5.7.0",
|
| 34 |
-
"peft>=0.12",
|
| 35 |
-
"accelerate>=0.33",
|
| 36 |
-
"ms-swift==4.3.0",
|
| 37 |
-
"sentencepiece",
|
| 38 |
-
"timm",
|
| 39 |
-
)
|
| 40 |
-
)
|
| 41 |
-
|
| 42 |
-
adapters = modal.Volume.from_name("blood-test-adapters", create_if_missing=True)
|
| 43 |
-
hf_cache = modal.Volume.from_name("blood-test-hf-cache", create_if_missing=True)
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
@app.function(
|
| 47 |
-
image=image,
|
| 48 |
-
gpu="A100",
|
| 49 |
-
timeout=6 * 60 * 60,
|
| 50 |
-
volumes={"/adapters": adapters, "/root/.cache/huggingface": hf_cache},
|
| 51 |
-
)
|
| 52 |
-
def train_medreason(n: int = 4000, epochs: int = 1, lr: float = 1e-4, n_eval: int = 500, seed: int = 13) -> str:
|
| 53 |
-
import json
|
| 54 |
-
import os
|
| 55 |
-
import subprocess
|
| 56 |
-
from pathlib import Path
|
| 57 |
-
|
| 58 |
-
from datasets import load_dataset
|
| 59 |
-
|
| 60 |
-
os.environ["USE_HF"] = "1" # pull dataset + weights from HF (fast on Modal), not ModelScope
|
| 61 |
-
|
| 62 |
-
# 1) medical-o1 reasoning data (English) -> text chat messages (Question -> CoT + Response)
|
| 63 |
-
ds = load_dataset("FreedomIntelligence/medical-o1-reasoning-SFT", "en", split="train")
|
| 64 |
-
ds = ds.shuffle(seed=seed).select(range(min(n + n_eval, len(ds))))
|
| 65 |
-
|
| 66 |
-
def to_messages(ex: dict) -> dict:
|
| 67 |
-
q = (ex.get("Question") or "").strip()
|
| 68 |
-
cot = (ex.get("Complex_CoT") or "").strip()
|
| 69 |
-
resp = (ex.get("Response") or "").strip()
|
| 70 |
-
answer = f"{cot}\n\n{resp}".strip() if cot else resp
|
| 71 |
-
return {"messages": [{"role": "user", "content": q}, {"role": "assistant", "content": answer}]}
|
| 72 |
-
|
| 73 |
-
rows = [to_messages(ex) for ex in ds]
|
| 74 |
-
val_rows, train_rows = rows[:n_eval], rows[n_eval:]
|
| 75 |
-
|
| 76 |
-
data_dir = Path("/root/data")
|
| 77 |
-
data_dir.mkdir(parents=True, exist_ok=True)
|
| 78 |
-
train_path = data_dir / "medreason_train.jsonl"
|
| 79 |
-
val_path = data_dir / "medreason_val.jsonl"
|
| 80 |
-
train_path.write_text("\n".join(json.dumps(r, ensure_ascii=False) for r in train_rows), encoding="utf-8")
|
| 81 |
-
val_path.write_text("\n".join(json.dumps(r, ensure_ascii=False) for r in val_rows), encoding="utf-8")
|
| 82 |
-
print(f"medical-o1: {len(train_rows)} train, {len(val_rows)} held-out eval examples")
|
| 83 |
-
|
| 84 |
-
# 2) LoRA the LLM only (freeze vision) on the reasoning text — keeps extraction untouched.
|
| 85 |
-
out_dir = "/adapters/medreason-lora"
|
| 86 |
-
cmd = [
|
| 87 |
-
"swift", "sft",
|
| 88 |
-
"--model", MODEL_ID,
|
| 89 |
-
"--dataset", str(train_path),
|
| 90 |
-
"--val_dataset", str(val_path),
|
| 91 |
-
"--num_train_epochs", str(epochs),
|
| 92 |
-
"--lora_rank", "16",
|
| 93 |
-
"--lora_alpha", "32",
|
| 94 |
-
"--learning_rate", str(lr),
|
| 95 |
-
"--warmup_ratio", "0.05",
|
| 96 |
-
"--per_device_train_batch_size", "2",
|
| 97 |
-
"--gradient_accumulation_steps", "8",
|
| 98 |
-
"--max_length", "4096", # medical CoT answers are long
|
| 99 |
-
"--freeze_vit", "true", # do not touch the vision encoder (extraction lives there)
|
| 100 |
-
"--eval_steps", "50", # report held-out reasoning loss during training
|
| 101 |
-
"--output_dir", out_dir,
|
| 102 |
-
"--save_total_limit", "1",
|
| 103 |
-
]
|
| 104 |
-
print("Running:", " ".join(cmd))
|
| 105 |
-
subprocess.run(cmd, check=True, env={**os.environ, "USE_HF": "1"})
|
| 106 |
-
|
| 107 |
-
adapters.commit()
|
| 108 |
-
return out_dir
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
@app.local_entrypoint()
|
| 112 |
-
def main(n: int = 4000, epochs: int = 1, lr: float = 1e-4) -> None:
|
| 113 |
-
# spawn() runs the job in the background on Modal and returns immediately, so it survives the
|
| 114 |
-
# local client disconnecting (.remote() + --detach can still be canceled on a dropped connection).
|
| 115 |
-
call = train_medreason.spawn(n=n, epochs=epochs, lr=lr)
|
| 116 |
-
print(f"\nLaunched medical-reasoning LoRA on Modal (call {call.object_id}) — runs in the background.")
|
| 117 |
-
print("Watch it in the Modal dashboard; it saves to the 'blood-test-adapters' volume. When done:")
|
| 118 |
-
print(" modal run train/modal_finetune.py::merge --adapter-dir /adapters/medreason-lora \\")
|
| 119 |
-
print(" --repo-id dimitriskl/blood-test-minicpmv-4_6-medreason")
|
| 120 |
-
print(" modal run train/modal_eval.py::compare --finetuned-id dimitriskl/blood-test-minicpmv-4_6-medreason")
|
| 121 |
-
print(" ^ Gate A: confirms extraction did NOT regress on the reasoning-tuned model.")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|