Spaces:
Running
Running
| title: MEDTRACE 🔬 | |
| emoji: 🧠 | |
| colorFrom: indigo | |
| colorTo: blue | |
| sdk: static | |
| app_file: index.html | |
| license: mit | |
| short_description: Longitudinal AI for brain-MRI disease evolution | |
| models: | |
| - AIOmarRehan/medtrace-brats-segresnet | |
| datasets: | |
| - AIOmarRehan/medtrace-rhuh-gbm-derived | |
| tags: | |
| - medical-imaging | |
| - mri | |
| - brain-tumour | |
| - segmentation | |
| - longitudinal | |
| - cornerstone3d | |
| - vtk | |
| [The Medium Article](https://medium.com/@ai.omar.rehan/medtrac-longitudinal-ai-for-brain-mri-analysis-tracking-disease-evolution-over-time-05808ebed764) | |
| --- | |
| <div align="center"> | |
| <img src="GIF/Brain.gif" alt="MEDTRACE: longitudinal brain MRI analysis" width="460" /> | |
| # MEDTRACE | |
| ### Longitudinal AI for brain-MRI disease evolution | |
| **An interactive, evidence-linked map of how a brain tumour changes over time.** | |
| <br /> | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
| </div> | |
| > [!WARNING] | |
| > **Research prototype. Not a medical device.** | |
| > MEDTRACE is not intended for diagnosis, treatment planning, or any clinical decision. It has | |
| > not been clinically validated. It reports **measured change only**, never a diagnosis, grade, | |
| > progression judgement, treatment recommendation, or prognosis. All clinical decisions remain | |
| > with qualified healthcare professionals. Built exclusively on public, de-identified research | |
| > data. | |
| > [!NOTE] | |
| > **What this hosted demo serves.** 28 glioblastoma patients from | |
| > [RHUH-GBM](https://doi.org/10.7937/4545-c905), 3 timepoints each, under CC BY 4.0. The figures | |
| > below describe the full local build across LUMIERE and BraTS 2023, which are covered by data use | |
| > agreements that grant use but not redistribution, so they are not published here. | |
| > | |
| > There is no server. This is a Static Space: the interface and its recorded responses are served | |
| > from this repository, imaging is range-fetched from the | |
| > [dataset repository](https://huggingface.co/datasets/AIOmarRehan/medtrace-rhuh-gbm-derived), and | |
| > both renderers run on your GPU. Nothing is computed on request. | |
| > | |
| > **The linked model is published alongside this demo, not run by it.** Every outline and every | |
| > measurement shown comes from the expert-corrected segmentations that ship with RHUH-GBM. That is | |
| > deliberate: RHUH-GBM is post-operative and post-treatment, and the model was trained on | |
| > pre-operative scans only, so the one dataset published here is the one regime the model was not | |
| > trained for. The model card gives the reasoning and the numbers behind it. | |
| > | |
| > [Model](https://huggingface.co/AIOmarRehan/medtrace-brats-segresnet) · | |
| > [Dataset](https://huggingface.co/datasets/AIOmarRehan/medtrace-rhuh-gbm-derived) · | |
| > [Code](https://github.com/AIOmarRehan/medtrace) | |
| --- | |
| <div align="center"> | |
| <table> | |
| <tr> | |
| <td align="center"><b>209</b><br />real patients</td> | |
| <td align="center"><b>874</b><br />MRI studies</td> | |
| <td align="center"><b>3,431</b><br />image series</td> | |
| <td align="center"><b>1,153</b><br />disease observations</td> | |
| </tr> | |
| <tr> | |
| <td align="center"><b>772</b><br />tracked lesions</td> | |
| <td align="center"><b>6,499</b><br />measurements</td> | |
| <td align="center"><b>599</b><br />atlas volumes</td> | |
| <td align="center"><b>59.9</b><br />3D fps measured</td> | |
| </tr> | |
| </table> | |
| </div> | |
| --- | |
| ## Contents | |
| <table> | |
| <tr> | |
| <td valign="top" width="33%"> | |
| **Understanding it** | |
| - [The clinical problem](#the-clinical-problem) | |
| - [What MEDTRACE answers](#what-medtrace-answers) | |
| - [The workstation](#the-workstation) | |
| - [Analysis pipeline](#analysis-pipeline) | |
| </td> | |
| <td valign="top" width="33%"> | |
| **How it works** | |
| - [Domain model](#domain-model-observations-not-images) | |
| - [AI segmentation](#ai-segmentation) | |
| - [Training notebook](#the-training-notebook) | |
| - [Validation & results](#validation--results) | |
| - [Measurement & change](#measurement-matching-and-change) | |
| - [3D disease evolution](#3d-disease-evolution) | |
| - [Evidence engine](#the-evidence-engine) | |
| </td> | |
| <td valign="top" width="33%"> | |
| **Running it** | |
| - [Architecture](#architecture) | |
| - [Technology stack](#technology-stack) | |
| - [Datasets](#datasets) | |
| - [Verification](#verification) | |
| - [Getting started](#getting-started) | |
| </td> | |
| </tr> | |
| </table> | |
| --- | |
| ## The clinical problem | |
| A patient with a brain tumour is imaged repeatedly: before surgery, after surgery, during | |
| radiotherapy and chemotherapy, then at follow-up for years. **Clinical decisions are made by | |
| comparing these examinations, not by reading any one of them.** | |
| Today that comparison is largely manual: | |
| ``` | |
| radiologist opens prior study ──► reads previous report ──► scrolls both studies side by side | |
| ──► re-measures the lesion by hand ──► mentally reconstructs the patient's history | |
| ``` | |
| Most medical-imaging AI does not help, because it analyses **one scan at one moment**. | |
| Longitudinal comparison is a recognised underdeveloped area of medical imaging AI, and it is | |
| exactly where the clinical decision actually happens. | |
| MEDTRACE exists to answer one question: | |
| <div align="center"> | |
| ### *What changed in this patient between examinations?* | |
| </div> | |
| --- | |
| ## What MEDTRACE answers | |
| Every screen exists to serve one of five questions. Nothing else is in scope. | |
| | Question | How MEDTRACE answers it | | |
| |---|---| | |
| | **What changed?** | Quantified, with units and direction: volume, diameter, surface area, growth rate | | |
| | **Where did it change?** | Highlighted in the image, per lesion, in 2D and 3D | | |
| | **When did it change?** | Located on the disease timeline, with the interval in days | | |
| | **How confident are we?** | Per pipeline stage, with the reason in plain language | | |
| | **Why do you believe that?** | The evidence, one click away | | |
| **The signature interaction is the time scrubber.** Dragging it moves imaging, segmentation, | |
| measurements, clinical events, the 3D surface and the evidence panel together. | |
| > [!NOTE] | |
| > An observation without evidence **cannot exist** in MEDTRACE. This is enforced in the service | |
| > layer, not by convention. It is why the system can always answer "why". | |
| --- | |
| ## The workstation | |
| <div align="center"> | |
| <img src="docs/screenshots/01-workstation.png" alt="MEDTRACE clinical workstation" width="100%" /> | |
| *The clinical workstation: patient timeline, synchronised prior/current comparison, measured | |
| findings, and the AI evidence strip.* | |
| </div> | |
| <table> | |
| <tr> | |
| <td width="50%" valign="top"> | |
| ### 2D comparison | |
| <img src="docs/screenshots/02-2d-comparison.png" alt="Synchronised prior and current study" width="100%" /> | |
| Prior and current study side by side, **linked by fractional depth** rather than world | |
| coordinates, because unregistered studies differ by tens of millimetres and copying a camera | |
| between them can place it outside the other volume entirely. | |
| <img src="GIF/gif-04-slice-sync.gif" alt="Scrolling one pane moves the other with it" width="100%" /> | |
| *Scroll one pane and the other follows.* | |
| </td> | |
| <td width="50%" valign="top"> | |
| ### Segmentation overlay | |
| <img src="docs/screenshots/03-segmentation-overlay.png" alt="Tumour mask overlaid on native image" width="100%" /> | |
| The tumour mask drawn in the **series' own voxel grid**, reoriented from the NIfTI affines with | |
| no resampling. Outline by default, fill on demand. | |
| </td> | |
| </tr> | |
| </table> | |
| --- | |
| ## Analysis pipeline | |
| Deliberately a pipeline of specialised stages, not one large model. **Every stage emits its | |
| prediction, its confidence, its quality flags and its model version.** | |
| ```mermaid | |
| %%{init: {"theme":"base","themeVariables":{"fontSize":"15px","textColor":"#1e293b","nodeTextColor":"#1e293b","lineColor":"#475569","edgeLabelBackground":"#ffffff"},"flowchart":{"nodeSpacing":34,"rankSpacing":50,"padding":10}}}%% | |
| flowchart LR | |
| A[("MRI studies")] --> B["Quality control"] | |
| B --> C["Segmentation"] | |
| C --> D["Lesion detection"] | |
| D --> E["Registration"] | |
| E --> F["Lesion matching"] | |
| F --> G["Change detection"] | |
| G --> H[("DiseaseObservation")] | |
| H --> I["Disease timeline"] | |
| H --> J["3D evolution map"] | |
| H --> K["Evidence engine"] | |
| classDef src fill:#e0f2fe,stroke:#0284c7,stroke-width:1px,color:#0c4a6e | |
| classDef stage fill:#ffffff,stroke:#64748b,stroke-width:1px,color:#1e293b | |
| classDef core fill:#ccfbf1,stroke:#0d9488,stroke-width:2px,color:#134e4a | |
| classDef view fill:#e0e7ff,stroke:#4f46e5,stroke-width:1px,color:#312e81 | |
| class A src | |
| class B,C,D,E,F,G stage | |
| class H core | |
| class I,J,K view | |
| ``` | |
| <details> | |
| <summary><b>What each stage actually does</b></summary> | |
| <br /> | |
| | Stage | Implementation | Output | | |
| |---|---|---| | |
| | **Quality control** | Rule-based validation of sequences, voxel spacing, orientation, protocol drift between timepoints | `QualityFlag[]` per study | | |
| | **Segmentation** | MONAI `SegResNet`, 18.8 M parameters, 4 sequences in → 3 overlapping compartments out | Tumour compartment masks | | |
| | **Lesion detection** | Connected-component extraction per compartment, with a measurability floor | `Lesion` candidates | | |
| | **Registration** | SimpleITK rigid then affine; atlas-space masks need none, native-space ones do | Transform + score | | |
| | **Lesion matching** | Similarity scoring over overlap (Dice), centroid distance, proximity and volume similarity | `Lesion` identity across time | | |
| | **Change detection** | Volumetric and morphological comparison with uncertainty propagation | Absolute + relative change, growth rate | | |
| | **Trajectory** | Longitudinal feature vectors per lesion across all timepoints | Per-lesion history | | |
| | **Evidence engine** | Structured findings → validated answer → linked evidence | `AIObservation` + `EvidenceItem[]` | | |
| </details> | |
| --- | |
| ## Domain model: observations, not images | |
| The core domain object is **`DiseaseObservation`**, not `MRI`. The timeline, the analytics, the | |
| AI answers and the audit trail are all *views over observations*. | |
| ```mermaid | |
| %%{init: {"theme":"base","themeVariables":{"fontSize":"15px","textColor":"#1e293b","nodeTextColor":"#1e293b","lineColor":"#475569","edgeLabelBackground":"#ffffff"}}}%% | |
| erDiagram | |
| PATIENT ||--o{ STUDY : "timepoints" | |
| PATIENT ||--o{ LESION : "identity" | |
| PATIENT ||--o{ CLINICAL_EVENT : "treatment" | |
| PATIENT ||--o{ AI_OBSERVATION : "answers" | |
| STUDY ||--o{ SERIES : "sequences" | |
| STUDY ||--o| STUDY_QUALITY : "validation" | |
| STUDY ||--o{ SEGMENTATION : "masks" | |
| STUDY ||--o{ LESION_OBSERVATION : "observed in" | |
| LESION ||--o{ LESION_OBSERVATION : "observed at" | |
| LESION_OBSERVATION ||--o{ MEASUREMENT : "quantified by" | |
| LESION_OBSERVATION ||--o{ EVIDENCE_ITEM : "supports" | |
| AI_OBSERVATION ||--o{ EVIDENCE_ITEM : "must cite" | |
| MODEL_RUN ||--o{ LESION_OBSERVATION : "produced" | |
| MODEL_RUN ||--o{ AI_OBSERVATION : "produced" | |
| ``` | |
| A **`Lesion` belongs to a patient, not a study**, and that is what makes a per-lesion trajectory | |
| possible, and it is precisely what a conventional viewer or segmentation tool does not provide. | |
| <details> | |
| <summary><b>The 14-table schema</b></summary> | |
| <br /> | |
| `patients` · `studies` · `series` · `study_quality` · `registrations` · `segmentations` · | |
| `lesions` · `lesion_observations` · `measurements` · `clinical_events` · `model_runs` · | |
| `ai_observations` · `evidence_items` · `audit_events` | |
| Managed with SQLAlchemy + Alembic. PostgreSQL stores **metadata only**. Imaging stays on the | |
| filesystem and is served through a path-allowlisted endpoint, never by a database blob. | |
| </details> | |
| --- | |
| ## AI segmentation | |
| A 3D `SegResNet` trained from scratch on BraTS 2023 GLI, on a Kaggle Tesla T4. | |
| ```mermaid | |
| %%{init: {"theme":"base","themeVariables":{"fontSize":"15px","clusterBkg":"#f1f5f9","clusterBorder":"#94a3b8","textColor":"#1e293b","nodeTextColor":"#1e293b","lineColor":"#475569","edgeLabelBackground":"#ffffff"},"flowchart":{"nodeSpacing":34,"rankSpacing":58,"padding":10}}}%% | |
| flowchart LR | |
| subgraph IN ["4 co-registered sequences"] | |
| direction TB | |
| A1["T1c"] | |
| A2["T1n"] | |
| A3["T2-FLAIR"] | |
| A4["T2w"] | |
| end | |
| B["Normalise"] | |
| C["Crop"] | |
| D["SegResNet 3D"] | |
| E["Sliding window"] | |
| subgraph OUT ["3 overlapping compartments"] | |
| direction TB | |
| F1["TC"] | |
| F2["WT"] | |
| F3["ET"] | |
| end | |
| G["Threshold"] | |
| H["BraTS labels"] | |
| A1 --> B | |
| A2 --> B | |
| A3 --> B | |
| A4 --> B | |
| B --> C --> D --> E | |
| E --> F1 | |
| E --> F2 | |
| E --> F3 | |
| F1 --> G | |
| F2 --> G | |
| F3 --> G | |
| G --> H | |
| classDef seq fill:#e0f2fe,stroke:#0284c7,stroke-width:1px,color:#0c4a6e | |
| classDef step fill:#ffffff,stroke:#64748b,stroke-width:1px,color:#1e293b | |
| classDef model fill:#ccfbf1,stroke:#0d9488,stroke-width:2px,color:#134e4a | |
| classDef out fill:#ede9fe,stroke:#7c3aed,stroke-width:1px,color:#4c1d95 | |
| class A1,A2,A3,A4 seq | |
| class B,C,E,G,H step | |
| class D model | |
| class F1,F2,F3 out | |
| ``` | |
| | Step | What happens | | |
| |---|---| | |
| | **T1c · T1n · T2-FLAIR · T2w** | Four sequences, in **this exact channel order** | | |
| | **Normalise** | Per case, per channel, zero mean unit variance over **non-zero voxels only** | | |
| | **Crop** | To the non-zero bounding box of the summed channels, 4-voxel margin | | |
| | **SegResNet 3D** | MONAI `SegResNet`, 18,798,627 parameters, 32 init filters, blocks down `[1,2,2,4]` | | |
| | **Sliding window** | 128³ patches, 0.5 overlap, gaussian blending | | |
| | **TC · WT · ET** | Independent sigmoid per channel, so the three compartments **overlap** rather than compete | | |
| | **Threshold** | 0.5, plus an enhancing-tumour floor of 200 voxels | | |
| | **BraTS labels** | Written WT→2, then TC→1, then ET→3, in that order | | |
| > [!IMPORTANT] | |
| > **Channel order is not recoverable from the weights.** `[t1c, t1n, t2f, t2w]` is part of the | |
| > model contract, recorded in `ml/artifacts/model_card.json`. Wrong order → wrong output → *no | |
| > error*. This is the kind of silent failure the model card exists to prevent. | |
| ### Held-out test performance | |
| **186 cases from 169 subjects never seen in training or tuning.** Splits are computed at | |
| **subject level**, because BraTS contains 1,251 cases from only 1,133 subjects, so a random split over | |
| cases would leak the same patient into train and test. | |
| | Compartment | Dice (mean) | Dice (median) | HD95 (median) | Sensitivity | Precision | Dice (mean) on a 0 to 1 scale | | |
| |---|---:|---:|---:|---:|---:|:---| | |
| | **Whole tumour** | **0.9216** | 0.9489 | 2.45 mm | 0.9248 | 0.9236 | ▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▱▱ | | |
| | **Tumour core** | **0.9078** | 0.9563 | 2.00 mm | 0.9166 | 0.9170 | ▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▱▱ | | |
| | **Enhancing tumour** | **0.8520** | 0.8984 | 1.41 mm | 0.8833 | 0.8487 | ▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▱▱▱ | | |
| | **Mean of the three** | **0.8938** | - | - | - | - | ▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▱▱ | | |
| > [!TIP] | |
| > **Test scored higher than validation** (0.8938 vs 0.8899 mean Dice). Since both epoch | |
| > selection and post-processing tuning used the validation split, the validation figures are | |
| > optimistic by construction, so the test figures are the honest ones, and they did not degrade. | |
| <details> | |
| <summary><b>Training configuration and honest limitations</b></summary> | |
| <br /> | |
| | Setting | Value | | |
| |---|---| | |
| | Architecture | `monai.networks.nets.SegResNet`, 3D, 32 init filters, blocks down `[1,2,2,4]` | | |
| | Loss | `DiceFocalLoss(sigmoid=True, squared_pred=True, batch=True)` | | |
| | Optimiser | AdamW, lr 2e-4, wd 1e-5, `CosineAnnealingLR` | | |
| | Precision | AMP float16 on Tesla T4 | | |
| | Patch sampling | 128³, 80% centred on whole tumour, 20% uniform | | |
| | Augmentation | Random axis flips, intensity scale ±10%, intensity shift ±10% | | |
| | Epochs | 37 of 60 completed (host RAM exhausted); **epoch 32 selected** on validation mean Dice | | |
| | Seed | `20260813` for both split and training | | |
| **Stated limitations** (from the model card): | |
| - Trained on **pre-operative** adult glioma only. Post-treatment appearances, such as resection | |
| cavities, radiation change, are not represented. | |
| - Requires all four sequences. Behaviour with a missing sequence is untested. | |
| - Assumes BraTS preprocessing: skull-stripped, co-registered, 1 mm isotropic. | |
| - Measures **agreement with one annotation protocol on one dataset**. That is not a measure of | |
| clinical accuracy. | |
| </details> | |
| --- | |
| ## The training notebook | |
| The model was trained in a single notebook on a Kaggle Tesla T4, and it is in the repository with | |
| its outputs intact: [`ml/notebooks/brats_segmentation_training_output.ipynb`](ml/notebooks/brats_segmentation_training_output.ipynb). | |
| GitHub renders it, so every number below can be traced to the cell that printed it. The clean | |
| unexecuted version is [`brats_segmentation_training.ipynb`](ml/notebooks/brats_segmentation_training.ipynb). | |
| Nineteen numbered stages, from configuration through to verifying the exported weights actually | |
| load. Twenty-two code cells, twenty-one of them executed. | |
| ### What the data looked like before any model existed | |
| <div align="center"> | |
| <img src="docs/notebook/01-tumour-volume-distribution.png" alt="Tumour volume distributions across the BraTS 2023 GLI training split" width="100%" /> | |
| </div> | |
| Across 1,251 cases the whole tumour has a median volume of 89.3 cm3 and a range of 2.8 to 361.8 | |
| cm3. The compartments are far smaller: enhancing tumour has a median of 17.3 cm3, and its minimum | |
| is **zero**, which is why the export applies a 200 voxel floor rather than reporting a | |
| one-voxel enhancing region as a finding. | |
| The single most consequential line the notebook printed: | |
| ``` | |
| tumour occupies 1.07% of all voxels | |
| -> uniform random patches would be almost pure background; sampling must be biased | |
| ``` | |
| That measurement is the reason patch sampling is 80% centred on the whole tumour and 20% uniform. | |
| It was not a hyperparameter guess. | |
| <div align="center"> | |
| <img src="docs/notebook/02-sequences-and-labels.png" alt="The four co-registered sequences with the reference labels overlaid" width="100%" /> | |
| </div> | |
| The four sequences for one case with the reference labels on T1C. Looking at the actual images is | |
| how the channel order was confirmed, and channel order is not recoverable from the weights: get it | |
| wrong and the model produces a plausible, wrong answer with no error anywhere. | |
| ### Training | |
| <div align="center"> | |
| <img src="docs/notebook/03-training-curve.png" alt="Training loss and validation Dice per epoch, with the selected epoch marked" width="100%" /> | |
| </div> | |
| Loss on the left, per-region validation Dice on the right, with the selected epoch marked. Thirty | |
| seven of a planned sixty epochs completed before host RAM was exhausted, and **epoch 32 was | |
| selected on mean validation Dice at 0.8867**. The curve is what justifies stopping there rather | |
| than at the last epoch: validation had flattened well before the run ended. | |
| ### The test set, run once | |
| <div align="center"> | |
| <img src="docs/notebook/04-test-dice-distribution.png" alt="Test Dice per region, and Dice against tumour size" width="100%" /> | |
| </div> | |
| 186 cases from 169 subjects, held out at subject level. Per-region Dice on the left, Dice against | |
| tumour size on the right, and that right-hand panel is the honest one: agreement collapses on the | |
| smallest tumours, where a few voxels of disagreement dominate the metric. | |
| | Split comparison | Validation | Test | Gap | | |
| |---|---:|---:|---:| | |
| | Mean Dice | 0.8899 | **0.8938** | **-0.0039** | | |
| The test score is marginally *higher* than validation. There is no overfitting to report, and the | |
| generalisation gap is smaller than the run-to-run noise. | |
| | Region | Cases below 0.5 Dice | Cases with empty ground truth | | |
| |---|---:|---:| | |
| | Whole tumour | 2 of 186 | 0 | | |
| | Tumour core | 5 of 186 | 1 | | |
| | Enhancing tumour | 8 of 186 | 5 | | |
| ### The cases it got wrong | |
| Most projects show the best cases. The notebook prints the worst four, because those are the ones | |
| that say something. | |
| <table> | |
| <tr> | |
| <td width="50%" valign="top"> | |
| <img src="docs/notebook/05-worst-test-case-1.png" alt="Worst test case by whole-tumour Dice" width="100%" /> | |
| `BraTS-GLI-00675-001`, whole tumour Dice **0.000**, yet tumour core and enhancing tumour both | |
| **1.000**. A whole tumour score of zero alongside perfect compartments is a labelling edge case, | |
| not a model that cannot see the tumour. | |
| </td> | |
| <td width="50%" valign="top"> | |
| <img src="docs/notebook/06-worst-test-case-2.png" alt="Second worst test case by whole-tumour Dice" width="100%" /> | |
| `BraTS-GLI-00493-000`, whole tumour Dice **0.184** on 34,451 labelled voxels, while core reaches | |
| 0.924 and enhancing 0.883. The oedema boundary is the disagreement, which is the least | |
| reproducible boundary between human annotators too. | |
| </td> | |
| </tr> | |
| </table> | |
| > [!NOTE] | |
| > The worst case is where the enhancing floor and the quality gate earn their place. A study that | |
| > produces a result like this is flagged rather than reported as a confident measurement, and | |
| > nothing from this model reaches the measurement pipeline at all. See | |
| > [Validation & results](#validation--results) for why. | |
| Regenerate the figures from the notebook at any time: | |
| ```bash | |
| make export-notebook-figures | |
| ``` | |
| ## Validation & results | |
| ### The decision not to ship the model into the measurement pipeline | |
| The trained model scores 0.894 mean Dice on held-out BraTS. It is **deliberately not** used for | |
| MEDTRACE's measurements, and the reason is measurement, not caution. | |
| Run against DeepBraTumIA on **12 randomly chosen real LUMIERE studies** | |
| (`ml/scripts/compare_on_lumiere.py`): | |
| | Region | Median Dice | Studies below 0.5 | Median Dice on a 0 to 1 scale | | |
| |---|---:|---:|:---| | |
| | Whole tumour | **0.923** | 0 of 12 | ▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▱▱ | | |
| | Tumour core | 0.816 | 1 of 12 | ▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▱▱▱▱ | | |
| | **Enhancing tumour** | **0.486** | **6 of 12** | ▰▰▰▰▰▰▰▰▰▰▱▱▱▱▱▱▱▱▱▱ | | |
| Splitting by how much enhancement is actually present shows this is not a uniform weakness: | |
| | DeepBraTumIA enhancing volume | n | Median ET Dice | Our volume vs theirs | | |
| |---|---:|---:|---:| | |
| | Bulky, ≥ 5 cm³ | 5 | **0.861** | 1.03× | | |
| | Small, < 5 cm³ | 7 | **0.193** | ~4× | | |
| **The cause was predicted before the comparison was run.** BraTS is pre-operative glioma, where | |
| enhancing tumour is a thick contrast-avid ring. LUMIERE is post-treatment: resection margins, | |
| radiation change and post-surgical enhancement all enhance, and none of it appears in BraTS. The | |
| model has never been shown a brain that has been operated on. | |
| > [!CAUTION] | |
| > Enhancing tumour is **the compartment MEDTRACE measures and reports change on**. Swapping the | |
| > pipeline over would inflate every enhancing volume, worst on exactly the small lesions where a | |
| > change of a few tenths of a cm³ decides whether progression is reported. So the pipeline keeps | |
| > DeepBraTumIA's masks, and this comparison is documented as *agreement between two automated | |
| > tools*, not as accuracy. | |
| --- | |
| ## Measurement, matching and change | |
| ### Measurement chosen by measurement | |
| Surface area is computed by marching cubes over a **signed distance field**, not over the binary | |
| mask directly. The estimator was selected by comparison against analytic shapes: it degrades far | |
| more gracefully with anisotropic voxels, and this data spans **0.36 mm to 6.0 mm** slice spacing. | |
| The reported surface area **is** the area of the mesh shipped to the 3D viewer, verified to | |
| within 0.009% across 1,153 meshes, so the number in the findings panel and the surface on screen | |
| cannot disagree. | |
| ### Cross-time lesion matching | |
| ```mermaid | |
| %%{init: {"theme":"base","themeVariables":{"fontSize":"15px","textColor":"#1e293b","nodeTextColor":"#1e293b","lineColor":"#475569","edgeLabelBackground":"#ffffff"},"flowchart":{"nodeSpacing":40,"rankSpacing":55,"padding":10}}}%% | |
| flowchart LR | |
| P["Prior lesions"] --> S{"Score each pair"} | |
| C["Current lesions"] --> S | |
| S --> O["Overlap Dice"] | |
| S --> D["Centroid distance"] | |
| S --> X["Proximity"] | |
| S --> V["Volume similarity"] | |
| O --> M["Optimal assignment"] | |
| D --> M | |
| X --> M | |
| V --> M | |
| M --> R1["Matched"] | |
| M --> R2["New lesion"] | |
| M --> R3["Disappeared"] | |
| M --> R4["Uncertain"] | |
| classDef inp fill:#e0f2fe,stroke:#0284c7,stroke-width:1px,color:#0c4a6e | |
| classDef comp fill:#ffffff,stroke:#64748b,stroke-width:1px,color:#1e293b | |
| classDef dec fill:#ccfbf1,stroke:#0d9488,stroke-width:2px,color:#134e4a | |
| classDef good fill:#dcfce7,stroke:#16a34a,stroke-width:1px,color:#14532d | |
| classDef warn fill:#ffedd5,stroke:#ea580c,stroke-width:2px,color:#7c2d12 | |
| class P,C inp | |
| class O,D,X,V comp | |
| class S,M dec | |
| class R1,R2,R3 good | |
| class R4 warn | |
| ``` | |
| | Outcome | Meaning | | |
| |---|---| | |
| | **Matched** | Keeps the same `Lesion` id, so the trajectory continues | | |
| | **New lesion** | An unmatched current lesion, so a new `Lesion` identity is created | | |
| | **Disappeared** | An unmatched prior lesion, recorded as an absence, not silently dropped | | |
| | **Uncertain** | `MATCH_UNCERTAIN`, `WEAK_SCORE` or `AMBIGUOUS_ALTERNATIVE`. The finding is de-emphasised and the 3D surface stays grey | | |
| A doubtful correspondence is **never presented as a confident one**. `MATCH_UNCERTAIN`, | |
| `WEAK_SCORE` and `AMBIGUOUS_ALTERNATIVE` de-emphasise the finding in the panel and force the 3D | |
| surface to render grey rather than in a change colour, because a colour that says "growing" is | |
| a claim, and it must not be made when the lesion it is compared against may be a different | |
| lesion. | |
| ### Change with propagated uncertainty | |
| <div align="center"> | |
| <img src="docs/screenshots/04-findings-panel.png" alt="Findings panel with measured change and confidence" width="80%" /> | |
| *Measured change per lesion, with confidence and the reason it is reduced.* | |
| </div> | |
| Confidence is reported **per pipeline stage**, and an answer's confidence is the **minimum** | |
| across the observations it rests on, never an average. A confident segmentation combined with an | |
| uncertain registration produces an uncertain change measurement, and averaging would let the | |
| reliable measurement hide the unreliable one. | |
| Eleven quality flags feed this, each translated into plain language: *"the two studies used | |
| different acquisition protocols"*, *"slice thickness was large enough to affect volume | |
| measurement"*, *"another lesion scored almost as well as this correspondence"*. | |
| --- | |
| ## 3D disease evolution | |
| <div align="center"> | |
| <img src="GIF/gif-01-timeline-3d.gif" alt="Dragging the time scrubber updates the 3D disease map" width="90%" /> | |
| *Dragging the time scrubber: the lesion surface, the measurements and the evidence move together.* | |
| <br /> | |
| <img src="GIF/gif-02-3d-rotate.gif" alt="Rotating the volume-rendered head" width="90%" /> | |
| *Rotation follows the pointer, and the head stays solid from every angle.* | |
| </div> | |
| ### The head is volume-rendered, and that was a hard-won decision | |
| The brain context was originally a **surface** extracted from the skull-strip mask. It was | |
| reported as having holes five separate times. Each round found something real: open edges from a | |
| crop that borrowed its margin from the source volume, front-face culling that erased deep | |
| concavities, a camera whose view-up was parallel to its view direction, lesions left unlit by a | |
| single headlight. Each round fixed it, measured the rendered image as clean, and the report still stood. | |
| > [!NOTE] | |
| > A surface leaves only two options, and **each has a failure mode invisible to a software | |
| > rasteriser**. Translucent, and the result depends on blending order and multisample resolve, | |
| > which vary by driver. Opaque with the near wall culled, and it is a hollow bowl that hides the | |
| > anatomy it exists to show. The checks ran under SwiftShader; the defect lived on the GPU. | |
| The head is now the patient's own **skull-stripped contrast-enhanced T1, ray-cast as a volume**. | |
| There is no surface to close, no winding, no culling, no blending order, so *a gap in the anatomy is | |
| not expressible*. And it is the real anatomy at full 1 mm resolution rather than a smoothed | |
| approximation of its outer boundary. | |
| | Measurement | Measured on the Intel Iris Plus iGPU | | |
| |---|---| | |
| | Median frame | **16.7 ms → 59.9 fps** *(16.7 ms is the vsync interval, so the renderer is not the limit)* | | |
| | 95th percentile frame | 16.8 ms | | |
| | First frame after load | 4.0 s | | |
| | Main thread after a 60-step drag | 2 ms | | |
| Benchmarked on real hardware rather than in software, because software rendering is exactly how | |
| five rounds of a rendering defect stayed invisible. Run it yourself: `make benchmark-3d`. | |
| <table> | |
| <tr> | |
| <td width="50%" valign="top"> | |
| <img src="docs/screenshots/05-3d-evolution.png" alt="3D disease evolution view" width="100%" /> | |
| **Coloured by measured change.** Red > +25%, blue < −25%, green stable, **grey for a baseline or | |
| a doubtful match**. The prior timepoint is drawn as a wireframe. | |
| </td> | |
| <td width="50%" valign="top"> | |
| <img src="docs/screenshots/06-3d-lesion-selected.png" alt="Single lesion selected in 3D" width="100%" /> | |
| Selecting a lesion highlights it simultaneously in the 2D panes, the findings list and the 3D | |
| view. Lesion meshes are **never smoothed**, because that mesh is the source of the reported surface area. | |
| </td> | |
| </tr> | |
| </table> | |
| <details> | |
| <summary><b>Why the lesions stay as surfaces while the head is a volume</b></summary> | |
| <br /> | |
| The lesions are the *measured* objects. Each needs its own colour for its own change, and a | |
| surface is the honest way to draw a boundary that came from a mask. They are extracted at full | |
| resolution (median 71 KB, max 715 KB per mesh) and shipped as binary PLY with per-vertex normals. | |
| `ml/scripts/check_mesh_integrity.py` verifies the shipped bytes rather than synthetic spheres: | |
| unit-length normals, outward orientation, triangle winding consistent with them, zero open edges, | |
| zero non-manifold edges, one connected component. | |
| </details> | |
| --- | |
| ## The evidence engine | |
| Five fixed clinical questions, answered from measurements already in the database, validated | |
| before delivery, and stored with the evidence that supports them. | |
| ```mermaid | |
| %%{init: {"theme":"base","themeVariables":{"fontSize":"15px","actorFontSize":"15px","noteFontSize":"14px","messageFontSize":"14px","textColor":"#1e293b","actorTextColor":"#1e293b","noteTextColor":"#1e293b","signalTextColor":"#1e293b","actorBkg":"#e0f2fe","actorBorder":"#0284c7","noteBkgColor":"#fef3c7","noteBorderColor":"#d97706","labelBoxBkgColor":"#e0f2fe","labelTextColor":"#1e293b"}}}%% | |
| sequenceDiagram | |
| autonumber | |
| actor U as Clinician | |
| participant F as findings | |
| participant A as answers | |
| participant L as LLM | |
| participant S as SafetyGuard | |
| participant D as Database | |
| U->>F: Ask one of five questions | |
| F->>D: Read recorded measurements | |
| D-->>F: Observations, confidence, flags | |
| F->>F: Assemble structured findings | |
| Note over F: No evidence means no observation | |
| F->>A: Compose deterministic answer | |
| A-->>S: Ground truth | |
| F->>L: Same findings, ask for prose | |
| L-->>S: Draft, or nothing at all | |
| S->>S: Forbidden claim? Invented number? | |
| S-->>U: Answer, evidence, confidence | |
| S->>D: Persist and audit | |
| ``` | |
| > [!IMPORTANT] | |
| > **The model never sees pixels, never computes a number, and never has the last word.** | |
| > `answers.py` composes the answer from findings alone with no model involved, and that sentence is | |
| > the ground truth. A language model may make it *more readable*; it may not make it *different*. | |
| <div align="center"> | |
| <img src="docs/screenshots/07-evidence-panel.png" alt="AI evidence panel with an answer and evidence chips" width="90%" /> | |
| <br /> | |
| <img src="GIF/gif-03-show-me-why.gif" alt="From an answer to its evidence to the study it came from" width="90%" /> | |
| *Every answer carries its evidence, and every piece of evidence navigates to the study it came from.* | |
| </div> | |
| An actual answer, generated from real measurements: | |
| > No measured enhancing volume changed by more than 25% between week-019-2 and week-033, 98 days | |
| > apart. 1 lesion changed by less: L06 measures 4.2 mm³, decreased by 5.7%. 5 lesions had no | |
| > prior to compare against… **Confidence 0.64; slice thickness was large enough to affect volume | |
| > measurement.** | |
| ### SafetyGuard | |
| Implements a fixed table of permitted and forbidden statements, and nothing beyond it. What a medical | |
| tool may and may not state is not an engineering decision. | |
| | Check | Outcome | | |
| |---|---| | |
| | Diagnosis, tumour type or grade | Draft discarded → measurement delivered | | |
| | Progression, response, improvement, recurrence | Draft discarded → measurement delivered | | |
| | Treatment recommendation | Draft discarded → measurement delivered | | |
| | Prognosis or survival | Draft discarded → measurement delivered | | |
| | Clinical urgency | Draft discarded → measurement delivered | | |
| | A number not present in the structured findings | Draft discarded → measurement delivered | | |
| | A lesion or study not in the evidence set | Draft discarded → measurement delivered | | |
| | **No linked evidence** | **Blocked, nothing delivered** | | |
| | Confidence below 0.6 | Delivered, marked low, reason in plain language | | |
| | Quality flags on the inputs | Delivered, flags surfaced alongside | | |
| The regular expressions are deliberately coarse and are **not** treated as a semantic filter. A | |
| pattern cannot understand a sentence, so the guarantee comes from the deterministic fallback, not | |
| from the cleverness of the patterns. A blocked draft is **kept**, because a block is a signal | |
| about the pipeline rather than just a filtered string. | |
| <details> | |
| <summary><b>A worked example</b></summary> | |
| <br /> | |
| ``` | |
| LLM draft: "The tumour is malignant and has progressed." | |
| SafetyGuard: MODIFIED, diagnostic or grading claim; progression or response judgement | |
| Delivered: "The segmented enhancing volume increased from 12.2 cm³ to 19.7 cm³ | |
| (+61.5%) between 2025-06-12 and 2025-09-04." | |
| ``` | |
| Two false positives were found by measurement and fixed: a timepoint called `week-012` parses as | |
| the number −12, and a lesion called `L01` as 1, so identifiers are stripped before the numeric | |
| scan. Rounding is not fabrication. 19.7 written as 20 is accepted, 42 is not. | |
| </details> | |
| ### Reproducibility and audit | |
| `findings_hash` digests the question and the entire findings payload. Asking the same question | |
| about unchanged findings returns the **stored** answer rather than generating a second one, so a | |
| past statement stays reconstructable. Every generation writes an `AuditEvent` carrying the model | |
| version, the input hash, the output hash and the safety outcome, with the patient referenced by | |
| UUID and no findings in the detail. | |
| --- | |
| ## Architecture | |
| ```mermaid | |
| %%{init: {"theme":"base","themeVariables":{"fontSize":"15px","clusterBkg":"#f8fafc","clusterBorder":"#94a3b8","textColor":"#1e293b","nodeTextColor":"#1e293b","lineColor":"#475569","edgeLabelBackground":"#ffffff"},"flowchart":{"nodeSpacing":40,"rankSpacing":62,"padding":12}}}%% | |
| flowchart TB | |
| subgraph BROWSER ["Browser"] | |
| W["Next.js · React"] | |
| CS["Cornerstone3D"] | |
| VTK["vtk.js"] | |
| end | |
| subgraph APILAYER ["FastAPI modular monolith"] | |
| R["Routers"] | |
| AN["Analysis"] | |
| EV["Evidence engine"] | |
| IG["Ingestion"] | |
| end | |
| subgraph MLLAYER ["ml, separate package"] | |
| MM["medtrace_ml"] | |
| ART["artifacts"] | |
| end | |
| subgraph SVC ["Docker Compose"] | |
| PG[("PostgreSQL")] | |
| MIO[("MinIO")] | |
| ORT[("Orthanc")] | |
| RD[("Redis")] | |
| end | |
| FS[("Filesystem")] | |
| W --- CS | |
| W --- VTK | |
| W -->|"REST · PLY · NIfTI"| R | |
| R --> AN | |
| R --> EV | |
| R --> IG | |
| AN --> MM | |
| EV --> MM | |
| IG --> MM | |
| MM --- ART | |
| AN --> PG | |
| IG --> PG | |
| EV --> PG | |
| IG --> ORT | |
| AN --> MIO | |
| R --> RD | |
| R -->|"path allowlist"| FS | |
| classDef ui fill:#e0f2fe,stroke:#0284c7,stroke-width:1px,color:#0c4a6e | |
| classDef api fill:#ccfbf1,stroke:#0d9488,stroke-width:1px,color:#134e4a | |
| classDef ml fill:#e0e7ff,stroke:#4f46e5,stroke-width:1px,color:#312e81 | |
| classDef svc fill:#f3e8ff,stroke:#9333ea,stroke-width:1px,color:#581c87 | |
| classDef fs fill:#fef3c7,stroke:#d97706,stroke-width:1px,color:#78350f | |
| class W,CS,VTK ui | |
| class R,AN,EV,IG api | |
| class MM,ART ml | |
| class PG,MIO,ORT,RD svc | |
| class FS fs | |
| ``` | |
| <table> | |
| <tr><td valign="top" width="25%"> | |
| **Browser** | |
| - `Next.js · React`: Zustand, TanStack Query | |
| - `Cornerstone3D`: 2D volumes + labelmaps | |
| - `vtk.js`: 3D volume ray-cast | |
| </td><td valign="top" width="25%"> | |
| **FastAPI** | |
| - `Routers`: patients, studies, timeline, files, evidence | |
| - `Analysis`: pipeline orchestration | |
| - `Evidence engine`: findings, safety, audit | |
| - `Ingestion`: BraTS, LUMIERE, quality control | |
| </td><td valign="top" width="25%"> | |
| **ml, a separate package** | |
| - `medtrace_ml`: measure, matching, change, mesh, volume, labels | |
| - `artifacts`: weights, TorchScript, model card | |
| </td><td valign="top" width="25%"> | |
| **Services** | |
| - `PostgreSQL`: metadata only | |
| - `MinIO`: object storage | |
| - `Orthanc`: DICOM | |
| - `Redis`: cache | |
| - `Filesystem`: `data/raw`, `data/derived` | |
| </td></tr> | |
| </table> | |
| **ML code stays out of application code.** The application depends on model *contracts*, never on | |
| training code. Two Python packages, `medtrace-api` and `medtrace-ml`, with the API importing the | |
| latter but never the reverse. | |
| <div align="center"> | |
| <img src="docs/screenshots/10-api-docs.png" alt="MEDTRACE OpenAPI documentation" width="90%" /> | |
| *Every endpoint is typed end to end: Pydantic v2 on the server, generated TypeScript contracts in | |
| the browser.* | |
| </div> | |
| <details> | |
| <summary><b>Repository layout</b></summary> | |
| <br /> | |
| ``` | |
| medtrace/ | |
| ├── apps/ | |
| │ ├── api/ FastAPI modular monolith | |
| │ │ ├── alembic/versions/ schema migrations | |
| │ │ ├── medtrace/ | |
| │ │ │ ├── analysis/ the measurement pipeline | |
| │ │ │ ├── domain/ models.py, enums.py | |
| │ │ │ ├── evidence/ findings · answers · llm · safety · service | |
| │ │ │ ├── ingestion/ BraTS and LUMIERE readers, quality control | |
| │ │ │ └── routers/ | |
| │ │ ├── scripts/ one-off data operations | |
| │ │ └── tests/ | |
| │ └── web/ Next.js clinical workstation | |
| │ ├── scripts/ browser verification and the 3D benchmark | |
| │ └── src/{app,components,lib,store} | |
| ├── ml/ | |
| │ ├── medtrace_ml/ measure · lesions · matching · change · trajectory | |
| │ │ registration · mesh · volume · labels | |
| │ ├── notebooks/ training notebook, its generator and its checks | |
| │ ├── scripts/ dataset preparation, model and mesh verification | |
| │ ├── artifacts/ trained weights, TorchScript, model card | |
| │ └── reports/ evaluation and agreement CSVs | |
| ├── packages/ reserved for shared contracts | |
| ├── infrastructure/ MLflow image, database init | |
| ├── data/ | |
| │ ├── raw/ the datasets | |
| │ ├── derived/ generated meshes and atlas volumes | |
| │ └── kaggle/ upload staging, manifest and splits kept | |
| └── docs/screenshots/ interface captures used in this README | |
| ``` | |
| </details> | |
| --- | |
| ## Technology stack | |
| <table> | |
| <tr><td valign="top" width="50%"> | |
| **Frontend** | |
| | Layer | Choice | | |
| |---|---| | |
| | Framework | Next.js 15.5 · React 19.1 | | |
| | Language | TypeScript 5.9 (strict) | | |
| | Styling | Tailwind CSS 4 | | |
| | Client state | Zustand 5 | | |
| | Server state | TanStack Query 5 | | |
| | 2D viewer | Cornerstone3D 5.7 | | |
| | 3D renderer | vtk.js 36.4 | | |
| **Backend** | |
| | Layer | Choice | | |
| |---|---| | |
| | API | FastAPI · Pydantic v2 | | |
| | ORM | SQLAlchemy 2 · Alembic | | |
| | Database | PostgreSQL 17 | | |
| | Object storage | MinIO | | |
| | DICOM | Orthanc 24.10 | | |
| | Cache | Redis 7 | | |
| </td><td valign="top" width="50%"> | |
| **Machine learning** | |
| | Layer | Choice | | |
| |---|---| | |
| | Framework | PyTorch 2.10 (cu128) | | |
| | Medical DL | MONAI 1.6 | | |
| | Registration | SimpleITK | | |
| | Imaging I/O | nibabel · NumPy · SciPy | | |
| | Meshing | scikit-image marching cubes | | |
| | Training | Kaggle Tesla T4 | | |
| | LLM | MedGemma (text-only) behind a provider adapter | | |
| **Engineering** | |
| | Layer | Choice | | |
| |---|---| | |
| | Infrastructure | Docker Compose, 5 services | | |
| | Testing | Pytest · Vitest · Playwright | | |
| | Linting | Ruff · ESLint | | |
| | CI | GitHub Actions | | |
| </td></tr> | |
| </table> | |
| > [!NOTE] | |
| > **On the LLM:** the default configuration is `MEDTRACE_LLM_PROVIDER=none`, and that is a | |
| > *working* configuration rather than a disabled one. MedGemma-27B does not fit on the | |
| > development GPU, and the five clinical questions must be answerable regardless, so the | |
| > deterministic composer answers them. Point `MEDTRACE_LLM_BASE_URL` at a vLLM, Ollama or | |
| > llama.cpp endpoint and it will rephrase; if that endpoint is unreachable you lose wording, not | |
| > correctness. | |
| --- | |
| ## Datasets | |
| Public, de-identified research data only. No real hospital patient data enters this repository | |
| under any circumstances. | |
| | Dataset | Role | Scale ingested | | |
| |---|---|---| | |
| | **BraTS 2023 GLI** | Segmentation training and held-out evaluation | 1,251 cases from 1,133 subjects · 118 with two labelled timepoints | | |
| | **LUMIERE** | The longitudinal dataset: timeline, matching, trajectory | 91 patients · 638 timepoints · 2,487 series | | |
| LUMIERE's acquisition metadata records **three field strengths, 21 scanner models and slice | |
| thickness from 0.8 mm to 6.0 mm**, so quality control and confidence reporting are built against | |
| measured heterogeneity rather than imagined inputs. | |
| <details> | |
| <summary><b>Data we hold and deliberately will not use</b></summary> | |
| <br /> | |
| LUMIERE ships **survival time in weeks, IDH status and MGMT methylation** for all 91 patients. | |
| That makes outcome and molecular prediction technically possible with the data already on disk. | |
| We are not building it. Prognosis is forbidden by the safety policy this project holds itself to, and a | |
| measurement system must be trustworthy before prediction built on top of it means anything. This | |
| is recorded as a deliberate decision rather than an oversight, so the temptation is resolved once | |
| instead of repeatedly. | |
| LUMIERE's **expert RANO ratings for 616 timepoints** are likewise used only as a reference | |
| standard for evaluation. Never a training target, never surfaced as a MEDTRACE output. | |
| </details> | |
| --- | |
| ## Verification | |
| Nothing here is asserted without measurement. **696 automated checks.** | |
| | Suite | Checks | What it proves | | |
| |---|---:|---| | |
| | `pytest` API | **155** | Contracts, ingestion, quality control, pipeline output, SafetyGuard | | |
| | `pytest` ML | **168** | Measurement, matching, change, meshing, volume windowing, label mappings | | |
| | `vitest` web | **17** | Camera conventions, hole detection geometry | | |
| | `verify-study-linkage` | **189** | Every pane displays the study it claims, at every timepoint | | |
| | `verify-evidence` | **42** | Answers carry evidence, cite only measured numbers, and navigate | | |
| | `verify-evolution` | **37** | The 3D view draws the right surfaces, in one coordinate frame | | |
| | `verify-workstation` | **19** | The clinical shell, the scrubber, and the intended-use notice | | |
| | `verify-brain-shell` | **15** | No holes in the rendered head, at five viewing angles | | |
| | `verify-overlay` | **15** | The 2D tumour overlay draws the right mask | | |
| | `verify-slice-sync` | **14** | The two panes really scroll together, and stop when unlinked | | |
| | `verify-findings` | **8** | The UI shows real measured findings, not seeded numbers | | |
| <details> | |
| <summary><b>Why the browser checks exist at all</b></summary> | |
| <br /> | |
| Because instrumentation has been wrong more than once, and each time it was wrong in a way that | |
| *passed*: | |
| - A hole detector that counted the gaps between the legend's **text glyphs**, because an element | |
| screenshot captures whatever is drawn over the element. | |
| - The same check running on the patient the app opens on, which has a single 96-pixel lesion, and | |
| and passing with zero holes while the defect was obvious on a patient with twelve. | |
| - A watertightness test that only ever ran at stride 1, while the shipped meshes used stride 2. | |
| - A "fit view" check that measured the brain instead of the lesion. | |
| Expected values are taken **from the API**, not from the page. The hole detector now lives in its | |
| own module with its own unit tests, because three wrong versions of a check is enough. | |
| </details> | |
| --- | |
| ## Getting started | |
| ### Prerequisites | |
| | Requirement | Needed | | |
| |---|---| | |
| | Docker Desktop | Running, with the Linux engine | | |
| | Python | ≥ 3.11 *(resolved against 3.14)* | | |
| | Node.js | ≥ 20 *(tested on 24)* | | |
| | Disk | ~50 GB for both datasets | | |
| ### First-time setup | |
| ```bash | |
| cp .env.example .env | |
| make up # PostgreSQL, Redis, MinIO, Orthanc, MLflow | |
| make install # API venv + ML venv + web dependencies | |
| make migrate # create the schema | |
| make seed # synthetic demonstration patient (labelled synthetic in the UI) | |
| ``` | |
| Then ingest and analyse. This is the slow part: | |
| ```bash | |
| make ingest-lumiere # 91 patients, 638 timepoints | |
| make ingest-brats # 118 subjects with two labelled timepoints | |
| make analyse # the full measurement pipeline (~20 min) | |
| make link-masks # reorient each series' mask for the 2D overlay | |
| ``` | |
| ### Running it | |
| Three terminals, in this order: | |
| ```bash | |
| # 1. services | |
| docker compose up -d | |
| # 2. API | |
| cd apps/api | |
| .venv/Scripts/python.exe -m uvicorn medtrace.main:app --host 127.0.0.1 --port 8000 | |
| # 3. web (production build, see the note below) | |
| cd apps/web | |
| npm run start | |
| ``` | |
| | Service | Address | | |
| |---|---| | |
| | **Workstation** | **http://localhost:3000** | | |
| | API documentation | http://localhost:8000/docs | | |
| | Health check | http://localhost:8000/health | | |
| | Orthanc | http://localhost:8042 | | |
| | MinIO console | http://localhost:9101 | | |
| **Stop cleanly:** `Ctrl+C` in the web and API terminals, then `docker compose stop`. | |
| > [!TIP] | |
| > Use `npm run start`, not `npm run dev`. Volume rendering is the heaviest part of MEDTRACE, and | |
| > development mode's HMR, source maps and double-invoked effects consume memory the renderer | |
| > needs. Never run `npm run build` while a dev server is running, because they share `.next`. | |
| > | |
| > The viewer caps its cache at **500 MB and six resident volumes**, shows its footprint in the | |
| > toolbar, and offers a `reset viewer` control if the graphics context is ever lost. | |
| > [!WARNING] | |
| > **No authentication.** The API is unauthenticated and binds to `127.0.0.1` only, with | |
| > development credentials from `.env`. It is for local use. Do not expose it. | |
| <details> | |
| <summary><b>Optional commands</b></summary> | |
| <br /> | |
| ```bash | |
| # Tests | |
| cd apps/api && .venv/Scripts/python.exe -m pytest -q # 155 | |
| cd ml && ../apps/api/.venv/Scripts/python.exe -m pytest -q # 168 | |
| cd apps/web && npm run test # 22 unit | |
| # Browser verification (needs both servers running) | |
| make verify # every browser suite | |
| make verify-evidence # the AI evidence engine | |
| make verify-brain-shell # no holes in the rendered head | |
| make benchmark-3d # 3D frame rate on the real GPU | |
| # Quality | |
| make lint # Ruff + ESLint | |
| make typecheck # tsc --noEmit | |
| # Data and model inspection | |
| make inspect p=Patient-006 # per-lesion trajectories | |
| make quality-report # what quality control found | |
| make check-mesh-integrity # normals, winding, watertightness of shipped meshes | |
| make check-volume-pockets # no enclosed pockets of air inside the 3D display volumes | |
| make verify-meshes # every stored mesh against its observation | |
| make check-readme-diagrams # render this README's diagrams, check no label overflows | |
| ``` | |
| Ports are shifted off the defaults, **PostgreSQL on 5433 and MinIO console on 9101**, so the stack | |
| does not collide with locally installed services. | |
| </details> | |
| --- | |
| ## Build status | |
| | Stage | Scope | State | | |
| |---|---|:---:| | |
| | **0** | Clinical problem, scope, safety boundaries, evaluation plan, architecture, dataset cards | Done | | |
| | **1** | Docker Compose infrastructure, FastAPI, 14-table observation model, clinical workstation | Done | | |
| | **2** | Dataset ingestion, quality control, Cornerstone3D medical viewer, slice synchronisation | Done | | |
| | **3** | Measurement, lesion extraction, registration, cross-time matching, change detection, trajectory | Done | | |
| | **4** | 3D disease evolution, volume rendering, surface extraction tied to the time scrubber | Done | | |
| | **5** | Evidence engine, SafetyGuard, five clinical questions, audit trail | Done | | |
| **Deliberately not built:** MLOps automation (DVC, MLflow registry integration, Great | |
| Expectations), authentication and role-based access, deformable registration, progression | |
| classification, prognosis. The first is a project decision; the rest are scoped to later versions. | |
| --- | |
| ## Honesty as a design principle | |
| MEDTRACE is presented as a prototype, because that is what it is. Understanding why clinical | |
| validation matters, and saying so plainly, is a strength when talking to clinicians, not a | |
| weakness to hide behind confident language. | |
| Every number in this README is measured and reproducible from the repository. Where a result is | |
| unflattering, it is stated: the model's enhancing-tumour agreement on post-treatment data is | |
| poor, and that is why it is not in the measurement pipeline. | |
| --- | |
| <div align="center"> | |
| **Datasets:** BraTS 2023 GLI and LUMIERE, used under their respective research licences. | |
| LUMIERE is non-commercial. | |
| <br /> | |
| *MEDTRACE reports measured change. It does not diagnose.* | |
| </div> | |
| --- | |
| <div align="center"> | |
| ### Omar Rehan | |
| [](https://omar-rehan.vercel.app/) | |
| [](https://github.com/AIOmarRehan) | |
| [](https://linkedin.com/in/omar-rehan-47b98636a) | |
| [](https://huggingface.co/AIOmarRehan) | |
| [](https://kaggle.com/aiomarrehan) | |
| [](https://medium.com/@ai.omar.rehan) | |
| [](https://public.tableau.com/app/profile/omar.rehan) | |
| </div> | |