| ---
|
| library_name: cbct-clinical-reasoner
|
| tags:
|
| - medical-imaging
|
| - cbct
|
| - report-generation
|
| - toothfairy4
|
| - odin2026
|
| ---
|
|
|
| # cbct-clinical-reasoner
|
|
|
| Finding predictor and calibrated report decoder for **ODIN 2026 Task 1
|
| (ToothFairy4)**: generating a maxillofacial surgical-planning report from a
|
| single 3D CBCT volume.
|
|
|
| For each of ~989 clinician-written statements the
|
| model predicts the probability that it applies to this scan; per-statement
|
| thresholds fitted on out-of-fold predictions then decide which to emit. Selecting
|
| from observed clinician phrasing means an entailment failure can only come from
|
| choosing the wrong finding, never from invented language.
|
|
|
| ## Contents
|
|
|
| | Path | Description |
|
| |---|---|
|
| | `bundle/prototypes.json` | Sentence-prototype label space built from the training corpus |
|
| | `bundle/decoder.json` | Per-prototype thresholds calibrated on out-of-fold predictions |
|
| | `bundle/checkpoints/` | Per-fold encoder checkpoints, ensembled at inference |
|
| | `bundle/config.json` | Preprocessing and training configuration |
|
| | `bundle/fallback_report.txt` | Prior-only report used if inference fails |
|
| | `plots/` | Diagnostic figures for every pipeline stage |
|
| | `results.json`, `RESULTS.md` | Metrics with provenance |
|
|
|
| ## Development metrics
|
|
|
| Out-of-fold over 622 public training cases.
|
|
|
| | metric | value |
|
| |---|---:|
|
| | **final** (0.8 clinical + 0.2 captioning) | 0.3544 |
|
| | clinical F1 (offline surrogate) | 0.3861 |
|
| | logical precision | 0.4910 |
|
| | logical recall | 0.3181 |
|
| | BLEU-4 (grader-exact) | 0.1493 |
|
| | METEOR (grader-exact) | 0.3064 |
|
|
|
| ## Ablation
|
|
|
| | variant | final | clinical | BLEU-4 | METEOR |
|
| |---|---:|---:|---:|---:|
|
| | corpus prior (no imaging) | 0.3575 | 0.3937 | 0.1170 | 0.3082 |
|
| | linear, 122 features | 0.3544 | 0.3861 | 0.1493 | 0.3064 |
|
| | fine-tuned encoder (29M params) | 0.3403 | 0.3752 | 0.1331 | 0.2684 |
|
|
|
| Mean out-of-fold mAP across folds: **0.0738**
|
|
|
| ## How to read these numbers
|
|
|
| * **BLEU-4 and METEOR are exact.** They reimplement the grader's own local
|
| implementations and match NLTK to machine precision, so they compare directly
|
| with the public leaderboard.
|
| * **The clinical score is a surrogate.** RadFact carries 80% of the challenge
|
| ranking, but computing it needs an LLM judge. The figure above comes from an
|
| offline lexical entailment model built to *rank* decoder variants cheaply. It
|
| is not the challenge metric — obtain that with
|
| `cbct-reasoner evaluate --radfact-lite`.
|
| * **These are in-domain.** Stratified folds over the public release, not the
|
| hidden 50-case external-centre test set. Leave-one-centre-out
|
| (`--strategy center`) is the honest external estimate and reads lower.
|
|
|
| ## Intended use and limitations
|
|
|
| Research artifact for a benchmark. **Not a medical device.** Generated text is a
|
| draft for review by a qualified clinician and must not be used for patient care.
|
| It has not been validated for any clinical purpose.
|
|
|
| ## Data provenance
|
|
|
| Derived from the access-controlled ToothFairy4 release. `prototypes.json`
|
| contains sentences taken verbatim from clinical reports, so this repository is
|
| **private by default** and must not be made public without checking the
|
| ToothFairy4 data-use agreement.
|
|
|
