Evidence-first Two-stage retrieval Auditable traces Distilled 7B

Multi-Agent Biomedical Reasoning Grounded in EMR & Guidelines

MedSwin is an evidence-constrained clinical QA stack: specialised agents coordinate retrieval, EMR summarisation, guideline synthesis, and safety critique—while a calibrated reranker enforces evidence sufficiency under a token budget.

Open Demo Explore Architecture

MedSwin Local-deployable by design Provenance-aware output

Replayable traces Evidence sufficiency checks Token budget control Hybrid ANN + BM25 Calibrated reranker Multi-agent coordination (MAC)

Trust Stack

Design primitives for clinical deployment readiness

System

Auditability

Typed artifacts + explicit provenance (document IDs, sections, timestamps, chunk offsets) enable replay and review.

Evidence sufficiency

Retrieval is accepted only when EMR/CPG coverage targets are met under a strict token budget.

Calibrated ranking

Long-context biomedical reranker outputs calibrated probabilities for deterministic inclusion policies.

Safety critique

Critic checks missing evidence, contraindications, and unsafe advice—then routes “retrieve-more” when needed.

Org Model Asset Rag Data Finetune Data

Research prototype — not a substitute for professional medical advice.

Overview

MedSwin frames clinical QA as an evidence-constrained decision pipeline. Every answer is gated by evidence sufficiency, bounded by a strict context budget, and accompanied by a replayable trace suitable for audit and safety review.

Answer

Clinically phrased, uncertainty-aware output generated only when evidence gates are satisfied.

Evidence bundle

Compact EMR + guideline passages selected under token and diversity constraints.

Trace

Structured artifact log: retrieval, ranking, policies, safety checks.

1) Auditable Multi-Agent Orchestration

Every agent produces artifacts with provenance metadata and logs tool calls + selected evidence. This creates a structured audit trail suitable for review and replay.

Typed artifacts: ids, sections, timestamps, offsets
Deterministic “retrieve-more” instead of guessing
Critique + safety checks before finalisation

2) Two-Stage Retrieval with Calibrated Reranking

Stage-1 retrieves candidates via dense retrieval + BM25 union. Stage-2 reranks with a long-context biomedical reranker (LoRA-adapted) and outputs calibrated probabilities used by policy constraints.

Hybrid candidate pool: ANN + lexical coverage
Evidence sufficiency thresholds for EMR/CPG
Budgeted, diverse selection (MMR-style)

3) Distilled 7B Medical LLM Pipeline

A compact student is built with SFT on augmented biomedical QA, then refined using hard+soft KD from a larger instructor. This targets deployability while preserving calibrated reasoning behavior.

Large-scale augmentation with semantic checks
Hard labels expand coverage; soft labels preserve uncertainty
PEFT (QLoRA/LoRA) enables modest GPU training

From question → audited answer

MedSwin outputs an answer only when the evidence bundle is sufficient. Otherwise it asks clarifying questions or expands retrieval.

Clarify Retrieve-more Safe final

1 Normalise

Canonicalise terms, expand abbreviations, form retrieval probes.
2 Retrieve + Rank

Hybrid candidates, rerank with calibrated probabilities, enforce sufficiency.
3 Synthesize + Critique

Summarise EMR, synthesise guideline actions, run safety critique, then answer.

Architecture Explorer

Explore MedSwin layers: multi-agent workflow, two-stage retrieval, evidence sufficiency checks, and MAC coordination.

Orchestrated Provenance Budgeted

High-Level System Diagram

Mermaid · interactive tab

flowchart LR
  U["Clinician UI / EMR"] -->|q + patient context| ORCH["Orchestrator (MAC)\nplanning · policy checks · logging"]
  ORCH --> QN["Query Normaliser"]
  ORCH --> RET["Evidence Retriever"]
  ORCH --> EMRS["EMR Summariser"]
  ORCH --> GS["Guideline Synthesiser"]
  ORCH --> SC["Safety Critic"]

  subgraph IR["Two-Stage Retrieval (Budgeted)"]
    DENSE["Stage 1: Dense ANN (MedEmbed)"] --> CAND["Candidates (dense OR BM25)"]
    BM25["Stage 1: Lexical (BM25)"] --> CAND
    CAND --> RER["Stage 2: Long-context Reranker\n(LoRA-adapted, calibrated)"]
    RER --> SEL["Policy-aware selection\nMMR + sufficiency constraints"]
  end

  RET --> IR
  SEL --> EVID["Evidence bundle M\nEMR + CPG + metadata"]

  EMRS --> STATE["Clinical state summary"]
  GS --> ACTIONS["Guideline actions + contraindications"]
  SC --> FLAGS["Safety flags / missing evidence"]

  EVID --> FUSE["Evidence-constrained synthesis"]
  STATE --> FUSE
  ACTIONS --> FUSE
  FLAGS --> FUSE
  FUSE --> OUT["Final answer + citations + cautions\n+ structured trace"]

Agents Exchange Typed Artifacts

Selectable “focus” cards

Artifact: EvidenceBundle

provenance

{
  "type": "EvidenceBundle",
  "query": "clinical question + patient context",
  "budget_tokens": 8192,
  "selected": [
    {"doc_id": "CPG:xyz", "section": "Recommendations", "timestamp": "YYYY-MM-DD", "chunk": [120, 420], "p_cal": 0.86},
    {"doc_id": "EMR:abc", "section": "Meds/Labs", "timestamp": "YYYY-MM-DD", "chunk": [0, 260], "p_cal": 0.81}
  ],
  "coverage": {"CPG": 1.0, "EMR": 1.0},
  "trace_id": "trace_...",
  "notes": ["deduped", "PII-scrubbed", "MMR-diversified"]
}

MAC: Multi-Agent Coordination

Parallel retrieval hypotheses

flowchart TB
  Q["Query q + patient context"] --> O["MAC Orchestrator"]

  subgraph A["Specialised Probes"]
    E["Exploration Agent\n(recall-normalised)"]
    P["Policy Agent\n(guideline coverage)"]
    S["Safety Agent\n(contraindications)"]
  end

  O --> E
  O --> P
  O --> S

  E --> CE["Candidate C_e + scores"]
  P --> CP["Candidate C_p + scores"]
  S --> CS["Candidate C_s + scores"]

  CE --> U["Weighted union + re-rank\nω_i · S_i(q,d)"]
  CP --> U
  CS --> U

  U --> M["Meta-bundle M_MAC\n(token budget B + sufficiency checks)"]
  M --> OUT["Evidence for synthesis + critique"]

Why MAC helps

Instead of a single retrieval hypothesis, multiple agents explore different constraints and then fuse evidence, preserving disagreements as inspectable signals.

Agent weights

Agents accumulate track-record weights (e.g., sufficiency compliance) to influence future selection in a transparent way.

Single-Turn Sequence

Retrieval → policy checks → synthesis

sequenceDiagram
  autonumber
  participant C as Clinician
  participant O as Orchestrator (MAC)
  participant QN as Query Normaliser
  participant IR as Two-stage Retrieval
  participant EMR as EMR Summariser
  participant GL as Guideline Synthesiser
  participant SC as Safety Critic
  participant L as LLM (Distilled 7B)

  C->>O: query + patient context
  O->>QN: normalise + expand probes
  QN-->>O: canonical query
  O->>IR: retrieve candidates + rerank
  IR-->>O: evidence bundle M + p_cal + provenance
  O->>EMR: build clinical state summary
  O->>GL: extract actions + contraindications
  O->>SC: check evidence sufficiency + safety

  alt Insufficient evidence
    SC-->>O: retrieve-more / clarify request
    O-->>C: request clarification / expand search
  else Sufficient evidence
    SC-->>O: ok + cautions
    O->>L: synthesize answer with citations
    L-->>C: answer + citations + trace
  end

Evidence Budget Simulator

Illustrative UI to explain budgeted selection + sufficiency thresholds.

interactive

Token budget (B)

2048 8192 16384

passages

1.0

CPG κ

1.0

EMR κ

Policy status

Accepted

Evidence sufficiency met → synthesis allowed.

Note: values are explanatory placeholders; the paper defines κ and sufficiency acceptance logic conceptually.

Safety Notice

MedSwin is for research and education. It must not be used as a stand-alone medical decision maker. Always involve qualified clinicians and local governance when trialing clinical AI.

Two-Stage Retrieval & Calibrated Reranking

Evidence selection is separated into recall-oriented candidate generation and precision-oriented reranking. This avoids early truncation while enabling deterministic, policy-aware inclusion decisions.

Stage 1 — Candidate generation

Dense ANN retrieval is unioned with BM25 to preserve rare clinical terms, abbreviations, and lab-specific phrasing.

Stage 2 — Long-context reranking

A biomedical LLM reranker scores each passage and outputs calibrated probabilities usable as policy thresholds.

Policy-aware selection

Final selection enforces EMR + guideline sufficiency, diversity (MMR-style), and a strict token budget.

Data, Training & Distillation

MedSwin’s deployable 7B model is trained for reliability rather than raw scale, combining large-scale augmentation, supervised fine-tuning, and knowledge distillation.

A · Data augmentation

Paraphrasing, formatting variants, deduplication, and medical consistency checks expand coverage without semantic drift.

B · Supervised fine-tuning

Aligns the student to clinical instruction style, tone control, and structured answers.

C · Knowledge distillation

Hard labels expand task coverage; soft labels preserve calibration and uncertainty from a larger instructor.

Evaluation & Safety

MedSwin evaluates clinical QA systems beyond answer accuracy, focusing on evidence quality, guideline compliance, and runtime safety behaviour.

Retrieval quality

Evidence relevance and coverage under a fixed token budget.

Guideline coverage

Presence of actionable recommendations and contraindications.

Faithfulness

Final answers remain grounded in cited evidence only.

Team

A multidisciplinary research team building an auditable medical AI system.

Swinburne Multi-role

🎖️

Liam

Leader

🧪

Henry

LLM

🔗

Hai

System