LuminaECG-2B

LuminaECG-2B is a compact multimodal ECG interpretation model designed around a simple principle:

Better diagnostic priors from cardiologists may matter before model scale.

Rather than relying on a larger or ECG-specific architecture, LuminaECG injects the structure of clinical ECG interpretation directly into the training data. The model is trained to localize waveform components, recover clinically relevant measurements, and generate structured diagnostic reports grounded in observable ECG evidence.

LuminaECG-2B is built on a general-purpose 2B vision-language backbone and fine-tuned using standard low-rank supervised adaptation, without architectural modification, reinforcement learning, or preference optimization.

Model overview

Cardiologists do not interpret an ECG by directly mapping the full tracing to a diagnostic label. They first identify waveform components, measure rhythm and intervals against the calibrated ECG grid, and then connect these observations to diagnostic criteria.

LuminaECG follows this structure through a clinician-informed data construction pipeline:

1. Twelve-lead ECG signals are rendered on standardized ECG grid paper.
2. P-wave, QRS-complex, and T-wave regions are explicitly delineated.
3. Signal-derived measurements and available annotations are organized into structured clinical reports.
4. The resulting image-measurement-report pairs are used to fine-tune a compact multimodal language model.

The goal is not only to generate fluent ECG reports, but to preserve the connection between waveform evidence, quantitative measurements, and diagnostic conclusions.

Intended use

LuminaECG-2B is intended for research on:

• multimodal ECG interpretation;
• structured ECG report generation;
• waveform measurement recovery;
• ECG diagnostic reasoning;
• cross-cohort generalization;
• clinically grounded multimodal learning.

The model is not intended for autonomous clinical diagnosis or direct patient care.

Key results

LuminaECG-2B was evaluated on held-out and external ECG cohorts.

Measurement-grounded reporting

On the held-out MIMIC-IV-ECG test set, LuminaECG-2B achieved:

• Heart-rate MAE: 0.43 bpm
• PR-interval MAE: 8.97 ms
• QRS-duration MAE: 4.48 ms
• BLEU-1: 0.853
• BLEU-4: 0.741
• ROUGE-L: 0.808
• CIDEr: 0.896

Critical-finding recovery

On PTB-XL, across 2,569 urgent findings spanning eleven clinically important diagnoses:

• Critical-finding recall: 50.8%
• No evaluated zero-shot frontier model exceeded 10.3%

LuminaECG-2B also showed a more balanced hallucination-omission profile than the evaluated general-purpose multimodal models.

Human-reader comparison

On CODE-test, LuminaECG-2B achieved:

• Macro-F1: 0.809
• Micro-F1: 0.818

These results approach the medical-student tier on the same benchmark.

Cross-cohort generalization

Without target-cohort training, LuminaECG-2B retained:

• CODE-test macro-F1: 0.809
• Chapman-Shaoxing macro-F1: 0.644
• PTB-XL macro-F1: 0.678

Emergent prognostic information

In an independent external Sami-Trop cohort, report-derived features improved an age-and-sex Cox model from:

• C-index 0.678 to 0.764
• Increment: ΔC = +0.086
• P = 0.005

The model was not trained on mortality outcomes.

Usage

from transformers import AutoProcessor, AutoModelForImageTextToText
from PIL import Image
import torch

model_id = "YOUR_ORG/LuminaECG-2B"

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

image = Image.open("example_ecg.png").convert("RGB")

prompt = """
Interpret this 12-lead electrocardiogram.

Please provide a structured ECG report including:
1. Rhythm
2. Heart rate
3. PR interval
4. QRS duration
5. QT/QTc interval
6. Electrical axis
7. Lead-specific morphology
8. Diagnostic impression
"""

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": prompt},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
)

inputs = {
    key: value.to(model.device)
    if hasattr(value, "to")
    else value
    for key, value in inputs.items()
}

with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=False,
    )

generated_ids = output_ids[:, inputs["input_ids"].shape[1]:]
report = processor.batch_decode(
    generated_ids,
    skip_special_tokens=True,
)[0]

print(report)