MedMO-8B-FP8

MedMO-8B-FP8 is an FP8-compressed variant built on top of MBZUAI/MedMO-8B. This edition applies BF16 · FP8 (F8_E4M3) precision formats to significantly reduce memory footprint and improve inference throughput while preserving the advanced medical multimodal reasoning and grounding capabilities of the original 8B architecture. The base MedMO-8B model from MBZUAI is an 8B-parameter open-source multimodal large language model built on the Qwen3-VL architecture. It is specialized for comprehensive medical image understanding and grounding across radiology including X-ray, CT, MRI, and ultrasound, as well as pathology, ophthalmology, dermatology, and nuclear medicine.

Trained on 26M+ diverse samples from 45 datasets through a three-stage pipeline, general medical supervised fine-tuning with 18.5M image-text pairs, high-resolution grounding with 3M samples at 1280×1280, and instruction tuning with 4.3M pairs, the model demonstrates strong performance in VQA, medical QA, report generation, disease localization, clinical reasoning, and multi-task medical understanding. This FP8 edition maintains high diagnostic reasoning fidelity while enabling more efficient deployment on compatible GPU hardware.

FP8 8-bit floating point weight and activation quantization using hardware acceleration on GPUs – FP8 W8A8. Quantization W8A8 FP8-dynamic recipe – examples.

Key Highlights

• BF16 · FP8 (F8_E4M3) Compression: Transformer Engine based FP8 quantization reduces VRAM usage and improves inference efficiency while maintaining medical reasoning accuracy.
• Medical Multimodal Specialization: Optimized for radiology, pathology, ophthalmology, dermatology, and nuclear medicine tasks.
• High-Resolution Grounding: Supports precise spatial localization at up to 1280×1280 resolution.
• Disease & Anatomical Localization: Bounding box grounding with strong IoU improvements over baseline and prior SOTA systems.
• Clinical Report Generation: Achieves strong CIDEr scores including 270.4 on Med-Trinity, significantly outperforming prior systems.
• Robust Cross-Modality Generalization: Strong performance across diverse imaging modalities and medical benchmarks.
• Transformers & vLLM Compatible: Deployable via standard Hugging Face Transformers and vLLM inference stacks.

Performance Highlights

• VQA: +13.7% over baseline
• Text-based Medical QA: +6.9% improvement
• Report Generation: CIDEr 270.4 on Med-Trinity, 4× prior SOTA
• Localization: IoU +40.4% over baseline, +37% over Fleming-VL
• Bacteria Segmentation: IoU 54.6%
• MedSG Multi-task Benchmark: 75.8% multi-view accuracy
• Grounding Benchmarks: SOTA on NIH Chest X-ray and DeepLesion datasets

Quick Start with Transformers

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import torch

# Load the FP8 MedMO model
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "MBZUAI/MedMO-8B-FP8",
    torch_dtype="auto",
    device_map="auto"
)

processor = AutoProcessor.from_pretrained(
    "MBZUAI/MedMO-8B-FP8"
)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "medical_scan.png",
            },
            {
                "type": "text",
                "text": "Identify abnormalities and provide a diagnostic summary."
            },
        ],
    }
]

text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

image_inputs, video_inputs = process_vision_info(messages)

inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
).to("cuda")

generated_ids = model.generate(**inputs, max_new_tokens=512)

generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]

output_text = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)

print(output_text)

Intended Use

• Clinical Decision Support Research
• Medical Image Grounding & Localization
• Radiology & Pathology Report Generation
• Medical VQA Systems
• Healthcare AI Research & Benchmarking

Limitations & Risks

Important: This model is intended for research and clinical decision support development only.

• Not a Substitute for Medical Professionals: Outputs must be reviewed by qualified clinicians.
• Execution Risk: If integrated into automated clinical workflows, safeguards and validation layers are mandatory.
• Data Distribution Sensitivity: Performance may vary across rare diseases or underrepresented modalities.
• Hardware Requirements: FP8 requires compatible GPU hardware for optimal acceleration.