munyew/meralion-oncology-lora

MERaLiON Oncology Transcription LoRA Adapter

Fine-tuned MERaLiON-2-3B for oncology tumour board transcription.

Overview

This LoRA adapter fine-tunes MERaLiON-2-3B to accurately transcribe complex oncology terminology used in Singapore multi-disciplinary tumour board meetings.

Training Details

Parameter	Value
Base model	MERaLiON/MERaLiON-2-3B
Training samples	500 synthetic oncology sentences
Epochs	3
LoRA rank	16
LoRA alpha	32
Training loss	2.26 → 0.72 (67.9% reduction)
Trainable params	20.7M (1.01% of total)

Target Vocabulary

Cancer types: carcinoma, adenocarcinoma, cholangiocarcinoma, hepatocellular carcinoma, non-small cell lung carcinoma

Procedures: Whipple procedure, hepatectomy, lymphadenectomy, resection, cholecystectomy

Drugs: pembrolizumab, cisplatin, carboplatin, gemcitabine, bevacizumab, oxaliplatin

Pathology: histopathology, immunohistochemistry, PD-L1, EGFR, Ki-67, microsatellite instability

Usage

from peft import PeftModel
from transformers import AutoProcessor

# Load base model
base_model = MERaLiON2ForConditionalGeneration.from_pretrained(
    "MERaLiON/MERaLiON-2-3B"
)

# Load fine-tuned adapter
model = PeftModel.from_pretrained(
    base_model,
    "munyew/meralion-oncology-lora"
)

processor = AutoProcessor.from_pretrained(
    "munyew/meralion-oncology-lora"
)

Developed By

Loh Mun Yew (IMDA) National Multimodal LLM Programme (NMLP) Business & Ecosystems, IMDA BTG

Built with MERaLiON by AI Singapore | Deployed on HuggingFace