tvgh-l-v2-ct2

Fine-tuned Whisper large-v2 for Traditional Chinese medical / nursing-record speech, exported to CTranslate2 (CT2) format for fast inference with faster-whisper.

This repository contains the CT2 model only (model.bin + tokenizer/config), ready to load directly without a subfolder.

Results

Metric	Value
Best eval CER	5.52% (char error rate, 100-sample subset @ step 9000)
Base model	openai/whisper-large-v2
Language	Traditional Chinese (zh)

CER fell from 12.2% (step 1000) to ~5.5% and plateaued in the 5.5–6.5% band; the released checkpoint is the best one (load_best_model_at_end).

Training

Setting	Value
Method	QLoRA (4-bit NF4) fine-tuning, then merged + CT2-converted
LoRA rank / alpha	r=64, alpha=128 (scale 2.0), dropout 0.05
LoRA target modules	q_proj, v_proj
Learning rate	5e-5
Max steps	20000 (~3 epochs)
Batch size	1 (grad accumulation 4)
Optimizer	paged_adamw_8bit, fp16, gradient checkpointing
Eval	CER every 1000 steps on a 100-sample subset

Training audio longer than Whisper's 30-second window was pre-segmented with a VAD + Whisper-segment-boundary chunker that re-aligns the ground-truth transcript to each chunk (so no audio/label truncation).

Usage

from faster_whisper import WhisperModel

model = WhisperModel("jeff7522553/tvgh-l-v2-ct2", device="cuda", compute_type="float16")
segments, info = model.transcribe("audio.wav", language="zh", beam_size=5)
print("".join(seg.text for seg in segments))

CPU:

model = WhisperModel("jeff7522553/tvgh-l-v2-ct2", device="cpu", compute_type="int8")

Limitations

• Specialised for Traditional Chinese medical/nursing dictation; general-domain accuracy may differ from the base model.
• The eval CER comes from a small (100-sample) subset and may not reflect every acoustic condition.
• Outputs may mix Simplified/Traditional characters depending on input; apply OpenCC normalization downstream if you need a fixed variant.