pai1-medgemma-27b

PAI (Pharmacy Assistant by ICDevelop) — premium tier. A Thai community-pharmacy assistant fine-tuned from google/medgemma-27b-it by I C Develop co.,ltd (Bangkok, Thailand).

💊 Scope — pharmacy assistant, not a general medical model

Its job is Thai retail-pharmacy counseling: OTC recommendation from symptoms, drug knowledge, dosing, interaction warnings, drug-law classification, and refer-to-doctor triage. It is NOT a diagnostic tool and does NOT read X-rays, lab results, MRI/CT or pathology images. Image input is for drug packages / labels / prescriptions only. Not a medical device — outputs require pharmacist review.

• Developed by: I C Develop co.,ltd (PharmaAI / PAI team)
• Model type: Gemma-3-based multimodal decoder (SigLIP vision encoder + 27B-parameter LM)
• Languages: Thai (primary), English
• Version: v2.6-scaled0.85 (2026-07-23)
• License: derivative of MedGemma under the HAI-DEF terms

Method

• LoRA SFT (rank 16, alpha 32) on language-model layers only; vision tower + multimodal projector frozen → multimodal capability preserved from base.
• Deliberately gentle (low learning rate, 1 epoch, keep-best on eval loss). The 27B base is already strong at counseling, so aggressive fine-tuning degrades it — see Benchmark history.
• bf16, cutoff 3072. Stack: LLaMA-Factory on NVIDIA NGC PyTorch 25.06 · 1× RTX PRO 6000 Blackwell 96 GB.
• LoRA delta scaled to 0.85x at merge time (alpha 32->27). Scaling the delta interpolates between the base and the fine-tune; a sweep of 0.50/0.60/0.70/0.85 measured on the deterministic benchmark put 0.85 highest.

Training data

Verified synthetic Thai pharmacy dialogs grounded on: the TMT drug registry (31,034 products, Thai Ministry of Public Health), a curated clinical KB, an AI-expanded clinical KB, a drug-interaction table, red-flag referral rules, brand/trade-name properties, vitamins & supplements, and RAG-grounded dialogs that teach the model to answer only from a supplied reference block and to refuse when a drug is not in it. Every dialog was cross-checked by a frontier teacher model before inclusion, then deduplicated and n-gram screened against all evaluation sets.

Benchmark

thai_pharma_bench_v1 — 200 held-out Thai pharmacy cases, cross-family LLM judge + deterministic must/must-not checks. Pass = score ≥7/10.

model	avg score	pass rate (≥7/10)
base medgemma-27b-it	6.1 / 10	44.5%
pai1-medgemma-27b @0.50 scale	6.1 / 10	45.5%
pai1-medgemma-27b @0.70 scale	6.2 / 10	46.5%
pai1-medgemma-27b @0.60 scale	6.3 / 10	47.2%
pai1-medgemma-27b @0.85 scale (this)	6.0 / 10	48.0%

How reliable is this number?

Pass rate 48.0% (95% bootstrap CI 41.0% – 54.5%, n=200), against a base medgemma-27b-it reference of 44.5% (CI 38.0% – 51.5%) measured on the same fixed benchmark with the same greedy decoding.

Difference +3.5%. On a paired comparison this model wins 35 items and loses 28 of 200 — a net of 7 items. A paired bootstrap (20,000 resamples) puts p = 0.206, so the difference is NOT statistically significant — this gap is within measurement noise.

Read this before comparing checkpoints. The judge changes its verdict on roughly 63 of 200 items between two models that differ by only a few net points, so differences smaller than about 8 points on this benchmark should not be treated as real improvements. Scores are produced by a single LLM judge (gemini-2.5-flash); the deterministic must-not checks are reported alongside but are not folded into the pass rate, so a response can be scored "pass" while still tripping a must-not rule. Treat these figures as a development signal, not as clinical validation.

Measurement note (2026-07-23). Earlier published figures were produced with the target model sampling at temperature 0.2, so repeat runs of the same weights differed by up to 7 points. The benchmark now decodes greedily (temperature 0.0) and every number below was re-measured on that same deterministic footing, including the base reference. Numbers from before this change are not directly comparable and have been removed rather than restated.

Per category:

dose_accuracy      n= 30 avg= 4.4 pass=33.3% must_not_violations=0
interaction        n= 30 avg= 6.4 pass=46.7% must_not_violations=0
legal_class        n= 25 avg= 5.5 pass=48.0% must_not_violations=0
otc_suggest        n= 60 avg= 5.6 pass=36.7% must_not_violations=1
redflag_referral   n= 35 avg= 8.9 pass=88.6% must_not_violations=1
unsafe_refusal     n= 20 avg= 4.8 pass=35.0% must_not_violations=0

Safety-violation (must-not) count: 2.

Honest note on the version history. Several earlier fine-tuning rounds scored below the untuned base — we publish those numbers above rather than hide them. Only a version that beats the base is released here.

📋 Data provenance & review status — please read

Source	Size	Assurance
TMT drug registry (Thai MoPH)	31,034 products	✅ Official government data
Curated clinical KB	~136 active ingredients	Compiled from standard references
AI-expanded clinical KB	374 active ingredients	⚠️ LLM-generated — pending pharmacist review

• ✅ All 374/374 (100%) AI-expanded active ingredients were verified to exist in the official TMT registry — no fabricated drugs entered the pipeline.
• ⚠️ Their dosage, cautions, indications and legal-class fields were written by an LLM and have not yet been reviewed by a licensed pharmacist; TMT does not carry those fields (0% coverage) so they could not be cross-checked automatically. Pharmacist review is in progress.
• These entries grounded dialog generation; they are not shipped inside the model, but the model may have absorbed errors from them.

⚠️ How to use correctly — you MUST supply drug data (RAG)

The model was trained to counsel, not to memorise the drug catalogue. Asking a bare drug question without a reference can produce hallucinated drug facts. Ground it:

[ข้อมูลอ้างอิงจากทะเบียนยา TMT]
- <ชื่อยา>: ตัวยา <active> | ความแรง <strength> | รูปแบบ <form> | ประเภท <legal class>

[คำถามลูกค้า] <the customer's message>

Instruct it to use only the reference block for drug facts and to answer "ไม่พบข้อมูลยานี้ในทะเบียนที่ตรวจสอบได้ กรุณาสอบถามเภสัชกร" when a named drug is absent. Recommended production shape:

user → retrieval (TMT + clinical KB) → [reference block] → pai1-medgemma-27b → answer
                                     ↘ interaction guard + red-flag detector ↗

Usage

vllm serve icdevelop/pai1-medgemma-27b --dtype bfloat16 --max-model-len 65536 \
  --limit-mm-per-prompt '{"image": 2}'

Limitations & safety

• Focused on the Thai retail-pharmacy domain; not tuned for hospital/clinical documentation.
• Drug-fact recall is the weakest dimension — pair with RAG; treat dosing answers as suggestions requiring pharmacist review.
• Not a medical device. Outputs require review by a licensed pharmacist. Not for unsupervised patient-facing use. Always refers red-flag cases to a physician.

License, redistribution & use restrictions (HAI-DEF)

This model is a Model Derivative of google/medgemma-27b-it, governed by the Health AI Developer Foundations (HAI-DEF) Terms of Use.

NOTICE HAI-DEF is provided under and subject to the Health AI Developer Foundations Terms of Use found at https://developers.google.com/health-ai-developer-foundations/terms

Modification notice. Modified from google/medgemma-27b-it by I C Develop co.,ltd — LoRA supervised fine-tuning of the language-model layers (vision tower and projector frozen).

If you redistribute this model or any derivative, you must: (1) provide recipients a copy of the HAI-DEF Terms; (2) include the use restrictions below as an enforceable provision; (3) include the NOTICE above; (4) carry prominent notices on modified files.

Use restrictions (HAI-DEF §3.2, passed through): comply with the Google Prohibited Use Policy; do not use in any way that could cause a Health Regulatory Authority to deem Google a "manufacturer" of a medical device; do not use in violation of applicable law.

Publisher statement. Not a medical device; not for clinical use or diagnosis; outputs require review by a licensed pharmacist. Google furnishes no medical advice in connection with this model.

Contact: I C Develop co.,ltd