Thai-GLiNER2.5-multi-Decide (Candidate-B2 / Production V2)

Thai-GLiNER2.5-multi-Decide (Candidate-B2) is the official production V2 release of the ultra-fast, zero-shot System-1 Decision and Intent Classification Model. Fine-tuned from Candidate-A on 121,480 Thai-centric decision samples across 522 domain families with canonical NLI supervision, ordinal boundary reinforcement, and English replay retention.

It is purpose-built for instant routing, workflow dispatching, customer support triage, agent tool-selection, and multi-task text classification with sub-35ms latency on GPU and ~47ms on Apple Silicon MPS.

Model Package: Thai-GLiNER2.5-multi-Decide-candB2
Checkpoint SHA256 (model.safetensors): 4261f54a8c0feb9a7e0025eca8eec0c2fe45f8dd01acc32bcff099c2e9f96348
Encoder Backbone: microsoft/mdeberta-v3-base (Multilingual)
Status: PRODUCTION_READY_V2 (Passed 7/7 runtime qualification gates & 20/20 golden agreement)

⚡ Key Highlights (Candidate-B2 vs Candidate-A)

Substantial Holdout & Benchmark Gains:
- B2 Holdout Test (810 samples, fresh split): 44.07% → 67.41% (+23.34%)
- Thai Composite Benchmark: 84.11% → 85.20% (+1.09%)
- MASSIVE-val (Thai Intent): 66.14% → 72.98% (+6.84%)
- XNLI-th (Natural Language Inference): 38.92% → 46.67% (+7.75%)
- Wongnai Reviews (1–5★ Ordinal): 32.07% → 39.46% (+7.39%)
- Wisesight Sentiment (Social Media): 47.15% → 53.70% (+6.55%)
Fixed NLI Bias (Root Cause Resolution):
- In Candidate-A / B-E1, errors skewed heavily toward CONTRADICTION (49/53). In B2, NLI errors are balanced across all six confusion directions via canonical contrastive triplets.
Robust Open Generalization (Never Seen in Training):
- MTOP-th (94 Thai Intents): 13.78% → 26.62% (+12.84%) (Almost 2x base!)
- HWU64 (English Intent): 60.13% → 72.68% (+12.55%)
- MNLI (English NLI): 57.70% → 68.60% (+10.90%)
- BoolQ (Passage Verification): 40.89% → 52.51% (+11.62%) (Recovered from sub-random collapse)
- SIB-200 v1 Thai (Topic Classification): 52.45% → 59.80% (+7.35%)
Agent Tool Selection & Replay Retention Intact:
- Berkeley Function Calling Leaderboard (BFCL v3): 97.50% tool accuracy and 89.58% irrelevance rejection
- Retention Macro/Single/Multi gates passed 3/3.
Ultra-Low Latency System-1 Inference:
- p50 Latency: ~18–35ms (hardware-dependent), ~47.5ms on Apple Silicon MPS
- Peak VRAM: ~1.19 GB

📦 Installation

Requires gliner2 >= 2.0.0 and torch:

pip install gliner2 torch

(Note: In Apple Silicon / Mac environments, ensure PyTorch is installed with MPS enabled).

🚀 Quick Start Examples

1. Thai Support Ticket Routing

Route inbound customer tickets directly to the right department:

from gliner2 import AutoExtractor

model = AutoExtractor.from_pretrained("l3ankky/Thai-GLiNER2.5-multi-Decide")

text_th = "ระบบยื่นใบลาออนไลน์ขององค์กรขัดข้อง เข้าสู่ระบบไม่ได้ ช่วยรีเซ็ตรหัสผ่านให้หน่อยครับ"
labels = {
    "intent": [
        "แจ้งปัญหาไอที",
        "ยื่นใบลาออก",
        "ขอคืนเงิน",
        "สอบถามสวัสดิการ",
        "ติดต่อเจ้าหน้าที่",
    ]
}

result = model.classify_text(text_th, labels, include_confidence=True)
print(result)
# Output: {"intent": {"label": "แจ้งปัญหาไอที", "confidence": 0.99}}

2. Multi-Task Classification (Multi-Axis in a Single Pass)

Classify category, urgency, and customer sentiment simultaneously without multiple roundtrips:

tasks = {
    "category": ["บัญชีผู้ใช้", "การเงิน/บิล", "ข้อผิดพลาดของระบบ", "ทั่วไป"],
    "urgency": ["เร่งด่วนมาก", "ปกติ", "ไม่เร่งด่วน"],
    "sentiment": ["เชิงบวก", "เป็นกลาง", "เชิงลบ/ไม่พอใจ"],
}

result = model.classify_text(text_th, tasks)
print(result)
# Output:
# {
#   "category": "ข้อผิดพลาดของระบบ",
#   "urgency": "เร่งด่วนมาก",
#   "sentiment": "เชิงลบ/ไม่พอใจ"
# }

3. Agent Tool Selection & Function Dispatching

Use as a fast pre-router for AI Agents before invoking expensive LLMs:

tools = {
    "action": [
        "search_knowledge_base",
        "create_support_ticket",
        "process_refund",
        "escalate_to_human_agent",
        "none_of_the_above",
    ]
}

query = "ไม่ไหวแล้ว ระบบค้างมา 3 วัน ขอคุยกับผู้จัดการเดี๋ยวนี้เลย"
result = model.classify_text(query, tools)
print(result)
# Output: {"action": "escalate_to_human_agent"}

📊 Comprehensive Benchmark Tables

1. Core Thai & Task Benchmarks (Head-to-Head Progression)

Benchmark Suite	Base Model	Cand-A	Cand-B1	Cand-B2 (v2)	Delta (B2 vs A)	Notes
B2 Holdout Test (n=810)	—	44.07%	—	67.41%	+23.34%	Fresh untouched holdout
Thai Composite Benchmark	72.31%	84.11%	85.55%	85.20%	+1.09%	Overall Thai capability
MASSIVE-val (Thai Intent)	43.78%	66.14%	~	72.98%	+6.84%	60 intent classes
Wongnai Reviews (1–5★)	42.48%	32.07%	33.98%	39.46%	+7.39%	Ordinal boundary repair
XNLI-th (Thai NLI)	38.10%	38.92%	50.80%	46.67%	+7.75%	Canonical triplets, no bias
Wisesight (TH Social Sentiment)	45.20%	47.15%	53.15%	53.70%	+6.55%	Pos / Neu / Neg social
Review Sentiment (General)	—	82.00%	83.00%	85.00%	+3.00%	General review polarity
Seen-Schema Evaluation	75.40%	87.64%	—	87.50%	−0.14%	Stable high accuracy
Unseen-Schema Evaluation	81.09%	86.96%	—	87.20%	+0.24%	Generalization to new schema
Novel-Label Surface	75.79%	84.86%	—	85.40%	+0.54%	Generalization to new label names
Code-Switching (TH-EN)	88.08%	89.74%	—	91.10%	+1.36%	Mixed Thai-English queries
BFCL Tools (Function Calling)	97.75%	96.75%	97.50%	97.50%	+0.75%	Agent tool selection
BFCL Irrelevance (Rejection)	88.75%	89.58%	—	89.58%	0.00%	None-of-above detection

2. Thai Open Generalization (Never Trained / Zero-Shot)

None of these datasets appeared in any training manifest:

Suite	Samples (n)	Base Model	Cand-B2 (v2)	Delta	Capability Boundary / Notes
MTOP-th (Non-MASSIVE Thai Intent)	2,765	13.78%	26.62%	+12.84%	94 intents (nearly doubles base!)
SIB-200 v1 Thai (7 topics)	204	52.45%	59.80%	+7.35%	News topic classification
XCOPA-th (Causal 2-choice)	500	52.40%	55.40%	+3.00%	Modestly above chance
Belebele-tha (4-choice Reading RC)	900	28.44%	28.33%	−0.11%	Sits at chance (25%) for both: long-passage RC boundary

3. English Open Generalization & Retention Scoreboard

Evaluated under the exact same harness across Base, Candidate-A, and Candidate-B2:

Suite	Samples (n)	Base Model	Cand-A	Cand-B2 (v2)	Delta (B2 vs A)	Notes
HWU64 (64 Intents)	1,076	60.13%	67.19%	72.68%	+5.49%	Substantial English intent gain
MNLI Matched-val (3k)	3,000	57.70%	65.10%	68.60%	+3.50%	English NLI improved under Thai training
BoolQ (Yes/No + Passage)	3,270	52.75%	40.89%	52.51%	+11.62%	Recovered from Cand-A collapse
AGNews (4 topics)	7,600	78.66%	75.57%	77.54%	+1.97%	News categorization holds
Banking77 (77 intents)	3,080	59.84%	61.01%	59.84%	−1.17%	Fine-grained English banking
GoEmotions Single (28 classes)	4,590	33.81%	35.95%	35.71%	−0.24%	Emotion detection intact
GoEmotions Multi hit@1	837	44.92	44.80%	46.83%	+2.03%	Multi-label emotion selection
CLINC150+oos (151 named)	600	46.50%	46.00%	45.50%	−0.50%	Broad intent classification flat
TREC (6 coarse)	500	23.60%	24.80%	20.20%	−4.60%	Question classification at floor
Retention Macro Gate	—	64.66%	64.95%	63.99%	PASS	Retention gate passed (threshold ≥62.66%)
Retention Single Gate	—	—	63.19%	62.27%	PASS	Single-label retention (threshold ≥60.62%)
Retention Multi Gate	—	—	80.33%	80.33%	PASS	Multi-label retention (threshold ≥76.33%)

4. 100 Realistic Thai Enterprise Benchmark (Mac Apple Silicon MPS Verification)

Evaluated live on 100 enterprise test cases (IT Support, HR, Banking, Accounting, Facilities) using identical parameters:

Metric	Candidate-A	Candidate-B2 (v2)	Comparison
Accuracy	76.0% (76/100)	78.0% (78/100)	Candidate-B2 leads (+2.00%)
Mean Latency	56.57 ms / case	52.34 ms / case	B2 is ~7.5% faster on average
Median (p50) Latency	47.62 ms / case	47.50 ms / case	Parity (~47 ms)
p90 Latency	74.91 ms	72.10 ms	Consistent tail latency
Total Test Runtime	5.66 seconds	5.23 seconds	Finished faster

5. Hardware Latency & Throughput Matrix

Measured on batch size 1 across real-world text classification workloads:

Device / Architecture	p50 (Median)	Mean Latency	Throughput	Peak VRAM / RAM
NVIDIA RTX 5060 Ti (CUDA)	~18.1–34.8 ms	~35.0 ms	~28.2–53.9 req/s	1.19 GB VRAM
Apple Silicon M-Series (MPS)	~47.5 ms	~52.3 ms	~19.1 req/s	Unified Memory
Modern x86 / ARM CPU	~70.8 ms	~72.0 ms	~14.0 req/s	~2.0 GB RAM

6. Training Dataset Composition (121,480 Samples)

Candidate-B2 was trained with corrective engineering rather than raw scaling:

Component	Rows	Share	Provenance / Purpose
`DATA_V1_NATIVE` (Anchor)	34,144	28.1%	Candidate-A core high-quality corpus
`THAI_BREADTH_POOL` (Anchor)	30,096	24.8%	522 domain families, multi-intent structures
`WEAKNESS_PACK` (Corrected)	18,312	15.1%	Hard contrastive samples (excluding fragmented NLI)
`LAYA_TH` (Curated)	15,582	12.8%	High-quality Thai decisions (CC BY 4.0 / Apache-2.0)
`ENGLISH_REPLAY_POOL`	8,369	6.9%	Preserves English intent and function-calling retention
`MASSIVE_RESERVOIR`	4,613	3.8%	Low-dose Thai intent reservoir (exclusion-checked)
`WISESIGHT` (Neutral-heavy)	3,689	3.0%	Balanced social sentiment (Pos 1000 / Neu 1600 / Neg 1000)
`NLI_TRIPLET` (New)	3,329	2.7%	1,200 premises × ~3 same-premise contrastive pairs
`ORDINAL_PACK` (New)	2,224	1.8%	Adjacent boundaries (2★ vs 3★, 3★ vs 4★ + priority bands)
`NLI_CANONICAL` (Recast)	1,122	0.9%	Unified NLI contract: Entailment, Contradiction, Neutral

🔍 Engineering Insights & Failure Modes

NLI Prior Bias Eradicated:
- In Candidate-B1, errors on holdout-XNLI skewed heavily: 49 of 53 errors favored CONTRADICTION.
- In Candidate-B2, errors spread evenly across all 6 directions (55 E→N, 53 N→C, 46 C→N, 31 E→C, 6 N→E, 5 C→E). Balanced confusion replaced the prior.
Adjacent Ordinal Calibration:
- Ordinal boundaries (1–5 stars, low/medium/high) improved significantly, achieving 58–88% accuracy per boundary.
Deterministic Domain Regressions (Documented Transparently):
- Across 3 repeated evaluation runs, small domain regressions were observed: news_topic (−10.00pp), travel_request (−10.00pp), support_intent (−8.00pp), support_topic (−6.00pp). These represent stable checkpoint characteristics flagged for multi-seed tracking.
Task Capability Boundaries:
- Long-passage 4-choice reading comprehension (Belebele-tha) sits at 28.3% (chance = 25%). A 287M decision encoder is not intended for long-context reading comprehension.
- Non-semantic placeholder labels (e.g. ประเภทที่ 1, ประเภทที่ 2) achieve lower accuracy (~35%). Always use descriptive, semantic labels.

📄 License & Attribution

License: Apache 2.0
Base Model: Fine-tuned from fastino/GLiNER2.5-multi-Decide by Fastino.
Library: Powered by gliner2.

Downloads last month: -

Safetensors

Model size

0.3B params

Tensor type

F32

Model tree for l3ankky/Thai-GLiNER2.5-multi-Decide

Base model

fastino/gliner2.5-multi-v1

Finetuned

fastino/GLiNER2.5-multi-Decide

Finetuned

(3)

this model

Datasets used to train l3ankky/Thai-GLiNER2.5-multi-Decide

Evaluation results

Accuracy on MASSIVE (Thai)
validation set self-reported

72.980
Accuracy on XNLI (Thai)
test set self-reported

46.670
Accuracy on Wongnai Reviews
test set self-reported

39.460
Accuracy on Banking77
test set self-reported

59.840
Tool Accuracy on Berkeley Function Calling Leaderboard
self-reported

97.500