bge-m3-thai-intent

BAAI/bge-m3 fine-tuned to pick a customer's intent from natural-language intent descriptions. Intents are data, not weights: embed the message, embed each intent description, softmax the similarities. Adding an intent means writing one line of description — no retraining.

Usage

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("SimonY0s/bge-m3-thai-intent")
intents = {
    "track_order": "Asking where a parcel is, delivery status, or when an order will arrive",
    "refund_return": "Wants to return an item, cancel an order, or get money back",
    "talk_to_human": "Explicitly asks to speak with a human agent instead of a bot",
}
msg = model.encode(["ของยังไม่มาเลยค่ะ สั่งไปตั้งแต่วันจันทร์"], normalize_embeddings=True)
lab = model.encode(list(intents.values()), normalize_embeddings=True)
logits = 30.0 * (msg @ lab.T)[0]                 # scale 30 was used in training
probs = np.exp(logits - logits.max()); probs /= probs.sum()
print(dict(zip(intents, probs.round(3))))

Descriptions were written in English during training; messages were Thai (some Thai-English mix).

Results

Synthetic Thai customer messages, 30 intents across e-commerce, banking and telecom. seen = 24 intents present in training (173 test messages); unseen = 6 held-out intents known only by description (241 messages).

Model	seen	unseen
Jev 1.13 (TypeSafe, via OpenRouter)	93.6%	87.6%
this model	93.6%	88.8%
BAAI/bge-m3 zero-shot	81.5%	80.9%
intfloat/multilingual-e5-small, same recipe	86.1%	75.9%

Differences under ~5 points are within noise at these test-set sizes.

Training

741 (message, intent description) pairs, 4 epochs, batch 16, lr 1e-5, AdamW, CLS pooling, cross-entropy over the intents of the message's own domain with similarity scale 30. About 5 minutes on an Apple M4 Pro.

Limitations — read before relying on it

• Synthetic data. Messages and labels were written by an LLM, not by real customers, and the training split kept only messages where a second model (Jev) agreed with the intended label. Expect lower accuracy on real traffic; validate on a human-labelled set.
• Probabilities are not calibrated. The softmax scores were not calibrated on held-out data; tune any confidence threshold yourself.
• Romanised Thai ("karaoke" spelling), numbers and dates are weak spots.
• It maps short messages to intents. It does not reason over structured state (for example "order number known, ticket not yet opened").