laya-indic

A multilingual typed-decision model with a focus on Indian languages and Hinglish. You give it a text (a message, ticket, review, headline or document) and one or more questions written in plain English. It answers every question in a single forward pass, with a calibrated probability for each option. Question schemas are defined at inference time, so it handles new labels and new tasks without retraining.

laya-indic is an independent personal release. It is not affiliated with or endorsed by HAL-X AI. It uses the THX-01 decision architecture and the open-source thx01 library (Apache-2.0, HAL-X AI), but it is trained from the raw jhu-clsp/mmBERT-base encoder (MIT), not from THX-01's weights. The training data is entirely public, permissively licensed data plus synthetic data generated by open-weights Qwen3 models.

Parameters	about 322M (mmBERT-base encoder plus a 2-layer decision head)
Input	up to 1,024 tokens per question (question, options and text)
Question types	choice, noul (yes/no), score (ordinal)
Training	2 epochs, 458,304 decisions: 285,078 from 17 public datasets and 173,226 verified Qwen3 synthetic questions
Languages	trained on 30+ languages; strongest on English, Hindi, Hinglish and 10 other Indian languages
Licence	Apache-2.0

Question types

type	returns	criteria
choice	one of N options, with a probability for each	{"key": "description", ...} (descriptions may be null)
noul	P(yes)	none
score	an ordinal level and its distribution	an ordered list of level descriptions

THX-01 also supports number, excerpt and "cite": true. This model was not trained for those and does not support them.

Quickstart

import thx01   # pip install thx01

agent = thx01.load("arunp0/laya-indic")          # or a local directory; GPU if available

text = "Kal se mera UPI payment fail ho raha hai, paise kat gaye par merchant ko nahi mile"
out = agent.system_one(text, {
    "topic":  {"type": "choice", "instructions": "What is the customer writing about?",
               "criteria": {"failed_payment": "payment failed or money debited but not received",
                            "refund": "asking for a refund", "account": "login or account access", "other": None}},
    "urgent": {"type": "noul", "instructions": "Does the customer need help urgently?"},
    "anger":  {"type": "score", "instructions": "How frustrated is the customer?",
               "criteria": ["calm", "mildly annoyed", "clearly frustrated", "furious"]},
})
# topic  -> failed_payment (0.97)
# urgent -> P(yes) 0.43
# anger  -> "clearly frustrated" 0.52 / "mildly annoyed" 0.41

The response's model field says "THX-01". That is the thx01 library's default label; the weights are this model's.

For many texts, use agent.predict_batch(texts, questions, batch_size=64). questions can be one schema shared by every text or a list with one schema per text.

Write questions in English and leave the text in its own language and script. That is how the model was trained.

Benchmarks

Zero-shot accuracy on 67 tasks, compared with THX-01 (322M, same architecture, roughly 2.1M training decisions). Both models were scored with the same harness, the same prompts and the same test samples. The harness is adapted from THX-01's open benchmark code (bench/tasks.py).

Task family	Tasks	THX-01	This model
Overall (all tasks)	67	0.757	0.789
English general (AG News, Emotion, Banking77, SMS, MASSIVE en)	7	0.813	0.851
MASSIVE scenario, THX's 14 languages	14	0.871	0.879
MASSIVE intent@20 + yes/no (az, hi)	4	0.916	0.914
SIB-200 topic, THX's 10 languages	10	0.766	0.751
SIB-200 topic, Indic (hi + 9)	10	0.628	0.626
MASSIVE scenario, Indic (bn ta te kn ml ur)	6	0.670	0.848
IndicSentiment (6 languages)	6	0.879	0.892
IndicCOPA (6 languages)	6	0.525	0.497
Hinglish-TOP (domain, intent@20)	2	0.790	0.935
Qwen3 synthetic held-out schemas	4	0.717	0.878

Per task: this model is ahead on 39 of 67 tasks, behind on 24, tied on 4.

Read these numbers with the training data in mind. Neither model saw these test sets. Some tasks, however, have their training splits in one model's data and not in the other's:

Eval family	This model trained on	THX-01
MASSIVE	train split, 20 languages (incl. hi bn ta te kn ml ur, az ka tr fa)	train splits ("public benchmarks' training splits")
Hinglish-TOP	train + validation splits	not stated
Banking77, SMS spam	Banking77 train; SMS messages outside the 1,000-message test sample	not stated
AG News	nothing (licence unclear, so excluded)	not stated individually
SIB-200	nothing (CC-BY-SA, so excluded); synthetic topic sentences instead	not stated
IndicSentiment, IndicCOPA, DAIR Emotion	nothing; synthetic reviews and emotion datasets instead	not stated

All training texts were deduplicated against every evaluation set, using normalised exact match.

Per-task results (accuracy and expected calibration error)

Task	n	THX-01 acc	This model acc	THX-01 ECE	This model ECE
AG News (4)	1000	0.931	0.878	0.008	0.035
DAIR Emotion (6)	1025	0.425	0.518	0.264	0.237
Banking77 (77)	1001	0.658	0.811	0.050	0.056
SMS spam (yes/no)	1000	0.881	0.957	0.098	0.025
MASSIVE-intent@20 (en)	1000	0.928	0.923	0.063	0.049
Intent yes/no (en)	1000	0.961	0.959	0.037	0.019
MASSIVE-intent@20 (az)	1000	0.908	0.876	0.053	0.045
Intent yes/no (az)	1000	0.948	0.947	0.028	0.024
MASSIVE-scenario (en)	500	0.906	0.914	0.090	0.053
MASSIVE-scenario (az)	500	0.878	0.844	0.059	0.064
MASSIVE-scenario (ru)	500	0.896	0.892	0.071	0.059
MASSIVE-scenario (tr)	500	0.874	0.882	0.089	0.065
MASSIVE-scenario (de)	500	0.884	0.894	0.068	0.044
MASSIVE-scenario (fr)	500	0.892	0.914	0.068	0.045
MASSIVE-scenario (es)	500	0.868	0.894	0.072	0.057
MASSIVE-scenario (ar)	500	0.832	0.846	0.073	0.068
MASSIVE-scenario (hi)	500	0.856	0.894	0.058	0.044
MASSIVE-scenario (zh-CN)	500	0.872	0.904	0.068	0.048
MASSIVE-scenario (fa)	500	0.894	0.892	0.077	0.056
MASSIVE-scenario (ka)	500	0.762	0.752	0.061	0.062
MASSIVE-scenario (ja)	500	0.910	0.910	0.066	0.058
MASSIVE-scenario (ko)	500	0.868	0.876	0.049	0.068
SIB-200 (az)	204	0.779	0.721	0.102	0.084
SIB-200 (en)	204	0.833	0.794	0.075	0.087
SIB-200 (ru)	204	0.779	0.774	0.072	0.070
SIB-200 (tr)	204	0.779	0.779	0.088	0.088
SIB-200 (de)	204	0.819	0.774	0.088	0.080
SIB-200 (ar)	204	0.770	0.740	0.079	0.104
SIB-200 (hi)	204	0.721	0.740	0.136	0.093
SIB-200 (zh)	204	0.794	0.784	0.067	0.075
SIB-200 (kk)	204	0.691	0.735	0.133	0.074
SIB-200 (uz)	204	0.696	0.667	0.133	0.102
Hinglish-TOP domain	1000	0.889	0.963	0.062	0.045
Hinglish-TOP intent@20	1000	0.692	0.908	0.050	0.040
MASSIVE-intent@20 (hi)	1000	0.877	0.883	0.034	0.054
Intent yes/no (hi)	1000	0.933	0.951	0.032	0.016
MASSIVE-scenario (bn)	500	0.700	0.836	0.074	0.067
MASSIVE-scenario (ta)	500	0.700	0.840	0.066	0.046
MASSIVE-scenario (te)	500	0.626	0.846	0.081	0.042
MASSIVE-scenario (kn)	500	0.614	0.854	0.065	0.062
MASSIVE-scenario (ml)	500	0.652	0.850	0.054	0.069
MASSIVE-scenario (ur)	500	0.728	0.862	0.061	0.059
SIB-200 (mr)	204	0.647	0.667	0.180	0.121
SIB-200 (bn)	204	0.618	0.652	0.179	0.104
SIB-200 (ta)	204	0.681	0.686	0.117	0.125
SIB-200 (te)	204	0.623	0.593	0.129	0.102
SIB-200 (gu)	204	0.618	0.544	0.128	0.134
SIB-200 (kn)	204	0.608	0.618	0.179	0.091
SIB-200 (ml)	204	0.529	0.593	0.238	0.123
SIB-200 (pa)	204	0.564	0.564	0.178	0.122
SIB-200 (ur)	204	0.672	0.603	0.137	0.171
IndicSentiment (hi)	998	0.918	0.923	0.096	0.019
IndicSentiment (bn)	998	0.866	0.888	0.065	0.031
IndicSentiment (mr)	998	0.896	0.891	0.082	0.043
IndicSentiment (ta)	998	0.871	0.900	0.065	0.020
IndicSentiment (te)	998	0.859	0.864	0.057	0.061
IndicSentiment (gu)	998	0.863	0.890	0.058	0.033
IndicCOPA (hi)	449	0.546	0.488	0.163	0.321
IndicCOPA (mr)	449	0.523	0.485	0.178	0.318
IndicCOPA (bn)	500	0.522	0.522	0.185	0.248
IndicCOPA (ta)	500	0.540	0.498	0.153	0.263
IndicCOPA (te)	500	0.530	0.478	0.177	0.293
IndicCOPA (gu)	448	0.489	0.509	0.221	0.279
Qwen3-synth held-out schemas (choice)	1008	0.727	0.845	0.118	0.064
Qwen3-synth held-out (Hinglish)	719	0.730	0.900	0.127	0.045
Qwen3-synth held-out schemas (noul)	1342	0.750	0.931	0.117	0.021
Qwen3-synth held-out schemas (score)	650	0.658	0.835	0.102	0.060
BoolQ-hi (yes/no) ‡	1000	0.643	0.622	0.190	0.212

ECE is expected calibration error with 15 equal-width bins; lower is better. Mean ECE over the 67 tasks is 0.087 for this model and 0.098 for THX-01.

‡ BoolQ-hi is left out of every mean. Our harness had a label-parsing bug in the first runs of this benchmark. The row shows THX-01 rescored after the fix (0.643, n = 1000) next to this model, which was scored after the fix.

Where this model is stronger: Indian-language intent and domain (MASSIVE Indic +0.18), Hinglish (+0.15), IndicSentiment, Banking77 (+0.15), SMS spam, emotion, Kazakh and Hindi topic classification, and new decision schemas it has never seen (synthetic held-out +0.16).

Where it is weaker: AG News (0.878 vs 0.931; THX-01 reports training on public benchmarks' training splits, while AG News is excluded here), SIB-200 in English and European languages, Uzbek and Azerbaijani SIB-200, and IndicCOPA. On IndicCOPA, a causal-reasoning task, both models are close to chance.

Training

Method. RLCD (Reinforcement Learning for Calibrated Decisions), as introduced by THX-01 and implemented in thx01:

• Policy-gradient term: sampling from noisy logits, rewarded by a strictly proper scoring rule.
• Soft cross-entropy term: on label-smoothed targets.
• Calibration: a temperature for each question type, fitted after training on held-out data.

Training started from raw jhu-clsp/mmBERT-base with a freshly initialised decision head.

Epochs	2
Batch	16 × 2 gradient accumulation = 32 decisions per update; 28,644 updates
Learning rates	encoder 5e-5, head 2e-4; AdamW, weight decay 0.01, 6% warmup, cosine decay
Precision	fp16 autocast, gradient checkpointing
Hardware / time	one NVIDIA T4 (16 GB), about 6.6 hours
Question mix	199k choice, 175k yes/no, 84k score

Public data (285,078 decisions; every source is CC0, CC-BY, Apache-2.0 or MIT; full attribution in LICENSES.md):

Source	Decisions	Licence	Used for
MoritzLaurer/synthetic_zeroshot_mixtral_v0.1	59,998	Apache-2.0	general zero-shot choice / yes-no over diverse tasks
MASSIVE (20 languages, train)	54,875	CC-BY-4.0	intent and domain choice, intent yes/no
HuffPost News Category	28,552	CC-BY-4.0	news-topic choice, coarse and fine
HelpSteer2	20,000	CC-BY-4.0	response-quality score questions
BRIGHTER emotion	19,344	CC-BY-4.0	emotion yes/no, human-labelled in-language (Hindi and Marathi in full, plus 8 other languages)
Civil Comments	12,999	CC0-1.0	toxicity yes/no and score
GoEmotions	11,998	Apache-2.0	fine-grained emotion
Nemotron Safety Guard v3	11,000	CC-BY-4.0	multilingual safety categories
Aegis 2.0	10,999	CC-BY-4.0	safety categories
LEDGAR (LexGLUE)	10,000	CC-BY-4.0	contract-clause type
Banking77 (train)	9,964	CC-BY-4.0	77-way intent
WMT DA human evaluation	9,000	Apache-2.0	translation-quality score
Measuring Hate Speech	8,000	CC-BY-4.0	hate-speech facets
IBM Argument Quality 30k	8,000	CC-BY-3.0	argument-quality score
Hinglish-TOP (train + validation)	6,936	Apache-2.0	Hinglish domain and intent
Aya Red-teaming	2,685	Apache-2.0	multilingual harm categories
SMS Spam Collection	728	CC-BY-4.0	spam vs ham

Synthetic data (173,226 verified questions over 62,263 texts). The generator is Qwen3-235B-A22B-Instruct-2507 and the verifier is Qwen3-30B-A3B-Instruct-2507, both Apache-2.0. The pipeline is label-first:

1. The generator proposes a decision schema.
2. Gold answers are sampled.
3. The generator writes a text in a chosen language, script and register that makes those answers true.
4. The verifier answers blind, and a question is kept only if the verifier agrees with the gold answer (about 72% were kept).

About 68% of the texts are in Indian languages or Hinglish. No proprietary-model outputs were used anywhere. The dataset is published as arunp0/laya-indic-synthetic; see also SYNTHETIC_DATA.md.

Limitations

• Text classification only. It does not do number, excerpt or citation questions, open-ended generation or multi-step reasoning. IndicCOPA results show it cannot do commonsense causal reasoning.
• Instructions in English. Questions and option descriptions should be in English; the text can be in any supported language.
• Long texts are truncated at 1,024 tokens per question.
• Calibration is good on average but not everywhere. Recalibrate on your own data before using probabilities as thresholds.
• About 38% of the training decisions are synthetic. Synthetic Indic text is cleaner and more formal than real user text.
• Content warning. The safety and toxicity datasets (Aegis 2.0, Nemotron Safety Guard, Measuring Hate Speech, Civil Comments, Aya Red-teaming) contain offensive and harmful text. The model can classify such content but is not a tested content-moderation system.

Acknowledgements and licence

• Architecture, training objective and inference library: THX-01 / thx01 by HAL-X AI (Apache-2.0).
• Benchmark harness: THX-01's bench/tasks.py, including its MASSIVE scenario descriptions and label humaniser, which were also used in some training instructions.
• Encoder: mmBERT-base, by JHU CLSP (MIT).

The model is released under the Apache License 2.0. See NOTICE for the required attributions and LICENSES.md for every dataset's authors and licence.