Indic-Delivery-Rider-SLM (40M parameters)

A small, from-scratch, Llama-style language model for offline assistance of delivery partners in India. Languages: English, Hindi, Bengali, Marathi, Tamil.

Experimental research model. Trained for ~28 minutes on one T4 GPU on a small dataset. Non-English domain data is machine-translated and has not been reviewed by native speakers. Not for payments, safety or legal advice.

Architecture

Decoder-only transformer, 10 layers, hidden 512, 8 attention heads / 4 KV heads (GQA), SwiGLU MLP (1408), RMSNorm, RoPE, tied embeddings, context 384. Tokenizer: SentencePiece BPE, 20000 tokens, byte fallback. Exports as LlamaForCausalLM (logits verified against the original implementation).

Tokenizer fertility on held-out text:

• English: 1.28 tokens/word (GPT-2 BPE: 1.30)
• Hindi: 1.26 tokens/word (GPT-2 BPE: 7.76)
• Bengali: 1.45 tokens/word (GPT-2 BPE: 12.79)
• Marathi: 1.57 tokens/word (GPT-2 BPE: 10.50)
• Tamil: 1.81 tokens/word (GPT-2 BPE: 23.44)

Prompt format

<s><|user|>{question}<|assistant|><|lang|> then the model writes the answer and </s>. <|lang|> is one of <|en|>, <|hi|>, <|bn|>, <|mr|>, <|ta|>; prefill it to force the reply language, or omit it to let the model choose.

Usage

import torch, sentencepiece as spm
from huggingface_hub import snapshot_download
from transformers import AutoModelForCausalLM

path = snapshot_download("learner-subodh/Indic-Delivery-Rider-SLM")
sp = spm.SentencePieceProcessor(model_file=f"{path}/tokenizer.model")
model = AutoModelForCausalLM.from_pretrained(path).eval()

q = "Customer wants to change the delivery address after pickup. What should I tell them?"
ids = [sp.bos_id(), sp.piece_to_id("<|user|>")] + sp.encode(q) + [sp.piece_to_id("<|assistant|>"), sp.piece_to_id("<|en|>")]
out = model.generate(torch.tensor([ids]), max_new_tokens=160, do_sample=True, temperature=0.6, top_k=40, top_p=0.9,
                     repetition_penalty=1.15, eos_token_id=sp.eos_id(), pad_token_id=sp.pad_id())
print(sp.decode(out[0][len(ids):].tolist()))

Training data

• Bitext customer-support dataset (English, bitext/Bitext-customer-support-llm-chatbot-training-dataset)
• Sentence-level machine translation of a stratified subset into the other languages with NLLB-200 (600M distilled), filtered for placeholders, script, repetition and length
• Native-script instruction data from the Aya dataset (CohereForAI/aya_dataset, Apache-2.0)

Evaluation (held-out)

Bits per byte of the answer text (lower is better; comparable across scripts). domain = support-style data, general = Aya.

lang	data	bits_per_byte	ppl	resp_tokens
bn	domain	0.204	6.614	10821
bn	general	0.825	193.146	5206
en	domain	0.258	2.297	52493
hi	domain	0.199	4.506	12750
hi	general	0.694	51.689	3170
mr	domain	0.217	7.363	11822
mr	general	0.668	36.486	4682
ta	domain	0.177	7.704	12050
ta	general	0.731	119.382	9661

Generation checks on hand-written rider prompts (3 per language; tiny sample, indicative only): lang_fidelity = reply is in the expected script (and Hindi/Marathi marker words), auto_lang_id = model picks the right language tag unprompted, distinct2 = 1 - repetition.

lang	lang_fidelity	auto_lang_id	distinct2	stop_rate	avg_words
bn	1.0	1.0	0.95	0.67	80.0
en	1.0	1.0	0.98	0.67	75.0
hi	1.0	1.0	0.94	1.0	91.67
mr	0.67	1.0	0.65	0.67	57.67
ta	1.0	1.0	0.99	0.67	89.67

Limitations

Small capacity, small data: it can state wrong policies fluently, blur Hindi and Marathi (same script), and repeat itself. Machine-translated training text carries translation errors. Route anything involving money, injuries or disputes to a human or a fixed template.

Licensing note

Mixed provenance: Bitext (CDLA-Sharing-1.0), Aya (Apache-2.0), and translations produced by NLLB-200 (model weights CC-BY-NC-4.0). Check these terms with your own counsel before any commercial use. Weights are released for research and evaluation.