Upload Indic Delivery Rider SLM (40M)

README.md

@@ -0,0 +1,87 @@
+---
+language: ["en", "hi", "bn", "mr", "ta"]
+license: other
+library_name: transformers
+pipeline_tag: text-generation
+tags: [small-language-model, slm, indic, multilingual, edge-ai, logistics, customer-support, llama]
+---
+
+# Indic-Delivery-Rider-SLM (40M parameters)
+
+A small, from-scratch, Llama-style language model for **offline** assistance of delivery partners in India.
+Languages: English, Hindi, Bengali, Marathi, Tamil.
+
+> **Experimental research model.** Trained for ~28 minutes on one T4 GPU on a small dataset. Non-English
+> domain data is **machine-translated** and has not been reviewed by native speakers. Not for payments, safety or legal advice.
+
+## Architecture
+Decoder-only transformer, 10 layers, hidden 512, 8 attention heads / 4 KV heads (GQA), SwiGLU MLP (1408),
+RMSNorm, RoPE, tied embeddings, context 384. Tokenizer: SentencePiece BPE, 20000 tokens, byte fallback.
+Exports as `LlamaForCausalLM` (logits verified against the original implementation).
+
+Tokenizer fertility on held-out text:
+- English: 1.28 tokens/word (GPT-2 BPE: 1.30)
+- Hindi: 1.26 tokens/word (GPT-2 BPE: 7.76)
+- Bengali: 1.45 tokens/word (GPT-2 BPE: 12.79)
+- Marathi: 1.57 tokens/word (GPT-2 BPE: 10.50)
+- Tamil: 1.81 tokens/word (GPT-2 BPE: 23.44)
+
+## Prompt format
+`<s><|user|>{question}<|assistant|><|lang|>` then the model writes the answer and `</s>`.
+`<|lang|>` is one of `<|en|>`, `<|hi|>`, `<|bn|>`, `<|mr|>`, `<|ta|>`; prefill it to force the reply language, or omit it to let the model choose.
+
+## Usage
+```python
+import torch, sentencepiece as spm
+from huggingface_hub import snapshot_download
+from transformers import AutoModelForCausalLM
+
+path = snapshot_download("learner-subodh/Indic-Delivery-Rider-SLM")
+sp = spm.SentencePieceProcessor(model_file=f"{path}/tokenizer.model")
+model = AutoModelForCausalLM.from_pretrained(path).eval()
+
+q = "Customer wants to change the delivery address after pickup. What should I tell them?"
+ids = [sp.bos_id(), sp.piece_to_id("<|user|>")] + sp.encode(q) + [sp.piece_to_id("<|assistant|>"), sp.piece_to_id("<|en|>")]
+out = model.generate(torch.tensor([ids]), max_new_tokens=160, do_sample=True, temperature=0.6, top_k=40, top_p=0.9,
+                     repetition_penalty=1.15, eos_token_id=sp.eos_id(), pad_token_id=sp.pad_id())
+print(sp.decode(out[0][len(ids):].tolist()))
+```
+
+## Training data
+- Bitext customer-support dataset (English, `bitext/Bitext-customer-support-llm-chatbot-training-dataset`)
+- Sentence-level machine translation of a stratified subset into the other languages with NLLB-200 (600M distilled), filtered for placeholders, script, repetition and length
+- Native-script instruction data from the Aya dataset (`CohereForAI/aya_dataset`, Apache-2.0)
+
+## Evaluation (held-out)
+Bits per byte of the answer text (lower is better; comparable across scripts). `domain` = support-style data, `general` = Aya.
+
+| lang | data | bits_per_byte | ppl | resp_tokens |
+|---|---|---|---|---|
+| bn | domain | 0.204 | 6.614 | 10821 |
+| bn | general | 0.825 | 193.146 | 5206 |
+| en | domain | 0.258 | 2.297 | 52493 |
+| hi | domain | 0.199 | 4.506 | 12750 |
+| hi | general | 0.694 | 51.689 | 3170 |
+| mr | domain | 0.217 | 7.363 | 11822 |
+| mr | general | 0.668 | 36.486 | 4682 |
+| ta | domain | 0.177 | 7.704 | 12050 |
+| ta | general | 0.731 | 119.382 | 9661 |
+
+Generation checks on hand-written rider prompts (3 per language; tiny sample, indicative only): `lang_fidelity` = reply is in the expected
+script (and Hindi/Marathi marker words), `auto_lang_id` = model picks the right language tag unprompted, `distinct2` = 1 - repetition.
+
+| lang | lang_fidelity | auto_lang_id | distinct2 | stop_rate | avg_words |
+|---|---|---|---|---|---|
+| bn | 1.0 | 1.0 | 0.95 | 0.67 | 80.0 |
+| en | 1.0 | 1.0 | 0.98 | 0.67 | 75.0 |
+| hi | 1.0 | 1.0 | 0.94 | 1.0 | 91.67 |
+| mr | 0.67 | 1.0 | 0.65 | 0.67 | 57.67 |
+| ta | 1.0 | 1.0 | 0.99 | 0.67 | 89.67 |
+
+## Limitations
+Small capacity, small data: it can state wrong policies fluently, blur Hindi and Marathi (same script), and repeat itself.
+Machine-translated training text carries translation errors. Route anything involving money, injuries or disputes to a human or a fixed template.
+
+## Licensing note
+Mixed provenance: Bitext (CDLA-Sharing-1.0), Aya (Apache-2.0), and translations produced by NLLB-200 (model weights CC-BY-NC-4.0).
+Check these terms with your own counsel before any commercial use. Weights are released for research and evaluation.

config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 512,
+  "initializer_range": 0.02,
+  "intermediate_size": 1408,
+  "max_position_embeddings": 384,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 8,
+  "num_hidden_layers": 10,
+  "num_key_value_heads": 4,
+  "pad_token_id": 3,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.22.2",
+  "use_cache": true,
+  "vocab_size": 20000
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 3,
+  "transformers_version": "5.22.2",
+  "use_cache": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12fb68f95eba1f4a4bfe4357d2c3dd3b4aa7cf1358df7892fda86bed0e370046
+size 158977928

slm_format.json

@@ -0,0 +1,15 @@
+{
+  "bos": 1,
+  "eos": 2,
+  "pad": 3,
+  "user": 4,
+  "assistant": 5,
+  "lang_tags": {
+    "en": 6,
+    "hi": 7,
+    "bn": 8,
+    "mr": 9,
+    "ta": 10
+  },
+  "format": "<s><|user|>{prompt}<|assistant|><|lang|>{reply}</s>"
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0c508b431f5c137fa61160753ce6d2ce0da32b26ca4901729b65ab82d789622
+size 472106