| --- |
| base_model: unsloth/Qwen2.5-7B-Instruct-bnb-4bit |
| library_name: peft |
| pipeline_tag: text-generation |
| language: |
| - bn |
| - en |
| tags: |
| - base_model:adapter:unsloth/Qwen2.5-7B-Instruct-bnb-4bit |
| - lora |
| - sft |
| - transformers |
| - trl |
| - bangla |
| - customer-support |
| license: mit |
| --- |
| |
| # 🇧🇩 BanglaSupport-LLM: Fine-Tuned Bangla Customer Support Model |
|
|
| **BanglaSupport-LLM** is a domain-adapted, fine-tuned Large Language Model optimized specifically for **Bangla E-Commerce Customer Support**. Fine-tuned from **Qwen2.5-7B-Instruct** using **Unsloth QLoRA**, this model eliminates cross-lingual Hindi-bleeding, offering natural, professional, and grammatically accurate customer support responses in native Bengali. |
|
|
| ## 📌 Model Details |
|
|
| - **Developed by:** Mahmudur Rahman ([mrshibly](https://github.com/mrshibly)) & Ferdous Hasan ([FHJibon](https://github.com/FHJibon)) |
| - **Model Type:** PEFT / LoRA Adapter for Causal Language Modeling |
| - **Language(s):** Bengali (`bn`), English (`en`) |
| - **License:** MIT |
| - **Finetuned from model:** `unsloth/Qwen2.5-7B-Instruct-bnb-4bit` |
|
|
| ### 🔗 Model Sources & Links |
|
|
| - **GitHub Repository:** [github.com/FHJibon/BanglaLLM](https://github.com/FHJibon/BanglaLLM) |
| - **Hugging Face Model Hub:** [huggingface.co/FHJibon/Bangla-LLM](https://huggingface.co/FHJibon/Bangla-LLM) |
|
|
| --- |
|
|
| ## Uses |
|
|
| ### Direct Use |
| - E-commerce customer service automation in Bangla. |
| - Answering queries regarding order tracking, shipping, return policies, payment options, and refund eligibility. |
| - Multi-turn conversational support with system persona framing. |
|
|
| ### Out-of-Scope Use |
| - Medical, legal, or high-risk financial advice. |
| - Generating non-Bengali support text where strict monolingual output is expected. |
|
|
| --- |
|
|
| ## How to Get Started with the Model |
|
|
| ```python |
| import torch |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| BASE_MODEL = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit" |
| ADAPTER_ID = "mrshibly/bangla-support-qwen3-8b" |
| |
| print("Loading model and tokenizer...") |
| tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) |
| base_model = AutoModelForCausalLM.from_pretrained( |
| BASE_MODEL, |
| device_map="auto", |
| trust_remote_code=True, |
| ) |
| |
| model = PeftModel.from_pretrained(base_model, ADAPTER_ID) |
| model.eval() |
| |
| system_prompt = "তুমি একজন সহায়ক বাংলা ই-কমার্স গ্রাহক সেবা সহকারী।" |
| user_question = "আমার অর্ডারটি ৩ দিন ধরে পেন্ডিং আছে, ডেলিভারি কখন পাব?" |
| |
| messages = [ |
| {"role": "system", "content": system_prompt}, |
| {"role": "user", "content": user_question}, |
| ] |
| |
| prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
| |
| outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7) |
| response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True) |
| |
| print("Response:", response) |
| ``` |
|
|
| --- |
|
|
| ## Training Details |
|
|
| ### Training Data |
| Trained on a curated dataset of **25,000 normalized Bangla instruction pairs** filtered from: |
| 1. `md-nishat-008/Bangla-Instruct` (ACL 2025 benchmark dataset) |
| 2. `CohereForAI/aya_dataset` (Bengali subset) |
|
|
| Dataset preprocessing included **NFC Unicode normalization**, MinHash LSH deduplication, and instruction-intent filtering. |
|
|
| ### Training Procedure |
|
|
| - **Framework:** PyTorch 2.11.0 + Unsloth `FastLanguageModel` + `SFTTrainer` |
| - **Method:** QLoRA 4-bit (`NormalFloat4` quantization) |
| - **Precision:** `bfloat16` |
| - **LoRA Parameters:** $r = 16$, $\alpha = 32$, Dropout = `0.0` |
| - **Target Modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
|
|
| --- |
|
|
| ## Evaluation Results |
|
|
| Evaluated against held-out test data using automated metrics & LLM-as-a-Judge benchmarking: |
|
|
| | Model Variant | BLEU-4 | ROUGE-L | BERTScore (F1) | LLM-Judge (Fluency) | LLM-Judge (Accuracy) | |
| |---|:---:|:---:|:---:|:---:|:---:| |
| | Base Qwen2.5-7B-Instruct | 0.1820 | 0.3840 | 0.7620 | 3.4 / 5.0 | 3.1 / 5.0 | |
| | **Fine-Tuned BanglaSupport-LLM** | **0.4280** | **0.6910** | **0.9140** | **4.8 / 5.0** | **4.7 / 5.0** | |
|
|
| *BERTScore evaluated using `sagorsarker/bangla-bert-base`.* |
|
|
| --- |
|
|
| ## Hardware & Compute |
|
|
| - **Hardware:** NVIDIA GeForce RTX 5060 Ti (16GB VRAM) |
| - **Platform:** Windows / CUDA 12.0 |
| - **Framework Versions:** PEFT 0.19.1, Transformers 5.5.0, Unsloth 2026.7.3 |
|
|
| --- |
|
|
| ## Authors & Contact |
|
|
| - **Mahmudur Rahman (mrshibly)** |
| - **GitHub:** [@mrshibly](https://github.com/mrshibly) |
|
|
| - **FHJibon** |
| - **GitHub:** [@FHJibon](https://github.com/FHJibon) |
