Create README.md

README.md

@@ -0,0 +1,241 @@
+---
+license: cc-by-4.0
+language:
+  - bn
+  - en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+  - code
+  - bangla
+  - bengali
+  - code-generation
+  - nlp
+  - low-resource
+datasets:
+  - md-nishat-008/Bangla-Code-Instruct
+base_model:
+  - md-nishat-008/TigerLLM-9B-it
+---
+
+<div align="center">
+
+<img src="https://img.shields.io/badge/🐯_TigerCoder-9B-orange?style=for-the-badge" alt="TigerCoder-9B"/>
+
+<h1 style="color: #2e8b57;">🐯 TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla</h1>
+
+<h3>Accepted at LREC 2026</h3>
+
+<h4>Nishat Raihan, Antonios Anastasopoulos, Marcos Zampieri</h4>
+<h5>George Mason University, Fairfax, VA, USA</h5>
+
+<br/>
+
+<table>
+<tr>
+<td>
+<a href="https://arxiv.org/abs/2509.09101">
+<img src="https://img.shields.io/badge/arXiv-2509.09101-b31b1b?style=for-the-badge&logo=arxiv" alt="arXiv"/>
+</a>
+</td>
+<td>
+<a href="https://arxiv.org/pdf/2509.09101">
+<img src="https://img.shields.io/badge/Paper-Read_PDF-blue?style=for-the-badge&logo=adobeacrobatreader" alt="Read PDF"/>
+</a>
+</td>
+<td>
+<a href="mailto:mraihan2@gmu.edu">
+<img src="https://img.shields.io/badge/Email-Contact_Us-green?style=for-the-badge&logo=gmail" alt="Contact Us"/>
+</a>
+</td>
+</tr>
+</table>
+
+
+
+<table>
+<tr>
+<td>
+<a href="https://huggingface.co/md-nishat-008/TigerCoder-1B">
+<img src="https://img.shields.io/badge/🤗_HuggingFace-TigerCoder--1B-yellow?style=for-the-badge" alt="TigerCoder-1B"/>
+</a>
+</td>
+<td>
+<a href="https://huggingface.co/md-nishat-008/TigerCoder-9B">
+<img src="https://img.shields.io/badge/🤗_HuggingFace-TigerCoder--9B-yellow?style=for-the-badge" alt="TigerCoder-9B"/>
+</a>
+</td>
+</tr>
+</table>
+
+<br/>
+
+**The first dedicated family of Code LLMs for Bangla, achieving 11-18% Pass@1 gains over prior baselines.**
+
+</div>
+
+---
+
+> **⚠️ Note:** Model weights will be released after the LREC 2026 conference. Stay tuned!
+
+## Overview
+
+Despite being the 5th most spoken language globally (242M+ native speakers), Bangla remains severely underrepresented in code generation. **TigerCoder** addresses this gap by introducing the first dedicated Bangla Code LLM family, available in 1B and 9B parameter variants.
+
+This model card is for **TigerCoder-9B**, the instruction-tuned 9B parameter variant, finetuned on **300K Bangla instruction-code pairs** from the Bangla-Code-Instruct dataset. TigerCoder-9B pushes the frontier of Bangla code generation to **0.82 Pass@1 on MBPP-Bangla**, achieving 11-18% gains over the strongest prior baselines (Gemma-3 27B and TigerLLM-9B) while being only one-third their size.
+
+## Key Contributions
+
+1. **Bangla-Code-Instruct**: A comprehensive 300K instruction-code dataset comprising three subsets: Self-Instruct (SI, 100K), Synthetic (Syn, 100K), and Translated+Filtered (TE, 100K).
+2. **MBPP-Bangla**: A 974-problem benchmark with expert-validated Bangla programming tasks across 5 programming languages (Python, C++, Java, JavaScript, Ruby).
+3. **TigerCoder Model Family**: Specialized Bangla Code LLMs (1B and 9B) that set a new state-of-the-art for Bangla code generation.
+
+## Performance
+
+### Python (Pass@K on Bangla Prompts)
+
+| Model | mHumanEval P@1 | mHumanEval P@10 | mHumanEval P@100 | MBPP P@1 | MBPP P@10 | MBPP P@100 |
+|:---|:---:|:---:|:---:|:---:|:---:|:---:|
+| GPT-3.5 | 0.56 | 0.56 | 0.59 | 0.60 | 0.62 | 0.62 |
+| Gemini-Flash 2.5 | 0.58 | 0.61 | 0.62 | 0.62 | 0.62 | 0.70 |
+| Gemma-3 (27B) | 0.64 | 0.65 | 0.69 | 0.69 | 0.70 | 0.70 |
+| TigerLLM (9B) | 0.63 | 0.69 | 0.72 | 0.61 | 0.68 | 0.73 |
+| TigerCoder (1B) | 0.69 | 0.73 | 0.77 | 0.74 | 0.74 | 0.81 |
+| **TigerCoder (9B)** | **0.75** | **0.80** | **0.84** | **0.82** | **0.84** | **0.91** |
+
+### Improvements over Strongest Prior Baseline (Δ)
+
+| Model | mHumanEval P@1 | mHumanEval P@10 | mHumanEval P@100 | MBPP P@1 | MBPP P@10 | MBPP P@100 |
+|:---|:---:|:---:|:---:|:---:|:---:|:---:|
+| TigerCoder (1B) | +0.05 | +0.04 | +0.05 | +0.05 | +0.04 | +0.08 |
+| **TigerCoder (9B)** | **+0.11** | **+0.11** | **+0.12** | **+0.13** | **+0.14** | **+0.18** |
+
+### Multi-Language Performance (TigerCoder-9B, Pass@1 on Bangla Prompts)
+
+| Language | mHumanEval P@1 | MBPP P@1 |
+|:---|:---:|:---:|
+| Python | 0.75 | 0.82 |
+| C++ | 0.67 | 0.72 |
+| Java | 0.62 | 0.67 |
+| JavaScript | 0.57 | 0.62 |
+
+## Usage
+
+### Quickstart
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model_name = "md-nishat-008/TigerCoder-9B"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto")
+
+# Bangla coding prompt
+chat = [{"role": "user", "content": "একটি ফাংশন লিখুন যা একটি সংখ্যার ফ্যাক্টরিয়াল গণনা করে।"}]
+
+inputs = tokenizer.apply_chat_template(chat, tokenize=True, return_tensors="pt").to(model.device)
+
+with torch.no_grad():
+    outputs = model.generate(
+        inputs=inputs,
+        max_new_tokens=512,
+        temperature=0.7,
+        top_p=0.95,
+        pad_token_id=tokenizer.eos_token_id,
+        eos_token_id=tokenizer.eos_token_id
+    )
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Training Details
+
+| Hyperparameter | Value |
+|:---|:---|
+| Base Model | TigerLLM-9B-it |
+| Training Data | Bangla-Code-Instruct (300K examples) |
+| Max Sequence Length | 2048 |
+| Batch Size (Train / Eval) | 32 |
+| Gradient Accumulation Steps | 8 |
+| Epochs | 3 |
+| Learning Rate | 1 × 10⁻⁶ |
+| Weight Decay | 0.04 |
+| Warm-up Steps | 15% |
+| Optimizer | AdamW |
+| LR Scheduler | Cosine |
+| Precision | BF16 |
+| Hardware | NVIDIA A100 (40GB) |
+
+## Datasets
+
+The **Bangla-Code-Instruct** dataset (300K total) consists of three complementary subsets:
+
+| Subset | Size | Method | Prompt Origin | Code Origin |
+|:---|:---:|:---|:---|:---|
+| SI (Self-Instruct) | 100K | 5000 expert seeds + GPT-4o expansion | Semi-Natural | Synthetic |
+| Syn (Synthetic) | 100K | GPT-4o + Claude 3.5 generation | Synthetic | Synthetic |
+| TE (Translated) | 100K | NLLB-200 MT from Evol-Instruct | Translated | Natural (Source) |
+
+All code in SI and Syn subsets is validated via syntax checking (`ast.parse`) and execution testing (Python 3.13.0, 10s timeout, 16GB memory).
+
+## Key Findings
+
+1. **LLMs exhibit a notable performance drop when coding prompts are in Bangla rather than English.** Most models lose 20-50+ percentage points.
+2. **Bangla → English machine translation does not help.** Translated prompts perform similarly or worse than native Bangla prompts due to mistranslation of code-specific keywords (e.g., "অক্ষর" (Character) → "Letter", "চলক" (Variable) → "Clever", "স্ট্রিং" (String) → "Rope").
+3. **High-quality, targeted data beats scale.** TigerCoder-1B surpasses models 27x its size, and TigerCoder-9B widens the lead to 11-18%, confirming that curated, domain-specific data outweighs model scale for low-resource code generation.
+
+## Limitations
+
+- TigerCoder is optimized primarily for Bangla code generation tasks. Performance on general NLU or non-code tasks may not match general-purpose models.
+- The training data is synthetically generated and/or machine-translated, which may introduce biases or artifacts.
+- Evaluation is currently limited to MBPP-Bangla and mHumanEval-Bangla; performance on real-world, production-level coding tasks has not been benchmarked.
+
+## Ethics Statement
+
+We adhere to the ethical guidelines outlined in the LREC 2026 CFP. Our benchmark creation involved careful translation and verification by qualified native speakers. We promote transparency through the open-source release of our models, datasets, and benchmark. We encourage responsible downstream use and community scrutiny.
+
+---
+
+## Citation
+
+If you find our work helpful, please consider citing our paper:
+
+```bibtex
+@article{raihan2025tigercoder,
+  title={Tigercoder: A novel suite of llms for code generation in bangla},
+  author={Raihan, Nishat and Anastasopoulos, Antonios and Zampieri, Marcos},
+  journal={arXiv preprint arXiv:2509.09101},
+  year={2025}
+}
+```
+
+You may also find our related work useful:
+
+```bibtex
+@inproceedings{raihan-zampieri-2025-tigerllm,
+    title = "{T}iger{LLM} - A Family of {B}angla Large Language Models",
+    author = "Raihan, Nishat and
+      Zampieri, Marcos",
+    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
+    month = jul,
+    year = "2025",
+    address = "Vienna, Austria",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2025.acl-short.69/",
+    doi = "10.18653/v1/2025.acl-short.69",
+    pages = "887--896",
+    ISBN = "979-8-89176-252-7"
+}
+```
+
+```bibtex
+@inproceedings{raihan-etal-2025-mhumaneval,
+    title = "m{H}uman{E}val - A Multilingual Benchmark to Evaluate Large Language Models for Code Generation",
+    author = "Raihan, Nishat and
+      Anastasopoulos, Antonios and
+      Zampieri, Marcos",
+    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
+    year = "2025",
+}
+```