Update README.md
Browse files
README.md
CHANGED
|
@@ -5,24 +5,38 @@ datasets:
|
|
| 5 |
- ai4bharat/naamapadam
|
| 6 |
language:
|
| 7 |
- bn
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
# Model Card for Model ID
|
| 11 |
|
| 12 |
-
|
|
|
|
| 13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
|
| 16 |
-
## Model Details
|
| 17 |
|
| 18 |
### Model Description
|
| 19 |
|
| 20 |
-
<!-- Provide a longer summary of what this model is. -->
|
| 21 |
|
| 22 |
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
|
| 23 |
|
| 24 |
-
- **Developed by:**
|
| 25 |
-
- **Funded by [optional]:**
|
| 26 |
- **Shared by [optional]:** [More Information Needed]
|
| 27 |
- **Model type:** [More Information Needed]
|
| 28 |
- **Language(s) (NLP):** [More Information Needed]
|
|
@@ -61,9 +75,7 @@ This is the model card of a 🤗 transformers model that has been pushed on the
|
|
| 61 |
|
| 62 |
## Bias, Risks, and Limitations
|
| 63 |
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
[More Information Needed]
|
| 67 |
|
| 68 |
### Recommendations
|
| 69 |
|
|
|
|
| 5 |
- ai4bharat/naamapadam
|
| 6 |
language:
|
| 7 |
- bn
|
| 8 |
+
base_model:
|
| 9 |
+
- openai-community/gpt2
|
| 10 |
---
|
| 11 |
|
| 12 |
# Model Card for Model ID
|
| 13 |
|
| 14 |
+
AddaGPT 2.0 is a Bengali language model based on GPT-2, fine-tuned using LoRA adapters for academic and low-resource applications. While GPT-2 was originally trained only on English data, this model has been adapted to Bengali using the AI4Bharat NaamaPadam dataset — a corpus focused on Named Entity Recognition (NER).
|
| 15 |
+
This project is intended as a proof of concept to explore how small, pretrained models like GPT-2 can be extended to Indic languages using low-rank adaptation (LoRA) techniques, even under limited compute settings (e.g., free Kaggle GPUs). It lays the foundation for future work in adapting language models for low-bandwidth, regional, and offline-first use cases — where even partial language understanding can support local communities.
|
| 16 |
|
| 17 |
+
## Model Details
|
| 18 |
+
| **Attribute** | **Description** |
|
| 19 |
+
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
| 20 |
+
| **Base Model** | GPT-2 (117M parameters) |
|
| 21 |
+
| **Fine-tuned Using** | [LoRA (Low-Rank Adaptation)](https://arxiv.org/abs/2106.09685) |
|
| 22 |
+
| **Language** | Bengali (`bn`) |
|
| 23 |
+
| **Training Dataset** | [`ai4bharat/naamapadam`](https://huggingface.co/datasets/ai4bharat/naamapadam) – Bengali NER corpus (train split only) |
|
| 24 |
+
| **Sentences Seen During Training** | \~9.6 million Bengali sentences |
|
| 25 |
+
| **Training Platform** | Kaggle (Free T4 GPUs) |
|
| 26 |
+
| **Frameworks** | 🤗 Transformers + PEFT (Parameter-Efficient Fine-Tuning) + Safetensors |
|
| 27 |
+
| **Trainable Parameters** | 294,912 |
|
| 28 |
+
| **Total Parameters** | 124,734,720 |
|
| 29 |
+
| **Percentage Fine-Tuned** | 0.2364% |
|
| 30 |
|
| 31 |
|
|
|
|
| 32 |
|
| 33 |
### Model Description
|
| 34 |
|
|
|
|
| 35 |
|
| 36 |
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
|
| 37 |
|
| 38 |
+
- **Developed by:** Swastik Guha Roy
|
| 39 |
+
- **Funded by [optional]:** Self Funded
|
| 40 |
- **Shared by [optional]:** [More Information Needed]
|
| 41 |
- **Model type:** [More Information Needed]
|
| 42 |
- **Language(s) (NLP):** [More Information Needed]
|
|
|
|
| 75 |
|
| 76 |
## Bias, Risks, and Limitations
|
| 77 |
|
| 78 |
+
This model is not capable of generating grammatically or syntactically correct Bengali sentences. Instead, it outputs individual Bengali words or word-like tokens that are often meaningful on their own — a direct result of training on a NER-style dataset rather than full natural language text.
|
|
|
|
|
|
|
| 79 |
|
| 80 |
### Recommendations
|
| 81 |
|