SwastikGuhaRoy commited on
Commit
dbae1dc
·
verified ·
1 Parent(s): 731e643

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -8
README.md CHANGED
@@ -5,24 +5,38 @@ datasets:
5
  - ai4bharat/naamapadam
6
  language:
7
  - bn
 
 
8
  ---
9
 
10
  # Model Card for Model ID
11
 
12
- <!-- Provide a quick summary of what the model is/does. -->
 
13
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
 
16
- ## Model Details
17
 
18
  ### Model Description
19
 
20
- <!-- Provide a longer summary of what this model is. -->
21
 
22
  This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
23
 
24
- - **Developed by:** [Swastik Guha Roy]
25
- - **Funded by [optional]:** [More Information Needed]
26
  - **Shared by [optional]:** [More Information Needed]
27
  - **Model type:** [More Information Needed]
28
  - **Language(s) (NLP):** [More Information Needed]
@@ -61,9 +75,7 @@ This is the model card of a 🤗 transformers model that has been pushed on the
61
 
62
  ## Bias, Risks, and Limitations
63
 
64
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
65
-
66
- [More Information Needed]
67
 
68
  ### Recommendations
69
 
 
5
  - ai4bharat/naamapadam
6
  language:
7
  - bn
8
+ base_model:
9
+ - openai-community/gpt2
10
  ---
11
 
12
  # Model Card for Model ID
13
 
14
+ AddaGPT 2.0 is a Bengali language model based on GPT-2, fine-tuned using LoRA adapters for academic and low-resource applications. While GPT-2 was originally trained only on English data, this model has been adapted to Bengali using the AI4Bharat NaamaPadam dataset — a corpus focused on Named Entity Recognition (NER).
15
+ This project is intended as a proof of concept to explore how small, pretrained models like GPT-2 can be extended to Indic languages using low-rank adaptation (LoRA) techniques, even under limited compute settings (e.g., free Kaggle GPUs). It lays the foundation for future work in adapting language models for low-bandwidth, regional, and offline-first use cases — where even partial language understanding can support local communities.
16
 
17
+ ## Model Details
18
+ | **Attribute** | **Description** |
19
+ | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
20
+ | **Base Model** | GPT-2 (117M parameters) |
21
+ | **Fine-tuned Using** | [LoRA (Low-Rank Adaptation)](https://arxiv.org/abs/2106.09685) |
22
+ | **Language** | Bengali (`bn`) |
23
+ | **Training Dataset** | [`ai4bharat/naamapadam`](https://huggingface.co/datasets/ai4bharat/naamapadam) – Bengali NER corpus (train split only) |
24
+ | **Sentences Seen During Training** | \~9.6 million Bengali sentences |
25
+ | **Training Platform** | Kaggle (Free T4 GPUs) |
26
+ | **Frameworks** | 🤗 Transformers + PEFT (Parameter-Efficient Fine-Tuning) + Safetensors |
27
+ | **Trainable Parameters** | 294,912 |
28
+ | **Total Parameters** | 124,734,720 |
29
+ | **Percentage Fine-Tuned** | 0.2364% |
30
 
31
 
 
32
 
33
  ### Model Description
34
 
 
35
 
36
  This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
37
 
38
+ - **Developed by:** Swastik Guha Roy
39
+ - **Funded by [optional]:** Self Funded
40
  - **Shared by [optional]:** [More Information Needed]
41
  - **Model type:** [More Information Needed]
42
  - **Language(s) (NLP):** [More Information Needed]
 
75
 
76
  ## Bias, Risks, and Limitations
77
 
78
+ This model is not capable of generating grammatically or syntactically correct Bengali sentences. Instead, it outputs individual Bengali words or word-like tokens that are often meaningful on their own — a direct result of training on a NER-style dataset rather than full natural language text.
 
 
79
 
80
  ### Recommendations
81