Hugging Face's logo

bharatgenai
/
AgriParam

Text Generation
Transformers
Safetensors
English
Hindi
parambharatgen
Multiturn
QnA
conversational
custom_code
Model card Files
xet
 Community
3
kundeshwar20 commited on
Commit
706f631
·
verified ·
1 Parent(s): b96d94b

Upload 2 files

Browse files
Files changed (2) hide show
  1. LICENSE +50 -0
  2. README.md +187 -0
LICENSE ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ BharatGen Research License (BRL)
2
+ Last Updated: August 7, 2025
3
+ Note: This license is subject to change. Please refer to the latest version before using any BharatGen resource.
4
+
5
+ 1. Scope and Acceptance
6
+ - This license applies to any use, modification, or distribution of BharatGen Models, Derivatives, or Outputs.
7
+ - By accessing, using, modifying, or distributing a BharatGen Model or its Derivatives, or by creating Outputs from them, you agree to be bound by this License.
8
+ - If you accept on behalf of an organization, you represent that you have authority to do so.
9
+
10
+ 2. Permitted Use
11
+ - You are granted a non-exclusive, non-transferable, royalty-free license to use, modify, and distribute BharatGen Models and Derivatives strictly for non-commercial research and academic purposes.
12
+ - You must retain the following notice in all copies and distributions:
13
+
14
+ "Licensed by BharatGen under the BharatGen Research License."
15
+ - If you create Derivatives, you must clearly indicate that modifications were made by you.
16
+ - You must not use BharatGen Models, Derivatives, or Outputs for any commercial purposes without prior written permission from the BharatGen legal team.
17
+
18
+ 3. Commercial Use & Modifications
19
+ - Commercial use is strictly prohibited unless you obtain explicit written permission from the BharatGen legal team.
20
+ - If you wish to release, share, publish, fine-tune, or deploy any BharatGen Models or Derivatives (including Outputs) publicly, you must request and receive written approval from the BharatGen team.
21
+ Please reach out via our contact form or designated email.
22
+
23
+ 4. Limitations
24
+ - You must not claim that your modifications, Derivatives, or Outputs are official BharatGen products.
25
+ - You may not use BharatGen resources for any unlawful or unethical purposes.
26
+ - Redistribution of BharatGen resources through SaaS, APIs, or any hosted service—even for free—requires permission.
27
+
28
+ 5. Outputs
29
+ - You own the content (Outputs) you generate using BharatGen models, subject to the non-commercial restriction.
30
+ - You are fully responsible for the Outputs you generate and their usage.
31
+
32
+ 6. Disclaimer & Liability
33
+ - BharatGen resources are provided "AS IS", without warranties of any kind.
34
+ - The BharatGen team are not liable for any damages arising from the use of BharatGen Models, Derivatives, or Outputs.
35
+
36
+ 7. Termination
37
+ - This license is valid until terminated.
38
+ - It will terminate immediately if you breach any of its terms. Upon termination, you must stop using and delete all copies of the Models and Derivatives.
39
+
40
+ 8. Jurisdiction
41
+ This License is governed by the laws of India. Any disputes shall be subject to the courts of India.
42
+
43
+ Definitions
44
+ BharatGen Model: Any model, code, data, or related resource released under this license.
45
+ Derivative: Any modification, fine-tuning, or adaptation of a BharatGen Model.
46
+ Output: Any result generated by running a BharatGen Model.
47
+ You: Any individual or organization using BharatGen resources.
48
+ Non-commercial research: Use in academic, personal, or scientific contexts not intended to generate revenue.
49
+
50
+ If you have questions or require a commercial license, please contact the BharatGen team.
README.md ADDED
@@ -0,0 +1,187 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <div align="center">
2
+ <img src="./BharatGen Logo (1).png" width="40%" alt="BharatGen" />
3
+ </div>
4
+ <hr>
5
+ <div align="center">
6
+ <a href="https://huggingface.co/bharatgenai/AgriParam" target="_blank" style="margin: 4px;">
7
+ <img alt="Model" src="https://img.shields.io/badge/Model-HuggingFace-ffda4b?style=flat&logo=huggingface&logoColor=black" />
8
+ </a>
9
+ <a href="https://huggingface.co/bharatgenai/AgriParam/blob/main/LICENSE" target="_blank" style="margin: 4px;">
10
+ <img alt="License" src="https://img.shields.io/badge/License-yellow.svg" />
11
+ </a>
12
+ </div>
13
+
14
+ # AgriParam
15
+
16
+ **BharatGen** introduces **AgriParam**, a domain-specialized large language model fine-tuned from **Param-1-2.9B-Instruct** on a high-quality, India-centric agriculture dataset.
17
+ AgriParam is designed to understand and generate contextually rich responses for agricultural queries, farmer advisories, policy information, research insights, and rural knowledge dissemination.
18
+
19
+ ---
20
+
21
+ ## 🌱 Motivation
22
+
23
+ Agriculture is the backbone of India’s economy, yet existing language models lack deep domain knowledge tailored to Indian contexts, languages, and cultural nuances.
24
+ AgriParam bridges this gap by combining **Param-1**’s bilingual capabilities with a meticulously curated agricultural knowledge base.
25
+
26
+ ---
27
+
28
+ ## 🏗 Model Architecture
29
+
30
+ AgriParam inherits the architecture of **Param-1-2.9B-Instruct**:
31
+
32
+ * **Hidden size**: 2048
33
+ * **Intermediate size**: 7168
34
+ * **Attention heads**: 16
35
+ * **Hidden layers**: 32
36
+ * **Key-value heads**: 8
37
+ * **Max position embeddings**: 2048
38
+ * **Activation**: SiLU
39
+ * **Positional Embeddings**: Rotary (RoPE, theta=10000)
40
+ * **Attention Mechanism**: Grouped-query attention
41
+ * **Precision**: bf16-mixed
42
+ * **Base model**: [Param-1-2.9B-Instruct](https://huggingface.co/bharatgenai/Param-1-2.9B-Instruct)
43
+
44
+ ---
45
+
46
+ ## 📚 Data Preparation
47
+
48
+ AgriParam’s training corpus was carefully crafted to ensure deep agricultural knowledge, cultural relevance, and bilingual (English-Hindi) accessibility.
49
+
50
+ **Steps involved:**
51
+
52
+ 1. **Source Gathering**
53
+ * 17k open-source, India-focused agricultural news & information passages.
54
+
55
+ 2. **Question Generation**
56
+ * Generated 5 curated Q&A pairs per passage using an open-source LLM.
57
+
58
+ 3. **Domain Taxonomy & Personas**
59
+ * Built an exhaustive, India-specific agricultural taxonomy.
60
+ * Defined farmer, policy-maker, scientist, and agri-business personas.
61
+
62
+ 4. **Dataset Construction**
63
+ * 2M Q&A pairs grounded in taxonomy and personas.
64
+ * Complete dataset translated into Hindi.
65
+ * 6M multi-turn conversation samples created.
66
+
67
+ ---
68
+
69
+ ## 🏋️ Training Setup
70
+
71
+ * **Base model**: Param-1-2.9B-Instruct
72
+ * **Training framework**: Hugging Face + `torchrun` multi-node setup
73
+ * **Prompt template**: Custom-designed for agricultural inference
74
+ * **Scheduler**: Linear with warmup
75
+ * **Epochs**: 3
76
+ * **Total training samples**: 12M
77
+ * **Test samples**: 1.2M
78
+ * **Base learning rate**: 5e-6
79
+ * **Minimum learning rate**: 0
80
+ * **Additional tokens**: `<user>`, `<assistant>`, `<context>`, `<system_prompt>`
81
+ * **Vocab size**: 256k + 4
82
+ * **Global batch size**: 1024
83
+ * **Micro batch size**: 4
84
+ * **Gradient accumulation steps**: 32
85
+
86
+ ---
87
+
88
+ ## 🚀 Inference Example
89
+
90
+ ```python
91
+ from transformers import AutoTokenizer, AutoModelForCausalLM
92
+ import torch
93
+
94
+ model_name = "bharatgenai/AgriParam"
95
+ tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=False)
96
+ model = AutoModelForCausalLM.from_pretrained(
97
+ model_name,
98
+ trust_remote_code=True,
99
+ torch_dtype=torch.bfloat16 if torch.cuda.is_available() else torch.bfloat32,
100
+ device_map="auto"
101
+ )
102
+
103
+ # Example agricultural query
104
+ user_input = "What are the best practices for organic wheat farming in Uttar Pradesh?"
105
+
106
+ prompt = f"<user> {user_input} <assistant>\n\n"
107
+ inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
108
+
109
+ with torch.no_grad():
110
+ output = model.generate(
111
+ **inputs,
112
+ max_new_tokens=300,
113
+ do_sample=True,
114
+ top_k=50,
115
+ top_p=0.95,
116
+ temperature=0.6,
117
+ eos_token_id=tokenizer.eos_token_id
118
+ )
119
+
120
+ print(tokenizer.decode(output[0], skip_special_tokens=True))
121
+ ```
122
+
123
+ 📊 Evaluation
124
+ * Crop-specific Q&A
125
+ * Policy & scheme awareness
126
+ * Rural advisory & extension services
127
+ * Bilingual (English/Hindi) capability
128
+
129
+ ### **BhashaBench Agriculture**
130
+ | Model | bhasha_bench_agri | bhasha_bench_agri_English | bhasha_bench_agri_Hindi |
131
+ |-------------------------------------|-------------------|---------------------------|-------------------------|
132
+ | Llama-3.2-1B | 28.91 | 29.71 | 25.21 |
133
+ | Llama-3.2-1B-Instruct | 28.65 | 29.16 | 26.33 |
134
+ | Llama-3.2-3B | 31.96 | 32.68 | 28.69 |
135
+ | granite-3.1-3b-a800m-base | 32.17 | 33.36 | 26.70 |
136
+ | sarvam-2b-v0.5 | 27.68 | 28.14 | 25.57 |
137
+ | sarvam-1 | 30.24 | 30.82 | 27.57 |
138
+ | **AgriParam** | **32.18** | **33.10** | **27.97** |
139
+
140
+ ---
141
+
142
+ ### **Subject Domain Performance**
143
+
144
+ | Subject Domain | Llama-3.2-1B | Llama-3.2-1B-Instruct | Llama-3.2-3B | granite-3.1-3b-a800m-base | sarvam-2b-v0.5 | sarvam-1 | AgriParam |
145
+ |----------------------------------------------------|--------------|-----------------------|--------------|---------------------------|----------------|----------|-----------|
146
+ | Agri-Environmental & Allied Disciplines | 31.82 | 32.95 | 25.00 | 36.93 | 29.55 | 30.11 | 27.27 |
147
+ | Agricultural Biotechnology | 31.11 | 28.63 | 34.35 | 43.13 | 30.34 | 36.64 | 36.64 |
148
+ | Agricultural Chemistry & Biochemistry | 27.05 | 22.78 | 31.32 | 35.94 | 27.05 | 34.52 | 34.16 |
149
+ | Agricultural Economics & Policy | 29.98 | 25.52 | 35.09 | 34.77 | 27.75 | 30.78 | 32.54 |
150
+ | Agricultural Engineering & Technology | 27.46 | 26.23 | 32.79 | 30.33 | 27.46 | 29.51 | 27.87 |
151
+ | Agricultural Extension Education | 30.88 | 29.46 | 32.30 | 29.84 | 28.17 | 29.97 | 34.50 |
152
+ | Agricultural Microbiology | 34.23 | 36.04 | 31.53 | 34.23 | 17.12 | 26.13 | 34.23 |
153
+ | Agriculture Communication | 33.07 | 28.35 | 29.53 | 34.25 | 25.59 | 33.07 | 32.68 |
154
+ | Agriculture Information Technology | 30.53 | 31.58 | 44.21 | 36.84 | 27.89 | 32.11 | 27.89 |
155
+ | Agronomy | 27.92 | 28.77 | 31.84 | 31.51 | 28.67 | 29.60 | 32.49 |
156
+ | Animal Sciences | 25.68 | 34.46 | 36.49 | 37.84 | 35.14 | 29.05 | **40.54** |
157
+ | Crop Sciences | 31.15 | 26.41 | 29.87 | 35.15 | 26.59 | 29.33 | 32.42 |
158
+ | Dairy & Poultry Science | 35.96 | 31.46 | 30.34 | 44.94 | 33.71 | 32.58 | 29.21 |
159
+ | Entomology | 29.02 | 27.59 | 35.49 | 29.31 | 27.59 | 27.87 | 31.75 |
160
+ | Fisheries and Aquaculture | 29.41 | 41.18 | 38.24 | 26.47 | 20.59 | 14.71 | 23.53 |
161
+ | General Knowledge & Reasoning | 28.44 | 27.53 | 33.13 | 32.38 | 26.17 | 30.56 | 31.92 |
162
+ | Genetics and Plant Breeding | 30.59 | 30.08 | 28.02 | 29.05 | 26.99 | 31.62 | 29.82 |
163
+ | Horticulture | 27.05 | 28.60 | 31.21 | 32.17 | 27.00 | 29.76 | 31.40 |
164
+ | Natural Resource Management | 28.50 | 26.42 | 29.02 | 32.64 | 26.42 | 26.94 | 27.46 |
165
+ | Nematology | 22.83 | 28.26 | 28.26 | 27.17 | 21.20 | 24.46 | 23.91 |
166
+ | Plant Pathology | 28.97 | 30.48 | 27.96 | 29.97 | 25.44 | 33.50 | 25.44 |
167
+ | Plant Sciences & Physiology | 28.68 | 31.78 | 37.98 | 26.36 | 20.93 | 30.23 | 31.01 |
168
+ | Seed Science and Technology | 29.70 | 28.71 | 27.72 | 29.21 | 29.70 | 34.65 | 27.23 |
169
+ | Soil Science | 31.25 | 29.92 | 31.69 | 29.99 | 27.49 | 30.21 | **34.93** |
170
+ | Veterinary Sciences | 27.08 | 14.58 | 37.50 | 39.58 | 20.83 | 41.67 | **43.75** |
171
+
172
+ ---
173
+
174
+ ### **Question Level Difficulty**
175
+
176
+ | Difficulty | Llama-3.2-1B | Llama-3.2-1B-Instruct | Llama-3.2-3B | granite-3.1-3b-a800m-base | sarvam-2b-v0.5 | sarvam-1 | AgriParam |
177
+ |------------|--------------|-----------------------|--------------|---------------------------|----------------|----------|-----------|
178
+ | Easy | 29.43 | 30.22 | 36.44 | 36.08 | 28.26 | 32.20 | **36.94** |
179
+ | Hard | 27.72 | 26.37 | 25.61 | 26.02 | 28.01 | 27.54 | 25.91 |
180
+ | Medium | 28.68 | 27.69 | 29.17 | 29.88 | 27.03 | 28.99 | 29.09 |
181
+
182
+ ---
183
+
184
+ ## 📜 License
185
+
186
+ This SFT checkpoint is released under the **BharatGen non-commercial license**.<br>
187
+ Please refer to the [LICENSE](./LICENSE) for terms and conditions.