---
language:
- en
- hi
license: apache-2.0
base_model: microsoft/Phi-3-mini-4k-instruct
tags:
- government-schemes
- rural-development
- scheme-recommendation
- india
- peft
- lora
- phi-3
library_name: peft
datasets:
- custom
metrics:
- perplexity
pipeline_tag: text-generation
---

# 🏛️ Government Scheme Recommendation Model

**Model ID:** `Manas281/scheme-recommendation-model-2`  
**Base Model:** `microsoft/Phi-3-mini-4k-instruct`  
**Fine-tuning Method:** LoRA (Low-Rank Adaptation via PEFT)

## 📘 Overview

This model has been fine-tuned to recommend appropriate **Indian government schemes** based on structured descriptions of developmental work in rural areas. It understands domain-specific infrastructure and welfare needs and provides targeted scheme recommendations with justifications.

### Key Capabilities

✅ **Structured Input Processing** - Accepts Domain, Indicator, and Description  
✅ **Multi-Domain Coverage** - Water & Sanitation, Education, Health, Roads, Electricity, etc.  
✅ **Scheme Identification** - Recommends both infrastructure and individual welfare schemes  
✅ **Contextual Justification** - Explains why each scheme is relevant  
✅ **JSON Output** - Structured format for easy integration

## 🎯 Model Objective

The model is designed to:

1. **Understand** structured domain descriptions (Domain, Indicator, Description)
2. **Identify** the underlying development need
3. **Recommend** the most relevant government scheme(s)
4. **Justify** the recommendation with clear reasoning

## 🧩 Input/Output Format

### Input Prompt Structure

```
### Instruction:
Domain: [Domain Name]
Indicator: [Indicator Code and Description]
Description: [Detailed description of the work/need]

Based on the above work description, recommend appropriate government schemes.

### Response:
```

### Example Input

```
### Instruction:
Domain: 1. Domain: Drinking Water and Sanitation
Indicator: 1.2 Household Tap Connections
Description: Extend pipeline to the Ambedkar Colony to cover 45 households

Based on the above work description, recommend appropriate government schemes.

### Response:
```

### Example Output

```json
{
  "infrastructure_schemes": [
    {
      "identified_need": "Extend water pipeline to Ambedkar Colony for 45 households.",
      "suggested_scheme": "Jal Jeevan Mission - Ensure tap connections for each household.",
      "justification": "This scheme directly addresses the need for tap connections in Ambedkar Colony."
    }
  ],
  "individual_schemes": [],
  "total_recommendations": 1
}
```

## 📊 Covered Domains

The model has been trained on the following rural development domains:

1. **Drinking Water and Sanitation** - Water supply, drainage, waste management, toilets
2. **Education** - School infrastructure, scholarships, enrollment
3. **Health and Nutrition** - Health facilities, insurance, maternal care, Anganwadis
4. **Social Security** - Pensions for elderly, widows, disabled persons
5. **Roads and Connectivity** - Road construction, bridges, culverts, footpaths
6. **Electricity** - Village electrification, household connections, street lighting
7. **Agriculture** - Irrigation, soil testing, organic farming
8. **Financial Inclusion** - Bank accounts, insurance schemes
9. **Digitization** - Internet access, CSCs, digital literacy
10. **Livelihood and Skill Development** - SHGs, employment generation, training

## ⚙️ Training Configuration

| Parameter | Value |
|-----------|-------|
| **Base Model** | microsoft/Phi-3-mini-4k-instruct |
| **Fine-tuning Type** | LoRA (PEFT) |
| **Trainable Parameters** | 8.9M (0.23% of total) |
| **Total Parameters** | 3.83B |
| **Training Epochs** | 2 |
| **Training Samples** | 179 |
| **Validation Samples** | 32 |
| **LoRA Rank (r)** | 16 |
| **LoRA Alpha** | 32 |
| **Learning Rate** | 2e-4 |
| **Max Sequence Length** | 4096 tokens |
| **Gradient Checkpointing** | Enabled |
| **Quantization** | 8-bit (during training) |

## 📈 Training Results

| Step | Training Loss | Validation Loss |
|------|---------------|-----------------|
| 5    | 1.7386        | 1.6543          |
| 10   | 1.5905        | 1.3575          |
| 15   | 1.1994        | 0.8755          |
| 20   | 0.7829        | 0.6408          |

**Observations:**
- ✅ Both training and validation losses decrease smoothly
- ✅ **Final validation loss: 0.64** indicates excellent convergence
- ✅ No signs of overfitting - validation loss tracks training loss well
- ✅ Model achieves strong generalization across domains

## 🧾 Evaluation Metrics

| Criterion | Result | Notes |
|-----------|--------|-------|
| **Model Accuracy** | High | Correctly identifies schemes for unseen examples |
| **JSON Consistency** | 95% | Occasional repetition issues, requires post-processing |
| **Generalization** | Strong | Works across multiple domains and scheme types |
| **Perplexity** | ~1.9 | Indicates confident text generation |

## 🚀 Usage

### Using Transformers Library

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model_name = "Manas281/scheme-recommendation-model-2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Prepare prompt
prompt = """### Instruction:
Domain: 1. Domain: Drinking Water and Sanitation
Indicator: 1.2 Household Tap Connections
Description: Extend pipeline to the Ambedkar Colony to cover 45 households

Based on the above work description, recommend appropriate government schemes.

### Response:
"""

# Generate recommendation
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    pad_token_id=tokenizer.pad_token_id
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

### Using with PEFT

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "Manas281/scheme-recommendation-model-2")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")




## 💡 Use Cases

✅ **Government Planning Automation** - Automated scheme mapping for development projects  
✅ **Rural Development Systems** - AI-powered recommendation engines for gram panchayats  
✅ **E-Governance Assistants** - Chatbots for scheme information and guidance  
✅ **Educational Tools** - Training materials for public policy and administration  
✅ **Grant Application Systems** - Automated scheme identification for funding proposals

## 🎯 Major Government Schemes Covered

### Infrastructure Schemes
- **Jal Jeevan Mission (JJM)** - Tap water connections
- **Swachh Bharat Mission - Gramin (SBM-G)** - Sanitation and waste management
- **Pradhan Mantri Gram Sadak Yojana (PMGSY)** - Rural roads
- **Saubhagya Scheme** - Household electrification
- **Integrated Child Development Services (ICDS)** - Anganwadi infrastructure
- **Samagra Shiksha** - School infrastructure
- **Common Service Centers (CSC)** - Digital infrastructure

### Individual/Household Schemes
- **Pradhan Mantri Awaas Yojana - Gramin (PMAY-G)** - Housing
- **Pradhan Mantri Ujjwala Yojana (PMUY)** - LPG connections
- **Ayushman Bharat (PM-JAY)** - Health insurance
- **PM Jan Dhan Yojana (PMJDY)** - Bank accounts
- **PM Suraksha Bima Yojana (PMSBY)** - Accident insurance
- **PM Jeevan Jyoti Bima Yojana (PMJJBY)** - Life insurance
- **National Social Assistance Programme (NSAP)** - Social pensions
- **Pre-Matric and Post-Matric Scholarships** - SC student support
- **PM Kaushal Vikas Yojana (PMKVY)** - Skill training
- **DAY-NRLM** - Self-Help Groups and livelihoods
- **Mahatma Gandhi NREGA** - Employment generation
- **Soil Health Card Scheme** - Agricultural support

## ⚠️ Limitations

1. **Dataset Size** - Trained on 179 examples; coverage may be incomplete for edge cases
2. **Geographic Focus** - Primarily focused on Indian government schemes
3. **Output Repetition** - Model may occasionally generate repeated recommendations (requires post-processing)
4. **JSON Parsing** - Some outputs may need cleaning to extract valid JSON
5. **Scheme Updates** - Does not reflect scheme changes after training cutoff date (2024)
6. **Language** - Primarily English; limited Hindi understanding

## 🔮 Future Improvements

- [ ] Expand dataset to 500+ examples covering more states and districts
- [ ] Add scheme eligibility criteria and application procedures
- [ ] Include budget allocation recommendations
- [ ] Multi-language support (Hindi, regional languages)
- [ ] Integration with live scheme databases for real-time updates
- [ ] Retrieval-Augmented Generation (RAG) for hybrid recommendations
- [ ] State-specific scheme variants and customizations
- [ ] Mobile-optimized version for field workers

## 🛠️ Technical Stack

- **Framework:** Hugging Face Transformers + PEFT
- **Base Model:** Microsoft Phi-3-mini-4k-instruct
- **Training:** LoRA (Low-Rank Adaptation)
- **Quantization:** 8-bit during training
- **Hardware:** GPU (CUDA-enabled)
- **Languages:** Python

## 📄 Citation

If you use this model in your research or applications, please cite:

```bibtex
@misc{scheme-recommendation-model-2024,
  author = {Manas Patil},
  title = {Government Scheme Recommendation Model},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Manas281/scheme-recommendation-model-2}}
}
```

## 👨‍💻 Author

**Manas Patil**  
🔗 [Hugging Face Profile](https://huggingface.co/Manas281)  


## 📜 License

This model is released under the **Apache 2.0 License**, same as the base Phi-3 model.

## 🙏 Acknowledgments

- Microsoft for the Phi-3-mini-4k-instruct base model
- Hugging Face for the Transformers and PEFT libraries
- The open-source AI community for tools and resources

---

**Model Card Version:** 1.0  
**Last Updated:** January 2025  
**Status:** Production-ready for testing and evaluation