---
language:
- en
license: apache-2.0
base_model: Qwen/Qwen2.5-3B-Instruct
tags:
- finance
- banking
- rbi
- regulation
- india
- qwen2.5
- lora
datasets:
- Vishva007/RBI-Circular-QA-Dataset
metrics:
- accuracy
---

# Qwen2.5-3B-Instruct Fine-tuned for RBI Regulations Q&A

A specialized model for answering questions about **Reserve Bank of India (RBI) regulations and banking policies**.

**Performance**: 57.6% accuracy (8.2x improvement over base model's 7%)

## Quick Facts

- 🎯 **Accuracy**: 57.6% on 1000-sample evaluation set
- 📚 **Coverage**: 100+ regulation areas (Basel III, FEMA, AML, PSL, etc.)
- 🚀 **Training**: 47K QA pairs with rephrased variants
- ⚡ **Efficient**: 3B parameters, optimized for deployment


## Performance Highlights

| Category | Base Model | Fine-tuned | Improvement |
|----------|-----------|-----------|-------------|
| Overall | 7.0% | **57.6%** | +50.6% |
| Fact-based | 6.8% | **57.6%** | +50.8% |
| Reasoning | 37.5% | **62.5%** | +25.0% |

**Top Categories** (70%+ accuracy): Anti-Money Laundering, Digital Payments, Government Banking, MSME Finance, Currency Management

## Training Details

- **Method**: LoRA fine-tuning (r=16, alpha=32)
- **Dataset**: [RBI-Circular-QA-Dataset](https://huggingface.co/datasets/Vishva007/RBI-Circular-QA-Dataset) (47K samples)
- **Training**: 1 epoch, 2 hours on NVIDIA L40S
- **Loss**: 0.79 → 0.57 (train), 0.58 (eval)

## Code & Resources

- 💻 **Training Code**: [GitHub Repository](https://github.com/vishvaRam/Unsloth-FineTuning.git)
- 📊 **Dataset**: [RBI-Circular-QA-Dataset](https://huggingface.co/datasets/Vishva007/RBI-Circular-QA-Dataset)
- 🤗 **Base Model**: [Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)

## 📝 Technical Deep Dive

Want to understand the theory and methodology behind this model?

**Read the full article:** [Fine-tuning Qwen 2.5 3B for RBI Regulations: Achieving 8x Performance with Smart Data Augmentation](https://dev.to/vishva_ram/fine-tuning-qwen-25-3b-for-rbi-regulations-achieving-8x-performance-with-smart-data-augmentation-5e38)

The article covers:
- 🧠 **LoRA Theory**: Why and how Low-Rank Adaptation works
- ⚡ **Unsloth Deep Dive**: Technical advantages and performance optimizations
- 📊 **Data Augmentation Strategy**: The rephrasing technique that delivered 40% of improvement
- 🎓 **Hyperparameter Analysis**: Detailed explanation of every training choice
- 📈 **Evaluation Methodology**: Stratified sampling and LLM-as-judge approach
- 🔬 **Ablation Studies**: What really mattered for the 8x improvement

Perfect for ML engineers who want to replicate or adapt this approach for their own domain-specific fine-tuning projects.

## Use Cases

✅ Banking compliance chatbots  
✅ Regulatory Q&A systems  
✅ Training tools for banking professionals  
✅ RBI circular analysis  

⚠️ Not for: Legal compliance decisions (requires expert review), real-time updates

## Limitations

- Knowledge cutoff: 2024 regulations
- 57.6% accuracy means ~42% of complex queries need verification
- Optimized for English only

## Citation

```
  @misc{
    qwen25-3b-rbi-qa,
    author = {Vishva007},
    title = {Qwen2.5-3B-Instruct Fine-tuned for RBI Regulations Q&A},
    year = {2025},
    publisher = {HuggingFace},
    howpublished = {\url{https://huggingface.co/Vishva007/Qwen2.5-3B-Instruct-RBI-QA}},
  }
```


## License

Apache 2.0 (inherits from base model)

---

**Author**: [@Vishva007](https://huggingface.co/Vishva007) | **Updated**: November 2025