README.md

---
base_model: meta-llama/Llama-4-Scout-17B-16E-Instruct
base_model_relation: adapter
library_name: peft
license: other
tags:
- lora
- peft
- sft
- finance
- devanagari
- llama-4
pipeline_tag: text-generation
---

# adaption_finance_local_devnagri_scrip

A LoRA adapter fine-tuned on top of [meta-llama/Llama-4-Scout-17B-16E-Instruct](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct) for finance in local Devanagari-script languages.

## Model Details

- **Base model:** [meta-llama/Llama-4-Scout-17B-16E-Instruct](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct) (109B)
- **Relation to base:** LoRA adapter (PEFT)
- **Training method:** Supervised fine-tuning (SFT)
- **Training type:** LoRA
- **Data format:** chat
- **Domain:** Finance, local Devanagari-script languages

### Training metrics

| Metric | base | adapted |
|---|---|---|
| Win rate (your dataset) | 19 | 81 |
| Win rate (Personal Finance category) | 26 | 74 |  



![image](https://cdn-uploads.huggingface.co/production/uploads/68ac99f6c49918962e55ddc2/kcN2rHg1b3uJbCIn22-_h.png) 


![image](https://cdn-uploads.huggingface.co/production/uploads/68ac99f6c49918962e55ddc2/cn9qY4fKP-v_jev28CJTe.png)

## LoRA Configuration

| Parameter | Value |
|---|---|
| lora_r | 64 |
| lora_alpha | 128 |
| lora_dropout | 0 |
| task type | CAUSAL_LM |
| trainable modules | k_proj, o_proj, q_proj, v_proj, shared_expert.gate_proj, shared_expert.up_proj, shared_expert.down_proj, feed_forward.gate_proj, feed_forward.up_proj, feed_forward.down_proj |

## Training Hyperparameters

| Parameter | Value |
|---|---|
| n_epochs | 5 |
| n_evals | 5 |
| batch_size | max |
| learning_rate | 0.0001 |
| lr_scheduler_type | cosine |
| scheduler_num_cycles | 0.5 |
| min_lr_ratio | 0.1 |
| warmup_ratio | 0.05 |
| weight_decay | 0.02 |
| max_grad_norm | 1 |
| train_on_inputs | false |

## How to Get Started

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "meta-llama/Llama-4-Scout-17B-16E-Instruct"
adapter = "sidddd625/adaption_finance_local_devnagri_scrip"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
```

## Job Metadata

- **finetune_job_id:** 032b1886-fea8-4c26-bea4-fdec65363a20
- **training_experiment_id:** 8e30f154-c295-47f8-a759-52f799cf36f9
- **trained_model_name:** adaption_finance_local_devnagri_scrip

### Framework versions

- PEFT 0.15.1