adaption_finance_local_devnagri_scrip
A LoRA adapter fine-tuned on top of meta-llama/Llama-4-Scout-17B-16E-Instruct for finance in local Devanagari-script languages.
Model Details
- Base model: meta-llama/Llama-4-Scout-17B-16E-Instruct (109B)
- Relation to base: LoRA adapter (PEFT)
- Training method: Supervised fine-tuning (SFT)
- Training type: LoRA
- Data format: chat
- Domain: Finance, local Devanagari-script languages
Training metrics
| Metric | base | adapted |
|---|---|---|
| Win rate (your dataset) | 19 | 81 |
| Win rate (Personal Finance category) | 26 | 74 |
LoRA Configuration
| Parameter | Value |
|---|---|
| lora_r | 64 |
| lora_alpha | 128 |
| lora_dropout | 0 |
| task type | CAUSAL_LM |
| trainable modules | k_proj, o_proj, q_proj, v_proj, shared_expert.gate_proj, shared_expert.up_proj, shared_expert.down_proj, feed_forward.gate_proj, feed_forward.up_proj, feed_forward.down_proj |
Training Hyperparameters
| Parameter | Value |
|---|---|
| n_epochs | 5 |
| n_evals | 5 |
| batch_size | max |
| learning_rate | 0.0001 |
| lr_scheduler_type | cosine |
| scheduler_num_cycles | 0.5 |
| min_lr_ratio | 0.1 |
| warmup_ratio | 0.05 |
| weight_decay | 0.02 |
| max_grad_norm | 1 |
| train_on_inputs | false |
How to Get Started
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "meta-llama/Llama-4-Scout-17B-16E-Instruct"
adapter = "sidddd625/adaption_finance_local_devnagri_scrip"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
Job Metadata
- finetune_job_id: 032b1886-fea8-4c26-bea4-fdec65363a20
- training_experiment_id: 8e30f154-c295-47f8-a759-52f799cf36f9
- trained_model_name: adaption_finance_local_devnagri_scrip
Framework versions
- PEFT 0.15.1
- Downloads last month
- 38
Model tree for sidddd625/adaption_finance_local_devnagri_scrip
Base model
meta-llama/Llama-4-Scout-17B-16E