Law QLoRA Adapter

This is a LoRA adapter finetuned on Sri Lankan Law instruction data.

Base Model

Qwen/Qwen2.5-1.5B-Instruct

Dataset

Shifaur/sri_lanka_constitutional_law_qa (subsample: 500)

Training Hyperparameters

LoRA rank: 16
LoRA alpha: 32
Learning rate: 2e-05
Max steps: 250
Batch size (effective): 64
Max length: 512

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "path/to/adapter")
tokenizer = AutoTokenizer.from_pretrained("path/to/adapter")

Limitations

Educational use only - NOT for actual law practises
Trained on limited data subsample
May produce incorrect or harmful information
Always consult qualified law professionals for actions

License & Attribution

This adapter inherits the license of the base model and dataset. Check those licenses before use or redistribution.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support