zo0302/indian_legal_bot

🇮🇳 Indian Legal Assistant (QLoRA Fine-tuned)

Model Description

This model is a QLoRA fine-tuned version of:

Base Model: unsloth/Llama-3.2-3B-Instruct-bnb-4bit

It has been fine-tuned on Indian legal and constitutional Q&A data to improve domain-specific understanding of:

• Indian Constitution articles
• Legal provisions
• Governance structures
• Real estate & regulatory references

---

Training Details

• Training Method: QLoRA (4-bit quantization + LoRA adapters)
• Framework: Unsloth + HuggingFace TRL (SFTTrainer)
• Sequence Length: 2048
• LoRA Rank (r): 16
• Learning Rate: 2e-4 with cosine decay
• Batch Size: 1 (with gradient accumulation)
• Training Steps: 100 (~0.1 epoch)
• Optimizer: paged_adamw_32bit

---

Training Behavior

During training:

• Training loss decreased from ~2.6 to ~1.5
• Gradient norms remained stable (1.1–1.8 range)
• Learning rate decayed smoothly using cosine scheduling
• No gradient explosions or instability observed

The training was stable and numerically well-behaved.

Note: The model was trained for a limited number of steps and may benefit from additional epochs for stronger domain adaptation.

---

Intended Use

This model is designed for:

• Educational purposes
• Legal information assistance
• Constitutional Q&A
• Research exploration

⚠️ This model does NOT provide legal advice. Always consult a qualified legal professional.

---

How To Use

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "zo0302/indian_legal_bot" 

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto",
)

prompt = "Explain Article 243W of Indian Constitution."

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Training Metrics