🪶 VedaGPT: LoRA Adapters (IndicVedas-LoRA)

This repository (shinigamiRaj/IndicVedas-LoRA) hosts the PEFT LoRA adapter weights for VedaGPT.

VedaGPT is fine-tuned on a rich corpus of ancient Indian Vedic literature (Rig Veda, Sama Veda, Yajur Veda, Atharva Veda) and classic Ayurvedic medicine treatises (Charaka Samhita, Sushruta Samhita, Rasa Jala Nidhi, IRJAY papers).

The base model is Qwen/Qwen2.5-14B-Instruct, and these adapters were trained using Unsloth on serverless Modal GPUs.

---

🏛️ Adapter Details

• Base Model: Qwen/Qwen2.5-14B-Instruct
• Adapter Type: LoRA (Low-Rank Adaptation)
• Max Sequence Length: 4096 tokens
• Training Framework: Unsloth & PEFT
• Parameters:
  • Rank (r): 64
  • Alpha: 64
  • Target Modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
  • Rank-Stabilized LoRA (rsLoRA): True

---

🛠️ Usage Instructions

1. Loading with Unsloth (Fastest & Easiest)

Unsloth is highly recommended for running inference or further training on these adapters.

from unsloth import FastLanguageModel

max_seq_length = 4096
dtype = None # None for auto-detection

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "shinigamiRaj/IndicVedas-LoRA", # Loads base model + adapter automatically
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = True, # Set to False for 16-bit
)
FastLanguageModel.for_inference(model)

# Generate using apply_chat_template
messages = [
    {
        "role": "system",
        "content": (
            "You are VedaGPT, an expert scholar of the ancient Vedic scriptures like RigVeda, SamaVeda, YajurVeda, AtharvaVeda, Charaka Samhita, Sushruta Samhita, Ayurveda, and Yoga. "
            "Answer questions accurately based on your knowledge of the Vedas, Upanishads, Charaka Samhita, Sushruta Samhita, and other classical Indian texts. "
            "Maintain the style of writing as per the ancient Vedic texts where required."
        )
    },
    {"role": "user", "content": "Tell me about the connection between Agni and Rig Veda Hymn 1."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
).to("cuda")

outputs = model.generate(
    input_ids=inputs,
    max_new_tokens=256,
    temperature=0.7,
    do_sample=True,
    top_p=0.9,
    repetition_penalty=1.15
)

response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)

2. Loading with standard PEFT & Transformers

If you aren't using Unsloth, load the base model and apply the adapters using the Hugging Face peft library.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-14B-Instruct"
adapter_id = "shinigamiRaj/IndicVedas-LoRA"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

# Build messages and apply chat template...

---

🔬 Training Configuration

• Hardware: Modal Serverless Cloud GPU (NVIDIA L40S)
• Quantization (during training): 4-bit NF4
• Parameters:
  • PEFT Rank (r): 64
  • LoRA Alpha: 64
  • Optimizer: adamw_8bit
  • Learning Rate: 2e-5 with cosine scheduler
  • Epochs: 1
  • Max Sequence Length: 4096 tokens