🙏 VedaGPT (IndicVedas) — 14B Merged & GGUF Model

VedaGPT (IndicVedas) is a specialized 14B parameter large language model designed for deep understanding, scholarly analysis, and interactive exploration of ancient Indian scriptures and classical texts. This model has undergone continuous pre-training/fine-tuning on a comprehensive, custom-scraped corpus of the four Vedas and foundational Ayurvedic texts.

This repository hosts the fully merged 16-bit bfloat16 weights along with optimized GGUF (Q4_K_M) quantizations for local deployment (e.g., via Ollama, llama.cpp).

If you are looking for the LoRA adapters, they can be found at shinigamiRaj/IndicVedas-LoRA.

---

🏛️ Corpus Composition & Sources

The model has been trained on a highly structured dataset containing:

• Rig Veda: Complete 10 Mandalas (Books), organized by Suktas (Hymns) and verses (Griffith Translation).
• Sama Veda: Part I (Mula/Decades) and Part II (Hymns) (Griffith Translation & Sanskrit/Hindi bilingual texts).
• Yajur Veda:
  • Black Yajur Veda (Taittiriya Samhita): English translation by Arthur Berriedale Keith.
  • White Yajur Veda (Vajasaneya Samhita): English translation by Ralph T.H. Griffith.
• Atharva Veda: 20 Books, Suktas, and Verses (Griffith Translation & bilingual texts).
• Ayurveda Corpus:
  • Charaka Samhita: Structural chapters on diagnosis, anatomy, therapeutics, and pharmacology.
  • Sushruta Samhita: Ancient surgical treatises, clinical procedures, and anatomy.
  • Rasa Jala Nidhi: Comprehensive works on Indian alchemy, mineralogy, and metallurgy.
  • IRJAY (International Research Journal of Ayurveda and Yoga).

---

⚙️ Model Details & Training Configurations

The model is based on Qwen2.5-14B-Instruct, optimized for fast patching and fine-tuning via Unsloth.

Hyperparameters & Fine-Tuning Recipe:

• Fine-Tuning Framework: Unsloth (2x faster, memory-efficient finetuning).
• Strategy: Aggressive knowledge overriding (resumed LoRA adaptation on 100% of the corpus with no train/val split).
• Epochs: 2.0 (to ensure deep familiarity with Sanskrit translations and terminology).
• Learning Rate: 5e-5 (Linear warm-up for 5% of training steps, followed by Cosine decay).
• Optimizer: adamw_8bit
• Weight Decay: 0.05 (for stable generalization across diverse prompts).
• Effective Batch Size: 8 (Batch size of 4 per device with gradient_accumulation_steps=2).
• Context Length: Trained with a context length of 4096 tokens (supports up to 16,384 during inference).
• Hardware: Fine-tuned on serverless Cloud GPUs (NVIDIA L40S) using Modal.

---

🚀 Deployment & Local Usage

The model is available in two formats:

1. Merged 16-bit (bfloat16): Best for cloud hosting, vLLM servers, or high-end GPU desktops.
2. Quantized GGUF (Q4_K_M): Optimized for fast CPU/GPU execution on consumer laptops (requires ~9GB VRAM/RAM).

Running with Ollama

An Ollama Modelfile is uploaded in the repository. To run VedaGPT locally:

1. Install Ollama.
2. Download unsloth.Q4_K_M.gguf from this repository.
3. Create a local file named Modelfile containing:

   FROM ./unsloth.Q4_K_M.gguf
   TEMPLATE """{{ if .System }}<|im_start|>system
   {{ .System }}<|im_end|>
   {{ end }}{{ if .Prompt }}<|im_start|>user
   {{ .Prompt }}<|im_end|>
   {{ end }}<|im_start|>assistant
   {{ .Response }}<|im_end|>
   """
   SYSTEM """You are VedaGPT, an expert scholar of the ancient Vedic scriptures like RigVeda, SamaVeda, YajurVeda, AtharvaVeda, Charaka Samhita, Sushruta Samhita, Rasa Jala Nidhi, IRJAY (International Research Journal of Ayurveda and Yoga). Answer questions accurately based on your knowledge of the Vedas, Upanishads, Charaka Samhita, Sushruta Samhita, and other classical Indian texts. Maintain the style of writing as per the ancient Vedic texts where required."""
   PARAMETER stop "<|im_end|>"
   PARAMETER stop "<|im_end|>"
   

4. Build the model:

   ollama create vedagpt -f Modelfile
   

5. Run the model:

   ollama run vedagpt
   

Running with vLLM

VedaGPT is compatible with vLLM for high-throughput serving:

python -m vllm.entrypoints.openai.api_server \
    --model shinigamiRaj/IndicVedas \
    --dtype bfloat16     --max-model-len 4096

---

📝 Prompt Template & ChatML Format

This model uses the standard ChatML template:

<|im_start|>system
You are VedaGPT, an expert scholar of the ancient Vedic scriptures...<|im_end|>
<|im_start|>user
What does the Charaka Samhita say about digestion?<|im_end|>
<|im_start|>assistant
...

---

🏛️ Acknowledgements & License

• We express deep gratitude to the digital Indology initiatives, translator projects, and open archiving systems that made the digitized translations available.
• License: Released under public domain/creative commons license for educational, cultural, and historical research purposes.