| --- |
| license: |
| - apache-2.0 |
| - mit |
| base_model: |
| - Qwen/Qwen2.5-0.5B-Instruct |
| - Qwen/Qwen2.5-Coder-3B-Instruct |
| - HuggingFaceTB/SmolVLM-Instruct |
| language: |
| - en |
| datasets: |
| - tatsu-lab/alpaca |
| - databricks/databricks-dolly-15k |
| - teknium/OpenHermes-2.5 |
| - stingning/ultrachat |
| metrics: |
| - accuracy |
| library_name: transformers |
| tags: |
| - text-generation |
| - custom-slm |
| - conversational |
| - agentic-router |
| - qlora |
| - chroma-rag |
| - local-ai |
| - pytorch |
| - vision |
| - multimodal |
| - code-generation |
| - vlm |
| - vqa |
| - ocr |
| --- |
| |
| # Zenthi-AI OS: Agentic Multi-Model Small Language Model (SLM) |
|
|
| Zenthi-AI is a production-grade, custom fine-tuned Small Language Model (SLM) conversational assistant. It is optimized for high-speed, local-first execution and acts as the intent-routing brain and synthesis engine of the **Zenthi-AI Multi-Model Operating System**. |
|
|
| This repository hosts the merged, full-precision model weights. |
|
|
| --- |
|
|
| ## 🚀 Key Features |
|
|
| * **Base Foundation**: Built on the highly capable `Qwen/Qwen2.5-0.5B-Instruct`. |
| * **Parameter-Efficient Finetuning**: Optimized via QLoRA (4-bit quantization) on a merged, cleaned dataset of Alpaca, Dolly 15K, OpenHermes, UltraChat, and ShareGPT. |
| * **Agentic Orchestrator Routing**: Tuned specifically to act as a Router and Planner Agent, classifying query intents with high accuracy (CODE, VISION, RAG, SEARCH, KNOWLEDGE, COMPLEX). |
| * **Quantization-Ready**: Quantized to GGUF format for local deployment (quantized size under 500 MB). |
| * **Local RAG Integration**: Built to work in tandem with local ChromaDB embedding vector stores. |
| * **Web Search Coordination**: Designed to synthesize real-time context fetched from local SearXNG search clients. |
| * **Memory Management**: Keeps a windowed session history for conversational continuity. |
|
|
| --- |
|
|
| ## 📊 Evaluation & Routing Performance |
|
|
| The model's semantic routing accuracy was benchmarked across **500 unique evaluation test queries** (100 queries per intent category) running on a local GPU: |
|
|
| * **Overall Routing Accuracy**: **72.60%** |
| * **Average Latency**: **651.54 ms** per query |
|
|
| | Intent Category | Accuracy (%) | Target Expert Model | |
| | :--- | :--- | :--- | |
| | **CODE** | **100.00%** | `qwen2.5-coder:3b` | |
| | **VISION** | **100.00%** | `riven/smolvlm:latest` | |
| | **SEARCH** | **99.00%** | `qwen2.5:1.5b-instruct` | |
| | **RAG** | **43.00%** | `qwen2.5:1.5b-instruct` | |
| | **KNOWLEDGE** | **21.00%** | `qwen2.5:1.5b-instruct` | |
|
|
| --- |
|
|
| ## 💻 Local Usage & Integration |
|
|
| ### 1. Ollama Deployment (GGUF) |
| To run Zenthi-AI locally in Ollama: |
|
|
| 1. Create a `Modelfile` with the system prompt: |
| ```dockerfile |
| FROM zenthi-ai:latest |
| PARAMETER temperature 0.7 |
| PARAMETER top_p 0.9 |
| SYSTEM """I am Zenthi-AI OS, a production-grade Agentic Multi-Model AI Operating System. I deliver accurate, secure, maintainable, and production-ready solutions by coordinating specialized AI capabilities.""" |
| ``` |
| 2. Build and run: |
| ```bash |
| ollama create Zenthi-AI -f Modelfile |
| ollama run Zenthi-AI |
| ``` |
|
|
| ### 2. Python Transformers API |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_name = "KATHIR2006/zenthi-ai" |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto") |
| |
| # Start conversation |
| messages = [ |
| {"role": "system", "content": "You are Zenthi-AI OS, a production-grade Agentic Multi-Model AI Operating System."}, |
| {"role": "user", "content": "Explain photosynthesis simply."} |
| ] |
| |
| text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| inputs = tokenizer([text], return_tensors="pt").to("cuda") |
| |
| generated_ids = model.generate(**inputs, max_new_tokens=512) |
| response = tokenizer.batch_decode(generated_ids[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0] |
| print(response) |
| ``` |
|
|
| --- |
|
|
| ## 🛠️ Fine-Tuned Expert Adapters |
|
|
| This repository also hosts the fine-tuned LoRA adapters for the specialized expert models of the Zenthi-AI OS: |
|
|
| ### 1. Code Expert Adapters (`code-adapters/`) |
| * **Base Model**: `Qwen/Qwen2.5-Coder-3B-Instruct` |
| * **Dataset**: Custom programming and instruction dataset (1,200 training steps) |
| * **Final Loss**: **0.1843** |
| * **Usage**: Optimized for React, Node.js, Python, MERN stack development, reviews, and refactoring. |
|
|
| ### 2. Vision Expert Adapters (`vision-adapters/`) |
| * **Base Model**: `HuggingFaceTB/SmolVLM-Instruct` |
| * **Dataset**: Synthetic VQA shape and color recognition dataset (100 training steps) |
| * **Final Loss**: **0.9077** |
| * **Usage**: Fine-tuned for OCR, visual question-answering, and image analysis. |
|
|
| --- |
|
|
| ## ⚖️ Licenses & Compliance |
|
|
| This project is dual-licensed: |
| * **LLM Model Weights & Adaptations**: Licensed under the **Apache License 2.0** (in compliance with the base Qwen2.5 license). |
| * **RAG Engine, Multi-Agent Framework, & Backend Codebase**: Licensed under the **MIT License**. |
|
|
|
|
