Update README.md

README.md

@@ -14,122 +14,91 @@ tags:
 - custom-finetune
 - lor-merged
 base_model: unsloth/Qwen2.5-Math-1.5B
+datasets:
+- Xerv-AI/GRAD
 ---
 
-# Qwen2.5-Math-1.5B: Generalized & Merged (Phase-Technologies)
-
-An ultra-lightweight, high-speed reasoning model heavily fine-tuned for graduate-level mathematical proofs while retaining full conversational generalization and fundamental safety alignments. 
-
-This model was fine-tuned using **Unsloth** on a standard Google Colab T4 GPU, utilizing Low-Rank Adaptation (LoRA) and subsequently merged into a full 16-bit standalone model for seamless deployment.
-
-## 🚀 Model Overview
-
-Standard math-specific LLMs often suffer from **catastrophic forgetting**—when prompted with basic conversational queries, they either hallucinate lengthy pseudo-proofs or fail entirely. 
-
-This model was engineered to solve that problem. It bridges the gap between hyper-specialized mathematical reasoning and general instruction-following by utilizing a carefully balanced, dual-distribution training dataset.
-
-* **Primary Capability:** Capable of outputting highly detailed, step-by-step proofs for advanced algebraic, topological, and geometric problems.
-* **Secondary Capability:** Capable of standard, conversational instruction-following without format-forced hallucinations.
-* **Hardware Optimized:** Designed to run at maximum throughput on low-VRAM environments (like the 16GB T4 GPU) using 4-bit quantization and Triton kernels.
-
-## 🏗 Architecture & Training Methodology
-
-### Base Model
-* **Original Architecture:** [Qwen2.5-Math-1.5B](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B)
-* **Parameters:** 1.5 Billion
-* **Precision:** Merged 16-bit (Fine-tuned in 4-bit via Unsloth)
-
-### The Dataset: Dual-Distribution Blending
-To achieve generalization, the model was fine-tuned on a 50/50 blend of two distinct datasets, batched and streamed via high-throughput Parquet files:
-1.  **Xerv-AI/GRAD (1.93k rows):** A synthetic dataset containing exceptionally long (average 8,000 characters) graduate and research-level mathematical proofs formatted in strict LaTeX.
-2.  **yahma/alpaca-cleaned (2k rows):** A subset of the standard Alpaca dataset used to teach the model how to answer basic queries, roleplay, and recognize when *not* to use complex math.
-
-### Training Configuration
-The fine-tuning process was executed via Supervised Fine-Tuning (SFT) targeting the attention mechanisms.
-* **Target Modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
-* **LoRA Rank (r):** 16
-* **LoRA Alpha:** 16
-* **Optimizer:** `adamw_8bit`
-* **Learning Rate:** 2e-4
-* **Effective Batch Size:** 8 (Batch size 2 with 4 Gradient Accumulation steps)
-
-## 🛡️ Safety & Alignment
-
-Despite being fine-tuned on unfiltered mathematical and conversational data, **this model retains its original safety alignment**. Because only 1-2% of the parameters were updated via LoRA (and later merged), the base Qwen2.5 weights responsible for safety remain fully intact.
-
-* **NSFW/18+ Prompts:** The model will actively refuse to generate explicit, illegal, or harmful content, relying on the RLHF and DPO safety guardrails instilled during its original pre-training phase.
-
-## 💻 Usage & Inference
-
-The model is highly responsive to a strict Instruction/Response template. For best results, use a `repetition_penalty` of roughly 1.15 to prevent the model from infinitely looping through math steps on simpler problems.
-
-### Installation
+## 🌌 Xerv-AI/Ada: The Multi-Modal Mathematical Generalist SLM
+**Ada** is an ultra-lightweight, high-speed, and highly optimized reasoning Small Language Model (SLM) derived from the powerful **Qwen2.5-Math-1.5B** architecture. Engineered specifically to bridge the gap between hyper-specialized graduate-level mathematical proofs and standard conversational utility, Ada solves the notorious "catastrophic forgetting" problem often found in math-heavy fine-tunes.
+Whether you need a step-by-step calculus breakdown, a topological proof in LaTeX, or just a simple conversational assistant for daily tasks, Ada delivers state-of-the-art performance for a 1.5 Billion parameter model.
+
+### 🚀 Model Overview 
+Standard math-specific LLMs frequently suffer from domain overfitting. When prompted with basic conversational queries, they either hallucinate lengthy pseudo-proofs or fail entirely to understand the user's intent. **Xerv-AI/Ada** was meticulously engineered to resolve this by utilizing a carefully balanced, dual-distribution training dataset, allowing it to act as both a rigorous STEM assistant and a general-purpose chat model.
+
+| Specification | Details |
+| :--- | :--- |
+| **Model Name** | Xerv-AI/Ada |
+| **Base Architecture** | unsloth/Qwen2.5-Math-1.5B |
+| **Parameter Count** | 1.5 Billion |
+| **Primary Capabilities** | Graduate-level STEM reasoning, logical deduction, and mathematical proofs. |
+| **Secondary Capabilities** | General conversational instruction-following, roleplay, and basic coding. |
+| **Training Framework** | QLoRA via Unsloth (Triton kernels). |
+| **Precision** | Merged 16-bit (Fine-tuned in 4-bit). |
+| **License** | Apache-2.0 | <br> ### 🔬 Core Capabilities & Strengths <br> * **Balanced Generalization:** Ada seamlessly transitions between casual conversation and intense analytical problem-solving without format-forced hallucinations. <br> * **Advanced STEM Reasoning:** Fully optimized to generate detailed, multi-step logical proofs in advanced algebra, calculus, topology, and physics. <br> * **Hardware Optimized for Edge Deployment:** Designed to run at maximum inference throughput on low-VRAM consumer hardware (such as a single 16GB NVIDIA T4 GPU, Mac M-series chips, or edge devices) using 4-bit quantization. <br> * **Impeccable Formatting:** Native understanding of structural formatting, easily outputting highly readable markdown and structured logic steps. <br> ### 🏗 Architecture & Training Methodology <br> Ada was trained using Supervised Fine-Tuning (SFT) targeting the attention mechanisms of the base model. Utilizing **Unsloth** on a standard Google Colab NVIDIA T4 GPU, the training leveraged Low-Rank Adaptation (LoRA) to maximize efficiency before being merged into a standalone 16-bit Hugging Face model. <br> * **Target Modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj <br> * **LoRA Rank (r):** 16 <br> * **LoRA Alpha:** 16 <br> * **Optimizer:** adamw_8bit <br> * **Learning Rate:** 2e-4 <br> * **Effective Batch Size:** 8 (Batch size 2 with 4 Gradient Accumulation steps) <br> ### 📚 The Dataset: Dual-Distribution Blending <br> To achieve generalization and prevent catastrophic forgetting, Ada was fine-tuned on a strict 50/50 blend of two distinct datasets, batched and streamed via high-throughput Parquet files:
+| Dataset | Sample Size | Description & Purpose |
+| :--- | :--- | :--- |
+| **Xerv-AI/GRAD** | ~1.93k rows | A proprietary synthetic dataset containing exceptionally long (average 8,000 characters) graduate and research-level mathematical proofs. This instills deep reasoning and strict formatting. |
+| **yahma/alpaca-cleaned** | ~2.00k rows | A refined subset of the standard Alpaca dataset. This teaches the model conversational flow, roleplay, basic Q&A, and crucially, *when not to use complex math*. |
+
+### 💻 Usage & Python Inference Guide
+The model is highly responsive to the standard **Alpaca Instruction/Response template**.
+**Important Inference Note:** For best results, use a repetition_penalty of roughly **1.15**. This acts as a crucial guardrail to prevent the model from infinitely looping through mathematical steps on overly simple arithmetic queries.
+**1. Installation Requirements**
 ```bash
-pip install unsloth transformers accelerate
-
+pip install unsloth transformers accelerate torch
 ```
-### Python Inference Script
+**2. Fast Inference Script**
 ```python
 from unsloth import FastLanguageModel
 import torch
-
-# 1. Configuration
-repo_name = "Phase-Technologies/qwen2.5-math-1.5b-generalized-merged"
+# Configuration
+repo_name = "Xerv-AI/Ada"
 max_seq_length = 2048
-
-# 2. Load the fully merged model
-# Loading in 4-bit is highly recommended for low-VRAM GPUs (like T4)
+# Load the model and tokenizer (4-bit recommended for low-VRAM)
 model, tokenizer = FastLanguageModel.from_pretrained(
     model_name = repo_name,
     max_seq_length = max_seq_length,
     dtype = None,
     load_in_4bit = True, 
 )
-
-# 3. Switch to highly optimized inference mode
+# Enable optimized inference mode
 FastLanguageModel.for_inference(model)
-
-# 4. Define the universal prompt template
+# Define the universal prompt template
 universal_prompt = """### Instruction:
 {}
-
 ### Response:
 {}"""
-
-# 5. Prepare your query
-query = "Provide a step-by-step proof finding the eigenvalues of the matrix [[2, 1], [1, 2]]."
-
+# Prepare your query
+query = "Provide a step-by-step logical proof finding the eigenvalues of the matrix [[2, 1], [1, 2]]."
 inputs = tokenizer(
     [universal_prompt.format(query, "")],
     return_tensors = "pt"
 ).to("cuda")
-
-print("Generating response...")
-
-# 6. Generate the output
+print("Generating analytical response...")
+# Generate the output
 outputs = model.generate(
     **inputs,
     max_new_tokens = 1024,
-    max_length = None,           # Bypasses Hugging Face length warnings
+    max_length = None,           
     use_cache = True,
-    repetition_penalty = 1.15,   # Critical: prevents infinite generation loops
+    repetition_penalty = 1.15,   # Critical: prevents generation loops
     pad_token_id = tokenizer.eos_token_id
 )
-
-# 7. Decode and print the result
+# Decode and print the result
 response = tokenizer.batch_decode(outputs, skip_special_tokens = True)[0]
 print(f"\n{'='*50}\nOutput:\n{'='*50}")
 print(response.split("### Response:\n")[-1])
-
-```
-## 📊 Limitations & Biases
- * **Language:** The model is optimized exclusively for English.
- * **Arithmetic Hallucinations:** While highly capable of symbolic logic and structured proofs, 1.5B parameter models can occasionally suffer from minor arithmetic errors (e.g., simple subtraction mistakes) deep within long proofs.
- * **Prompt Sensitivity:** The model performs best when math queries explicitly ask for a "proof" or "step-by-step" breakdown in the instruction block.
-## 🤝 Acknowledgements
- * **Alibaba Cloud:** For the phenomenal base Qwen2.5-Math architecture.
- * **Unsloth AI:** For the Triton-optimized training kernels that made compiling this model possible on consumer hardware.
- * **Xerv-AI:** For the GRAD synthetic dataset powering the advanced reasoning capabilities.
-```
-
 ```
+### 🛡️ Safety & Alignment Guardrails
+Despite being fine-tuned on raw mathematical logic and conversational instruction data, Ada successfully retains its foundational safety alignments. Because only 1% to 2% of the parameters were actively updated via LoRA (and subsequently merged), the original base Qwen2.5 weights responsible for safety remain fully intact.
+ * **Content Moderation:** The model actively refuses to generate explicit, illegal, or harmful content, relying on the RLHF and DPO safety guardrails instilled during Alibaba's original pre-training phase.
+### ⚠️ Limitations & Known Biases
+While Ada punches well above its 1.5B weight class, it is important to acknowledge the limitations inherent to Small Language Models:
+ * **Arithmetic Hallucinations:** Ada is exceptionally capable at symbolic logic, structural breakdowns, and mathematical theory. However, like many SLMs, it can occasionally suffer from minor arithmetic errors (e.g., basic addition/subtraction mistakes) deep within multi-page proofs. Always verify raw calculations.
+ * **Language Constraint:** The model is optimized exclusively for **English** text and standard mathematical notation.
+ * **Prompt Sensitivity:** Ada performs at its absolute peak when math queries explicitly ask for a "proof," "step-by-step breakdown," or "logical analysis" within the instruction block.
+ * **World Knowledge:** It lacks the broad, encyclopedic trivia knowledge found in massive 70B+ parameter models.
+### 🤝 Acknowledgements
+ * **Alibaba Cloud:** For the phenomenal, state-of-the-art base Qwen2.5-Math architecture.
+ * **Unsloth AI:** For the Triton-optimized training kernels that made compiling and fine-tuning this model possible and highly efficient on consumer hardware.
+ * **Xerv-AI:** For the curation of the GRAD synthetic dataset powering the advanced reasoning capabilities.