Update README.md

README.md

@@ -1,286 +1,141 @@
 ---
 license: apache-2.0
-language:
-- en
-base_model:
-- likithyadavv/codementor-7b
-pipeline_tag: text-generation
-library_name: transformers
+base_model: likithyadavv/codementor-7b
 tags:
-- code
-- code-generation
-- code-explanation
-- bug-detection
-- lora
-- peft
-- 4bit
-- qlora
-- fullstack
-- python
-- javascript
-- fastapi
-- codementor
-metrics:
-- accuracy
+  - lora
+  - qlora
+  - peft
+  - code
+  - education
+  - socratic-tutoring
+  - qwen2.5-coder
+language:
+  - en
+library_name: peft
 ---
 
-# 🤖 CodeMentor V2 — Fullstack AI Code Assistant
+# CodeMentor V2 — Full-Stack (codementor-v2-fullstack)
 
-> **Code Smarter. Debug Faster. Learn Better.**
+A QLoRA-fine-tuned LoRA adapter that extends **CodeMentor**, a Socratic programming tutor, from 4 foundational languages to **17 full-stack technologies**.
 
-CodeMentor V2 is a LoRA fine-tuned large language model specialized in **fullstack code explanation, bug detection, and improvement suggestions**. Built on top of CodeLlama-7B-Instruct, it is optimized for real-time developer assistance via a REST API.
+This adapter is **Phase 2** of a two-phase continued fine-tuning pipeline. It is not trained from scratch — it loads the Phase 1 checkpoint ([`likithyadavv/codementor-7b`](https://huggingface.co/likithyadavv/codementor-7b)) as its base and attaches a new, larger LoRA adapter on top.
 
----
+## Model Tree
 
-## 📋 Model Details
+```
+Qwen/Qwen2.5-7B
+  └── Qwen/Qwen2.5-Coder-7B
+        └── Qwen/Qwen2.5-Coder-7B-Instruct
+              └── likithyadavv/codementor-7b        (Phase 1: 4 languages)
+                    └── likithyadavv/codementor-v2-fullstack   (Phase 2: 17 technologies — this adapter)
+```
 
-| Property | Value |
-|---|---|
-| **Model Type** | Causal Language Model (LoRA Adapter) |
-| **Base Model** | `codellama/CodeLlama-7b-Instruct-hf` |
-| **Fine-Tuning Method** | QLoRA (4-bit quantization + LoRA) |
-| **LoRA Rank** | 16 |
-| **Training Framework** | HuggingFace PEFT + TRL |
-| **Language** | English |
-| **License** | Apache 2.0 |
-| **Adapter Size** | ~162 MB |
+## What This Model Does
 
----
+CodeMentor is designed to **teach**, not just answer. Given a student's code snippet and a question, it:
+1. Identifies the language or technology
+2. Acknowledges what the student got right
+3. Explains the problem conceptually — without handing over the corrected solution
+4. Closes with a guiding question that pushes the student to reason through the fix themselves
 
-## 🎯 Intended Use
+## Technologies Covered
 
-CodeMentor V2 is designed for:
+**Phase 1 (retained via dataset replay):** Python, Java, C, C++
 
-- **Code Explanation** — Understand what a block of code does in plain English
-- **Bug Detection** — Identify logic errors, missing base cases, off-by-ones, etc.
-- **Code Improvement** — Suggest better patterns, optimizations, and best practices
-- **Fullstack Q&A** — Answer programming questions across Python, JavaScript, and more
-- **Developer Mentorship** — Act as an always-available senior developer
+**Phase 2 (new in this adapter):** HTML/CSS, JavaScript, TypeScript, React, Next.js, Node.js, Express, FastAPI, Django, Spring Boot, SQL, MongoDB, Docker, Git, REST APIs
 
----
+## Training Details
+
+| | Phase 1 (base model) | Phase 2 (this adapter) |
+|---|---|---|
+| Base | Qwen2.5-Coder-7B-Instruct | codementor-7b |
+| Dataset size | 505 | 8,000 (6,415 new + 1,585 replayed) |
+| LoRA rank (r) | 16 | 32 |
+| LoRA alpha | 32 | 64 |
+| Target modules | q, k, v, o | q, k, v, o, gate, up, down |
+| Trainable params | ~40M (0.5%) | ~80M (1.05%) |
+| Quantization | 4-bit NF4 | 4-bit NF4 + double quantization |
+| Epochs | 4 | 1 |
+| Final training loss | 0.370 | ~0.24 |
+| Adapter size | ~40 MB | ~120 MB |
+| Hardware | 2× NVIDIA T4 (Kaggle) | 2× NVIDIA T4 (Kaggle) |
+| Training time | ~91.5 min | ~30 hours |
+
+**Why load Phase 1 as the base instead of raw Qwen?** The Phase 1 checkpoint already encodes Socratic tutoring behavior for 4 languages, so Phase 2 only needs to learn new technology-specific vocabulary and error patterns — not the tutoring format itself. This warm start is reflected in training loss falling below 1.0 within 50 steps, despite Phase 2 having 16× more training examples than Phase 1.
+
+**Dataset replay:** 1,585 Phase 1 examples (~19.8% of the Phase 2 training mix) were included to prevent catastrophic forgetting of the original 4 languages. Post-training evaluation confirmed 100% retention on all Phase 1 language test cases.
+
+## Evaluation
+
+Manual evaluation across all 17 technologies, one representative test prompt each, against 5 pass/fail criteria (technology identification, correct error diagnosis, Socratic closing question, no direct solution given, Phase 1 retention):
 
-## 🚀 Quick Start
+- **Overall: 97.2% (70/72 checks passed)**
+- Technology identification: 100% (17/17)
+- Socratic closing question present: 100% (17/17)
+- Phase 1 language retention: 100% (4/4)
 
-### Load with PEFT (Recommended)
+This is a manual smoke-test evaluation, not a held-out benchmark — see Limitations below.
+
+## Usage
 
 ```python
-from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+from transformers import AutoModelForCausalLM, BitsAndBytesConfig, AutoTokenizer
 from peft import PeftModel
 import torch
 
-# 4-bit quantization config
 bnb = BitsAndBytesConfig(
     load_in_4bit=True,
-    bnb_4bit_compute_dtype=torch.float16
+    bnb_4bit_quant_type='nf4',
+    bnb_4bit_compute_dtype=torch.bfloat16,
 )
 
-BASE_MODEL = "codellama/CodeLlama-7b-Instruct-hf"
-ADAPTER    = "likithyadavv/codementor-v2-fullstack"
-
-# Load base model
-base_model = AutoModelForCausalLM.from_pretrained(
-    BASE_MODEL,
+base = AutoModelForCausalLM.from_pretrained(
+    'likithyadavv/codementor-7b',
     quantization_config=bnb,
-    device_map="auto"
+    device_map='auto',
 )
 
-# Load LoRA adapter
-model     = PeftModel.from_pretrained(base_model, ADAPTER)
-tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
-
-print("✅ CodeMentor loaded!")
+model = PeftModel.from_pretrained(base, 'likithyadavv/codementor-v2-fullstack')
+tokenizer = AutoTokenizer.from_pretrained('likithyadavv/codementor-v2-fullstack')
 ```
 
-### Run Inference
+Prompt format:
 
-```python
-def ask_codementor(instruction, code_input="", max_new_tokens=512):
-    prompt = f"### Instruction:\n{instruction}\n\n### Input:\n{code_input}\n\n### Response:\n"
-    
-    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
-    with torch.no_grad():
-        outputs = model.generate(
-            **inputs,
-            max_new_tokens=max_new_tokens,
-            temperature=0.2,
-            do_sample=True,
-            pad_token_id=tokenizer.eos_token_id,
-        )
-    response = tokenizer.decode(
-        outputs[0][inputs["input_ids"].shape[1]:],
-        skip_special_tokens=True
-    )
-    return response.strip()
-
-# Example usage
-print(ask_codementor(
-    instruction="Explain this code and identify any bugs.",
-    code_input="def factorial(n): return n * factorial(n-1)"
-))
 ```
+### System:
+You are CodeMentor, a patient programming tutor for Python, Java, C, C++, HTML, CSS,
+JavaScript, TypeScript, React, Next.js, Node.js, Express, FastAPI, Django, Spring Boot,
+SQL, MongoDB, Git, REST APIs, and Docker. Always identify the language or technology
+first. Acknowledge what is correct, then guide with hints and questions -- never give
+away the full solution.
 
-**Output:**
-```
-This is a recursive factorial function. However, it has a critical bug —
-there is no base case, so it will recurse infinitely and raise a
-RecursionError. Fix:
-
-def factorial(n):
-    if n == 0:      # ← base case added
-        return 1
-    return n * factorial(n - 1)
-```
+### Instruction:
+{instruction}
 
----
+### Input:
+{code_snippet}
 
-## 💬 Interactive Chat Loop
-
-```python
-chat_history = []
-
-while True:
-    user_input = input("\n👤 You: ").strip()
-    if user_input.lower() in ["exit", "quit"]:
-        break
-
-    # Build context from last 3 exchanges
-    context = ""
-    for u, b in chat_history[-3:]:
-        context += f"User: {u}\nAssistant: {b}\n\n"
-
-    is_code = any(x in user_input for x in ["def ", "class ", "import ", "return ", "=>"])
-    instruction = (
-        "Explain this code, identify any bugs, and suggest improvements."
-        if is_code else
-        "Answer this programming question clearly and concisely."
-    )
-
-    full_input = f"{context}User: {user_input}" if context else user_input
-    response = ask_codementor(instruction, full_input)
-
-    print(f"\n🤖 CodeMentor: {response}")
-    chat_history.append((user_input, response))
+### Response:
 ```
 
----
+## Limitations
 
-## 🌐 Deploy as REST API (FastAPI + ngrok)
+- Evaluated manually on one prompt per technology — no automated benchmark (BLEU/ROUGE/BERTScore) or held-out test set yet
+- Single-turn only — no conversation memory across exchanges
+- Training data was authored against a fixed schema and quality-reviewed, but at hand-crafted scale (8,000 examples), which limits further growth without a more scalable data pipeline
 
-```python
-from fastapi import FastAPI
-from pydantic import BaseModel
-import uvicorn, nest_asyncio, threading
-from pyngrok import ngrok
-
-app = FastAPI(title="CodeMentor API")
-
-class AskRequest(BaseModel):
-    instruction: str
-    input: str = ""
-
-@app.get("/")
-def root():
-    return {"status": "CodeMentor API is live 🚀"}
-
-@app.get("/health")
-def health():
-    return {"status": "ok"}
-
-@app.post("/ask")
-def ask(req: AskRequest):
-    response = ask_codementor(req.instruction, req.input)
-    return {"response": response}
-
-# Launch
-nest_asyncio.apply()
-public_url = ngrok.connect(8000)
-print(f"🚀 Live at: {public_url}/docs")
-
-threading.Thread(
-    target=lambda: uvicorn.run(app, host="0.0.0.0", port=8000, log_level="warning"),
-    daemon=True
-).start()
-```
+## Citation
 
-**Example curl:**
-```bash
-curl -X POST https://YOUR-NGROK-URL/ask \
-  -H "Content-Type: application/json" \
-  -d '{"instruction": "Explain and fix this code", "input": "def f(n): return n*f(n-1)"}'
-```
-
----
-
-## 📊 Evaluation
-
-| Metric | Score |
-|---|---|
-| Code Explanation Accuracy | **92.6%** |
-| Bug Detection Rate | **89.3%** |
-| Improvement Suggestion Quality | **4.1 / 5.0** |
-| Avg. Response Latency (T4 GPU) | **~3.2s** |
-
-> Evaluated on a held-out set of 500 fullstack coding tasks across Python, JavaScript, and SQL.
-
----
-
-## 🗂️ Training Details
+If you use this model, please cite:
 
 ```
-Dataset:        Custom fullstack coding instruction dataset
-                (code explanations, bug fixes, Q&A pairs)
-Format:         Alpaca-style (### Instruction / ### Input / ### Response)
-Base Model:     codellama/CodeLlama-7b-Instruct-hf
-Method:         QLoRA — 4-bit NF4 quantization + LoRA adapters
-LoRA Config:    r=16, alpha=32, dropout=0.05
-                target_modules: q_proj, v_proj, k_proj, o_proj
-Epochs:         3
-Batch Size:     4 (gradient accumulation: 4)
-Learning Rate:  2e-4 with cosine scheduler
-Hardware:       Google Colab A100 (40GB)
-Training Time:  ~4 hours
+CodeMentor LLM: A QLoRA Fine-Tuned Socratic Programming Tutor
+Mohammad Yusha G.N., Likith Yadav, Suhas R, Dhananjay M. Hiremath
+(Guide: Prof. Sanjivani D. Tipe)
+Dept. of Artificial Intelligence & Machine Learning, MVJ College of Engineering, Bengaluru
+SYNERGY 2026 — IC-SIIT, MS Ramaiah University of Applied Sciences
 ```
 
----
-
-## ⚙️ Hardware Requirements
-
-| Setup | Minimum | Recommended |
-|---|---|---|
-| GPU VRAM | 8 GB (4-bit) | 16 GB+ |
-| RAM | 12 GB | 24 GB |
-| GPU | T4 | A100 / RTX 3090+ |
-| Storage | 15 GB | 20 GB |
-
-> ✅ Runs on **free Google Colab T4** with 4-bit quantization.
-
----
-
-## ⚠️ Limitations
-
-- Responses may occasionally hallucinate for very niche or obscure APIs
-- Best results on Python and JavaScript; other languages have lower coverage
-- Long code blocks (>200 lines) may exceed context window — chunk inputs
-- Not suitable for security-critical code auditing without human review
-
----
-
-## 📚 Citation
-
-```bibtex
-@misc{codementor-v2-fullstack,
-  author       = {Likith Yadav},
-  title        = {CodeMentor V2: A LoRA Fine-Tuned Fullstack Code Assistant},
-  year         = {2025},
-  publisher    = {HuggingFace},
-  howpublished = {\url{https://huggingface.co/likithyadavv/codementor-v2-fullstack}},
-}
-```
-
----
-
-## 🔗 Links
+## Acknowledgements
 
-- 🤗 **Model Repo:** [likithyadavv/codementor-v2-fullstack](https://huggingface.co/likithyadavv/codementor-v2-fullstack)
-- 📖 **Base Model:** [codellama/CodeLlama-7b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf)
-- 🏫 **Institution:** MVJ College of Engineering, Bengaluru, India
+Built using the Hugging Face ecosystem — Transformers, PEFT, TRL, BitsAndBytes, and Datasets. Compute provided via Kaggle's free-tier GPU program.