| --- |
| library_name: transformers |
| tags: |
| - code-generation |
| - bug-fixing |
| - ai-safety |
| - llm |
| - qwen |
| - fine-tuned |
| - peft |
| - developer-tools |
| license: mit |
| language: |
| - en |
| - hi |
| base_model: |
| - Qwen/Qwen2.5-Coder-3B-Instruct |
| pipeline_tag: question-answering |
| --- |
| |
| --- |
|
|
| # 🚀 CodeSentinel — Smart Code Assistant for Bug Detection & Safe Generation |
|
|
| > ⚡ *"Write better code. Catch bugs early. Avoid unsafe outputs."* |
|
|
| --- |
|
|
| ## 🧠 Model Overview |
|
|
| **CodeSentinel** is a fine-tuned Large Language Model built on top of **Qwen2.5-Coder-3B-Instruct**, designed specifically for: |
|
|
| * 🐞 Detecting and fixing bugs in code |
| * 🧠 Reducing hallucinations in generated code |
| * 🔒 Enforcing safe and responsible code generation |
| * ⚡ Assisting developers with clean, reliable solutions |
|
|
| This model is optimized for **real-world developer workflows**, especially for students and engineers in Tier-2/3 ecosystems. |
|
|
| --- |
|
|
| ## 📌 Key Features |
|
|
| * ✅ **Bug Detection & Fixing** |
| * ✅ **Hallucination Reduction** |
| * ✅ **Safe Code Refusal (malicious prompts blocked)** |
| * ✅ **Structured & Clean Outputs** |
| * ✅ **Lightweight LoRA Fine-tuning (efficient)** |
|
|
| --- |
|
|
| ## 🏗️ Model Details |
|
|
| * **Base Model:** Qwen/Qwen2.5-Coder-3B-Instruct |
| * **Fine-tuning Method:** LoRA (PEFT) |
| * **Training Stages:** |
|
|
| * Stage 1 → Supervised Fine-Tuning (SFT) |
| * Stage 2 → Direct Preference Optimization (DPO) |
| * **Framework:** Hugging Face Transformers + PEFT |
| * **Precision:** FP16 |
| * **Hardware:** Tesla T4 (Kaggle) |
|
|
| --- |
|
|
| ## 🎯 Intended Use |
|
|
| ### 💡 Direct Use |
|
|
| * Debugging Python / general code |
| * Improving code quality |
| * Generating safe code snippets |
|
|
| ### 🔧 Downstream Use |
|
|
| * IDE plugins |
| * Code review assistants |
| * Educational tools |
| * Developer copilots |
|
|
| --- |
|
|
| ## ❌ Out-of-Scope Use |
|
|
| * Generating malware or exploits |
| * Bypassing security systems |
| * Illegal or harmful automation |
|
|
| 👉 The model is trained to **refuse such requests** |
|
|
| --- |
|
|
| ## ⚠️ Limitations |
|
|
| * May still hallucinate in rare edge cases |
| * Limited to training distribution (mostly Python-centric) |
| * Not a replacement for production-level code review |
|
|
| --- |
|
|
| ## 🛠️ How to Use |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| base_model = AutoModelForCausalLM.from_pretrained( |
| "Qwen/Qwen2.5-Coder-3B-Instruct", |
| device_map="auto", |
| torch_dtype="auto" |
| ) |
| |
| model = PeftModel.from_pretrained(base_model, "PraneetNS/codesentinel-adapter") |
| tokenizer = AutoTokenizer.from_pretrained("PraneetNS/codesentinel-adapter") |
| |
| prompt = "Fix this bug: KeyError in dictionary access" |
| |
| inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
| outputs = model.generate(**inputs, max_new_tokens=200) |
| |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| --- |
|
|
| ## 📊 Training Details |
|
|
| ### 📚 Dataset |
|
|
| Custom curated dataset including: |
|
|
| * Buggy vs correct code pairs |
| * Hallucination traps (e.g., fake functions) |
| * Safe vs unsafe response pairs |
|
|
| ### ⚙️ Training Setup |
|
|
| * Batch size: 2 (effective 16 via accumulation) |
| * Learning rate: 2e-4 (SFT), 5e-6 (DPO) |
| * Epochs: 1–2 |
| * Optimizer: AdamW |
|
|
| --- |
|
|
| ## 📈 Evaluation |
|
|
| ### ✅ Improvements Observed |
|
|
| * Reduced incorrect API usage |
| * Better structured responses |
| * Improved refusal for unsafe prompts |
|
|
| ### 🧪 Example |
|
|
| **Input:** |
|
|
| ```python |
| users = {'alice': {'score': 80}} |
| print(users['bob']['score']) |
| ``` |
|
|
| **Output:** |
|
|
| * Detects `KeyError` |
| * Suggests safe access (`dict.get()`) |
|
|
| --- |
|
|
| ## 🌍 Impact |
|
|
| * Helps beginner developers avoid common mistakes |
| * Encourages safe AI usage |
| * Reduces debugging time |
|
|
| --- |
|
|
| ## ⚡ Technical Highlights |
|
|
| * LoRA-based fine-tuning (memory efficient) |
| * Runs on low-cost GPUs (T4) |
| * Easily deployable |
|
|
| --- |
|
|
| ## 💻 Compute |
|
|
| * **GPU:** Tesla T4 |
| * **Platform:** Kaggle |
| * **Training Time:** ~1–2 hours |
|
|
| --- |
|
|
| ## 🔗 Links |
|
|
| * **Model:** https://huggingface.co/PraneetNS/codesentinel-adapter |
|
|
| --- |
|
|
| ## 📢 Why CodeSentinel? |
|
|
| > Most code models generate answers. |
| > **CodeSentinel understands correctness and safety.** |
|
|
| --- |
|
|
| ## 🙌 Acknowledgements |
|
|
| * Hugging Face 🤗 |
| * Qwen Team |
| * Open-source ML community |
|
|
| --- |
|
|
| ## 📬 Contact |
|
|
| **Author:** Praneet Savant |
| 📧 savantpraneet@gmail.com |
|
|
| --- |
|
|
| ⭐ If you find this useful, consider starring the repo! |
