🐛 BugFixer-DeepSeek-6.7B

A Fine-Tuned Code Correction Model

---

🔍 Overview

BugFixer-DeepSeek-6.7B is a fine-tuned version of the base model DeepSeek-Coder-6.7B-Instruct designed to automatically detect and fix bugs in source code.

The model takes buggy code as input and outputs a corrected version without explanation, making it suitable for automation pipelines and IDE integrations.

---

🧠 Model Details

• Base Model: deepseek-ai/deepseek-coder-6.7b-instruct
• Model Type: Causal Language Model (Decoder-only Transformer)
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Task: Code Refinement / Bug Fixing
• Framework: Hugging Face Transformers
• License: Same as base model (check DeepSeek license)

---

🎯 Intended Use

✅ Direct Use

• Fix Python bugs automatically

• Code debugging assistant

• Integration in:

  • VS Code extensions
  • CI/CD pipelines
  • AI coding assistants

---

🔄 Downstream Use

• Code auto-repair systems
• Educational tools (learning debugging)
• Static analysis augmentation

---

❌ Out-of-Scope Use

• Not designed for:

  • Production-critical code validation
  • Security-sensitive systems
  • Non-code natural language tasks

---

📊 Training Details

📂 Dataset

• Dataset: CodeXGLUE — Code Refinement

• Contains:

  • Buggy code snippets
  • Corrected versions

---

⚙️ Training Procedure

• Hardware: Kaggle (2× T4 GPUs)
• Precision: FP16
• Method: LoRA fine-tuning
• Epochs: 1
• Batch Size: 1 (with gradient accumulation)
• Max Sequence Length: 512

---

🔧 Hyperparameters

Parameter	Value
Learning Rate	2e-4
LoRA Rank (r)	16
LoRA Alpha	32
Dropout	0.05
Scheduler	Cosine
Warmup	3%

---

🧪 Evaluation

📌 Metrics Used

• Training Loss
• Qualitative Code Correctness

📈 Observations

• Strong performance on:

  • Syntax errors
  • Index errors
  • Simple logical bugs

• Moderate performance on:

  • Complex algorithms
  • Multi-file dependencies

---

⚠️ Limitations

• Primarily trained on Python → weaker on other languages

• May:

  • Produce partially correct fixes
  • Miss deep logical bugs

• No guarantee of optimal or efficient solutions

---

⚠️ Risks & Bias

• Model may:

  • Introduce new bugs
  • Generate insecure code

• Should always be reviewed by a developer

---

🚀 Usage

🔹 Load Model

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "<your-username>/bugfixer-deepseek-6.7b",
    torch_dtype="auto",
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(
    "<your-username>/bugfixer-deepseek-6.7b"
)

---

🔹 Example

prompt = """### Instruction:
Fix the bug in the following code.

### Buggy Code:
def is_even(n):
    return n % 2 = 0

### Fixed Code:
"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

🧩 Model Variants

🔹 Adapter Version

• Repo: <your-username>/bugfixer-deepseek-6.7b-adapter
• Requires base model

🔹 Full Model

• Repo: <your-username>/bugfixer-deepseek-6.7b
• Fully merged and ready to use

---

🌍 Environmental Impact

• Hardware: NVIDIA T4 ×2
• Platform: Kaggle
• Estimated Duration: Few hours
• Precision: FP16 (reduced energy vs FP32)

---

🏗️ Technical Architecture

• Transformer-based decoder model

• Self-attention layers

• LoRA applied to:

  • Query (q_proj)
  • Key (k_proj)
  • Value (v_proj)
  • Output (o_proj)

---

📚 Citation

If you use this model, please cite:

@misc{bugfixer_deepseek_6_7b,
  title={BugFixer-DeepSeek-6.7B},
  author={<your-name>},
  year={2026},
  note={Fine-tuned on CodeXGLUE dataset using LoRA}
}

---

📬 Contact

• Developer: praveen
• Hugging Face: https://huggingface.co/prav-974

---

⭐ Acknowledgements

• DeepSeek AI for base model
• Hugging Face for ecosystem
• CodeXGLUE dataset contributors

---