---
library_name: transformers
base_model: Qwen/Qwen2.5-1.5B-Instruct
language:
- en
license: apache-2.0
tags:
- llm
- qlora
- python
- code-generation
- instruction-tuning
- transformers
---

# CodeMentor-LLM

CodeMentor-LLM is a lightweight coding assistant fine-tuned from Qwen2.5-1.5B-Instruct using QLoRA. The model is designed to assist with Python programming tasks, algorithm explanations, code generation, and beginner-friendly coding guidance.

## Model Details

### Developed By
Soumya Singh

### Base Model
Qwen/Qwen2.5-1.5B-Instruct

### Model Type
Causal Language Model (LLM)

### Language
English

## Training Data

The model was fine-tuned on 100 instruction-response examples from the Python Code Instructions Alpaca dataset.

**Dataset:** `iamtarun/python_code_instructions_18k_alpaca`

## Training Method

- QLoRA Fine-Tuning
- 4-bit Quantization
- PEFT (Parameter Efficient Fine-Tuning)
- Transformers Library
- Hugging Face Trainer

## Training Configuration

| Parameter | Value |
|------------|--------|
| Epochs | 3 |
| Batch Size | 2 |
| Learning Rate | 2e-4 |
| Gradient Accumulation | 4 |
| Precision | FP16 |
| GPU | NVIDIA Tesla T4 |

## Intended Use

This model can be used for:

- Python code generation
- Algorithm explanations
- Programming tutoring
- Beginner coding assistance
- Educational demonstrations of LLM fine-tuning

## Example Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "soumya-006/CodeMentor-LLM"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = """
Instruction:
Write a Python function to check if a number is prime.

Response:
"""

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=150
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Limitations

- Trained on only 100 examples.
- Intended as a demonstration project.
- May generate incorrect or inefficient code.
- Should not be used for production systems without additional training and evaluation.

## Future Improvements

- Increase training dataset to 5,000+ examples.
- Add multi-language support.
- Improve reasoning capabilities.
- Evaluate on standard coding benchmarks.
- Deploy an interactive web application.

## Author

Soumya Singh

B.Tech Computer Science Student

## Hugging Face Repository

https://huggingface.co/soumya-006/CodeMentor-LLM