| --- |
| library_name: transformers |
| base_model: Qwen/Qwen2.5-1.5B-Instruct |
| language: |
| - en |
| license: apache-2.0 |
| tags: |
| - llm |
| - qlora |
| - python |
| - code-generation |
| - instruction-tuning |
| - transformers |
| --- |
| |
| # CodeMentor-LLM |
|
|
| CodeMentor-LLM is a lightweight coding assistant fine-tuned from Qwen2.5-1.5B-Instruct using QLoRA. The model is designed to assist with Python programming tasks, algorithm explanations, code generation, and beginner-friendly coding guidance. |
|
|
| ## Model Details |
|
|
| ### Developed By |
| Soumya Singh |
|
|
| ### Base Model |
| Qwen/Qwen2.5-1.5B-Instruct |
|
|
| ### Model Type |
| Causal Language Model (LLM) |
|
|
| ### Language |
| English |
|
|
| ## Training Data |
|
|
| The model was fine-tuned on 100 instruction-response examples from the Python Code Instructions Alpaca dataset. |
|
|
| **Dataset:** `iamtarun/python_code_instructions_18k_alpaca` |
|
|
| ## Training Method |
|
|
| - QLoRA Fine-Tuning |
| - 4-bit Quantization |
| - PEFT (Parameter Efficient Fine-Tuning) |
| - Transformers Library |
| - Hugging Face Trainer |
|
|
| ## Training Configuration |
|
|
| | Parameter | Value | |
| |------------|--------| |
| | Epochs | 3 | |
| | Batch Size | 2 | |
| | Learning Rate | 2e-4 | |
| | Gradient Accumulation | 4 | |
| | Precision | FP16 | |
| | GPU | NVIDIA Tesla T4 | |
|
|
| ## Intended Use |
|
|
| This model can be used for: |
|
|
| - Python code generation |
| - Algorithm explanations |
| - Programming tutoring |
| - Beginner coding assistance |
| - Educational demonstrations of LLM fine-tuning |
|
|
| ## Example Usage |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| model_name = "soumya-006/CodeMentor-LLM" |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| model = AutoModelForCausalLM.from_pretrained(model_name) |
| |
| prompt = """ |
| Instruction: |
| Write a Python function to check if a number is prime. |
| |
| Response: |
| """ |
| |
| inputs = tokenizer(prompt, return_tensors="pt") |
| |
| outputs = model.generate( |
| **inputs, |
| max_new_tokens=150 |
| ) |
| |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| ## Limitations |
|
|
| - Trained on only 100 examples. |
| - Intended as a demonstration project. |
| - May generate incorrect or inefficient code. |
| - Should not be used for production systems without additional training and evaluation. |
|
|
| ## Future Improvements |
|
|
| - Increase training dataset to 5,000+ examples. |
| - Add multi-language support. |
| - Improve reasoning capabilities. |
| - Evaluate on standard coding benchmarks. |
| - Deploy an interactive web application. |
|
|
| ## Author |
|
|
| Soumya Singh |
|
|
| B.Tech Computer Science Student |
|
|
| ## Hugging Face Repository |
|
|
| https://huggingface.co/soumya-006/CodeMentor-LLM |
