README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-7B-Instruct
tags:
- qlora
- peft
- fine-tune
- code-optimization
- qwen
- code-generation
- llm
---

# Code Optimization Fine-tuned Qwen2.5-Coder-7B-Instruct (LoRA Adapter)

This repository contains a fine-tuned LoRA adapter for the `Qwen/Qwen2.5-Coder-7B-Instruct` model, specialized for Python code optimization. The model was fine-tuned using QLoRA on the `SeifElden2342532/Code-Optimization` dataset.

## Model Description

This model is a LoRA adapter of the `Qwen2.5-Coder-7B-Instruct` base model. It has been fine-tuned to act as an expert Python code optimizer, capable of taking user-provided Python code and optimizing it for performance, readability, or conciseness, along with providing explanations and complexity comparisons.

## Training Details

*   **Base Model**: `Qwen/Qwen2.5-Coder-7B-Instruct`
*   **Fine-tuning Framework**: QLoRA (Parameter-Efficient Fine-Tuning)
*   **Dataset**: `SeifElden2342532/Code-Optimization`
*   **Training Parameters:**
    *   `num_train_epochs`: 3
    *   `lora_r`: 64
    *   `lora_alpha`: 128
    *   `max_seq_length`: 2048
    *   `per_device_train_batch_size`: 4
    *   `gradient_accumulation_steps`: 4
    *   `learning_rate`: 2e-4
    *   `bf16`: True
    *   `gradient_checkpointing`: True

## Github Repo
https://github.com/SeifEldenOsama/Code-Optimizer/tree/master

## How to Use

To use this LoRA adapter, you need to load the base model and then apply the adapter. Here's a complete example to test the model:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# 1. Configuration
base_model_id = "Qwen/Qwen2.5-Coder-7B-Instruct"
adapter_repo_id = "SeifElden2342532/Code-Optimizer"

# 2. Load Tokenizer and Base Model
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id, 
    torch_dtype=torch.bfloat16, 
    device_map="auto"
)

# 3. Load and Merge the LoRA Adapter
model = PeftModel.from_pretrained(model, adapter_repo_id)
model = model.merge_and_unload() # Merging for faster inference

# 4. Prepare the Input
messages = [
    {
        "role": "system", 
        "content": "You are an expert Python code optimizer. Your goal is to take user-provided Python code and optimize it for performance, readability, or conciseness, based on the user's specified category. Provide the optimized code, a brief explanation of the changes, and a complexity comparison table (e.g., time and space complexity before and after optimization)."
    },
    {
        "role": "user", 
        "content": "Original Code:\n```python\ndef factorial(n):\n    if n == 0:\n        return 1\n    else:\n        return n * factorial(n-1)\n```\nCategory: Performance"
    }
]

# 5. Generate Optimization
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(**model_inputs, max_new_tokens=1024)
output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

# 6. Print Result
print(output)