File size: 3,168 Bytes
fd8149c 21814f8 fd8149c 21814f8 fd8149c 21814f8 fd8149c 21814f8 fd8149c 21814f8 fd8149c 21814f8 fd8149c 21814f8 7a0fbc3 21814f8 7a5fffd 21814f8 7a0fbc3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-7B-Instruct
tags:
- qlora
- peft
- fine-tune
- code-optimization
- qwen
- code-generation
- llm
---
# Code Optimization Fine-tuned Qwen2.5-Coder-7B-Instruct (LoRA Adapter)
This repository contains a fine-tuned LoRA adapter for the `Qwen/Qwen2.5-Coder-7B-Instruct` model, specialized for Python code optimization. The model was fine-tuned using QLoRA on the `SeifElden2342532/Code-Optimization` dataset.
## Model Description
This model is a LoRA adapter of the `Qwen2.5-Coder-7B-Instruct` base model. It has been fine-tuned to act as an expert Python code optimizer, capable of taking user-provided Python code and optimizing it for performance, readability, or conciseness, along with providing explanations and complexity comparisons.
## Training Details
* **Base Model**: `Qwen/Qwen2.5-Coder-7B-Instruct`
* **Fine-tuning Framework**: QLoRA (Parameter-Efficient Fine-Tuning)
* **Dataset**: `SeifElden2342532/Code-Optimization`
* **Training Parameters:**
* `num_train_epochs`: 3
* `lora_r`: 64
* `lora_alpha`: 128
* `max_seq_length`: 2048
* `per_device_train_batch_size`: 4
* `gradient_accumulation_steps`: 4
* `learning_rate`: 2e-4
* `bf16`: True
* `gradient_checkpointing`: True
## Github Repo
https://github.com/SeifEldenOsama/Code-Optimizer/tree/master
## How to Use
To use this LoRA adapter, you need to load the base model and then apply the adapter. Here's a complete example to test the model:
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch
# 1. Configuration
base_model_id = "Qwen/Qwen2.5-Coder-7B-Instruct"
adapter_repo_id = "SeifElden2342532/Code-Optimizer"
# 2. Load Tokenizer and Base Model
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# 3. Load and Merge the LoRA Adapter
model = PeftModel.from_pretrained(model, adapter_repo_id)
model = model.merge_and_unload() # Merging for faster inference
# 4. Prepare the Input
messages = [
{
"role": "system",
"content": "You are an expert Python code optimizer. Your goal is to take user-provided Python code and optimize it for performance, readability, or conciseness, based on the user's specified category. Provide the optimized code, a brief explanation of the changes, and a complexity comparison table (e.g., time and space complexity before and after optimization)."
},
{
"role": "user",
"content": "Original Code:\n```python\ndef factorial(n):\n if n == 0:\n return 1\n else:\n return n * factorial(n-1)\n```\nCategory: Performance"
}
]
# 5. Generate Optimization
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(**model_inputs, max_new_tokens=1024)
output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
# 6. Print Result
print(output) |