|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct |
|
|
tags: |
|
|
- lora |
|
|
- code-generation |
|
|
- deep-instruction |
|
|
- chain-of-thought |
|
|
--- |
|
|
|
|
|
# Qwen2.5-Coder-1.5B - Deep Instruction LoRA |
|
|
|
|
|
This model is a LoRA fine-tuned version of **Qwen2.5-Coder-1.5B-Instruct**. It was trained on the **CodeGen-Deep-5K** dataset to enhance code reasoning capabilities using Chain-of-Thought (CoT) traces. |
|
|
|
|
|
## Model Details |
|
|
- **Base Model:** Qwen/Qwen2.5-Coder-1.5B-Instruct |
|
|
- **Training Dataset:** CodeGen-Deep-5K (Reasoning-focused) |
|
|
- **Method:** LoRA (Rank: 32, Alpha: 64) |
|
|
- **Epochs:** 3 |
|
|
|
|
|
## Performance (Pass@1 on LiveCodeBench - AtCoder Easy) |
|
|
- **Base Model:** 26.83% |
|
|
- **This Model:** **34.15%** (+7.32% Improvement) 🚀 |
|
|
|
|
|
This model specializes in algorithmic problems requiring multi-step reasoning and state tracking. |
|
|
|
|
|
## Usage |
|
|
```python |
|
|
from peft import PeftModel |
|
|
from transformers import AutoModelForCausalLM |
|
|
|
|
|
base_model_id = "Qwen/Qwen2.5-Coder-1.5B-Instruct" |
|
|
adapter_model_id = "deneme0001/Qwen2.5-Coder-Deep-Instruct-LoRA" |
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained(base_model_id, device_map="auto") |
|
|
model = PeftModel.from_pretrained(model, adapter_model_id) |
