File size: 4,286 Bytes
7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 7a1b97f 0169e28 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 | ---
license: mit
base_model: deepseek-ai/deepseek-coder-6.7b-base
tags:
- code
- leetcode
- python
- fine-tuned
- lora
- qlora
datasets:
- LongQ/leetcode_python
language:
- en
pipeline_tag: text-generation
---
# DeepSeek-Coder-6.7B LeetCode Fine-Tuned
A fine-tuned version of [DeepSeek-Coder-6.7B-Base](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base) specialized for solving LeetCode-style algorithmic problems in Python.
## Model Details
| Attribute | Value |
|-----------|-------|
| **Base Model** | deepseek-ai/deepseek-coder-6.7b-base |
| **Fine-tuning Method** | QLoRA (4-bit quantization + LoRA) |
| **Training Data** | [LongQ/leetcode_python](https://huggingface.co/datasets/LongQ/leetcode_python) (2,369 problems) |
| **Epochs** | 3 |
| **Hardware** | NVIDIA T4 (16GB VRAM) |
| **Training Time** | ~5 hours |
## Performance
Evaluated on 100 LeetCode problems with automated code execution:
| Metric | Base Model | Fine-Tuned | Improvement |
|--------|------------|------------|-------------|
| **Overall Accuracy** | 24% | 34% | +42% |
| **Easy Problems** | 30.3% | 52% | +72% |
| **Medium Problems** | 32.4% | 27.8% | -14% |
| **Hard Problems** | 9.1% | 28.6% | +214% |
### Key Findings
- **Significant gains on Easy and Hard problems** — model learned both fundamental patterns and complex algorithms
- **Slight regression on Medium** — possible overfitting to extremes of difficulty distribution
- **Domain-specific data matters** — initial training on general coding data degraded performance
## Intended Use
- Solving algorithmic coding challenges
- LeetCode practice and learning
- Code generation for competitive programming
- Educational tool for understanding algorithmic solutions
## Limitations
- Optimized specifically for LeetCode-style problems, may not generalize to other coding tasks
- Python-only (not trained on other languages)
- May produce syntactically correct but logically incorrect solutions
- Struggles with problems requiring complex data structure implementations (LinkedList, Trees)
## How to Use
### With Hugging Face Transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"Jerry-lin23/deepseek-leetcode-fp16",
torch_dtype=torch.float16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Jerry-lin23/deepseek-leetcode-fp16")
prompt = """### Problem:
Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.
### Starter Code:
```python
class Solution:
def twoSum(self, nums: List[int], target: int) -> List[int]:
```
### Solution:
```python
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.2)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
### With Ollama (Local Deployment)
1. Convert to GGUF format
2. Create a Modelfile:
```
FROM ./deepseek-leetcode-q8.gguf
PARAMETER temperature 0.2
PARAMETER top_p 0.95
```
3. Import: `ollama create deepseek-leetcode -f Modelfile`
4. Run: `ollama run deepseek-leetcode`
## Training Details
### LoRA Configuration
```python
LoraConfig(
r=16,
lora_alpha=32,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
)
```
### Training Arguments
```python
SFTConfig(
num_train_epochs=3,
per_device_train_batch_size=1,
gradient_accumulation_steps=8,
learning_rate=2e-4,
warmup_ratio=0.03,
fp16=True,
gradient_checkpointing=True,
max_seq_length=2048,
dataset_text_field="text"
)
```
## Citation
```bibtex
@misc{deepseek-leetcode-finetuned,
author = {Jerry Lin},
title = {DeepSeek-Coder-6.7B LeetCode Fine-Tuned},
year = {2024},
publisher = {Hugging Face},
url = {https://huggingface.co/Jerry-lin23/deepseek-leetcode-fp16}
}
```
## Links
- 📊 [Benchmark Repository](https://github.com/jerrylin-23/DeepSeek-LeetCode-Oriented-Training)
- 🤗 [Base Model](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base)
- 📚 [Training Dataset](https://huggingface.co/datasets/LongQ/leetcode_python)
|