Jerry-lin23/deepseek-leetcode-p3

DeepSeek-Coder-6.7B LeetCode Fine-Tuned

A fine-tuned version of DeepSeek-Coder-6.7B-Base specialized for solving LeetCode-style algorithmic problems in Python.

Model Details

Attribute	Value
Base Model	deepseek-ai/deepseek-coder-6.7b-base
Fine-tuning Method	QLoRA (4-bit quantization + LoRA)
Training Data	LongQ/leetcode_python (2,369 problems)
Epochs	3
Hardware	NVIDIA T4 (16GB VRAM)
Training Time	~5 hours

Performance

Evaluated on 100 LeetCode problems with automated code execution:

Metric	Base Model	Fine-Tuned	Improvement
Overall Accuracy	24%	34%	+42%
Easy Problems	30.3%	52%	+72%
Medium Problems	32.4%	27.8%	-14%
Hard Problems	9.1%	28.6%	+214%

Key Findings

• Significant gains on Easy and Hard problems — model learned both fundamental patterns and complex algorithms
• Slight regression on Medium — possible overfitting to extremes of difficulty distribution
• Domain-specific data matters — initial training on general coding data degraded performance

Intended Use

• Solving algorithmic coding challenges
• LeetCode practice and learning
• Code generation for competitive programming
• Educational tool for understanding algorithmic solutions

Limitations

• Optimized specifically for LeetCode-style problems, may not generalize to other coding tasks
• Python-only (not trained on other languages)
• May produce syntactically correct but logically incorrect solutions
• Struggles with problems requiring complex data structure implementations (LinkedList, Trees)

How to Use

With Hugging Face Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Jerry-lin23/deepseek-leetcode-fp16",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Jerry-lin23/deepseek-leetcode-fp16")

prompt = """### Problem:
Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.

### Starter Code:
```python
class Solution:
    def twoSum(self, nums: List[int], target: int) -> List[int]:

Solution:

"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.2)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

With Ollama (Local Deployment)

1. Convert to GGUF format
2. Create a Modelfile:

FROM ./deepseek-leetcode-q8.gguf
PARAMETER temperature 0.2
PARAMETER top_p 0.95

3. Import: ollama create deepseek-leetcode -f Modelfile
4. Run: ollama run deepseek-leetcode

Training Details

LoRA Configuration

LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

Training Arguments

SFTConfig(
    num_train_epochs=3,
    per_device_train_batch_size=1,
    gradient_accumulation_steps=8,
    learning_rate=2e-4,
    warmup_ratio=0.03,
    fp16=True,
    gradient_checkpointing=True,
    max_seq_length=2048,
    dataset_text_field="text"
)

Citation

@misc{deepseek-leetcode-finetuned,
  author = {Jerry Lin},
  title = {DeepSeek-Coder-6.7B LeetCode Fine-Tuned},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Jerry-lin23/deepseek-leetcode-fp16}
}

Links

• 📊 Benchmark Repository
• 🤗 Base Model
• 📚 Training Dataset