nicholashat/qwen2.5-coder-3b-oss-instruct-lora

Qwen2.5-Coder-3B-Instruct — OSS-Instruct LoRA

A LoRA adapter for mlx-community/Qwen2.5-Coder-3B-Instruct-4bit, fine-tuned on synthetic instruction/solution pairs generated with the OSS-Instruct method (Magicoder). Trained end to end on Apple Silicon using mlx-lm — no CUDA, no bitsandbytes.

Full pipeline, training log, and evaluation methodology: github.com/NicholasHat/oss-instruct-py.

Training data

~28.3k (instruction, solution) pairs, generated by prompting claude-haiku-4-5 to write a self-contained Python problem and solution inspired by a real open-source code snippet (not copied), sourced from bigcode/the-stack-dedup (permissively-licensed Python files under 2KB). Post-generation quality filter (ast.parse validity, MinHash near-duplicate removal) and a 95/5 train/valid split. Total generation cost: $33.60.

Training config

• LoRA, rank 8, all linear/embedding layers (num_layers: -1), scale 20.0
• AdamW, cosine-decay LR schedule (peak 5e-5), 1 epoch (28,304 iters, batch size 1 + gradient accumulation 8)
• Wall-clock: ~17.2 hours; peak memory 10.97GB; final train loss 0.282, final val loss 0.326

Evaluation

Base vs. tuned, identical mlx_lm.generate invocation (greedy, max_tokens=512, seed=0):

Task	Base 3B	Tuned 3B (this adapter)	Δ
HumanEval pass@1	0.677 (111/164)	0.707 (116/164)	+3.0 pts
MBPP pass@1	0.582 (291/500)	0.596 (298/500)	+1.4 pts

Scored with bigcode-evaluation-harness (sandboxed execution + pass@1 scoring only — completions were generated separately with mlx_lm.generate, not the harness).

Usage

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load(
    "mlx-community/Qwen2.5-Coder-3B-Instruct-4bit",
    adapter_path="<path to this repo>",
)
messages = [{"role": "user", "content": "Write a Python function that reverses a linked list."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

Limitations

• ~28k training pairs (vs. Magicoder's original ~75k), capped by a $50 synthetic-generation budget — not a reflection of the pipeline's ceiling.
• MLX 4-bit quantization + mlx_lm.lora, not the original QLoRA/bitsandbytes recipe — absolute numbers aren't directly comparable to a from-scratch reproduction on that stack.
• Evaluated at pass@1 only (n_samples=1, greedy), not pass@10.
• 1 epoch only; a second epoch was never tested.

See the repo's plan.md for the full decision log, including a training-stability investigation on a 7B ablation (shelved, unresolved) and a scoring-stability issue found and corrected during evaluation.