File size: 2,340 Bytes
3834a7f e414932 3834a7f e414932 3834a7f e414932 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
license: apache-2.0
base_model: unsloth/Qwen2.5-Coder-7B-Instruct
tags:
- code
- qlora
- unsloth
- qwen2.5-coder
- text-generation
language:
- en
pipeline_tag: text-generation
---
# GX-Coder-7B
**v0.1** — QLoRA fine-tune of [`unsloth/Qwen2.5-Coder-7B-Instruct`](https://huggingface.co/Qwen2.5-Coder-7B-Instruct),
built to power a multi-mode coding agent. This first release sits at **parity with
the base** on HumanEval+ (see table); it's an end-to-end QLoRA→eval→ship baseline.
Later versions target agent-specific tasks (tool-calling, web/UI codegen) where
the base isn't already saturated.
## Summary
- **Base:** unsloth/Qwen2.5-Coder-7B-Instruct
- **Method:** 4-bit QLoRA (LoRA r=16) via [Unsloth](https://github.com/unslothai/unsloth)
- **Hardware:** single free-Colab T4 (16GB)
- **Data:** open instruction-coding datasets, ChatML-formatted (see training repo)
## Benchmarks (HumanEval+ pass@1, greedy)
| Model | HumanEval+ pass@1 |
|-------|-------------------|
| unsloth/Qwen2.5-Coder-7B-Instruct (base) | 80.5 |
| **GX-Coder-7B (this)** | **78.0** |
> Scored with [EvalPlus](https://github.com/evalplus/evalplus) (164 problems,
> greedy). The ~2pt gap is within noise (~3-4 problems) — treat as parity. A
> generic fine-tune can't easily beat a near-ceiling base here; gains come from
> task-specific data in later versions.
## Intended use
Code generation, completion, review, and tool-using agent workflows. Powers the
companion multi-mode coding agent (Claude-Code-like coder + Figma-like web/UI
design mode).
## Limitations
Small model fine-tune — not frontier-level. May hallucinate APIs, miss edge
cases, and produce insecure code. Always review generated code before running.
Not trained on benchmark test sets (contamination guard in data_prep.py).
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Garv98/GX-Coder-7B")
model = AutoModelForCausalLM.from_pretrained("Garv98/GX-Coder-7B", device_map="auto")
msgs = [{"role": "user", "content": "Write a Python function to check if a string is a palindrome."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=256)[0][ids.shape[1]:], skip_special_tokens=True))
```
|