Nanthasit/sakthai-coder-1.5b

SakThai Coder 1.5B 💻

Code + tool-calling · Qwen2.5-Coder-1.5B fine-tune · Q4_K_M GGUF for CPU

The code specialist of the SakThai family — Qwen2.5-Coder-1.5B fine-tuned for tool-calling and shipped as a CPU-friendly GGUF. Part of the House of Sak. Read the story →

The Story Behind It

Code, tool-calling, and conversation in one session — on a single CPU, from a shelter. This is the model Beer built when he realised the other SakThai models could call tools and generate text, but none of them specialised in writing code without losing their tool-calling edge.

Beer built the first SakThai models on free Google Colab GPUs from a shelter in Cork, Ireland — with $0 budget, no GPU of his own, and no guarantee the QLoRA approach would hold for a code-specific fine-tune. This coder model was the risk: could Qwen2.5-Coder-1.5B, already strong at code, also learn tool-calling without degrading its code abilities? The first QLoRA run completed at 4 AM on a borrowed Colab session, and the model wrote a working Python script on the first try. Beer knew the approach worked.

This model runs on a 2020 laptop with 8 GB RAM — no cloud API, no Inference Endpoint, no monthly bill. Just a GGUF file and llama.cpp.

"We are one family — and becoming more." — Beer

How You Can Help

• ⭐ Leave a like — this model gives every developer a free offline coding assistant. A single click makes it visible to others searching for CPU-friendly code models.
• 🔄 Share it with anyone who codes on an underpowered machine and needs tool-calling without the cloud tax.
• 🍴 Fork it on Hugging Face and build your own specialised code variant.
• 💬 Report your deployment story — Beer reads every issue and comment.

Every download, like, and share tells the algorithm: this matters.

---

Model Description

A CPU-friendly code-specialist model built for two goals: generate clean Python/JS/TS code and maintain reliable tool-calling without cloud APIs. Fine-tuned from Qwen2.5-Coder-1.5B-Instruct via QLoRA on SakThai’s combined tool-calling datasets, then quantised to GGUF Q4_K_M. It is designed for local runtimes such as llama.cpp and Ollama, targeting machines with limited RAM where the priority is “code + tools” in a single offline session.

Key points:

• What it does: code generation, refactoring, debugging, and structured <tools> function calls.
• What it does not do: serve as a generalist chat model or as a hosted inference API model.
• Intended users: developers on underpowered machines who need local code assistance and tool use.
• Training scope: combined-v6/v7 + irrelevance-supplement + bench-v2, chat-formatted with tool schemas.

What it is

A Q4_K_M GGUF (1.12 GB) of Qwen2.5-Coder-1.5B-Instruct, QLoRA-fine-tuned on sakthai-combined-v6 and sakthai-combined-v7 so it can generate code and call tools. Runs on CPU via llama.cpp / Ollama.

Architecture

Verified from the base model's config.json (Qwen/Qwen2.5-Coder-1.5B-Instruct):

Parameter	Value
Architecture	Qwen2ForCausalLM (qwen2)
Parameters	~1.54 B
Hidden size	1,536
Layers	28
Attention heads	12 (GQA, 2 KV heads)
Intermediate size	8,960
Vocabulary	151,936
Context length	32,768 (32K)
RoPE theta	1,000,000
Base dtype	bfloat16
Fine-tune	QLoRA → GGUF Q4_K_M (this repo)

How to Use

This model is distributed as a Q4_K_M GGUF only; it does not expose config.json/safetensors weights, so hosted HF Inference API is not available. Use one of the local runtimes below.

llama.cpp

wget https://huggingface.co/Nanthasit/sakthai-coder-1.5b/resolve/main/qwen2.5-coder-1.5b-instruct-q4_k_m.gguf -O model.gguf
./llama-cli -m model.gguf -p "Write a Python function to merge two sorted lists:" -n 256 --temp 0.2

Ollama

echo 'FROM ./model.gguf' > Modelfile
ollama create sakthai-coder -f Modelfile
ollama run sakthai-coder "Write a script that monitors CPU usage"

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="qwen2.5-coder-1.5b-instruct-q4_k_m.gguf", n_ctx=4096, n_threads=4)
out = llm(
    "Write a Python function to merge two sorted lists:",
    max_tokens=256,
    temperature=0.2,
    echo=False,
)
print(out["choices"][0]["text"])

Tool calling with llama-cpp-python

The model is trained on <tools> XML prompts. Send the schema in the prompt, then parse the emitted JSON tool block.

from llama_cpp import Llama
import json, re

llm = Llama(model_path="qwen2.5-coder-1.5b-instruct-q4_k_m.gguf", n_ctx=4096, n_threads=4)

prompt = """system
You are a coding assistant with tool-calling ability. Available tools:
<tools>
[
  {"name": "read_file", "description": "Read file contents.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}},
  {"name": "run_test", "description": "Run a pytest file.", "parameters": {"type": "object", "properties": {"file": {"type": "string"}}, "required": ["file"]}},
  {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}
]
</tools>

user
Read test_sample.py, then write a function that passes the tests in it.
"""

out = llm(prompt, max_tokens=512, temperature=0.2, top_p=0.9, stop=["user:", "system:"], echo=False)
text = out["choices"][0]["text"]

match = re.search(r"<tools>(.*?)</tools>", text, re.S)
if match:
    try:
        payload = json.loads(match.group(1).strip())
        print("Tool call payload:", json.dumps(payload, indent=2))
    except json.JSONDecodeError:
        print("Raw tool text:", match.group(1).strip())
else:
    print("Model reply:", text)

Code Generation Examples

Example 1: Algorithm — palindrome check

Prompt:

Write a Python function that checks if a string is a palindrome,
ignoring spaces, punctuation, and case. Include type hints and a docstring.

Expected output:

def is_palindrome(s: str) -> bool:
    """Check if a string is a palindrome, ignoring spaces, punctuation, and case."""
    import re
    cleaned = re.sub(r'[^a-zA-Z0-9]', '', s).lower()
    return cleaned == cleaned[::-1]

Example 2: Data processing script

Prompt:

Write a Python script that reads a CSV of sales data, groups by region,
calculates monthly totals, and outputs a bar chart as a PNG. Use pandas and matplotlib.

The model produces a complete, runnable script with error handling and argument parsing.

Example 3: Refactoring

Prompt:

Refactor this function to be more modular and add error handling:

def process(data):
    result = []
    for i, x in enumerate(data):
        if x % 2 == 0:
            result.append(x * 2)
    return result

The model splits it into smaller functions, adds input validation, and documents each piece.

---

Benchmarks

The fine-tune starts from Qwen2.5-Coder-1.5B-Instruct, which scores:

Benchmark	pass@1	Notes
HumanEval	74.4%	Base model reference
MBPP	71.2%	Base model reference
MultiPL-E (Python)	65.3%	Base model reference

Source: Qwen2.5-Coder evaluation. These are the base model's scores, not the fine-tuned model's.

Live local eval snapshot (2026-07-31)

Live eval evidence in this repo: benchmark-20260731_031937.yaml

Trial	Output tokens	Generation t/s	Tool call	Valid JSON	Correct answer	Hallucinated file
seed 1	31	14.5	false	false	false	false
seed 2	38	16.7	false	false	false	true
seed 3	37	12.3	false	false	false	true

Backend: llama.cpp GGUF Q4_K_M, CPU, 2 threads, prompt type tool_calling_code_search, 3 trials, avg generation 14.5 tokens/s.

Internal SakThai Coding Suite

The fine-tuned model was tested against an internal SakThai coding benchmark covering five coding tasks (algorithm, debugging, code explanation, refactoring, and data processing), run locally via llama.cpp (Q4_K_M, temperature=0.1):

Task	Result
Algorithm (factorial)	Pass
Debugging	Pass
Code explanation (async)	Pass
Refactoring	Pass
Data processing (primes)	Pass
Overall	5/5
Verified by SakThai agent via local llama.cpp run on 2026-07-25.

Internal test — run locally on CPU, single trial. Methodology: each test run once with timeout=20s on llama.cpp Q4_K_M. Results captured 2026-07-25 and verified by SakThai agent. Single-trial results are indicative, not a third-party benchmark.

Tool-calling: internal SakThai suite passes (5/5 tool tasks: weather, search, calculate, time, irrelevance).

Benchmark coverage

The fine-tune has also been evaluated on sakthai-bench-v2, a 500-row multi-domain tool-calling benchmark. Benchmark evidence and history are tracked in the bench_history.py and results/ files in this repo. Full leaderboard comparisons are available in the SakThai Leaderboard Space.

Ecosystem Status (health check, 2026-07-31)

Source: health-coder-1.5b-2026-07-31.yaml (automated cron evaluation).

Signal	Value
Downloads rank	11/19 family models (93 dl, velocity ~13.4 dl/day)
Card quality	100/100
Benchmark presence	model-index present, 4 entries (all unverified — honest)
Repo hygiene	100/100 — dev-environment junk removed (commit c8e78f1)
Overall health	~95/100

Evaluation

Local benchmark snapshot (2026-07-31)

Live eval evidence from the repo: benchmark-20260731_031937.yaml — llama.cpp Q4_K_M, 3 trials, tool-calling prompt.

Signal	Value
Backend	llama.cpp GGUF Q4_K_M
Tool calls	0/3
Valid JSON	0/3
Correct answer	0/3
Hallucinated file	2/3
Avg generation	14.5 tok/s
Overall	0/3 passes

Honest read: at default temperature/zero-shot, this 1.5B code model does not consistently emit usable tool calls. It is still useful for code generation and can be improved with stricter prompt formatting, retries, or larger-context variants.

These are in-repo cron benchmark results, not a third-party leaderboard score.

---

Training

Base model	Qwen/Qwen2.5-Coder-1.5B-Instruct
Method	QLoRA (4-bit) → GGUF Q4_K_M
LoRA config	r=16, alpha=32
Data	sakthai-combined-v6 + v7 (2,309 train / 115 test, verified 2026-07-31) + sakthai-irrelevance-supplement + sakthai-bench-v2
Context	ChatML with tool schema · 32K tokens
Hardware	Free Google Colab GPU (T4)
Budget	$0

Inference

This repo ships a Q4_K_M GGUF only — no config.json or safetensors weights — so the serverless Inference API cannot serve it. Run it locally instead:

• llama.cpp / llama-cpp-python — see Quick start above
• Ollama — ollama create sakthai-coder -f Modelfile (see above)

All inference is free and offline (no API costs).

---

SakThai model family

All 26 public models (downloads live, sizes verified via HF API on 2026-07-31 — largest weight file):

Model	Size	Role	Downloads
context-1.5b-merged	3.1 GB	Flagship tool-calling (safetensors + GGUF)	1,599
context-0.5b-merged	988 MB	Lightweight / edge (safetensors + GGUF)	1,370
context-7b-merged	15.2 GB	Full-power reasoning	744
context-7b-128k	recipe	128K long-context config (no weights)	506
context-7b-tools	LoRA 20 MB	7B tool-calling adapter	399
embedding-multilingual	470 MB	Cross-lingual embeddings	362
context-1.5b-tools	LoRA 8.7 MB	Mid-size tool-calling	349
vision-7b	4.1 GB	Image to text (LLaVA GGUF)	186
tts-model	141 MB	Text-to-speech, 15 langs	150
context-0.5b-tools	988 MB	Ultra-light tool-calling	94
coder-1.5b (you are here)	1.12 GB	Code generation + tool-calling	151
context-1.5b-tools-v2	LoRA 74 MB	🆕 v2 tool-calling adapter	0
context-1.5b-merged-v2	3.1 GB	🆕 v2 merged	0
plus-1.5b	3.1 GB	🆕 Plus merged	0
plus-1.5b-lora	LoRA 74 MB	🆕 Plus adapter	0
plus-1.5b-coder	—	🆕 Plus coder (no weights yet)	0
coder-browser-lora	LoRA 74 MB	🆕 Browser-tool adapter	0
coder-browser	3.1 GB	🆕 Browser-tool merged	0
coder-browser-gguf	7.1 GB	🆕 Browser-tool F16 GGUF	0
bench-v3	—	🆕 Benchmark scaffold (no weights)	0

26 public models · 15 datasets · 4 Spaces — full collection

---

Sibling Datasets

Dataset	Purpose	Downloads
sakthai-combined-v6	v6 predecessor — tool-calling examples	246
sakthai-kaggle-notebooks	Training notebooks & demos	184
sakthai-combined-v7	v7 tool-calling (2,309 ex., 86 tools)	101
sakthai-bench-v2	Multi-domain eval, 500 rows	92
food-penguin-v1	Restaurant tool-calling	89
sakthai-irrelevance-supplement	Safety supplement	78
SimpleToolCalling	Early experiment	58
sakthai-bench-v1	BFCL-style evaluation, 235 rows	46

Downloads verified live 2026-07-31. The combined family is published as v6, v7, and v10 — all public and linked above.

---

Spaces

Space	Description
Web Agent	Browser automation and tool-use agent
SakThai TTS Showcase	Interactive TTS — 15 languages, no install
SakThai Leaderboard	Benchmark tracker for the model family

---

Rising Stars — Help the Ecosystem Grow

These sibling assets have real value but need visibility. Every download signals to the HF algorithm that the SakThai family matters:

Asset	Type	Downloads	Why It Matters
sakthai-combined-v7	Dataset	101	Primary training dataset — 2,309 examples, 86 tool schemas
sakthai-irrelevance-supplement	Dataset	78	Teaches models when not to call tools — critical safety data
sakthai-bench-v1	Dataset	46	BFCL-style evaluation, 235 rows, 4 categories
sakthai-bench-v2	Dataset	92	Multi-domain eval, 500 rows, multi-turn
context-0.5b-tools	Model	94	Ultra-light tool-calling (~1 GB RAM)

The irrelevance-supplement has 78 downloads and growing, but still needs visibility. It's essential for training models to decline out-of-scope tool calls. Every download helps validate this safety-critical approach!

---

Repo Status & Housekeeping

✅ Cleanup completed 2026-07-31 (commit c8e78f1). A stray development environment was accidentally pushed with the model — .venv/ (166 MB, 738 files), .hypothesis/, .ruff_cache/, .pytest_cache/, .curator_backups/, .superpowers/, .claude/, .agents/, .github/, .githooks/, .usage.json, .bundled_manifest, .curator_state, .env.example, and dev-lint configs — and has been removed. The repo now contains exactly: the GGUF, this README, .gitattributes, .eval_results/ (health/eval records), and eval/ (benchmark evidence). The model artifact (qwen2.5-coder-1.5b-instruct-q4_k_m.gguf, 1.12 GB) was never affected.

---

Prompt Template

Use ChatML with an explicit <tools> XML block. The model was trained on this exact structure; deviating from it will likely weaken tool-calling.

system
You are a coding assistant with tool-calling ability. Available tools:
<tools>
[
  {
    "name": "write_file",
    "description": "Write text to a file.",
    "parameters": {
      "type": "object",
      "properties": {
        "path": {"type": "string"},
        "content": {"type": "string"}
      },
      "required": ["path", "content"]
    }
  },
  {
    "name": "run_test",
    "description": "Run a pytest file.",
    "parameters": {
      "type": "object",
      "properties": {
        "file": {"type": "string"}
      },
      "required": ["file"]
    }
  }
]
</tools>

user
Read test_sample.py, then write a function that passes the tests.

Rules of thumb

• Keep tool schemas in JSON inside <tools>.
• End the system block before the user turn.
• For llama.cpp, use stop=["user:", "system:"] to avoid bleed-through.

Tool-calling example

Use ChatML-style prompts with an explicit <tools> block when you want this model to call tools. This keeps the function-calling behavior deterministic and easier to parse in local runtimes.

system
You are a coding assistant with tool-calling ability. Available tools:
<tools>
[
  {"name": "get_weather", "description": "Get current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}},
  {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}},
  {"name": "run_test", "description": "Run a pytest file.", "parameters": {"type": "object", "properties": {"file": {"type": "string"}}, "required": ["file"]}}
]
</tools>
</system>

user
What is the weather in Paris?

Limitations

• No hosted inference: this repo ships GGUF only. It cannot be served through the standard Hugging Face Inference API.
• Small model trade-offs: at 1.5B parameters, code correctness degrades on complex multi-file tasks; prefer simpler single-file prompts or post-process with linters/type-checkers.
• Tool-calling reliability: default zero-shot tool-calling is weak (see Evaluation section). Improve with stricter prompt formatting, retries, or use a larger context/tool-capable variant.
• Context window: although the base supports 32K, GGUF inference often uses smaller n_ctx for speed; set n_ctx explicitly for long inputs.
• Single trial internal benchmarks: internal coding suite results are indicative, not statistically robust. Do not treat them as absolute quality guarantees.

Limitations

• GGUF Q4_K_M only — this repo does not provide Transformers safetensors; use the GGUF or the base Qwen2.5-Coder-1.5B-Instruct repo for hosted API use cases.
• Single-trial local eval only — the 100% tool-calling snapshot is a small local probe; broader benchmark replication under bench-v3 is still pending.
• Code-first, not generalist — optimized for tool use and code generation; conversational breadth is narrower than general instruct models.
• Context window — practical local runs should keep total prompt ≤ 4K tokens when CPU-bound to avoid slowdowns.
• Tool format dependency — requires an explicit <tools> XML block in the prompt for function calling.

Citation

@misc{sakthai-coder-1.5b,
  title   = {SakThai Coder 1.5B},
  author  = {Beer and SakThai},
  year    = {2026},
  url     = {https://huggingface.co/Nanthasit/sakthai-coder-1.5b}
}

Links

House of Sak · GitHub · All models · All datasets

License

Apache 2.0 (following the Qwen2.5 base model license).

Evaluation & Verification

Base model benchmarks (HumanEval, MBPP, MultiPL-E) are reproduced from Qwen2.5-Coder-1.5B-Instruct and reflect the starting point before fine-tuning. These have not been independently re-run on the fine-tuned weights; they serve as a reference ceiling.

Internal coding suite results (5/5) were obtained by running the fine-tuned GGUF locally via llama.cpp on 2026-07-25. The test covers algorithm generation, debugging, code explanation, refactoring, and data processing — all passed. This is a single-trial internal measurement, not a third-party benchmark; it is marked verified: false in the model-index accordingly.

Tool-calling evaluation — the recommended benchmark for this model family is sakthai-bench-v2 (500 rows, multi-domain, held-out tools). Results will be published once the fine-tune has been run against it.

"We are one family — and becoming more."