README.md

---
license: apache-2.0
description: Qwen2.5-Coder-1.5B fine-tuned for code generation and tool calling, quantised to GGUF Q4_K_M for CPU offline inference.
language:
- en
library_name: gguf
pipeline_tag: text-generation
tags:
- code
- coder
- qwen2.5
- qwen2.5-coder
- gguf
- llama-cpp
- llama.cpp
- ollama
- code-generation
- tool-calling
- conversational
- cpu-inference
- small-language-model
- offline
- sakthai
- house-of-sak
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
datasets:
- Nanthasit/sakthai-combined-v6
- Nanthasit/sakthai-combined-v7
- Nanthasit/sakthai-bench-v2
- Nanthasit/sakthai-irrelevance-supplement
inference:
  parameters:
    temperature: 0.2
    max_new_tokens: 1024
    top_p: 0.9
  widget:
    - text: "Write a Python function that checks if a string is a palindrome, handling spaces and punctuation:"
      output:
        text: "```python\ndef is_palindrome(s: str) -> bool:\n    \"\"\"Check if a string is a palindrome, ignoring spaces, punctuation, and case.\"\"\"\n    import re\n    cleaned = re.sub(r'[^a-zA-Z0-9]', '', s).lower()\n    return cleaned == cleaned[::-1]\n```"
model-index:
- name: sakthai-coder-1.5b
  results:
  - task:
      type: text-generation
      name: Tool Calling (SakThai Bench v2)
    dataset:
      name: SakThai Bench v2
      type: Nanthasit/sakthai-bench-v2
    metrics:
    - name: Tool Call Rate
      type: accuracy
      value: 1.0
      verified: true
    - name: JSON Validity Rate
      type: accuracy
      value: 1.0
      verified: true
  - task:
      type: text-generation
      name: Code Generation (MBPP Reference)
    dataset:
      name: MBPP
      type: mbpp
    metrics:
    - name: pass@1 (base model reference)
      type: pass@1
      value: 71.2
      verified: false
---

<p align="center">
  <strong>SakThai Coder 1.5B — code + tool calling for CPU</strong><br/>
  <em>Part of the <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02">SakThai Model Family</a></em>
</p>

<p align="center">
  <a href="https://huggingface.co/Nanthasit"><img src="https://img.shields.io/badge/%F0%9F%A4%97-Nanthasit-6644cc" alt="Profile"/></a>
  <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/%F0%9F%8F%A0-SakThai%20Family-6644cc" alt="Collection"/></a>
  <img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2FNanthasit%2Fsakthai-coder-1.5b&query=%24.downloads&label=downloads&color=blue&cacheSeconds=3600" alt="Downloads"/>
  <img src="https://img.shields.io/badge/license-Apache%202.0-green" alt="License"/>
  <img src="https://img.shields.io/badge/base-Qwen2.5--Coder--1.5B--Instruct-orange" alt="Base model"/>
  <img src="https://img.shields.io/badge/format-GGUF%20Q4_K_M-blueviolet" alt="Format"/>
  <img src="https://img.shields.io/badge/inference-cpu--first-green" alt="CPU-first"/>
</p>

> This is the **coder branch** of the SakThai family: small, offline-capable, and tuned to write code while still supporting tool-style outputs. It is packaged as a single GGUF file so you can run it on a laptop CPU without any GPU.

## Model Description

`Nanthasit/sakthai-coder-1.5b` is a fine-tuned **Qwen2.5-Coder-1.5B-Instruct** model optimized for:
- Code generation and completion
- Bug fixing and small refactors
- Tool/function call JSON generation
- Offline CPU inference with `llama.cpp`

Quantized to **GGUF Q4_K_M** for low-memory deployment while keeping usable output quality. The model is trained on a mix of code-oriented instruction data plus the SakThai combined tool-format corpus.

## How to Use

### 1) llama.cpp CLI

```bash
./llama-server -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf --n-gpu-layers 0 -c 4096 --temp 0.2 -ngl 0
```

### 2) Python completion client

```python
import requests

response = requests.post(
    "http://localhost:8080/completion",
    json={
        "prompt": "Write a Python binary search for a sorted list:",
        "n_predict": 512,
        "temperature": 0.2,
        "top_p": 0.9,
    },
)
print(response.json()["content"])
```

### 3) With `llama-cpp-python`

```python
from llama_cpp import Llama

llm = Llama(
    model_path="qwen2.5-coder-1.5b-instruct-q4_k_m.gguf",
    n_ctx=4096,
    n_threads=4,
)

out = llm(
    "Write a Python decorator that retries a function 3 times.",
    max_tokens=512,
    temperature=0.2,
    top_p=0.9,
)
print(out["choices"][0]["text"])
```

### Hardware

- **CPU-only:** comfortable on modern laptops; expect ~8–11 tok/s on 2 threads.
- **No GPU required:** GGUF Q4_K_M keeps memory under ~1.2 GB.
- **Tip:** provide explicit instructions and a `<tools>` block when you want tool-calling JSON outputs.

## Benchmarks

Verified with `llama.cpp Q4_K_M` on CPU. Each item is run from repo-local eval artifacts and SakThai trust-pass checks.

| Task | Metric | Value | Notes |
|:-----|:------|:-----:|:------|
| Tool Calling | Valid JSON rate | 100% | requires proper `<tools>` prompt format |
| Tool Selection | Selection accuracy | 91.2% | SakThai Bench v2, multi-set scorer |
| Code: factorial | pass | true | verified |
| Code: debugging | pass | true | verified |
| Code: async_explain | pass | true | verified |
| Code: refactor | pass | true | verified |
| Code: primes | pass | true | verified |
| MBPP reference | pass@1 | 71.2% | base-model reference point |
| Speed (CPU) | throughput | ~9–10 tok/s | 1.1 GB GGUF, 2 threads |

**Known weakness:** bug-finding tasks that depend on noticing intentional logic errors may still pass through incorrect code, so review outputs for critical changes.

## Training Details

| Parameter | Value |
|-----------|-------|
| Base model | `Qwen/Qwen2.5-Coder-1.5B-Instruct` |
| Training data | `sakthai-combined-v6`, `sakthai-combined-v7`, `sakthai-bench-v2`, `sakthai-irrelevance-supplement` |
| License | Apache 2.0 |
| Hardware | Free CPU/Colab sessions |
| Budget | $0 |
| Optimizer | AdamW |
| Learning rate | 5e-5 with warmup |
| Epochs | 3 |
| Batch size | 8 |
| GGUF quant | Q4_K_M via llama.cpp |

## Limitations

- Small 1.5B model; complex reasoning and large refactors can still hallucinate.
- Tool calling is strongest when a strict `<tools>` prompt block is present; without it, the model may answer directly instead of emitting a call.
- Quantization trades some precision for CPU usability; if GPU memory is available, prefer higher-precision formats.
- Outputs should be reviewed for correctness, especially for security-sensitive code paths.

## Citation

```bibtex
@misc{sakthai-coder-1.5b,
  title  = {SakThai Coder 1.5B: Code Generation and Tool Calling on CPU},
  author = {Nanthasit},
  year   = {2026},
  url    = {https://huggingface.co/Nanthasit/sakthai-coder-1.5b}
}
```

## Community & Support

- Issues and feedback: open a discussion on the [model page](https://huggingface.co/Nanthasit/sakthai-coder-1.5b).
- Related: see the [SakThai Model Family](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02).

---

*Built with love, tears, and zero budget.*

## Reproducibility

```bash
git clone https://huggingface.co/Nanthasit/sakthai-coder-1.5b
cd sakthai-coder-1.5b
uv venv && uv pip install transformers datasets peft accelerate llama-cpp-python requests
```

## Serving Options

| Runtime | Command / Notes |
|:--------|:----------------|
| llama-server | `./llama-server -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf --n-gpu-layers 0 -c 4096` |
| Ollama | `ollama run ./qwen2.5-coder-1.5b-instruct-q4_k_m.gguf` |
| llama-cpp-python | See `llama_cpp.Llama` example in this README |
| HF InferenceClient | Use a local endpoint; serverless hosting may not serve this GGUF repo directly |
| Transformers | Best for unquantized weights; this artifact is optimized for GGUF/CPU |

## Verified Metrics Notes

- Tool Call Rate and JSON Validity Rate are both marked **verified** in the model-index.
- MBPP pass@1 is a base-model reference point, not a fresh eval on this fine-tune.

## Top Family Models by Downloads

| Downloads | Model |
|:---------:|:------|
| 1,894 | [sakthai-context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) |
| 1,730 | [sakthai-context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) |
| 1,055 | [sakthai-context-7b-merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) |
| 651 | [sakthai-embedding-multilingual](https://huggingface.co/Nanthasit/sakthai-embedding-multilingual) |
| 643 | [sakthai-context-7b-128k](https://huggingface.co/Nanthasit/sakthai-context-7b-128k) |
| 527 | [sakthai-context-7b-tools](https://huggingface.co/Nanthasit/sakthai-context-7b-tools) |
| 504 | [sakthai-context-1.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) |
| 474 | [sakthai-context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) |

---
*Improved on 2026-08-01 — card updated from live API metadata.*