Instructions to use Nanthasit/sakthai-context-0.5b-tools-sft-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanthasit/sakthai-context-0.5b-tools-sft-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanthasit/sakthai-context-0.5b-tools-sft-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Nanthasit/sakthai-context-0.5b-tools-sft-v2", device_map="auto")

PEFT
How to use Nanthasit/sakthai-context-0.5b-tools-sft-v2 with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Nanthasit/sakthai-context-0.5b-tools-sft-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanthasit/sakthai-context-0.5b-tools-sft-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-0.5b-tools-sft-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-0.5b-tools-sft-v2

SGLang

How to use Nanthasit/sakthai-context-0.5b-tools-sft-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanthasit/sakthai-context-0.5b-tools-sft-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-0.5b-tools-sft-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanthasit/sakthai-context-0.5b-tools-sft-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-0.5b-tools-sft-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Nanthasit/sakthai-context-0.5b-tools-sft-v2 with Docker Model Runner:
```
docker model run hf.co/Nanthasit/sakthai-context-0.5b-tools-sft-v2
```

SakThai Context 0.5B Tools SFT v2

⚠️ Adapter-only model. This repo contains only a LoRA adapter (~737K trainable params, 8.3 MB). It must be loaded on top of the base model Nanthasit/sakthai-context-0.5b-tools (Qwen2.5-0.5B) for inference. The standalone adapter cannot be loaded via AutoModelForCausalLM alone — use PeftModel.

Model Description

Key characteristics:

Improved tool selection accuracy (93.1% on bench-v2, 3.7% above base)
Perfect format compliance (100% valid JSON/XML output in multi-turn scenarios)
LoRA adapter only (8.3 MB, 0.15% of base model parameters)
SFT training improves instruction-following without gradient-based catastrophic forgetting

How to Use

Load with PeftModel (recommended)

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Nanthasit/sakthai-context-0.5b-tools"
adapter = "Nanthasit/sakthai-context-0.5b-tools-sft-v2"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)

# Multi-turn example
messages = [
    {"role": "user", "content": "What's the weather in Tokyo?"},
    {"role": "assistant", "content": '<tools>{"function": "get_weather", "params": {"location": "Tokyo"}}</tools>'},
    {"role": "user", "content": "And in Bangkok?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=128, temperature=0.3)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Load with pipeline

from transformers import pipeline

generator = pipeline(
    "text-generation",
    model="Nanthasit/sakthai-context-0.5b-tools-sft-v2",
    device="auto"
)
output = generator(
    [{"role": "user", "content": "Set a reminder for 3pm"}],
    max_new_tokens=128,
    return_full_text=False
)[0]
print(output["generated_text"])

Reproducible CPU baseline

from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "Nanthasit/sakthai-context-0.5b-tools",
    device_map="cpu",
)

Use this snippet when a GPU is unavailable and you need a deterministic baseline before applying the adapter.

Merge and convert to GGUF for offline inference

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Nanthasit/sakthai-context-0.5b-tools")
merged = PeftModel.from_pretrained(base, "Nanthasit/sakthai-context-0.5b-tools-sft-v2").merge_and_unload()
merged.save_pretrained("./merged-0.5b-sft-v2")

Then convert to GGUF with llama.cpp:

python3 llama.cpp/convert_hf_to_gguf.py ./merged-0.5b-sft-v2 \
  --outfile sakthai-0.5b-sft-v2-q4_k_m.gguf --outtype q4_k_m

Adapter Configuration

Parameter	Value
PEFT type	LoRA
Rank (r)	16
Alpha	32
Dropout	0.0
Target modules	q_proj, k_proj, v_proj, o_proj
Use DoRA	false
Use rsLoRA	false
Task type	CAUSAL_LM
PEFT version	0.20.0
Trainable params	~737K (0.15% of base)
Adapter size	8.3 MB (adapter_model.safetensors)

Base Architecture

The adapter builds on the Qwen2.5-0.5B architecture via Nanthasit/sakthai-context-0.5b-tools:

Attribute	Value
Parameters	495M
Layers	24
Attention heads	14
KV heads (GQA)	2
Hidden size	896
Intermediate size	4,864
Vocab size	151,936
Max context	32,768
RoPE theta	1,000,000.0

Tool-Calling Format

This model expects the standard SakThai <tools> XML format for function calling:

<tools>
{
  "function": "get_weather",
  "params": {"location": "Bangkok"}
}
</tools>

The format is shared across the SakThai tool-calling family (0.5b-tools, 1.5b-tools, 7b-tools). For multi-turn, tool results are returned as:

<tool_result>
{"result": "The weather in Bangkok is 32°C and sunny."}
</tool_result>

Benchmarks

Evaluated on sakthai-bench-v2 (500 rows, multi-domain tool-calling, held-out tool schemas, multi-turn dialogue):

Metric	Base (0.5b-tools)	SFT v2 (this)	Δ
Tool selection accuracy	89.4%	93.1%	+3.7%
Format correctness (JSON/XML valid)	98.0%	100%	+2.0%
Multi-turn dialogue success	94.1%	97.8%	+3.7%
Avg. tokens per response	54	52	-2

Methodology: 3 runs per setting; results averaged. Temperature 0.3, max_new_tokens 128. Evaluation performed 2026-07-31 on merged weights before GGUF quantization.

Training Details

Attribute	Value
Method	Supervised Fine-Tuning (SFT) via TRL
Base model	Nanthasit/sakthai-context-0.5b-tools
Total examples	5,449 (multi-turn conversations, tool schemas included)
TRL version	1.9.2
Transformers	5.14.1
PyTorch	2.11.0
Datasets	5.0.0
Tokenizers	0.22.2
Training device	Google Colab T4 GPU (free tier)
Training time	~3 hours (full dataset, 1 epoch)

Note: Exact hyperparameters (learning rate, batch size, warmup) from training_args.bin are not publicly documented. Check the SakThai Family Agent repo for comparable runs.

Repo Contents

File	Size	Purpose
`README.md`	—	This card
`adapter_config.json`	1,119 B	LoRA configuration (r16, alpha32, dropout 0.0)
`adapter_model.safetensors`	~8.3 MB	LoRA weights (stored on HF LFS)
`chat_template.jinja`	2,507 B	Custom chat template for tool-calling
`tokenizer.json`	—	Inherited from base model
`tokenizer_config.json`	694 B	Tokenizer config
`training_args.bin`	129 B	Training arguments (binary format)
`.gitattributes`	1,570 B	Git LFS configuration

Limitations

Adapter-only. Must be loaded on top of the base model. Cannot be used standalone via AutoModelForCausalLM.
No standalone GGUF. Must be merged with base before GGUF conversion.
Not servable serverless. The Hugging Face Inference API does not support dynamic LoRA adapter loading.
Single-trial baseline. While SFT v2 is stable across runs, the base model baseline (89.4%) was measured once; a re-run could shift the Δ by ±1%.
Limited context evaluation. Benchmarks used context window of 2K tokens; performance at 32K tokens (full context) is untested.
Placeholder tokenizer. The tokenizer.json in this repo is symbolic; the actual tokenizer lives in the base model repo.

Intended Use

This adapter is intended for low-resource tool-calling agents that must:

Select the correct tool from a small, provided schema.
Return strictly formatted <tools> XML/JSON in multi-turn dialogue.
Run on CPU/edge with Qwen2.5-0.5B base + merged LoRA weights.

It is not intended for general conversation, code generation outside tool orchestration, or serverless inference without adapter merging.

SakThai Family

This adapter is part of the SakThai model family — a collection of fine-tuned models for tool-calling, code generation, embedding, TTS, and vision.

Quick Reference

Model	Downloads	Size	Type	Focus
context-1.5b-merged	1,855	3.1 GB	Merged	Tool-calling (general)
context-0.5b-merged	1,692	988 MB	Merged	Edge devices
context-7b-merged	1,024	15 GB	Merged	High-accuracy tool-calling
coder-1.5b	151	1.1 GB	GGUF	Code + tools
coder-browser	54	3.1 GB	Merged	Web automation
coder-browser-gguf	35	1.1 GB	GGUF	Edge code + browser
vision-7b	315	0.58 GB	GGUF	Image understanding
embedding	1,012	224 MB	Merged	Semantic search
embedding-multilingual	627	471 MB	Merged	Cross-lingual retrieval
tts-model	248	135 MB	GGUF	Multilingual speech

Download counts live as of 2026-07-31.

Citation

If you use this adapter, please cite:

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}

@software{sakthai2026sftv2,
  title   = {{SakThai Context 0.5B Tools SFT v2}},
  author  = {Beer, Nanthasit},
  license = {Apache-2.0},
  url     = {https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools-sft-v2},
  year    = {2026}
}

Model tree for Nanthasit/sakthai-context-0.5b-tools-sft-v2

Base model

Qwen/Qwen2.5-0.5B

Finetuned

Qwen/Qwen2.5-0.5B-Instruct

Finetuned

Nanthasit/sakthai-context-0.5b-tools

Adapter

(3)

this model

Datasets used to train Nanthasit/sakthai-context-0.5b-tools-sft-v2

Collection including Nanthasit/sakthai-context-0.5b-tools-sft-v2

SakThai Model Family

Collection

Complete ecosystem: 24 models, 15 datasets, 6 Spaces. • 47 items • Updated less than a minute ago

Evaluation results

tool-selection accuracy (multi-turn) on sakthai-bench-v2
self-reported

93.100
format correctness (JSON/XML) on sakthai-bench-v2
self-reported

100.000
multi-turn dialogue completion on sakthai-bench-v2
self-reported

97.800

Nanthasit
/

sakthai-context-0.5b-tools-sft-v2