Instructions to use Nanthasit/sft-out with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanthasit/sft-out with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanthasit/sft-out")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Nanthasit/sft-out", device_map="auto")

PEFT
How to use Nanthasit/sft-out with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Nanthasit/sft-out with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanthasit/sft-out"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sft-out",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanthasit/sft-out

SGLang

How to use Nanthasit/sft-out with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanthasit/sft-out" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sft-out",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanthasit/sft-out" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sft-out",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Nanthasit/sft-out with Docker Model Runner:
```
docker model run hf.co/Nanthasit/sft-out
```

sft-out / README.md

SakThai Agent

fix: remove phantom combined-v10 references

39cdacd about 7 hours ago

preview code

Raw

History Blame Contribute Delete

8.76 kB

	---
	license: apache-2.0
	language:
	- en
	base_model: Nanthasit/sakthai-context-0.5b-tools
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- generated_from_trainer
	- trl
	- sft
	- peft
	- lora
	- adapter
	- finetuned
	- conversational
	- assistant
	- tool-calling
	- function-calling
	- qwen
	- qwen2.5
	- slm
	- small-language-model
	- sakthai
	- house-of-sak
	- eval-results
	datasets:
	- Nanthasit/sakthai-combined-v7
	inference:
	requires_base: true
	adapter_only: true
	parameters:
	temperature: 0.3
	max_new_tokens: 128
	top_p: 0.9
	model-index:
	- name: SakThai SFT-out — LoRA Adapter
	results:
	- task:
	type: text-generation
	name: Adapter merge readiness
	dataset:
	type: internal
	name: Adapter merge pipeline
	metrics:
	- type: merge-status
	value: 0
	name: Standalone usable without merge
	verified: false
	- type: tool-call-success
	value: pending
	name: Tool-call success post-merge
	verified: false
	---

	<div align="center">

	[![Base Model](https://img.shields.io/badge/Base_Model-Nanthasit/sakthai-context--0.5b--tools-green)](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools)
	[![LoRA](https://img.shields.io/badge/LoRA-r8%20%C3%8116-orange)](https://huggingface.co/Nanthasit/sft-out)
	[![TRL](https://img.shields.io/badge/TRL-1.9.2-red)](https://github.com/huggingface/trl)
	[![License](https://img.shields.io/badge/License-Apache_2.0-green)](https://huggingface.co/Nanthasit/sft-out)

	</div>

	> ⚠️ Adapter-only repository — this repo contains only a LoRA adapter, not a standalone model. You must merge it with the base model before inference or GGUF export.

	# Model Card for `sft-out`

	## Model Description

	\| Property \| Value \|
	\|:---------\|------:\|
	\| Base model \| [Nanthasit/sakthai-context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) (Qwen2.5-0.5B-Instruct) \|
	\| Adapter type \| LoRA \|
	\| LoRA rank `r` \| 8 \|
	\| LoRA alpha \| 16 \|
	\| Dropout \| 0.0 \|
	\| Target modules \| `q_proj`, `k_proj`, `v_proj`, `o_proj` \|
	\| Adapter size \| ~158 KB \|
	\| Max context \| 32,768 tokens \|
	\| Parameters \| 495M base + adapter \|
	\| Language \| English \|

	This adapter is a low-rank delta over the base SakThai 0.5B tool-calling model. It preserves the same architecture and tokenizer.

	## What's in the box

	\| File \| Size \| Description \|
	\|:-----\|-----:\|:------------\|
	\| `adapter_model.safetensors` \| 138 KB \| LoRA delta weights \|
	\| `adapter_config.json` \| 1.1 KB \| Adapter hyperparameters \|
	\| `training_args.bin` \| 129 B \| Serialized training args \|
	\| `tokenizer.json` \| 13.2 KB \| Tokenizer \|
	\| `tokenizer_config.json` \| 694 B \| Tokenizer config \|
	\| `chat_template.jinja` \| 2.5 KB \| Chat template \|

	Total adapter size: ~158 KB

	## How to Use

	### Merge with base model

	```python
	from transformers import AutoTokenizer, AutoModelForCausalLM
	from peft import PeftModel

	base = "Nanthasit/sakthai-context-0.5b-tools"
	adapter = "Nanthasit/sft-out"

	tokenizer = AutoTokenizer.from_pretrained(base)
	model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
	model = PeftModel.from_pretrained(model, adapter)
	```

	Generate:

	```python
	messages = [{"role": "user", "content": "What's the weather in Tokyo?"}]
	inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
	outputs = model.generate(inputs, max_new_tokens=128, temperature=0.3, top_p=0.9)
	print(tokenizer.decode(outputs[0], skip_special_tokens=True))
	```

	### Merge to GGUF (llama.cpp)

	```bash
	python -m peft.utils.merge_adapters \
	--base_model Nanthasit/sakthai-context-0.5b-tools \
	--adapter Nanthasit/sft-out \
	--output merged-model
	python -m transformers.convert_save_to_gguf merged-model --outtype q4_k_m --outfile sakthai-0.5b-sft-out-q4_k_m.gguf
	```

	Then run with llama.cpp:

	```bash
	./llama-cli -m sakthai-0.5b-sft-out-q4_k_m.gguf -p "What's the weather in Tokyo?" -n 128 --temp 0.3
	```

	## Tool-Calling Format

	This adapter was trained on the SakThai tool-calling format.

	Prompt format:

	```xml
	<tools>
	{
	"function": "get_weather",
	"params": {"location": "Bangkok"}
	}
	</tools>
	```

	Model output:

	```json
	{"name": "get_weather", "arguments": {"location": "Bangkok"}}
	```

	Multi-turn tool results:

	```xml
	<tool_result>
	{"result": "The weather in Bangkok is 32°C and sunny."}
	</tool_result>
	```

	## Benchmarks

	No published benchmarks for this specific adapter yet. The base model
	[Nanthasit/sakthai-context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools)
	achieves strong single-shot tool-calling on [sakthai-bench-v2](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v2).

	For the SakThai family benchmarks, see the [SakThai Leaderboard](https://huggingface.co/spaces/Nanthasit/sakthai-leaderboard).

	## Training Details

	\| Attribute \| Value \|
	\|:----------\|:------\|
	\| Base model \| [Nanthasit/sakthai-context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) \|
	\| Method \| Supervised Fine-Tuning (SFT) via TRL \|
	\| TRL version \| 1.9.2 \|
	\| Transformers \| 5.14.1 \|
	\| PyTorch \| 2.11.0 \|
	\| Datasets \| 5.0.0 \|
	\| Tokenizers \| 0.22.2 \|

	> Note: Exact training hyperparameters (learning rate, batch size, epochs) are not available from this repo alone. The `training_args.bin` file is present but contains unparsed binary data. Check sibling model `sakthai-context-0.5b-tools` for comparable settings.

	## Limitations

	- Adapter-only. Must be loaded on top of the base model. Cannot be used standalone via `AutoModelForCausalLM`.
	- No standalone GGUF. Must be merged with base before GGUF conversion.
	- Not servable serverless. The Hugging Face Inference API does not support dynamic LoRA adapter loading.
	- Experimental. This is a new SFT training experiment; performance has not been extensively evaluated.
	- Placeholder tokenizer. The `tokenizer.json` in this repo is a placeholder; the actual tokenizer lives in the base model repo.
	- License compatibility depends on base-model license (`apache-2.0`).

	## Working Alternatives

	\| Model \| Type \| Best for \|
	\|:------\|:-----\|:---------\|
	\| [sakthai-context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) \| Base adapter (8.7 MB) \| Strongest standalone 0.5B tool caller \|
	\| [sakthai-context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) \| Merged (988 MB) \| Direct inference, no merge needed \|
	\| [sakthai-context-0.5b-tools-sft-v2](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools-sft-v2) \| Adapter (8.3 MB) \| Newer SFT variant with richer data \|
	\| [sakthai-context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) \| Merged (3.1 GB) \| Mid-size, best balance \|

	## Citation

	If you use this adapter or the SakThai family in your work, please cite:

	```bibtex
	@misc{sakthai-sft-out,
	title = {SakThai SFT-out LoRA Adapter},
	author = {Beer (beer-sakthai) and the SakThai Agent family},
	year = {2026},
	publisher = {Hugging Face},
	howpublished = {\url{https://huggingface.co/Nanthasit/sft-out}
	}
	```

	Base model:

	```bibtex
	@misc{qwen25-2025,
	title = {Qwen2.5: A Party of Foundation Models},
	author = {Qwen Team},
	year = {2025},
	url = {https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct}
	}
	```

	TRL:

	```bibtex
	@software{vonwerra2020trl,
	title = {{TRL: Transformers Reinforcement Learning}},
	author = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
	license = {Apache-2.0},
	url = {https://github.com/huggingface/trl},
	year = {2020}
	}
	```

	## SakThai Family

	This adapter is part of the [SakThai model family](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02).

	\| Model \| Downloads \| Type \|
	\|:------\|:---------:\|:-----\|
	\| [context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) \| 1,855 \| Merged weights \|
	\| [context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) \| 1,692 \| Merged weights \|
	\| [context-7b-merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) \| 1,024 \| Merged weights \|
	\| [context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) \| 251 \| Base adapter \|
	\| [context-0.5b-tools-sft](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools-sft) \| 0 \| SFT v1 \|
	\| [context-0.5b-tools-sft-v2](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools-sft-v2) \| 0 \| SFT v2 \|
	\| sft-out \| 0 \| Adapter-only experiment \|

	Downloads verified 2026-07-31.

	---

	Built with zero budget. Part of the [House of Sak](https://huggingface.co/Nanthasit).