Instructions to use applegrew/support-125M-slm-sft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use applegrew/support-125M-slm-sft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="applegrew/support-125M-slm-sft")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("applegrew/support-125M-slm-sft")
model = AutoModelForCausalLM.from_pretrained("applegrew/support-125M-slm-sft", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use applegrew/support-125M-slm-sft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "applegrew/support-125M-slm-sft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "applegrew/support-125M-slm-sft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/applegrew/support-125M-slm-sft

SGLang

How to use applegrew/support-125M-slm-sft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "applegrew/support-125M-slm-sft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "applegrew/support-125M-slm-sft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "applegrew/support-125M-slm-sft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "applegrew/support-125M-slm-sft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use applegrew/support-125M-slm-sft with Docker Model Runner:
```
docker model run hf.co/applegrew/support-125M-slm-sft
```

Support 125M SLM - SFT (Instruction-Tuned + RAFT)

A 125M parameter Llama-style language model trained from scratch on ~2.6B tokens of IT support data, then instruction-tuned on ~17K examples including RAFT (Retrieval-Augmented Fine-Tuning) data.

Built from the pretrained base model (applegrew/support-125M-slm-base).

Capabilities

Capability	Example
IT troubleshooting	"My VPN keeps disconnecting" → step-by-step help
Casual chat	"Hi" → "Hello! How can I help you today?"
Follow-ups	"That didn't work" → "Let's try another approach"
Grounded answers (RAFT)	Given a KB article, answers only from it
Refusals (RAFT)	"Not in the context" → "I don't have enough information to answer"

Training Data (17,002 records)

Source	Pairs	Method	Cost
StackExchange (filtered IT)	10,372	Direct extraction	$0
Ubuntu IRC + Gemini	2,821	Teacher distillation	$1.50
Casual interactions	732	Seed + Gemini expansion	$0.04
RAFT dataset	3,077	Gemini 3.5 Flash-Lite	$0.40
Total	17,002		~$1.94

RAFT Pairs Breakdown

Type	Count	Behavior taught
Answerable (grounded)	2,306	Answer strictly from provided context
Unanswerable (refusal)	771	Say "not enough information"

Training Details

Setting	Value
Epochs	3
Learning rate	2e-4 (cosine decay)
Batch size	8
Loss masking	Assistant tokens only
Hardware	1x H100
Best val loss	2.099
Val perplexity	8.83
Total SFT cost	~$2.20

Usage

Plain chat

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("applegrew/support-125M-slm-sft")
tokenizer = AutoTokenizer.from_pretrained("applegrew/support-125M-slm-sft")

chat = "<|bos|><|system|>\nYou are a helpful IT support technician.<|eos|>\n<|user|>\nMy VPN keeps disconnecting every 5 minutes<|eos|>\n<|assistant|>\n"
inputs = tokenizer(chat, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))

RAG / Grounded (RAFT) style

context = "WiFi drops on Ubuntu 22.04. Run 'iwconfig', check Power Management, disable with 'sudo iwconfig wlan0 power off'."
question = "Why does my wifi keep dropping?"

chat = (
    "<|bos|><|system|>\nYou are a helpful IT support technician. Answer using ONLY the "
    "provided context. If the answer is not in the context, say you do not have enough "
    "information to answer.<|eos|>\n"
    f"<|user|>\n\n{context}\n\n\nQuestion: {question}<|eos|>\n<|assistant|>\n"
)

Chat Template

<|bos|><|system|>
{system_prompt}<|eos|>
<|user|>
{question}<|eos|>
<|assistant|>
{answer}<|eos|>

Only the assistant's answer contributes to the loss during training.

Web Demo

Try it live: https://vercel-liart-xi.vercel.app

Project Summary

Phase	Cost	Description
Data pipeline	$0	Clean → dedup → tokenize 2.6B IT tokens
Tokenizer	$0	16K BPE trained on corpus
Pretraining (6 epochs)	$31	125M Llama, 8x H100
SFT data	~$1.94	Gemini distillation + filtering + RAFT
SFT training (3 rounds)	~$2.20	1x H100
Total	~$35

Downloads last month: 55

Safetensors

Model size

0.1B params

Tensor type

BF16