Instructions to use Nanthasit/sakthai-context-7b-128k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanthasit/sakthai-context-7b-128k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanthasit/sakthai-context-7b-128k")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-7b-128k")
model = AutoModelForCausalLM.from_pretrained("Nanthasit/sakthai-context-7b-128k")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Nanthasit/sakthai-context-7b-128k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanthasit/sakthai-context-7b-128k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-7b-128k",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-7b-128k

SGLang

How to use Nanthasit/sakthai-context-7b-128k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanthasit/sakthai-context-7b-128k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-7b-128k",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanthasit/sakthai-context-7b-128k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-7b-128k",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Nanthasit/sakthai-context-7b-128k with Docker Model Runner:
```
docker model run hf.co/Nanthasit/sakthai-context-7b-128k
```

SakThai Context 7B — 128K Extended

A context-extended version of Nanthasit/sakthai-context-7b-merged, adapted to 128K tokens using YaRN (Yet another RoPE extensioN method).

Context Window

Property	Original	Extended
`max_position_embeddings`	32,768	131,072
YaRN factor	—	4.0x
RoPE theta	1,000,000	1,000,000

Method

Base model: Nanthasit/sakthai-context-7b-merged (Qwen2.5-7B-Instruct + tool-calling LoRA)
Extension: YaRN with factor=4.0, original_max_position_embeddings=32768
Approach: Config-only — no training required. Qwen2.5 supports YaRN natively.
Deployment: Use with vLLM, TGI, or transformers. Note: vLLM uses static YaRN — scaling factor is fixed, so short-context accuracy (<32K) has a minor trade-off. Use the 32K main branch for fast tool-calling, and this 128K branch for heavy document retrieval.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Nanthasit/sakthai-context-7b-128k",
    torch_dtype="bfloat16",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-7b-128k")

# Use with long prompts up to 128K tokens
prompt = "Your long document here..."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)