gittensor-model-hub/Qwythos-9B-nsys-SFT

Qwythos-9B nsys SFT (self-taught)

LoRA + GGUF fine-tune of Qwythos-9B for Nsight Systems (nsys) / CUDA-L1 tool-calling.

Self-taught (not API distillation)

The SFT trajectories were generated by this same local model stack, not by an external teacher LLM API (no GPT/Claude/OpenRouter demos):

1. Qwythos-9B served locally with sparkinfer on an RTX 5090
2. Autonomous agent loop (prof_dataset_gen) + real nsys / CUDA-L1 tools
3. Quality-filtered dataset → cuda-nsys-training
4. QLoRA SFT back into the same Qwythos base

Base pretrained weights still come from Empero/Qwen; the tool-use teaching signal is self-generated.

Contents

Path	Description
adapter/	PEFT LoRA (r=16, α=32)
gguf/Qwythos-9B-nsys-SFT-Q4_K_M.gguf	Merged no-MTP Q4_K_M (~5.3G) for sparkinfer
gguf/Qwythos-9B-nsys-SFT-noMTP-BF16.gguf	Merged no-MTP BF16 (~17G), non-quantized
eval/	Holdout A/B results: base Q4, SFT Q4, SFT BF16

Eval (first-turn XML tool calls, n=20)

Same holdout prompts from sft_eval.jsonl, temp=0.2, max_tokens=768:

Model	Runtime	parse_rate	valid tool names
Base Mythos Q4_K_M	sparkinfer	0%	0%
SFT Q4_K_M	sparkinfer	30%	30%
SFT BF16 (non-quantized)	llama.cpp (enable_thinking=false)	85%	85%

See eval/summary.json, eval/sft_bf16_llamacpp_n20_v2.json.

Serve

Q4 (sparkinfer):

sparkinfer_server -m Qwythos-9B-nsys-SFT-Q4_K_M.gguf --tokenizer tokenizer.json --ctx 65536 --model-name qwythos-9b --port 8080

BF16 (llama.cpp):

llama-server -m Qwythos-9B-nsys-SFT-noMTP-BF16.gguf -c 4096 -ngl 99 --jinja -fa on --port 8081
# API: chat_template_kwargs={"enable_thinking": false}

Load LoRA (transformers)

from transformers import Qwen3_5ForConditionalGeneration, AutoTokenizer
from peft import PeftModel

base_id = "empero-ai/Qwythos-9B-Claude-Mythos-5-1M"
tok = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
model = Qwen3_5ForConditionalGeneration.from_pretrained(base_id, torch_dtype="bfloat16", trust_remote_code=True)
model = PeftModel.from_pretrained(model, "gittensor-model-hub/Qwythos-9B-nsys-SFT", subfolder="adapter")

License

Apache-2.0 (inherits Qwen / Empero base terms). Dataset: see cuda-nsys-training.