torchlight-qwen3-8b

This repo has the full model, with the fine-tuning merged in.

Model Overview

torchlight is a small model that helps with PyTorch questions. It is Qwen3-8B (8.2B parameters), fine-tuned on 6,902 chats from the PyTorch forums. Most end with the answer that solved the thread; the rest teach it to ask for missing details.

It was trained not to guess: when a question has enough detail, suggest a fix; when it doesn't, ask for the code or the full error.

• Trained on real answers. Each answer is the post that solved the thread. Most were written by ptrblck.
• Tested on new questions. 163 forum questions that are newer than anything the model saw in training.
• Trained for free. 4.1 hours on one Tesla T4 GPU on Kaggle.

Model Details

Base model	Qwen/Qwen3-8B
Size	8.2B parameters (43.6M trained with LoRA)
Training data	samadpls/torchlight-threads
Language	English
License	Apache-2.0
Author	Abdul Samad
Code	github.com/samadpls/torchlight

The model comes in three versions:

Repo	What it is	Use it with
samadpls/torchlight-qwen3-8b	Full model (16-bit)	transformers, mlx-lm
samadpls/torchlight-qwen3-8b-lora	LoRA adapter only	PEFT
samadpls/torchlight-qwen3-8b-GGUF	GGUF files (Q4_K_M, Q8_0)	llama.cpp, Ollama, LM Studio

⚡ Quickstart

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

SYSTEM_PROMPT = "You are a careful PyTorch helper. Give specific, correct guidance with code where it helps. If a question lacks the details needed to answer it, ask for them, for example a minimal executable code snippet or the full error message, instead of guessing."

repo = "samadpls/torchlight-qwen3-8b"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")
messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "How do I check which device a tensor is on?"},
]
# The model was trained with thinking mode off.
text = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False, enable_thinking=False
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

mlx-lm (Apple Silicon)

pip install mlx-lm
mlx_lm.generate --model samadpls/torchlight-qwen3-8b --max-tokens 512 \
  --system-prompt "You are a careful PyTorch helper. Give specific, correct guidance with code where it helps. If a question lacks the details needed to answer it, ask for them, for example a minimal executable code snippet or the full error message, instead of guessing." \
  --prompt "How do I check which device a tensor is on?" \
  --chat-template-config '{"enable_thinking": false}'

Use the same system prompt as in the examples. The model was trained with it.

📊 Evaluation

We asked 163 forum questions from 2025-04-01 to 2026-10-08 to four models. They are newer than Qwen3-8B's training data, which reportedly runs to March 2025. Claude Opus 5.5, acting as judge, scored each answer from 1 to 5 by comparing it with the answer that solved the thread.

Model	Average grade (out of 5)	Graded 4 or 5	Wrong answers (graded 1)	Code with only real torch APIs	Asked for details, when they were missing	Asked for details, when not needed
Phi-4-mini (base model)	1.96	5%	33%	108 of 109	0 of 4	1%
torchlight-phi4-mini (fine-tuned)	2.09	8%	16%	5 of 5	0 of 4	5%
Qwen3-8B (base model)	2.17	17%	31%	114 of 120	0 of 4	1%
torchlight-qwen3-8b (fine-tuned)	2.25	9%	15%	11 of 11	3 of 4	13%

What the results show

• Half as many wrong answers. Fine-tuning cut answers graded 1 (wrong) from 31% to 15% for Qwen3-8B, and from 33% to 16% for Phi-4-mini. Both gaps are real, not chance (p < 0.001).
• The bigger model helps. Qwen3-8B beats Phi-4-mini both before fine-tuning (2.17 against 1.96) and after (2.25 against 2.09). Both gaps are real (95% intervals +0.04 to +0.39 and +0.03 to +0.29).
• It asks for missing details. On the 4 questions where the expert first asked for more information, torchlight-qwen3-8b asked too on 3. None of the other models asked on any. It also asks when it doesn't need to on 13% of the other questions.
• But it gives fewer complete fixes than Qwen3-8B itself. 9% of its answers were graded 4 or 5, against 17% for the base model (p = 0.02). The base model writes long answers (a median of 1,024 tokens, often cut off) that more often contain the exact fix. The fine-tune learned the forum's short style (a median of 42 tokens).
• On average, the fine-tune is a little ahead of Qwen3-8B (2.25 against 2.17), but that gap is too small to be sure of (95% interval −0.09 to +0.23).
• The test is hard by design. These 163 questions are newer than Qwen3-8B's training data, and a 4 or 5 needs the same fix that solved the thread.

How the test works

• Models. Qwen3-8B and this model, plus Phi-4-mini and torchlight-phi4-mini, the earlier and smaller fine-tune. None of them searches the forum here.
• Settings. Greedy decoding (no sampling), up to 1,024 new tokens, and Qwen3's thinking mode off. The Qwen models run in 16-bit on a Kaggle GPU.
• Scoring. The judge sees all four answers to a question together, shuffled and without names. It follows prompts/judge.md.
• Valid torch APIs. A script checks that every torch.* name in an answer's code exists in PyTorch 2.15.
• Asks for details. How often an answer asks for more information. This is measured on the 4 questions where the expert also asked, and on the other 159, where asking less is better.

Training Details

Data. 6,902 chats from forum threads created before 2024-07-01. Each chat ends with the answer that solved the thread. 23 chats longer than 4,096 tokens were left out. The 169 validation chats come from 2024-07-01 to 2025-01-01. The test questions are newer than both.

Settings.

Method	LoRA on every linear layer of all layers; the base weights stay frozen in 4-bit (QLoRA)
LoRA rank / alpha	16 / 32
Learning rate	2e-4, cosine schedule, 5% warmup
Batch	2 chats, 4 accumulation steps (8 chats per update)
Epochs	1 (863 steps)
Max length	4,096 tokens
Loss	On the final answer only
Optimizer	AdamW 8-bit, weight decay 0.01
Hardware	One Tesla T4 GPU on Kaggle, 4.1 hours
Software	Unsloth 2026.10.11, Transformers 5.16.1

Training results.

Epoch	Step	Training loss	Validation loss
0.00	0	-	3.059
0.23	200	1.540	1.606
0.46	400	1.399	1.572
0.70	600	1.398	1.538
0.93	800	1.380	1.532
1.00	863	1.358	1.532

Training loss is the average since the previous row. Step 0 is the base model before training; each step is 8 chats.

Intended Use

Good for: PyTorch questions about errors, tensor shapes, training loops, data loading and CUDA.

Not for: code you run without reading, other frameworks, PyTorch features from after mid-2024, or anything where a wrong answer can cause harm.

⚠️ Limitations

• It is a small model and can be wrong while sounding sure. Check its code.
• Its answers are short, like the forum's. It gives fewer complete fixes than Qwen3-8B itself (9% of answers graded 4 or 5, against 17%), which writes much longer answers.
• It sometimes asks for details it doesn't need (on 13% of questions that could be answered).
• It knows PyTorch up to early 2025.
• The training data leans toward ptrblck's topics: debugging, CUDA, training loops and shape errors.
• English only.

License and Credits

The model is released under the Apache-2.0 license, like the base model. The training data, samadpls/torchlight-threads, is under CC BY 4.0. The answers in it were written by the PyTorch forum community, above all ptrblck, and every row links to its thread. This model is not made or endorsed by PyTorch or the Linux Foundation.

Citation

@misc{samad2026torchlight_qwen3,
  author = {Abdul Samad},
  title = {torchlight: a PyTorch helper fine-tuned on resolved forum threads},
  year = {2026},
  url = {https://huggingface.co/samadpls/torchlight-qwen3-8b}
}

Contact

Made by Abdul Samad (samadpls.com · GitHub @samadpls). Questions or corrections: abdulsamadsid1@gmail.com.