Instructions to use samadpls/torchlight-phi4-mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use samadpls/torchlight-phi4-mini with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="samadpls/torchlight-phi4-mini")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("samadpls/torchlight-phi4-mini")
model = AutoModelForCausalLM.from_pretrained("samadpls/torchlight-phi4-mini", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

MLX

How to use samadpls/torchlight-phi4-mini with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("samadpls/torchlight-phi4-mini")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

vLLM

How to use samadpls/torchlight-phi4-mini with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "samadpls/torchlight-phi4-mini"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "samadpls/torchlight-phi4-mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/samadpls/torchlight-phi4-mini

SGLang

How to use samadpls/torchlight-phi4-mini with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "samadpls/torchlight-phi4-mini" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "samadpls/torchlight-phi4-mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "samadpls/torchlight-phi4-mini" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "samadpls/torchlight-phi4-mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

How to use samadpls/torchlight-phi4-mini with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "samadpls/torchlight-phi4-mini"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "samadpls/torchlight-phi4-mini"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use samadpls/torchlight-phi4-mini with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "samadpls/torchlight-phi4-mini"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "samadpls/torchlight-phi4-mini"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "samadpls/torchlight-phi4-mini",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Docker Model Runner
How to use samadpls/torchlight-phi4-mini with Docker Model Runner:
```
docker model run hf.co/samadpls/torchlight-phi4-mini
```

Hermes Agent

How to use samadpls/torchlight-phi4-mini with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "samadpls/torchlight-phi4-mini"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default samadpls/torchlight-phi4-mini

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use samadpls/torchlight-phi4-mini with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "samadpls/torchlight-phi4-mini"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "samadpls/torchlight-phi4-mini" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

torchlight-phi4-mini / README.md

samadpls

Point to the Qwen3-8B fine-tune in the results

00d01cc verified 2 days ago

preview code

Raw History Blame Contribute Delete

10.8 kB

---
license: mit
language:
- en
base_model: microsoft/Phi-4-mini-instruct
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-generation
datasets:
- samadpls/torchlight-threads
tags:
- pytorch
- code
- debugging
- lora
- mlx
- phi3
---
<p align="center">
  <img alt="torchlight" src="https://huggingface.co/samadpls/torchlight-phi4-mini/resolve/main/assets/hero.png" width="100%">
</p>

<p align="center">
  <a href="https://github.com/samadpls/torchlight"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-samadpls%2Ftorchlight-181717?logo=github"></a>
  <a href="https://huggingface.co/datasets/samadpls/torchlight-threads"><img alt="Dataset" src="https://img.shields.io/badge/dataset-torchlight--threads-FFD21E?logo=huggingface&logoColor=000"></a>
  <a href="https://huggingface.co/microsoft/Phi-4-mini-instruct"><img alt="Base model: Phi-4-mini-instruct" src="https://img.shields.io/badge/base-Phi--4--mini--instruct-555"></a>
  <img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-EE4C2C">
  <a href="https://samadpls.com"><img alt="samadpls.com" src="https://img.shields.io/badge/by-samadpls.com-EE4C2C"></a>
</p>

# torchlight-phi4-mini

> This repo has the full model, with the fine-tuning merged in.

## Model Overview

torchlight is a small model that helps with PyTorch questions. It is [Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct) (3.8B parameters), fine-tuned on 6,918 chats from the [PyTorch forums](https://discuss.pytorch.org). Most end with the answer that solved the thread; the rest teach it to ask for missing details.

It was trained not to guess: when a question has enough detail, suggest a fix; when it doesn't, ask for the code or the full error.

- **Trained on real answers.** Each answer is the post that solved the thread. Most were written by [ptrblck](https://discuss.pytorch.org/u/ptrblck).
- **Tested on new questions.** 260 forum questions that are newer than anything the model saw in training.
- **Trained on a laptop.** 6.3 hours on a MacBook Pro (M3 Pro).

## Model Details

| | |
|---|---|
| **Base model** | [`microsoft/Phi-4-mini-instruct`](https://huggingface.co/microsoft/Phi-4-mini-instruct) |
| **Size** | 3.8B parameters (11.5M trained with LoRA) |
| **Training data** | [`samadpls/torchlight-threads`](https://huggingface.co/datasets/samadpls/torchlight-threads) |
| **Language** | English |
| **License** | MIT |
| **Author** | [Abdul Samad](https://samadpls.com) |
| **Code** | [github.com/samadpls/torchlight](https://github.com/samadpls/torchlight) |

The model comes in three versions:

| Repo | What it is | Use it with |
|---|---|---|
| [`samadpls/torchlight-phi4-mini`](https://huggingface.co/samadpls/torchlight-phi4-mini) | Full model (16-bit) | transformers, mlx-lm |
| [`samadpls/torchlight-phi4-mini-lora`](https://huggingface.co/samadpls/torchlight-phi4-mini-lora) | LoRA adapter only | PEFT, mlx-lm |
| [`samadpls/torchlight-phi4-mini-GGUF`](https://huggingface.co/samadpls/torchlight-phi4-mini-GGUF) | GGUF files (Q4_K_M, Q8_0) | llama.cpp, Ollama, LM Studio |
## ⚡ Quickstart
### Transformers
```python
from transformers import pipeline
SYSTEM_PROMPT = "You are a careful PyTorch helper. Give specific, correct guidance with code where it helps. If a question lacks the details needed to answer it, ask for them, for example a minimal executable code snippet or the full error message, instead of guessing."

chat = pipeline("text-generation", model="samadpls/torchlight-phi4-mini", dtype="auto", device_map="auto")
messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "How do I check which device a tensor is on?"},
]
print(chat(messages, max_new_tokens=512)[0]["generated_text"][-1]["content"])
```
### mlx-lm (Apple Silicon)

```bash
pip install mlx-lm
mlx_lm.generate --model samadpls/torchlight-phi4-mini --max-tokens 512 \
  --system-prompt "You are a careful PyTorch helper. Give specific, correct guidance with code where it helps. If a question lacks the details needed to answer it, ask for them, for example a minimal executable code snippet or the full error message, instead of guessing." \
  --prompt "How do I check which device a tensor is on?"
```

Use the same system prompt as in the examples. The model was trained with it.

## 📊 Evaluation

We asked 260 forum questions from 2025-01-01 to 2026-10-08 to four setups. Claude Opus 5.5, acting as judge, scored each answer from 1 to 5 by comparing it with the answer that solved the thread.

| Model | Average grade (out of 5) | Graded 4 or 5 | Wrong answers (graded 1) | Code with only real torch APIs | Asked for details, when they were missing | Asked for details, when not needed | Searches per question |
|---|---|---|---|---|---|---|---|
| Phi-4-mini (base model) | 1.88 | 5% | 39% | 170 of 175 | 0 of 7 | 0% | 0.0 |
| Phi-4-mini + forum search | 1.75 | 5% | 45% | 145 of 157 | 1 of 7 | 2% | 0.1 |
| torchlight-phi4-mini (fine-tuned) | 1.97 | 7% | 27% | 9 of 10 | 0 of 7 | 7% | 0.0 |
| torchlight-phi4-mini + forum search | 1.95 | 6% | 26% | 4 of 4 | 0 of 7 | 2% | 0.0 |

**What the results show**

- **Far fewer wrong answers.** 27% of torchlight-phi4-mini's answers were graded 1 (wrong), against 39% for the base model: 71 against 102. That gap is real, not chance (p = 0.003).
- **Better than giving the base model forum search.** torchlight-phi4-mini averages 1.97 against 1.75 (95% interval +0.09 to +0.35). Search pulled the small base model off topic: on the 23 questions where it searched, it averaged 1.35.
- **Short and fast.** The median answer is 31 tokens, against 690 for the base model, and it answered all 260 questions in 9 minutes, against 32.
- **The test is hard by design.** The questions are real forum posts from 2025 and 2026, after the model's knowledge ends, and a 4 or 5 needs the same fix that solved the thread. Every small setup gets a 4 or 5 on only 5–7% of questions. torchlight-phi4-mini raises the average grade from 1.88 to 1.97 and good answers from 14 to 19, but those gaps are too small to be sure of.
- **torchlight-phi4-mini never uses the search tool.** None of its training chats included tools, so adding search changes nothing.
- **A bigger model helps.** The same training on Qwen3-8B (8.2B parameters) scored higher on the newest questions. See [`torchlight-qwen3-8b`](https://huggingface.co/samadpls/torchlight-qwen3-8b).

<details>
<summary><b>How the test works</b></summary>

- **Setups.** Phi-4-mini (the base model) and torchlight, each with and without forum search. Search covers 7,438 solved forum threads, up to twice per question.
- **Settings.** The local models use greedy decoding (no sampling) and up to 1,024 new tokens.
- **Scoring.** The judge sees all the answers to a question together, shuffled and without names. It follows [`prompts/judge.md`](https://github.com/samadpls/torchlight/blob/main/prompts/judge.md).
- **Valid torch APIs.** A script checks that every `torch.*` name in an answer's code exists in PyTorch 2.15.
- **Asks for details.** How often an answer asks for more information. This is measured on the 7 questions where the expert also asked, and on the other 253, where asking less is better.
- **Reference answer.** For each question, the judge also scored an answer from Claude Sonnet 5.5 with the same search. It isn't in the table, which compares models of the same size.

</details>

## Training Details

**Data.** 6,918 chats from forum threads created before 2024-07-01. Each chat ends with the answer that solved the thread. 7 chats longer than 4,096 tokens were left out. The 169 validation chats come from 2024-07-01 to 2025-01-01. The test questions are newer than both.

**Settings.**

| | |
|---|---|
| Method | LoRA on the top 16 of 32 layers; the base weights stay frozen |
| LoRA rank / alpha | 16 / 32 |
| Learning rate | 2e-4, cosine schedule, 43 warmup steps |
| Batch | 2 chats, 4 accumulation steps (8 chats per update) |
| Epochs | 1 (3,459 batches) |
| Max length | 4,096 tokens |
| Loss | On the final answer only |
| Optimizer | AdamW, weight decay 0.01 |
| Hardware | MacBook Pro (M3 Pro), 6.3 hours |
| Software | mlx-lm 0.32.2, MLX 0.32.4 |

**Training results.**

<p align="center">
  <img alt="Training and validation loss" src="https://huggingface.co/samadpls/torchlight-phi4-mini/resolve/main/assets/training-loss.png" width="100%">
</p>

| Epoch | Batch | Training loss | Validation loss |
|---|---|---|---|
| 0.00 | 0 | - | 3.280 |
| 0.14 | 500 | 1.995 | - |
| 0.29 | 1,000 | 1.738 | 1.554 |
| 0.43 | 1,500 | 1.713 | - |
| 0.58 | 2,000 | 1.674 | 1.511 |
| 0.72 | 2,500 | 1.665 | - |
| 0.87 | 3,000 | 1.623 | 1.497 |
| 1.00 | 3,459 | 1.687 | 1.495 |

Training loss is the average since the previous row. Batch 0 is the base model before training.

## Intended Use

**Good for:** PyTorch questions about errors, tensor shapes, training loops, data loading and CUDA.

**Not for:** code you run without reading, other frameworks, PyTorch features from after mid-2024, or anything where a wrong answer can cause harm.

## ⚠️ Limitations

- It is a small model and can be wrong while sounding sure. Check its code.
- On new forum questions it rarely finds the real fix on its own (see Evaluation). Use it for quick first ideas, not final answers.
- It asks for missing details less often than it should.
- With greedy decoding, about 1 answer in 65 gets stuck repeating a phrase. A repetition penalty or sampling avoids this.
- It learned the forum's style, so its answers are short. Sometimes they are too short to help on their own.
- It knows PyTorch up to mid-2024, not version 2.4 or later.
- The training data leans toward ptrblck's topics: debugging, CUDA, training loops and shape errors.
- English only.

## License and Credits

The model is released under the MIT license, like the base model. The training data, [`samadpls/torchlight-threads`](https://huggingface.co/datasets/samadpls/torchlight-threads), is under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). The answers in it were written by the [PyTorch forum](https://discuss.pytorch.org) community, above all [ptrblck](https://discuss.pytorch.org/u/ptrblck), and every row links to its thread. This model is not made or endorsed by PyTorch or the Linux Foundation.

## Citation

```bibtex
@misc{samad2026torchlight_model,
  author = {Abdul Samad},
  title = {torchlight: a PyTorch helper fine-tuned on resolved forum threads},
  year = {2026},
  url = {https://huggingface.co/samadpls/torchlight-phi4-mini}
}
```

## Contact

Made by **Abdul Samad** ([samadpls.com](https://samadpls.com) · [GitHub @samadpls](https://github.com/samadpls)). Questions or corrections: abdulsamadsid1@gmail.com.

	---
	license: mit
	language:
	- en
	base_model: microsoft/Phi-4-mini-instruct
	base_model_relation: finetune
	library_name: transformers
	pipeline_tag: text-generation
	datasets:
	- samadpls/torchlight-threads
	tags:
	- pytorch
	- code
	- debugging
	- lora
	- mlx
	- phi3
	---

	<p align="center">
	<img alt="torchlight" src="https://huggingface.co/samadpls/torchlight-phi4-mini/resolve/main/assets/hero.png" width="100%">
	</p>

	<p align="center">
	<a href="https://github.com/samadpls/torchlight"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-samadpls%2Ftorchlight-181717?logo=github"></a>
	<a href="https://huggingface.co/datasets/samadpls/torchlight-threads"><img alt="Dataset" src="https://img.shields.io/badge/dataset-torchlight--threads-FFD21E?logo=huggingface&logoColor=000"></a>
	<a href="https://huggingface.co/microsoft/Phi-4-mini-instruct"><img alt="Base model: Phi-4-mini-instruct" src="https://img.shields.io/badge/base-Phi--4--mini--instruct-555"></a>
	<img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-EE4C2C">
	<a href="https://samadpls.com"><img alt="samadpls.com" src="https://img.shields.io/badge/by-samadpls.com-EE4C2C"></a>
	</p>

	# torchlight-phi4-mini

	> This repo has the full model, with the fine-tuning merged in.

	## Model Overview

	torchlight is a small model that helps with PyTorch questions. It is [Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct) (3.8B parameters), fine-tuned on 6,918 chats from the [PyTorch forums](https://discuss.pytorch.org). Most end with the answer that solved the thread; the rest teach it to ask for missing details.

	It was trained not to guess: when a question has enough detail, suggest a fix; when it doesn't, ask for the code or the full error.

	- Trained on real answers. Each answer is the post that solved the thread. Most were written by [ptrblck](https://discuss.pytorch.org/u/ptrblck).
	- Tested on new questions. 260 forum questions that are newer than anything the model saw in training.
	- Trained on a laptop. 6.3 hours on a MacBook Pro (M3 Pro).

	## Model Details

	\| \| \|
	\|---\|---\|
	\| Base model \| [`microsoft/Phi-4-mini-instruct`](https://huggingface.co/microsoft/Phi-4-mini-instruct) \|
	\| Size \| 3.8B parameters (11.5M trained with LoRA) \|
	\| Training data \| [`samadpls/torchlight-threads`](https://huggingface.co/datasets/samadpls/torchlight-threads) \|
	\| Language \| English \|
	\| License \| MIT \|
	\| Author \| [Abdul Samad](https://samadpls.com) \|
	\| Code \| [github.com/samadpls/torchlight](https://github.com/samadpls/torchlight) \|

	The model comes in three versions:

	\| Repo \| What it is \| Use it with \|
	\|---\|---\|---\|
	\| [`samadpls/torchlight-phi4-mini`](https://huggingface.co/samadpls/torchlight-phi4-mini) \| Full model (16-bit) \| transformers, mlx-lm \|
	\| [`samadpls/torchlight-phi4-mini-lora`](https://huggingface.co/samadpls/torchlight-phi4-mini-lora) \| LoRA adapter only \| PEFT, mlx-lm \|
	\| [`samadpls/torchlight-phi4-mini-GGUF`](https://huggingface.co/samadpls/torchlight-phi4-mini-GGUF) \| GGUF files (Q4_K_M, Q8_0) \| llama.cpp, Ollama, LM Studio \|

	## ⚡ Quickstart

	### Transformers

	```python
	from transformers import pipeline

	SYSTEM_PROMPT = "You are a careful PyTorch helper. Give specific, correct guidance with code where it helps. If a question lacks the details needed to answer it, ask for them, for example a minimal executable code snippet or the full error message, instead of guessing."

	chat = pipeline("text-generation", model="samadpls/torchlight-phi4-mini", dtype="auto", device_map="auto")
	messages = [
	{"role": "system", "content": SYSTEM_PROMPT},
	{"role": "user", "content": "How do I check which device a tensor is on?"},
	]
	print(chat(messages, max_new_tokens=512)[0]["generated_text"][-1]["content"])
	```

	### mlx-lm (Apple Silicon)

	```bash
	pip install mlx-lm
	mlx_lm.generate --model samadpls/torchlight-phi4-mini --max-tokens 512 \
	--system-prompt "You are a careful PyTorch helper. Give specific, correct guidance with code where it helps. If a question lacks the details needed to answer it, ask for them, for example a minimal executable code snippet or the full error message, instead of guessing." \
	--prompt "How do I check which device a tensor is on?"
	```

	Use the same system prompt as in the examples. The model was trained with it.

	## 📊 Evaluation

	We asked 260 forum questions from 2025-01-01 to 2026-10-08 to four setups. Claude Opus 5.5, acting as judge, scored each answer from 1 to 5 by comparing it with the answer that solved the thread.

	\| Model \| Average grade (out of 5) \| Graded 4 or 5 \| Wrong answers (graded 1) \| Code with only real torch APIs \| Asked for details, when they were missing \| Asked for details, when not needed \| Searches per question \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| Phi-4-mini (base model) \| 1.88 \| 5% \| 39% \| 170 of 175 \| 0 of 7 \| 0% \| 0.0 \|
	\| Phi-4-mini + forum search \| 1.75 \| 5% \| 45% \| 145 of 157 \| 1 of 7 \| 2% \| 0.1 \|
	\| torchlight-phi4-mini (fine-tuned) \| 1.97 \| 7% \| 27% \| 9 of 10 \| 0 of 7 \| 7% \| 0.0 \|
	\| torchlight-phi4-mini + forum search \| 1.95 \| 6% \| 26% \| 4 of 4 \| 0 of 7 \| 2% \| 0.0 \|

	What the results show

	- Far fewer wrong answers. 27% of torchlight-phi4-mini's answers were graded 1 (wrong), against 39% for the base model: 71 against 102. That gap is real, not chance (p = 0.003).
	- Better than giving the base model forum search. torchlight-phi4-mini averages 1.97 against 1.75 (95% interval +0.09 to +0.35). Search pulled the small base model off topic: on the 23 questions where it searched, it averaged 1.35.
	- Short and fast. The median answer is 31 tokens, against 690 for the base model, and it answered all 260 questions in 9 minutes, against 32.
	- The test is hard by design. The questions are real forum posts from 2025 and 2026, after the model's knowledge ends, and a 4 or 5 needs the same fix that solved the thread. Every small setup gets a 4 or 5 on only 5–7% of questions. torchlight-phi4-mini raises the average grade from 1.88 to 1.97 and good answers from 14 to 19, but those gaps are too small to be sure of.
	- torchlight-phi4-mini never uses the search tool. None of its training chats included tools, so adding search changes nothing.
	- A bigger model helps. The same training on Qwen3-8B (8.2B parameters) scored higher on the newest questions. See [`torchlight-qwen3-8b`](https://huggingface.co/samadpls/torchlight-qwen3-8b).

	<details>
	<summary><b>How the test works</b></summary>

	- Setups. Phi-4-mini (the base model) and torchlight, each with and without forum search. Search covers 7,438 solved forum threads, up to twice per question.
	- Settings. The local models use greedy decoding (no sampling) and up to 1,024 new tokens.
	- Scoring. The judge sees all the answers to a question together, shuffled and without names. It follows [`prompts/judge.md`](https://github.com/samadpls/torchlight/blob/main/prompts/judge.md).
	- Valid torch APIs. A script checks that every `torch.*` name in an answer's code exists in PyTorch 2.15.
	- Asks for details. How often an answer asks for more information. This is measured on the 7 questions where the expert also asked, and on the other 253, where asking less is better.
	- Reference answer. For each question, the judge also scored an answer from Claude Sonnet 5.5 with the same search. It isn't in the table, which compares models of the same size.

	</details>

	## Training Details

	Data. 6,918 chats from forum threads created before 2024-07-01. Each chat ends with the answer that solved the thread. 7 chats longer than 4,096 tokens were left out. The 169 validation chats come from 2024-07-01 to 2025-01-01. The test questions are newer than both.

	Settings.

	\| \| \|
	\|---\|---\|
	\| Method \| LoRA on the top 16 of 32 layers; the base weights stay frozen \|
	\| LoRA rank / alpha \| 16 / 32 \|
	\| Learning rate \| 2e-4, cosine schedule, 43 warmup steps \|
	\| Batch \| 2 chats, 4 accumulation steps (8 chats per update) \|
	\| Epochs \| 1 (3,459 batches) \|
	\| Max length \| 4,096 tokens \|
	\| Loss \| On the final answer only \|
	\| Optimizer \| AdamW, weight decay 0.01 \|
	\| Hardware \| MacBook Pro (M3 Pro), 6.3 hours \|
	\| Software \| mlx-lm 0.32.2, MLX 0.32.4 \|

	Training results.

	<p align="center">
	<img alt="Training and validation loss" src="https://huggingface.co/samadpls/torchlight-phi4-mini/resolve/main/assets/training-loss.png" width="100%">
	</p>

	\| Epoch \| Batch \| Training loss \| Validation loss \|
	\|---\|---\|---\|---\|
	\| 0.00 \| 0 \| - \| 3.280 \|
	\| 0.14 \| 500 \| 1.995 \| - \|
	\| 0.29 \| 1,000 \| 1.738 \| 1.554 \|
	\| 0.43 \| 1,500 \| 1.713 \| - \|
	\| 0.58 \| 2,000 \| 1.674 \| 1.511 \|
	\| 0.72 \| 2,500 \| 1.665 \| - \|
	\| 0.87 \| 3,000 \| 1.623 \| 1.497 \|
	\| 1.00 \| 3,459 \| 1.687 \| 1.495 \|

	Training loss is the average since the previous row. Batch 0 is the base model before training.

	## Intended Use

	Good for: PyTorch questions about errors, tensor shapes, training loops, data loading and CUDA.

	Not for: code you run without reading, other frameworks, PyTorch features from after mid-2024, or anything where a wrong answer can cause harm.

	## ⚠️ Limitations

	- It is a small model and can be wrong while sounding sure. Check its code.
	- On new forum questions it rarely finds the real fix on its own (see Evaluation). Use it for quick first ideas, not final answers.
	- It asks for missing details less often than it should.
	- With greedy decoding, about 1 answer in 65 gets stuck repeating a phrase. A repetition penalty or sampling avoids this.
	- It learned the forum's style, so its answers are short. Sometimes they are too short to help on their own.
	- It knows PyTorch up to mid-2024, not version 2.4 or later.
	- The training data leans toward ptrblck's topics: debugging, CUDA, training loops and shape errors.
	- English only.

	## License and Credits

	The model is released under the MIT license, like the base model. The training data, [`samadpls/torchlight-threads`](https://huggingface.co/datasets/samadpls/torchlight-threads), is under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). The answers in it were written by the [PyTorch forum](https://discuss.pytorch.org) community, above all [ptrblck](https://discuss.pytorch.org/u/ptrblck), and every row links to its thread. This model is not made or endorsed by PyTorch or the Linux Foundation.

	## Citation

	```bibtex
	@misc{samad2026torchlight_model,
	author = {Abdul Samad},
	title = {torchlight: a PyTorch helper fine-tuned on resolved forum threads},
	year = {2026},
	url = {https://huggingface.co/samadpls/torchlight-phi4-mini}
	}
	```

	## Contact

	Made by Abdul Samad ([samadpls.com](https://samadpls.com) · [GitHub @samadpls](https://github.com/samadpls)). Questions or corrections: abdulsamadsid1@gmail.com.