---
license: mit
language:
- en
base_model: microsoft/Phi-4-mini-instruct
base_model_relation: adapter
library_name: peft
pipeline_tag: text-generation
datasets:
- samadpls/torchlight-threads
tags:
- pytorch
- code
- debugging
- lora
- mlx
- phi3
---
# torchlight-phi4-mini-lora
> This repo has only the LoRA adapter, in PEFT format. The mlx-lm version is in `mlx/`. For the full model, see [`samadpls/torchlight-phi4-mini`](https://huggingface.co/samadpls/torchlight-phi4-mini).
## Model Overview
torchlight is a small model that helps with PyTorch questions. It is [Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct) (3.8B parameters), fine-tuned on 6,918 chats from the [PyTorch forums](https://discuss.pytorch.org). Most end with the answer that solved the thread; the rest teach it to ask for missing details.
It was trained not to guess: when a question has enough detail, suggest a fix; when it doesn't, ask for the code or the full error.
- **Trained on real answers.** Each answer is the post that solved the thread. Most were written by [ptrblck](https://discuss.pytorch.org/u/ptrblck).
- **Tested on new questions.** 260 forum questions that are newer than anything the model saw in training.
- **Trained on a laptop.** 6.3 hours on a MacBook Pro (M3 Pro).
## Model Details
| | |
|---|---|
| **Base model** | [`microsoft/Phi-4-mini-instruct`](https://huggingface.co/microsoft/Phi-4-mini-instruct) |
| **Size** | 3.8B parameters (11.5M trained with LoRA) |
| **Training data** | [`samadpls/torchlight-threads`](https://huggingface.co/datasets/samadpls/torchlight-threads) |
| **Language** | English |
| **License** | MIT |
| **Author** | [Abdul Samad](https://samadpls.com) |
| **Code** | [github.com/samadpls/torchlight](https://github.com/samadpls/torchlight) |
The model comes in three versions:
| Repo | What it is | Use it with |
|---|---|---|
| [`samadpls/torchlight-phi4-mini`](https://huggingface.co/samadpls/torchlight-phi4-mini) | Full model (16-bit) | transformers, mlx-lm |
| [`samadpls/torchlight-phi4-mini-lora`](https://huggingface.co/samadpls/torchlight-phi4-mini-lora) | LoRA adapter only | PEFT, mlx-lm |
| [`samadpls/torchlight-phi4-mini-GGUF`](https://huggingface.co/samadpls/torchlight-phi4-mini-GGUF) | GGUF files (Q4_K_M, Q8_0) | llama.cpp, Ollama, LM Studio |
## ⚡ Quickstart
### PEFT
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("microsoft/Phi-4-mini-instruct", dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "samadpls/torchlight-phi4-mini-lora")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-4-mini-instruct")
model = model.merge_and_unload() # optional: merge the adapter into the model
```
### mlx-lm (Apple Silicon)
```bash
hf download samadpls/torchlight-phi4-mini-lora --include "mlx/*" --local-dir torchlight-lora
mlx_lm.generate --model microsoft/Phi-4-mini-instruct --adapter-path torchlight-lora/mlx --max-tokens 512 \
--system-prompt "You are a careful PyTorch helper. Give specific, correct guidance with code where it helps. If a question lacks the details needed to answer it, ask for them, for example a minimal executable code snippet or the full error message, instead of guessing." \
--prompt "How do I check which device a tensor is on?"
```
Use the same system prompt as in the examples. The model was trained with it.
## 📊 Evaluation
We asked 260 forum questions from 2025-01-01 to 2026-10-08 to four setups. Claude Opus 5.5, acting as judge, scored each answer from 1 to 5 by comparing it with the answer that solved the thread.
| Model | Average grade (out of 5) | Graded 4 or 5 | Wrong answers (graded 1) | Code with only real torch APIs | Asked for details, when they were missing | Asked for details, when not needed | Searches per question |
|---|---|---|---|---|---|---|---|
| Phi-4-mini (base model) | 1.88 | 5% | 39% | 170 of 175 | 0 of 7 | 0% | 0.0 |
| Phi-4-mini + forum search | 1.75 | 5% | 45% | 145 of 157 | 1 of 7 | 2% | 0.1 |
| torchlight-phi4-mini (fine-tuned) | 1.97 | 7% | 27% | 9 of 10 | 0 of 7 | 7% | 0.0 |
| torchlight-phi4-mini + forum search | 1.95 | 6% | 26% | 4 of 4 | 0 of 7 | 2% | 0.0 |
**What the results show**
- **Far fewer wrong answers.** 27% of torchlight-phi4-mini's answers were graded 1 (wrong), against 39% for the base model: 71 against 102. That gap is real, not chance (p = 0.003).
- **Better than giving the base model forum search.** torchlight-phi4-mini averages 1.97 against 1.75 (95% interval +0.09 to +0.35). Search pulled the small base model off topic: on the 23 questions where it searched, it averaged 1.35.
- **Short and fast.** The median answer is 31 tokens, against 690 for the base model, and it answered all 260 questions in 9 minutes, against 32.
- **The test is hard by design.** The questions are real forum posts from 2025 and 2026, after the model's knowledge ends, and a 4 or 5 needs the same fix that solved the thread. Every small setup gets a 4 or 5 on only 5–7% of questions. torchlight-phi4-mini raises the average grade from 1.88 to 1.97 and good answers from 14 to 19, but those gaps are too small to be sure of.
- **torchlight-phi4-mini never uses the search tool.** None of its training chats included tools, so adding search changes nothing.
- **A bigger model helps.** The same training on Qwen3-8B (8.2B parameters) scored higher on the newest questions. See [`torchlight-qwen3-8b`](https://huggingface.co/samadpls/torchlight-qwen3-8b).
How the test works
- **Setups.** Phi-4-mini (the base model) and torchlight, each with and without forum search. Search covers 7,438 solved forum threads, up to twice per question.
- **Settings.** The local models use greedy decoding (no sampling) and up to 1,024 new tokens.
- **Scoring.** The judge sees all the answers to a question together, shuffled and without names. It follows [`prompts/judge.md`](https://github.com/samadpls/torchlight/blob/main/prompts/judge.md).
- **Valid torch APIs.** A script checks that every `torch.*` name in an answer's code exists in PyTorch 2.15.
- **Asks for details.** How often an answer asks for more information. This is measured on the 7 questions where the expert also asked, and on the other 253, where asking less is better.
- **Reference answer.** For each question, the judge also scored an answer from Claude Sonnet 5.5 with the same search. It isn't in the table, which compares models of the same size.
## Training Details
**Data.** 6,918 chats from forum threads created before 2024-07-01. Each chat ends with the answer that solved the thread. 7 chats longer than 4,096 tokens were left out. The 169 validation chats come from 2024-07-01 to 2025-01-01. The test questions are newer than both.
**Settings.**
| | |
|---|---|
| Method | LoRA on the top 16 of 32 layers; the base weights stay frozen |
| LoRA rank / alpha | 16 / 32 |
| Learning rate | 2e-4, cosine schedule, 43 warmup steps |
| Batch | 2 chats, 4 accumulation steps (8 chats per update) |
| Epochs | 1 (3,459 batches) |
| Max length | 4,096 tokens |
| Loss | On the final answer only |
| Optimizer | AdamW, weight decay 0.01 |
| Hardware | MacBook Pro (M3 Pro), 6.3 hours |
| Software | mlx-lm 0.32.2, MLX 0.32.4 |
**Training results.**
| Epoch | Batch | Training loss | Validation loss |
|---|---|---|---|
| 0.00 | 0 | - | 3.280 |
| 0.14 | 500 | 1.995 | - |
| 0.29 | 1,000 | 1.738 | 1.554 |
| 0.43 | 1,500 | 1.713 | - |
| 0.58 | 2,000 | 1.674 | 1.511 |
| 0.72 | 2,500 | 1.665 | - |
| 0.87 | 3,000 | 1.623 | 1.497 |
| 1.00 | 3,459 | 1.687 | 1.495 |
Training loss is the average since the previous row. Batch 0 is the base model before training.
## Intended Use
**Good for:** PyTorch questions about errors, tensor shapes, training loops, data loading and CUDA.
**Not for:** code you run without reading, other frameworks, PyTorch features from after mid-2024, or anything where a wrong answer can cause harm.
## ⚠️ Limitations
- It is a small model and can be wrong while sounding sure. Check its code.
- On new forum questions it rarely finds the real fix on its own (see Evaluation). Use it for quick first ideas, not final answers.
- It asks for missing details less often than it should.
- With greedy decoding, about 1 answer in 65 gets stuck repeating a phrase. A repetition penalty or sampling avoids this.
- It learned the forum's style, so its answers are short. Sometimes they are too short to help on their own.
- It knows PyTorch up to mid-2024, not version 2.4 or later.
- The training data leans toward ptrblck's topics: debugging, CUDA, training loops and shape errors.
- English only.
## License and Credits
The model is released under the MIT license, like the base model. The training data, [`samadpls/torchlight-threads`](https://huggingface.co/datasets/samadpls/torchlight-threads), is under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). The answers in it were written by the [PyTorch forum](https://discuss.pytorch.org) community, above all [ptrblck](https://discuss.pytorch.org/u/ptrblck), and every row links to its thread. This model is not made or endorsed by PyTorch or the Linux Foundation.
## Citation
```bibtex
@misc{samad2026torchlight_model,
author = {Abdul Samad},
title = {torchlight: a PyTorch helper fine-tuned on resolved forum threads},
year = {2026},
url = {https://huggingface.co/samadpls/torchlight-phi4-mini}
}
```
## Contact
Made by **Abdul Samad** ([samadpls.com](https://samadpls.com) · [GitHub @samadpls](https://github.com/samadpls)). Questions or corrections: abdulsamadsid1@gmail.com.