---
license: apache-2.0
base_model: Qwen/Qwen3-0.6B
library_name: transformers
pipeline_tag: text-generation
language: [en]
tags: [decision-making, classification, calibration, selective-prediction, gguf, llama.cpp, ollama, pocketsflow]
---
# Pocketsflow Penny 1
A small open decision model. Give it a situation and one question; it answers
**choose** (one of 2–8 options), **yes / no**, or a **1–5 score**, with a
calibrated confidence, in a single forward pass. It never writes paragraphs.
The confidence is the point: send the answers it is sure about straight
through, and the rest to a person.
- 0.6B parameters (Qwen3-0.6B, LoRA fine-tune, merged) · Apache-2.0
- Runs on a CPU: Ollama, LM Studio, llama.cpp
- Paper, charts and playground: https://www.pocketsflow.com/research/penny
## Quick start
```bash
ollama run hf.co/pocketsflow/penny-1:Q4_K_M
```
Penny reads the next-token probabilities of one answer token, so call it in
raw mode with the prompt format below, or use the bundled clients
(`penny.py`, `penny.mjs`, zero dependencies, calibration baked in):
```python
from penny import Penny
penny = Penny(model="hf.co/pocketsflow/penny-1:Q4_K_M") # Ollama on localhost:11434
penny.yes_no("From: security@paypa1-support.com\nSubject: Your account is suspended\n\"We noticed unusual activity. Verify your password within 24 hours at http://paypa1-verify.co/login or your account will be closed.\"", "Is this email a phishing attempt?")
# {'yes': True, 'confidence': 0.8315}
```
### Prompt format
```
any text or JSON about the situation
What should we do with this refund request?
A. Refund in full
B. Offer store credit
C. Decline politely
```
The answer is the next token: a letter `A`–`H` (`yes_no`: `A` = yes, `B` = no)
or a digit `1`–`5` for `score`. Take the softmax over the allowed tokens only
and divide the logits by the per-type temperature in `penny_config.json`.
## Results
Accuracy on held-out test sets (a score counts as right within one point).
"Unseen domains" are task families that were not in training at all. API
models were asked the same prompts; local models are read from logprobs.
| Model | General, trained domains | General, unseen domains | Money, trained | Money, unseen | Calibration error (unseen) |
| --- | --- | --- | --- | --- | --- |
| **Pocketsflow Penny 1** | 80.2% | 79.9% | 71.5% | 66.6% | 5.2% |
| Qwen3 0.6B (base, untrained) | 55.8% | 61.8% | 45.6% | 47.5% | 25.0% |
| Llama 3.2 1B | 50.2% | 45.8% | 44.4% | 40.7% | – |
| Llama 3.2 3B | 76.0% | 78.6% | 65.2% | 69.5% | – |
| Llama 3.1 8B | 82.7% | 82.8% | 68.8% | 70.1% | – |
| Llama 3.3 70B | 88.2% | 87.9% | 81.9% | 81.3% | – |
| gpt-oss-20b | 89.7% | 92.2% | 83.5% | 83.3% | – |
**Automatic decisions.** With the thresholds in `penny_config.json`
(choice ≥ 0.79, yes_no ≥ 0.72, score ≥ 0.8): general, trained: 68.2% answered automatically at 89.2% accuracy; general, unseen: 64.5% answered automatically at 90.8% accuracy; money, trained: 46.4% answered automatically at 84.9% accuracy; money, unseen: 47.5% answered automatically at 82.4% accuracy.
**Cost and speed.** Q4_K_M is 397 MB. On a 4 vCPU / 8 GB
server ($48/month) Penny does about
2.4 decisions a second, about
$7.59 per million decisions, against
$27 for Llama 3.1 8B and
$63 for Llama 3.3 70B through a hosted API.
Penny is less accurate than 8B and 70B models. It is meant for the cheap,
high-volume decisions where a calibrated "I'm not sure" is worth more than a
few points of accuracy.
## Training
- Base: Qwen/Qwen3-0.6B. LoRA rank 16 on the top 14 of 28 layers, merged.
- Loss on the single answer token only. Trained on a CPU.
- Data: about 10,101 synthetic decisions written to a known answer by open
models, kept only when a blind judge model agreed. No customer data.
- Calibration: one temperature per question type, fitted on validation.
## Limits
- English only. Synthetic training data: real inputs will be messier.
- Not for credit or lending, insurance, hiring, medical or legal decisions.
- One question per call; long states are truncated.
- Scores (1–5) cluster near the middle on unfamiliar inputs, and a score counts
as right within one point. Prefer choose or yes / no for decisions that matter.
- Not a security control: it can be confidently wrong about a dangerous agent
command. Keep a hard allow-list in front of anything destructive.
- Confidence is calibrated on our test sets, not on your data. Check it on
a sample of your own before automating anything.
## Files
| File | What |
| --- | --- |
| `model.safetensors` + config/tokenizer | Merged model, bf16 (transformers, vLLM, TGI) |
| `pocketsflow-penny-1-q4_k_m.gguf` | 4-bit GGUF (recommended) |
| `pocketsflow-penny-1-q8_0.gguf` | 8-bit GGUF |
| `Modelfile` | Ollama model file for the Q4_K_M GGUF |
| `penny.py`, `penny.mjs` | Clients with calibration built in |
| `penny_config.json` | Per-type temperatures and auto thresholds |
## Citation
```
@misc{pocketsflow2026penny,
title = {Pocketsflow Penny 1: a small open decision model},
author = {Pocketsflow Research},
year = {2026},
url = {https://www.pocketsflow.com/research/penny}
}
```