--- license: apache-2.0 base_model: Qwen/Qwen3-0.6B library_name: transformers pipeline_tag: text-generation language: [en] tags: [decision-making, classification, calibration, selective-prediction, gguf, llama.cpp, ollama, pocketsflow] --- # Pocketsflow Penny 1 A small open decision model. Give it a situation and one question; it answers **choose** (one of 2–8 options), **yes / no**, or a **1–5 score**, with a calibrated confidence, in a single forward pass. It never writes paragraphs. The confidence is the point: send the answers it is sure about straight through, and the rest to a person. - 0.6B parameters (Qwen3-0.6B, LoRA fine-tune, merged) · Apache-2.0 - Runs on a CPU: Ollama, LM Studio, llama.cpp - Paper, charts and playground: https://www.pocketsflow.com/research/penny ## Quick start ```bash ollama run hf.co/pocketsflow/penny-1:Q4_K_M ``` Penny reads the next-token probabilities of one answer token, so call it in raw mode with the prompt format below, or use the bundled clients (`penny.py`, `penny.mjs`, zero dependencies, calibration baked in): ```python from penny import Penny penny = Penny(model="hf.co/pocketsflow/penny-1:Q4_K_M") # Ollama on localhost:11434 penny.yes_no("From: security@paypa1-support.com\nSubject: Your account is suspended\n\"We noticed unusual activity. Verify your password within 24 hours at http://paypa1-verify.co/login or your account will be closed.\"", "Is this email a phishing attempt?") # {'yes': True, 'confidence': 0.8315} ``` ### Prompt format ``` any text or JSON about the situation What should we do with this refund request? A. Refund in full B. Offer store credit C. Decline politely ``` The answer is the next token: a letter `A`–`H` (`yes_no`: `A` = yes, `B` = no) or a digit `1`–`5` for `score`. Take the softmax over the allowed tokens only and divide the logits by the per-type temperature in `penny_config.json`. ## Results Accuracy on held-out test sets (a score counts as right within one point). "Unseen domains" are task families that were not in training at all. API models were asked the same prompts; local models are read from logprobs. | Model | General, trained domains | General, unseen domains | Money, trained | Money, unseen | Calibration error (unseen) | | --- | --- | --- | --- | --- | --- | | **Pocketsflow Penny 1** | 80.2% | 79.9% | 71.5% | 66.6% | 5.2% | | Qwen3 0.6B (base, untrained) | 55.8% | 61.8% | 45.6% | 47.5% | 25.0% | | Llama 3.2 1B | 50.2% | 45.8% | 44.4% | 40.7% | – | | Llama 3.2 3B | 76.0% | 78.6% | 65.2% | 69.5% | – | | Llama 3.1 8B | 82.7% | 82.8% | 68.8% | 70.1% | – | | Llama 3.3 70B | 88.2% | 87.9% | 81.9% | 81.3% | – | | gpt-oss-20b | 89.7% | 92.2% | 83.5% | 83.3% | – | **Automatic decisions.** With the thresholds in `penny_config.json` (choice ≥ 0.79, yes_no ≥ 0.72, score ≥ 0.8): general, trained: 68.2% answered automatically at 89.2% accuracy; general, unseen: 64.5% answered automatically at 90.8% accuracy; money, trained: 46.4% answered automatically at 84.9% accuracy; money, unseen: 47.5% answered automatically at 82.4% accuracy. **Cost and speed.** Q4_K_M is 397 MB. On a 4 vCPU / 8 GB server ($48/month) Penny does about 2.4 decisions a second, about $7.59 per million decisions, against $27 for Llama 3.1 8B and $63 for Llama 3.3 70B through a hosted API. Penny is less accurate than 8B and 70B models. It is meant for the cheap, high-volume decisions where a calibrated "I'm not sure" is worth more than a few points of accuracy. ## Training - Base: Qwen/Qwen3-0.6B. LoRA rank 16 on the top 14 of 28 layers, merged. - Loss on the single answer token only. Trained on a CPU. - Data: about 10,101 synthetic decisions written to a known answer by open models, kept only when a blind judge model agreed. No customer data. - Calibration: one temperature per question type, fitted on validation. ## Limits - English only. Synthetic training data: real inputs will be messier. - Not for credit or lending, insurance, hiring, medical or legal decisions. - One question per call; long states are truncated. - Scores (1–5) cluster near the middle on unfamiliar inputs, and a score counts as right within one point. Prefer choose or yes / no for decisions that matter. - Not a security control: it can be confidently wrong about a dangerous agent command. Keep a hard allow-list in front of anything destructive. - Confidence is calibrated on our test sets, not on your data. Check it on a sample of your own before automating anything. ## Files | File | What | | --- | --- | | `model.safetensors` + config/tokenizer | Merged model, bf16 (transformers, vLLM, TGI) | | `pocketsflow-penny-1-q4_k_m.gguf` | 4-bit GGUF (recommended) | | `pocketsflow-penny-1-q8_0.gguf` | 8-bit GGUF | | `Modelfile` | Ollama model file for the Q4_K_M GGUF | | `penny.py`, `penny.mjs` | Clients with calibration built in | | `penny_config.json` | Per-type temperatures and auto thresholds | ## Citation ``` @misc{pocketsflow2026penny, title = {Pocketsflow Penny 1: a small open decision model}, author = {Pocketsflow Research}, year = {2026}, url = {https://www.pocketsflow.com/research/penny} } ```