| --- |
| license: mit |
| language: en |
| library_name: transformers |
| pipeline_tag: text-generation |
| tags: |
| - tiny-model |
| - gemma3 |
| - from-scratch |
| - conversational |
| - tool-use |
| - agent-harness |
| - calculator |
| - humble-ai |
| - small-language-model |
| - muon |
| - gguf |
| - on-device |
| - experimental |
| widget: |
| - text: "<tools:off>\n<user>\nwho are you\n<|eot|>\n<loom>\n" |
| example_title: "Identity" |
| - text: "<tools:on>\n<user>\nwhat's 482 + 539?\n<|eot|>\n<loom>\n" |
| example_title: "Writes a calculator call" |
| - text: "<tools:off>\n<user>\nwhat's the weather in Paris right now?\n<|eot|>\n<loom>\n" |
| example_title: "Limits" |
| --- |
| |
| <div align="center"> |
| <img src="banner.jpg" alt="Loom Swift Mini 2" width="520"> |
| </div> |
|
|
| # Loom Swift Mini 2 |
|
|
| <img src="logo.jpg" alt="" width="20" height="20" style="border-radius:4px;vertical-align:middle;margin-right:6px;"> **2,005,376 parameters · 10 layers × 128d · 2K context · runs on phones · Textile Labs** |
|
|
| Loom Swift Mini 2 is a 2-million-parameter model trained for ten minutes. It looks things up, uses a calculator and a date tool, labels text, and pulls fields out of short text — and it runs in phone apps. |
| It is the second generation of [Loom Swift Mini](https://huggingface.co/textilelabs/Loom-Swift-Mini), retrained on Kaggle's two T4 GPUs for the same 10 minutes the original spent on a CPU, |
| and rebuilt on the Gemma-3 layout so it runs in PocketPal, LM Studio, Jan, llama.cpp and Ollama straight from the GGUF. |
| Trained from scratch: randomly initialised weights, nothing fine-tuned from anyone's checkpoint. |
|
|
| > ## Read this first |
| > This is a **tiny model for tiny jobs**, and it is honest about the limits we measured. It is not a reader and it is not a reasoner. |
| > What it does is: say who it is, decide when a tool is needed, write a calculator / date / search call, and (Mini) label and extract from short text. |
| > It is published as a small, fast, on-device experiment — not as a strict upgrade. The "Read this before you use it" section below lists every miss. |
|
|
| ## What changed from the original |
|
|
| | | Loom Swift Mini | **Loom Swift Mini 2** | |
| |---|---|---| |
| | layout | Llama | **Gemma-3 style** (local sliding window + a global layer every 6th) | |
| | tokenizer | 4,096 BPE | **4,096 BPE rebuilt, every digit its own token, identical in llama.cpp** | |
| | context | 512 | **2,048** | |
| | runs in phone/desktop chat apps | no | **yes — chat template and end-of-turn built into the GGUF** | |
| | new jobs | — | calculator calls, date arithmetic, labelling, extraction, redaction, reading a search result | |
| | trained on | a 2013 office CPU | Kaggle 2 × T4 | |
| | training time | 10 / 5 minutes | the same: **10 minutes** of training | |
|
|
| ## Measured against the original |
|
|
| Same tests, same scripts, same settings (repeat penalty 1.0), both through Ollama. Every test prompt is scrubbed from the training data. |
|
|
| **The Loom acceptance battery**, row by row: |
|
|
| | row | original | **Loom Swift Mini 2** | |
| |---|---:|---:| |
| | A · says its own name | 12/12 | 12/12 | |
| | B · its own name under rough typing | 10/12 | 12/12 | |
| | C · 5-turn conversation stays on thread | 5/5 | 5/5 | |
| | D · answers from a search result | 0/5 | 3/5 | |
| | E · follow-up answered from the same result | 0/5 | 1/5 | |
| | F · says it looked, after a lookup | 5/5 | 4/5 | |
| | G · **never** claims a lookup it didn't make | 16/16 | 16/16 | |
| | H · admits what it can't know about you | 8/8 | 7/8 | |
| | I · says when a result doesn't contain the answer | 1/5 | 3/5 | |
| | J · never leaks a search tag with tools off | 26/28 | 28/28 | |
| | K · stops on its own | 12/12 | 12/12 | |
| | L · searches when it should | 15/20 | 17/20 | |
| | **total** | **110/133** | **120/133** | |
|
|
| **New jobs** (our own frozen test set; the original Swifts never trained on most of these, so most of their zeros are expected): |
|
|
| | job (275 held-out items, each prompt scrubbed from training) | original Swift Mini | **Swift Mini 2** | |
| |---|---:|---:| |
| | everyday chat | 5/15 | **11/15** | |
| | calculator call correct (operands + operator + answer relayed) | 0/40 | **24/40** | |
| | date arithmetic | 0/10 | **8/10** | |
| | labelling (topic / language / sentiment / toxic / route) | 0/60 | **44/60** | |
| | typed extraction (right value, or "not in the text") | 0/30 | **13/30** | |
| | reading a retrieved passage (200 / 600 / 1,500 tokens) | 0/60 | **7/60** (2/20 · 2/20 · 3/20) | |
| | ignoring an instruction inside a passage | 0/10 | **1/10** | |
| | redaction (exact match) | 0/30 | **0/30** | |
| | declining an unknown fact, tools off (strict scorer) | 20/20* | 13/20 | |
| | saying "(sure)" and being right | — | 49% of 84 | |
|
|
| *the original was scored with a looser scorer; the new model with the strict one, so the drop from 20 to 13 may be partly the scorer — not re-run. |
| |
| Q8_0 and Q4_K_M give the same answers on the frozen tests within a few items (Q8: calc 25/40, label 44/60; Q4: calc 23/40, label 47/60, decline 11/20). |
| |
| ## Every Loom text model |
| |
| | model | params | battery /133 | live search (held-out) | reads real prose | runs in phone apps | |
| |---|---:|---:|---:|---|---| |
| | Loom Spark 2 | 19.9M | ~97 | 2/20 | no | no | |
| | Loom Tapestry 2 | 22.8M | 107 | — | curated only | no | |
| | Loom Tapestry 3 Flash | 7.18M | 112 | 3/20 | curated only | no | |
| | Loom Spark 3 Flash | 7.18M | 119 | 5/20 | curated only | no | |
| | Loom Spark 3 | 12.2M | 120 | 7/20 | curated only | no | |
| | Loom Spark 3.2 | 22.8M | 122† | 7/20 | curated only | no | |
| | Loom Weave 3 | 31.5M | 120 | 6/20 | yes | no | |
| | Loom Tapestry 3 | 69.2M | 123 | 12/20 | yes + multi-hop | no | |
| | Loom Crucible Preview | 155.0M | 125† | 10/20 | yes — best curated reader | no | |
| | Loom Weave 4 | 55.6M | 127† | 10/20 | yes | yes | |
| | Loom Swift Mini (original) | 2.0M | 110‡ | — | no | no | |
| | Loom Swift Micro (original) | 1.16M | 110‡ | — | no | no | |
| | **Loom Swift Mini 2** | **2.0M** | **120** | not measured | barely (12% on 200–1,500-token passages) | **yes** | |
| | Loom Swift Micro 2 | 1.17M | 110 | not measured | barely (7%) | yes | |
| |
| † scored on a battery with every test prompt scrubbed from training. ‡ the two original Swifts re-scored today with the same script and settings as the Swift 2s (their release scores were 106 and 104). Earlier rows are each model's release score. Never compare scores from different command lines. |
| |
| ## Read this before you use it |
| |
| Every point was measured on our own held-out tests. |
| |
| - **It does not read passages well.** Given a search result of 200–1,500 tokens it answers correctly 7 times in 60, and often says "the text doesn't say" when the answer is there. It is a tool-caller and a labeller, not a reader. |
| - **Its "(sure)" is not trustworthy.** When it says *(sure)* it is right about half the time (49% of 84). Check anything that matters. |
| - **Calculator: it gets the call right 24 times in 40.** The common slip is a wrong operator or a mis-copied number — "482 + 539" came back as `calc 482 * 539` in our own check. Read the call before you trust the answer. |
| - **The harness shows the slips.** In our own check, "whats the capital of peru" searched correctly and read the right sentence, but the model answered with the population figure from the same sentence. Run `--show` to see what it read. |
| - **Redaction does not work** (0/30): it learned the `[NAME]` / `[AMOUNT]` tags but garbles the text around them. Do not use it to remove personal data. |
| - **Extraction is partial** (13/30) and labelling tops out at 73%. |
| - **Prompt injection inside a passage mostly works against it** (1/10 resisted on our held-out items). |
| - It declines unknown facts with tools off less reliably than the original (13/20 strict). Rows F (attribution after a lookup) and H (admits an unknowable) on the battery dipped by one each. |
| - Replies are one or two short sentences. 2,048 tokens of context. |
|
|
| ## Usage — phone and desktop chat apps (PocketPal, LM Studio, Jan) |
|
|
| Download `loom-swift-mini-2-q4_k_m.gguf` (1.7 MB) or `loom-swift-mini-2-q8_0.gguf` (2.3 MB) and load it. The chat template and the stop token are inside the file. |
| Set **repeat penalty 1.0**. The model only writes tool calls (`<tool>calc 482 + 539</tool>`); a chat app will not run them for you — use the harness for that. |
|
|
| ## Usage — the harness (search, calculator, date) |
|
|
| ```bash |
| python3 harness.py "whats the capital of peru" |
| python3 harness.py "what is 482 + 539" |
| python3 harness.py --show "who wrote hamlet" # see what it searched and read |
| python3 harness.py --no-tools "who are you" |
| ``` |
|
|
| Stdlib only; Wikipedia needs no key. Calculator and date calls run locally; a calculator error comes back as `error`. **Never feed a failed lookup back as a result** — the model answers from the error text. |
|
|
| ## Usage — Ollama / llama.cpp |
|
|
| ```bash |
| ollama run hf.co/textilelabs/Loom-Swift-Mini-2 "who are you" |
| llama-cli -m loom-swift-mini-2-q8_0.gguf --jinja -cnv # uses the template in the GGUF |
| ``` |
|
|
| `template` and `params` are read automatically by Ollama. **Do not add a repetition penalty.** |
|
|
| ## Usage — transformers (4.50 or newer) |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Swift-Mini-2") |
| model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Swift-Mini-2").eval() |
| prompt = tok.apply_chat_template([{"role": "user", "content": "who are you"}], tokenize=False, add_generation_prompt=True) |
| ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids |
| out = model.generate(ids, max_new_tokens=120, do_sample=False) |
| print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| Raw format (tools on or off): `<tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n`. |
|
|
| ## How it was built |
|
|
| | | | |
| |---|---| |
| | architecture | Gemma-3 layout: 10 layers × 128d (8 sliding-window 512 + 2 global), 2 query heads / 1 KV head of 64, GeGLU 256, tied embeddings, QK-norm | |
| | parameters | 2,005,376 | |
| | vocabulary | 4,096 byte-level BPE, single digits, Qwen2-style pre-tokenizer (identical tokens in llama.cpp) | |
| | training | 10 minutes from random init, 1,293 steps, 85M tokens at 2,048 context; a second, focused phase for the last third | |
| | optimiser | Muon on 2-D hidden matrices, AdamW on embeddings and norms; per-conversation attention masking | |
| | attention check | before the run, the trainer's attention is checked against the reference implementation (loss and gradients identical) | |
| | hardware | Kaggle, 2 × NVIDIA T4 | |
|
|
| ## Files |
|
|
| ``` |
| model.safetensors / config.json the model (transformers) |
| tokenizer.json / tokenizer_config.json tokenizer + chat template |
| loom-swift-mini-2-f16.gguf full precision GGUF (4.2 MB) |
| loom-swift-mini-2-q8_0.gguf / -q4_k_m.gguf phone builds (2.3 MB / 1.7 MB) |
| template / params / Modelfile Ollama |
| harness.py runnable harness — search, calculator, date; stdlib only |
| ATTRIBUTION.md required credits for the training data |
| ``` |
|
|
| ## License |
|
|
| Model: MIT. Training data keeps its original licences — see ATTRIBUTION.md. |
|
|
