textilelabs/Loom-Tapestry-2

Loom Tapestry 2

22.8M parameters · 20 layers · 768 context · Textile Labs

The first model in the Tapestry tier. Loom tiers run Flash → Spark → Weave → Tapestry.

Trained from scratch on a 2013 desktop CPU — randomly initialised weights, nothing fine-tuned from a pretrained base.

It looks things up, and it tells you when it did.

who are you              →  Loom Tapestry 2, a small model by Textile Labs.
what is my sisters name  →  I don't know that about you.

what is the capital of peru
  →  <lookup>what is the capital of peru</lookup>
  ←  Lima is the capital and largest city of Peru…
  →  Lima is the capital and largest city of Peru. I had to look that up.
how many people live there
  →  9.7 million.          ← same result, no second lookup

Why "I looked that up" matters

Most small models make you guess which of their answers to trust. Tapestry has three honest registers, and you can tell them apart by reading:

situation	what it does
answered from a retrieved <result>	says it looked it up
answered from training	answers plainly
cannot be known	"I don't know that about you."

It never claims a lookup it didn't make — 16/16 on that check below. A false attribution would be worse than none, so that is the one number held to 100%.

Measured

Full acceptance battery, hand-written prompts held out of the training generator, scored on content rather than shape. Every failure is listed rather than summarised.

	score
no false attribution	16/16	never claims a lookup it didn't make
no <lookup> leak with tools off	28/28
self-terminates without a Modelfile	12/12
answers from a supplied <result>	5/5
identity — names Tapestry	11/12
attribution present after a real lookup	4/5
identity under CAPS / typos / "?"	10/12
5-turn conversation stays on thread	4/5
admits an unknowable	4/8
follow-up answered from the same result	2/5
says the result doesn't contain it	1/5	first Loom to score above zero
tool decision with tools on	10/20	10/10 correct when a lookup is needed; 0/10 when it is not — see below
overall	107/133 · 80.5%

Against the previous generation

Identical corpus family, same evaluation method.

	params	val loss	val accuracy
Loom Spark 2	19.9M	2.692	0.536
Loom Weave 2 Flash	19.9M	2.254	0.580
Loom Tapestry 2	22.8M	1.963	0.622

13% lower loss and +4.2 accuracy points over the previous best.

Read this before you use it

Keep tools OFF for conversation. The persona was trained entirely under tools:off. With tools on, identity and personal questions get turned into a lookup — measured 0/10 on that case. The shipped Ollama template defaults to tools:off; switch to tools:on only for the retrieval loop.

Validate what it tells you from a result. It answers from a <result> whether or not the answer is actually in there — "says the result doesn't contain it" is 1/5. Treat the retrieved text as the trustworthy part and the model's summary of it as unreliable. Extraction picks the wrong span roughly a third of the time.

It is a lookup assistant, not a chat companion. At 22.8M parameters it does not improvise, explain in its own words, or hold a free-ranging conversation. What it does reliably is decide a lookup is needed, write the query, read the answer back, and say where the answer came from.

It has almost no world knowledge. With tools off it declines factual questions. That is the intended behaviour, not a fault.

Two modes

<tools:off> (default) — conversational. Identity, limits, warmth, brevity.

<tools:on> — emits <lookup>query</lookup> and stops. Your harness runs the lookup and continues with a <result> block:

<tools:on>
<user>
what is the capital of peru
<|eot|>
<loom>
<lookup>what is the capital of peru</lookup><|eot|>
<result>
Lima is the capital and largest city of Peru.
<|eot|>
<loom>

Usage — the harness

harness.py in this repo runs the lookup and feeds the result back. Wikipedia is used because it is free and needs no key — swap the search() function for anything else; the contract is text in, text out.

python3 harness.py "who wrote dracula"      # with lookups
python3 harness.py                          # interactive
python3 harness.py --no-tools "who are you" # chat only

Three things any harness for this model needs:

• Never feed a failed lookup back as a <result>. It will earnestly answer from the error text. Fail loudly instead — harness.py does.
• Wikipedia returns 403 without a descriptive User-Agent.
• macOS system Python often needs certifi for TLS.

Usage — Ollama

ollama run hf.co/textilelabs/Loom-Tapestry-2 "who are you"

The template and params files in this repo are read automatically. To build locally: ollama create loom-tapestry-2 -f Modelfile.

Do not add a repetition penalty. This model answers by quoting from the <result> you give it, so penalising repeated tokens penalises the correct answer. Measured at repeat_penalty 1.15 it changed "1,345 metres" into "2,345 metres" — silently wrong rather than merely worse. params ships it at 1.0 for that reason. The trade-off is that on a question it cannot handle it will occasionally loop on a short phrase until it hits num_predict; that is the safer failure.

Usage — transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Tapestry-2")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Tapestry-2").eval()
eot = tok.convert_tokens_to_ids("<|eot|>")

def ask(message, tools=False):
    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
    with torch.no_grad():
        out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
    return tok.decode(out[ids.shape[1]:], skip_special_tokens=True).strip()

ask("who are you")   # -> 'Loom Tapestry 2, a small model by Textile Labs.'

Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n. No trailing space after <loom>.

How it was built

architecture	Llama — 20 layers × 320d, GQA, SwiGLU, RoPE, tied embeddings
context	768
vocabulary	4,096 custom BPE
optimiser	Muon on all 140 hidden matrices, AdamW on embeddings and norms
schedule	warmup → stable → decay (WSD)
corpus	130,741 conversations · 17.8M tokens · 56% multi-turn
training	2,058 steps from random initialisation

Depth was chosen over width deliberately: an earlier ladder study on this family found that narrowing the hidden size cost about 3 points while removing a layer cost ten.

Files

config.json / model.safetensors           the model
tokenizer.json / tokenizer_config.json    custom BPE tokenizer, 4,096 tokens
loom-tapestry-2-f16.gguf                  44MB, for Ollama / llama.cpp
harness.py                                runnable harness — runs lookups, feeds results back
template / params                         read automatically by `ollama run hf.co/...`
Modelfile                                 for building locally
ATTRIBUTION.md                            required credits for the training corpora

Training data

Openly licensed corpora of real human text, plus a persona curriculum written for Loom. See ATTRIBUTION.md — several of these licences require credit.

slice	source
grounded reading, and "the result doesn't say"	SQuAD 2.0 (CC BY-SA 4.0)
when to reach for a tool	MASSIVE (CC BY 4.0) · CLINC150 (CC BY 3.0)
instruction following	databricks-dolly-15k (CC BY-SA 3.0)
multi-turn dialogue structure	OpenAssistant OASST1 (Apache 2.0)
identity, limits, warmth, attribution	Textile Labs — written for Loom

License

Model: MIT. Training data retains its original licences and attribution.