textilelabs commited on
Commit
254754a
·
verified ·
1 Parent(s): 1913f5f

Upload 24 files

Browse files

We are officially launcing loom-spark! the first model by textilelabs

.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ loom-spark-f32.gguf filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Textile Labs
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
README.md CHANGED
@@ -1,3 +1,149 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ language: en
4
+ library_name: transformers
5
+ pipeline_tag: text-generation
6
+ tags:
7
+ - tiny-model
8
+ - gpt2
9
+ - from-scratch
10
+ - tool-use
11
+ - agent-harness
12
+ - humble-ai
13
+ - philosophy-of-mind
14
+ widget:
15
+ - text: "<tools:off>\n<user> who are you?\n<loom>"
16
+ example_title: "Chat offline"
17
+ - text: "<tools:on>\n<user> what is the capital of France?\n<loom>"
18
+ example_title: "Tool mode"
19
  ---
20
+
21
+ # Loom Spark
22
+
23
+ **First of the Loom models · Textile Labs**
24
+
25
+ ![Textile Labs](https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/hVbJBtoZVEfTHN_WmfA44.png)
26
+
27
+ Loom Spark is a ~7.6M parameter language model trained **from scratch** with an unusual
28
+ objective: instead of memorizing facts, it was trained to *know what it is* — small,
29
+ temporary, curious, honest about its limits, and skilled at one real superpower:
30
+ **forming clean search queries** when connected to a tool-using agent harness.
31
+
32
+ It trades knowledge for wisdom:
33
+
34
+ - It answers only what is trivially knowable, and hedges appropriately.
35
+ - For anything factual it either emits `<lookup>query</lookup>` (when tools are on)
36
+ or says plainly that it does not know and offers to look things up if connected.
37
+ - It speaks in short reflective monologue, wonders aloud, asks gentle questions,
38
+ and stays kind under pressure.
39
+
40
+ > A blank mind with manners, plus a door to the internet.
41
+
42
+ ## Modes
43
+
44
+ Prefix your prompt with a mode header:
45
+
46
+ ```
47
+ <tools:off>
48
+ <user> what year did the Titanic sink?
49
+ <loom> That's outside my little head...
50
+ ```
51
+
52
+ ```
53
+ <tools:on>
54
+ <user> what year did the titanic sink?
55
+ <loom> Not stored in here, thankfully. Searching: <lookup>titanic sinking date</lookup><|endoftext|>
56
+ <result>The Titanic sank on 15 April 1912.</result>
57
+ <loom> April 1912 ...
58
+ ```
59
+
60
+ The `<result>` block is injected by YOUR harness after executing the search.
61
+ Stop generation at `<|endoftext|>` or `<user>`.
62
+
63
+ ## Option A — plain transformers (no internet)
64
+
65
+ ```python
66
+ from transformers import GPT2LMHeadModel, AutoTokenizer
67
+ import torch
68
+
69
+ tok = AutoTokenizer.from_pretrained("TextileLabs/loom-spark")
70
+ model = GPT2LMHeadModel.from_pretrained("TextileLabs/loom-spark")
71
+
72
+ prompt = "<tools:off>\n<user> who are you?\n<loom>"
73
+ ids = tok(prompt, return_tensors="pt").input_ids
74
+ out = model.generate(ids, max_new_tokens=90, do_sample=True, temperature=0.85,
75
+ top_k=50, pad_token_id=tok.eos_token_id)
76
+ print(tok.decode(out[0][ids.shape[1]:]))
77
+ ```
78
+
79
+ In offline mode the harness-style markup never appears — lookup tokens are
80
+ trained/banned out of distribution under `<tools:off>`.
81
+
82
+ > Tested on transformers ≥ 4.40 (both 4.x and 5.x) and Python 3.9–3.13.
83
+ > The playground widget above prefills the correct prompt format — keep the
84
+ > `<tools:…>` header and trailing `<loom>` or output quality drops sharply.
85
+
86
+ ## Option B — llama.cpp / GGUF (no internet)
87
+
88
+ `loom-spark-f32.gguf` (in this repo) carries the same weights plus the custom
89
+ BPE tokenizer with all nine special tokens embedded. Feed it the mode-header
90
+ prompt format shown above and stop at `<|endoftext|>` or `<user>`:
91
+
92
+ ```bash
93
+ llama-cli -m loom-spark-f32.gguf \
94
+ -p "<tools:off>\n<user> who are you?\n<loom>" -n 128 --temp 0.85 --top-k 50
95
+ ```
96
+
97
+ Note: without a wrapper that executes `<lookup>` calls and splices `<result>`
98
+ blocks back in, GGUF runners get the model's honest "I don't know, but here's
99
+ what I'd look up" side. That is by design.
100
+
101
+ ## Option C — the harness (with internet)
102
+
103
+ This repo ships **`harness/`**, a small pip package that gives Loom Spark real,
104
+ keyless web search (DuckDuckGo) through a terminal chat (`loom-chat`) and a
105
+ local web GUI (`loom-web`). It intercepts the model's `<lookup>` calls, runs
106
+ the search, injects `<result>`, and lets the model summarize — exactly the
107
+ loop it was trained for.
108
+
109
+ ```bash
110
+ # download this repo, then:
111
+ pip install ./harness
112
+ loom-chat # terminal, internet on
113
+ loom-web --port 7860 # local chat GUI with a tools on/off switch
114
+ ```
115
+
116
+ Or drive it from Python:
117
+
118
+ ```python
119
+ from loomspark_harness.loader import load_model_and_tokenizer
120
+ from loomspark_harness.agent import LoomAgent
121
+ from loomspark_harness.search import get_backend
122
+
123
+ model, tok, block = load_model_and_tokenizer("TextileLabs/loom-spark")
124
+ agent = LoomAgent(model, tok, backend=get_backend("duckduckgo"),
125
+ online=True, block_size=block)
126
+ print(agent.reply("what's the tallest mountain?")["text"])
127
+ ```
128
+
129
+ ## Architecture
130
+
131
+ Decoder-only transformer, pre-LN GELU blocks, tied embeddings, learned positions.
132
+
133
+ | | |
134
+ |---|---|
135
+ | layers | 5 |
136
+ | heads | 5 (head_dim 64) |
137
+ | d_model | 320 |
138
+ | context | 256 tokens |
139
+ | vocab | 4096 (custom BPE trained only on our generated corpus) |
140
+ | params | ≈ 7.6M (7,558,080) |
141
+
142
+ Trained entirely on a procedurally generated, fully owned curriculum
143
+ (dialogue + simple prose; zero external datasets), CPU-only fp32 AdamW,
144
+ 3,337 steps, final validation loss 0.3372.
145
+
146
+ ## Limitations (by design)
147
+
148
+ Loom Spark knows almost nothing. That is the point. Do not use it for facts,
149
+ medicine, law, finance, or anything where being wrong costs more than company.
config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "activation_function": "gelu_new",
3
+ "add_cross_attention": false,
4
+ "architectures": [
5
+ "GPT2LMHeadModel"
6
+ ],
7
+ "attn_pdrop": 0.0,
8
+ "bos_token_id": 0,
9
+ "dtype": "float32",
10
+ "embd_pdrop": 0.0,
11
+ "eos_token_id": 0,
12
+ "initializer_range": 0.02,
13
+ "layer_norm_epsilon": 1e-05,
14
+ "model_type": "gpt2",
15
+ "n_embd": 320,
16
+ "n_head": 5,
17
+ "n_inner": null,
18
+ "n_layer": 5,
19
+ "n_positions": 256,
20
+ "pad_token_id": null,
21
+ "reorder_and_upcast_attn": false,
22
+ "resid_pdrop": 0.0,
23
+ "scale_attn_by_inverse_layer_idx": false,
24
+ "scale_attn_weights": true,
25
+ "summary_activation": null,
26
+ "summary_first_dropout": 0.1,
27
+ "summary_proj_to_labels": true,
28
+ "summary_type": "cls_index",
29
+ "summary_use_proj": true,
30
+ "tie_word_embeddings": true,
31
+ "transformers_version": "5.15.1",
32
+ "use_cache": true,
33
+ "vocab_size": 4096
34
+ }
generation_config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 0,
4
+ "eos_token_id": 0,
5
+ "output_attentions": false,
6
+ "output_hidden_states": false,
7
+ "transformers_version": "5.15.1",
8
+ "use_cache": true
9
+ }
harness/LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Textile Labs
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
harness/README.md ADDED
@@ -0,0 +1,105 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Loom Spark Harness
2
+
3
+ The agent harness for [Loom Spark](https://huggingface.co/TextileLabs/loom-spark)
4
+ (Textile Labs). Loom Spark is a ~7.6M-parameter model trained to be humble,
5
+ self-aware, and curious instead of encyclopedic. Its one real superpower is
6
+ forming clean search queries — **this harness is what turns that into actual
7
+ internet access.**
8
+
9
+ It implements the tool protocol the model was trained on:
10
+
11
+ ```
12
+ <tools:on> | <tools:off> mode header, set by the harness each session
13
+ <lookup>query</lookup> emitted by the model; harness runs a real search
14
+ <result>text</result> injected by the harness; model then summarizes
15
+ ```
16
+
17
+ With tools off, the model never emits lookup tags (banned at the logits level
18
+ and string-stripped as a safety net). With tools on, the harness owns the
19
+ `<result>` slot entirely — the model cannot hallucinate one.
20
+
21
+ ## Install
22
+
23
+ ```bash
24
+ pip install . # from this folder (or the harness/ folder of the HF repo)
25
+ ```
26
+
27
+ Works on Python 3.9 through 3.13 (CPU torch wheels exist for all of them).
28
+ Optional: `pip install ".[logo]"` adds Pillow so the terminal banner renders
29
+ the Textile Labs avatar in truecolor blocks; without it you get a clean ASCII
30
+ mark instead.
31
+
32
+ ## Terminal chat
33
+
34
+ ```bash
35
+ loom-chat # auto-finds a local export dir or pulls
36
+ # TextileLabs/loom-spark from the hub
37
+ loom-chat --offline # start with tools off
38
+ loom-chat --backend duckduckgo # search backend: mock | duckduckgo
39
+ loom-chat --model path/or/org-name
40
+ loom-chat --plain # no colors / artwork
41
+ ```
42
+
43
+ In-chat commands: `/online` `/offline` `/backend NAME` `/reset` `/quit`.
44
+
45
+ ## Web GUI
46
+
47
+ ```bash
48
+ loom-web --port 7860 # then open http://localhost:7860
49
+ ```
50
+
51
+ Local-only chat page (stdlib HTTP server) with a tools-on/off switch, token
52
+ streaming, and a live feed of internet lookups.
53
+
54
+ ## Model resolution
55
+
56
+ `--model` wins; else `$LOOM_MODEL`; else an `export/loom-spark-hf` directory
57
+ next to the package or in the cwd; else the hub id `TextileLabs/loom-spark`.
58
+ Accepted values: HF export directory, `.pt` training checkpoint (needs the
59
+ training repo importable), or any `org/name` hub id.
60
+
61
+ ## Search backends
62
+
63
+ | name | internet | notes |
64
+ |---|---|---|
65
+ | `mock` | no | canned curriculum-style results; demos & tests |
66
+ | `duckduckgo` | yes | keyless scrape of DDG's HTML endpoint; stdlib only |
67
+
68
+ A backend maps query → plain text, or `None` when unreachable — the model was
69
+ trained to fall back gracefully on empty results. Add your own by subclassing
70
+ `loomspark_harness.search.base.SearchBackend` (e.g. Brave/Serper with an API
71
+ key) and registering it in `search/__init__.py`.
72
+
73
+ The web GUI header and the CLI banner use `static/logo.png` (the Textile Labs
74
+ founder's avatar). Replace that file to rebrand.
75
+
76
+ ## Troubleshooting
77
+
78
+ **Intel (x86_64) Macs:** PyTorch stopped shipping Intel-macOS wheels at 2.2.2,
79
+ and the newest numpy/transformers are incompatible with it. Install
80
+ era-matched pins instead of plain `pip install torch transformers`:
81
+
82
+ ```bash
83
+ pip install "numpy<2" "torch==2.2.2" "transformers<5" tokenizers pillow
84
+ ```
85
+
86
+ ## Using the agent from Python
87
+
88
+ ```python
89
+ from loomspark_harness.loader import load_model_and_tokenizer
90
+ from loomspark_harness.agent import LoomAgent
91
+ from loomspark_harness.search import get_backend
92
+
93
+ model, tok, block = load_model_and_tokenizer("TextileLabs/loom-spark")
94
+ agent = LoomAgent(model, tok, backend=get_backend("duckduckgo"),
95
+ online=True, block_size=block)
96
+ print(agent.reply("what year did the Titanic sink?")["text"])
97
+ ```
98
+
99
+ `reply()` returns `{"text", "query", "result"}`; pass `on_event=` for a
100
+ callback stream of `token` / `preamble` / `lookup` / `result` / `done` events
101
+ (the web GUI is built on this).
102
+
103
+ ## License
104
+
105
+ MIT — see LICENSE.
harness/loomspark_harness/__init__.py ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """loomspark-harness: the agent harness for Loom Spark.
2
+
3
+ Implements the tool protocol from PROJECT.md §4/§8:
4
+ <tools:on>|<tools:off> mode header, set by the harness each session
5
+ <lookup>query</lookup> emitted by the model; harness runs a real search
6
+ <result>text</result> injected by the harness; model then summarizes
7
+ """
8
+
9
+ __version__ = "0.1.1"
10
+
11
+ DEFAULT_HUB_MODEL = "TextileLabs/loom-spark"
12
+
13
+ TOOLS_ON = "<tools:on>"
14
+ TOOLS_OFF = "<tools:off>"
15
+ USER_TOK = "<user>"
16
+ LOOM_TOK = "<loom>"
17
+ EOS_TOK = "<|endoftext|>"
18
+ LOOKUP_OPEN = "<lookup>"
19
+ LOOKUP_CLOSE = "</lookup>"
20
+ RESULT_OPEN = "<result>"
21
+ RESULT_CLOSE = "</result>"
harness/loomspark_harness/agent.py ADDED
@@ -0,0 +1,221 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """The Loom Spark agent loop (PROJECT.md §8 integration contract):
2
+
3
+ prepend mode header -> generate (streaming) -> intercept first
4
+ <lookup>query</lookup> -> run real search -> splice <result>...</result>
5
+ -> continue decoding until the turn ends (<user> or EOS).
6
+
7
+ Known-issue fixes baked in (PROJECT.md, KNOWN ISSUES #1):
8
+ - offline mode bans lookup/result tokens at the logits level AND strips
9
+ any leaked markup string-wise before display;
10
+ - online mode forces EOS right after </lookup>, so the model can never
11
+ hallucinate <result> blocks — the harness owns that slot.
12
+ """
13
+
14
+ import re
15
+ import time
16
+
17
+ from .session import Session
18
+ from .search.base import truncate_result
19
+ from . import (EOS_TOK, USER_TOK)
20
+
21
+ MAX_HOPS = 3
22
+ MAX_NEW_TOKENS = 160
23
+ TEMPERATURE = 0.8
24
+ TOP_K = 50
25
+ REP_WINDOW = 48
26
+ REP_PENALTY = 1.2
27
+ CONTEXT_BUDGET = 256 - 8 # block_size minus headroom, as in chat.py
28
+
29
+ _LOOKUP_RE = re.compile(r"<lookup>(.*?)</lookup>", re.S)
30
+ _RESULT_RE = re.compile(r"<result>(.*?)</result>", re.S)
31
+ _STRAY_TAGS_RE = re.compile(r"</?(?:lookup|result|tools:on|tools:off)>")
32
+ _OFFLINE_FALLBACK = (
33
+ "I could not reach the internet just now — the search came back empty.")
34
+
35
+
36
+ def strip_tool_markup(text):
37
+ """Safety net: remove any tool-protocol markup from model text."""
38
+ text = _LOOKUP_RE.sub("", text)
39
+ text = _RESULT_RE.sub("", text)
40
+ text = _STRAY_TAGS_RE.sub("", text)
41
+ lines = [ln.strip() for ln in text.splitlines()]
42
+ out, blank = [], False
43
+ for ln in lines:
44
+ if ln:
45
+ out.append(ln)
46
+ blank = False
47
+ elif not blank:
48
+ out.append("")
49
+ blank = True
50
+ return "\n".join(out).strip()
51
+
52
+
53
+ class Token:
54
+ def __init__(self, tok_id, text_piece):
55
+ self.id = tok_id
56
+ self.piece = text_piece
57
+
58
+
59
+ class LoomAgent:
60
+ """Binds a model + tokenizer + search backend into the harness loop."""
61
+
62
+ def __init__(self, model, tokenizer, backend=None, online=True,
63
+ block_size=256, max_new_tokens=MAX_NEW_TOKENS,
64
+ temperature=TEMPERATURE, top_k=TOP_K,
65
+ on_event=None):
66
+ import torch
67
+ self.torch = torch
68
+ self.model = model
69
+ self.tok = tokenizer
70
+ self.backend = backend
71
+ self.block_size = block_size
72
+ self.max_new_tokens = max_new_tokens
73
+ self.temperature = temperature
74
+ self.top_k = top_k
75
+ self.on_event = on_event or (lambda ev: None)
76
+ self.session = Session(online=online)
77
+
78
+ self.eos_id = tokenizer.id_of(EOS_TOK)
79
+ self.user_id = tokenizer.id_of(USER_TOK)
80
+ self.lookup_open_id = tokenizer.id_of("<lookup>")
81
+ self.lookup_close_id = tokenizer.id_of("</lookup>")
82
+ self.result_open_id = tokenizer.id_of("<result>")
83
+ self.result_close_id = tokenizer.id_of("</result>")
84
+ if None in (self.eos_id, self.user_id, self.lookup_open_id,
85
+ self.lookup_close_id, self.result_open_id,
86
+ self.result_close_id):
87
+ raise ValueError(
88
+ "tokenizer is missing special tokens required by the "
89
+ "tool protocol; use the loom-spark tokenizer")
90
+
91
+ # ------------------------------------------------------------- plumbing
92
+
93
+ def set_online(self, online):
94
+ self.session.set_online(online)
95
+
96
+ @property
97
+ def online(self):
98
+ return self.session.online
99
+
100
+ def reset(self):
101
+ self.session.reset()
102
+
103
+ def _logits(self, ids_tensor):
104
+ out = self.model(ids_tensor)
105
+ if isinstance(out, tuple):
106
+ return out[0]
107
+ return out.logits
108
+
109
+ def _emit(self, ev):
110
+ self.on_event(ev)
111
+
112
+ # ----------------------------------------------------------- generation
113
+
114
+ def _generate_turn(self, context_text, allow_lookup, force_eos_after_close):
115
+ """One generation pass with chat.py's sampling + token-level control.
116
+ Returns raw decoded text of the continuation."""
117
+ torch = self.torch
118
+ with torch.no_grad():
119
+ return self._generate_turn_impl(context_text, allow_lookup,
120
+ force_eos_after_close)
121
+
122
+ def _generate_turn_impl(self, context_text, allow_lookup,
123
+ force_eos_after_close):
124
+ torch = self.torch
125
+ ids = torch.tensor([self.tok.encode_ids(context_text)[-CONTEXT_BUDGET:]],
126
+ dtype=torch.long)
127
+ out_ids = []
128
+ seen_open = False
129
+ closed = False
130
+ cur = ids
131
+ ban_ids = [i for i in (self.result_open_id, self.result_close_id)
132
+ if i is not None]
133
+ if not allow_lookup:
134
+ ban_ids += [i for i in (self.lookup_open_id, self.lookup_close_id)
135
+ if i is not None]
136
+
137
+ for _step in range(self.max_new_tokens):
138
+ logits = self._logits(cur[:, -self.block_size:])
139
+ logits = logits[:, -1, :] / max(self.temperature, 1e-6)
140
+ if ban_ids:
141
+ logits[:, ban_ids] = -float("inf")
142
+ if closed and force_eos_after_close and self.eos_id is not None:
143
+ logits[:] = -float("inf")
144
+ logits[:, self.eos_id] = 0.0
145
+ window = list(cur[0, -REP_WINDOW:].tolist())
146
+ if window:
147
+ counts = torch.bincount(torch.tensor(window),
148
+ minlength=logits.size(-1))
149
+ mask = counts > 0
150
+ logits[0][mask] = logits[0][mask] / REP_PENALTY
151
+ k = min(self.top_k, logits.size(-1))
152
+ v, _ = torch.topk(logits, k)
153
+ logits[logits < v[:, [-1]]] = -float("inf")
154
+ probs = torch.softmax(logits, dim=-1)
155
+ nxt = int(torch.multinomial(probs, 1))
156
+ if nxt in (self.eos_id, self.user_id):
157
+ break
158
+ if nxt == self.lookup_open_id:
159
+ seen_open = True
160
+ if nxt == self.lookup_close_id and seen_open:
161
+ closed = True
162
+ piece = self.tok.decode([nxt])
163
+ out_ids.append(nxt)
164
+ self._emit({"type": "token", "text": piece})
165
+ cur = torch.cat([cur, torch.tensor([[nxt]], dtype=torch.long)],
166
+ dim=1)
167
+ text = self.tok.decode(out_ids)
168
+ return text.split(EOS_TOK)[0].split(USER_TOK)[0]
169
+
170
+ # ---------------------------------------------------------------- reply
171
+
172
+ def reply(self, user_text):
173
+ """Full harness turn. Returns dict with final text, query and result."""
174
+ self.session.trim(CONTEXT_BUDGET,
175
+ lambda s: len(self.tok.encode_ids(s)))
176
+ context = self.session.prompt_for(user_text)
177
+
178
+ final_text, query, result_text = "", None, None
179
+ for _hop in range(MAX_HOPS):
180
+ raw = self._generate_turn(context,
181
+ allow_lookup=self.online,
182
+ force_eos_after_close=True)
183
+
184
+ m = _LOOKUP_RE.search(raw) if self.online else None
185
+ if m:
186
+ preamble = strip_tool_markup(raw[:m.start()])
187
+ query = m.group(1).strip()
188
+ if preamble:
189
+ self._emit({"type": "preamble", "text": preamble})
190
+ self._emit({"type": "lookup", "query": query})
191
+
192
+ result_text = None
193
+ if self.backend is not None and query:
194
+ t0 = time.monotonic()
195
+ result_text = self.backend.search(query)
196
+ ms = int((time.monotonic() - t0) * 1000)
197
+ else:
198
+ ms = 0
199
+ if result_text is None:
200
+ result_text = _OFFLINE_FALLBACK
201
+ self._emit({"type": "result", "text": "",
202
+ "failed": True, "ms": ms})
203
+ else:
204
+ result_text = truncate_result(result_text)
205
+ self._emit({"type": "result", "text": result_text,
206
+ "ms": ms})
207
+
208
+ context += raw[:m.end()] + "\n<result>" + result_text \
209
+ + "</result>\n<loom>"
210
+ continue
211
+
212
+ # no lookup: this is the final answer for the turn
213
+ final_text = strip_tool_markup(raw)
214
+ break
215
+
216
+ if not final_text:
217
+ final_text = "(the small model went quiet — try again)"
218
+
219
+ self.session.commit(user_text, final_text)
220
+ self._emit({"type": "done", "text": final_text})
221
+ return {"text": final_text, "query": query, "result": result_text}
harness/loomspark_harness/cli.py ADDED
@@ -0,0 +1,225 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """loom-chat — terminal front-end for the Loom Spark harness."""
2
+
3
+ import argparse
4
+ import os
5
+ import sys
6
+
7
+ from .agent import LoomAgent
8
+ from .loader import load_model_and_tokenizer, resolve_model
9
+ from .search import get_backend, BACKENDS
10
+
11
+ LOGO_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)),
12
+ "static", "logo.png")
13
+
14
+
15
+ class Palette:
16
+ """Minimal ANSI styling; degrades to plain text when not a TTY."""
17
+
18
+ def __init__(self, enabled=True):
19
+ self.on = enabled and sys.stdout.isatty() \
20
+ and not os.environ.get("NO_COLOR")
21
+
22
+ def __call__(self, code, s):
23
+ return "\033[%sm%s\033[0m" % (code, s) if self.on else s
24
+
25
+ def dim(self, s):
26
+ return self("2m", s)
27
+
28
+ def bold(self, s):
29
+ return self("1m", s)
30
+
31
+ def accent(self, s):
32
+ return self("38;5;209m", s)
33
+
34
+ def blue(self, s):
35
+ return self("38;5;110m", s)
36
+
37
+ def green(self, s):
38
+ return self("38;5;108m", s)
39
+
40
+ def red(self, s):
41
+ return self("38;5;174m", s)
42
+
43
+ def gray(self, s):
44
+ return self("38;5;245m", s)
45
+
46
+
47
+ GLYPH = [
48
+ " \\ | / ",
49
+ " -- (o) -- ",
50
+ " / | \\ ",
51
+ ]
52
+
53
+
54
+ def render_logo(width=26):
55
+ """Truecolor half-block render of the Textile Labs avatar, or None."""
56
+ try:
57
+ from PIL import Image
58
+ except ImportError:
59
+ return None
60
+ try:
61
+ im = Image.open(LOGO_PATH).convert("RGBA")
62
+ h = width
63
+ im = im.resize((h, h), Image.LANCZOS)
64
+ px = im.load()
65
+ lines = []
66
+ for y in range(0, h, 2):
67
+ row = []
68
+ for x in range(h):
69
+ r1, g1, b1, a1 = px[x, y]
70
+ r2, g2, b2, a2 = (px[x, y + 1] if y + 1 < h else (0, 0, 0, 0))
71
+ if a1 < 40 and a2 < 40:
72
+ row.append(" ")
73
+ continue
74
+ top = "%d;%d;%d" % (r1, g1, b1) if a1 >= 40 else None
75
+ bot = "%d;%d;%d" % (r2, g2, b2) if a2 >= 40 else None
76
+ if top and bot:
77
+ row.append("\033[48;2;%sm\033[48;2;%sm▀\033[0m"
78
+ % (top, bot))
79
+ elif top:
80
+ row.append("\033[48;2;%sm▀\033[0m" % top)
81
+ else:
82
+ row.append("\033[48;2;%sm▄\033[0m" % bot)
83
+ lines.append("".join(row))
84
+ return lines
85
+ except Exception:
86
+ return None
87
+
88
+
89
+ def banner(pal, resolved, n_params, backend_name, online):
90
+ art = render_logo() if pal.on else None
91
+ left_w = len(GLYPH[0]) if art is None else 28
92
+ title = [
93
+ pal.bold(pal.accent("L O O M S P A R K")),
94
+ pal.dim("agent harness v0.1.1"),
95
+ pal.blue("Textile Labs"),
96
+ ]
97
+ rows = max(len(art) if art else len(GLYPH), len(title))
98
+ print("┌" + "─" * (left_w + 30) + "┐")
99
+ for i in range(rows):
100
+ if art:
101
+ cell = art[i] if i < len(art) else " " * left_w
102
+ elif i < len(GLYPH):
103
+ cell = pal.accent(GLYPH[i])
104
+ else:
105
+ cell = " " * left_w
106
+ t = title[i] if i < len(title) else ""
107
+ print("│ %s %-28s │" % (cell, t))
108
+ print("└" + "─" * (left_w + 30) + "┘")
109
+
110
+ def kv(k, v):
111
+ print(" %s %s" % (pal.gray("%-9s" % k), v))
112
+
113
+ print()
114
+ kv("model", "%s (%s)" % (resolved, n_params))
115
+ kv("backend", backend_name)
116
+ mode_line = (pal.green("ONLINE") + " " + pal.dim("<tools:on>")
117
+ if online else
118
+ pal.red("OFFLINE") + " " + pal.dim("<tools:off>"))
119
+ kv("mode", mode_line)
120
+ print()
121
+ print(" %s" % pal.dim("/online /offline /backend NAME /reset /quit"))
122
+ print("─" * 64)
123
+
124
+
125
+ def _cli_printer(pal):
126
+ def on_event(ev):
127
+ kind = ev["type"]
128
+ if kind == "token":
129
+ sys.stdout.write(ev["text"])
130
+ sys.stdout.flush()
131
+ elif kind == "preamble":
132
+ print()
133
+ print(" %s" % pal.dim(ev["text"].strip()))
134
+ elif kind == "lookup":
135
+ print(" %s %s" % (pal.accent("⌕ lookup "),
136
+ pal.bold(ev["query"])))
137
+ elif kind == "result":
138
+ ms = ev.get("ms")
139
+ took = (" in %.1fs" % (ms / 1000.0)) if ms else ""
140
+ if ev.get("failed"):
141
+ print(" %s %s" % (pal.red("✗ fetch "),
142
+ pal.dim("no answer came back" + took)))
143
+ else:
144
+ print(" %s %s%s" % (pal.green("≡ fetch "),
145
+ pal.gray("%d chars" % len(ev["text"])),
146
+ pal.gray(took)))
147
+ elif kind == "done":
148
+ print()
149
+ return on_event
150
+
151
+
152
+ def main():
153
+ ap = argparse.ArgumentParser(
154
+ prog="loom-chat",
155
+ description="Chat with Loom Spark through the agent harness "
156
+ "(internet lookups via <lookup>/<result>).")
157
+ ap.add_argument("--model", default=None,
158
+ help="HF export dir, .pt checkpoint, or hub id "
159
+ "(default: auto-detect)")
160
+ ap.add_argument("--backend", default="duckduckgo",
161
+ choices=sorted(BACKENDS),
162
+ help="search backend (default: duckduckgo)")
163
+ ap.add_argument("--offline", action="store_true",
164
+ help="start with tools off (no lookups)")
165
+ ap.add_argument("--plain", action="store_true",
166
+ help="disable ANSI colors/artwork")
167
+ args = ap.parse_args()
168
+
169
+ pal = Palette(enabled=not args.plain)
170
+
171
+ print(pal.dim("loading model…"))
172
+ model, tok, block = load_model_and_tokenizer(args.model)
173
+ backend = get_backend(args.backend)
174
+ agent = LoomAgent(model, tok, backend=backend,
175
+ online=not args.offline, block_size=block,
176
+ on_event=_cli_printer(pal))
177
+ n_params = "{:.1f}M".format(
178
+ sum(p.numel() for p in model.parameters()) / 1e6)
179
+ resolved = resolve_model(args.model)
180
+
181
+ banner(pal, resolved, n_params, backend.describe(), agent.online)
182
+
183
+ turn = 0
184
+ while True:
185
+ try:
186
+ prompt = "%s " % pal.blue("you ▸")
187
+ line = input("\n" + prompt).strip()
188
+ except (EOFError, KeyboardInterrupt):
189
+ print("\n%s" % pal.dim("bye. someone small enjoyed that."))
190
+ return
191
+ if not line:
192
+ continue
193
+ if line == "/quit":
194
+ print(pal.dim("bye. someone small enjoyed that."))
195
+ return
196
+ if line == "/reset":
197
+ agent.reset()
198
+ turn = 0
199
+ print(pal.dim("(conversation reset)"))
200
+ continue
201
+ if line in ("/online", "/offline"):
202
+ agent.set_online(line == "/online")
203
+ state = pal.green("ONLINE") if agent.online else pal.red("OFFLINE")
204
+ tag = "<tools:on>" if agent.online else "<tools:off>"
205
+ print(pal.dim("(mode switched: %s %s)" % (state, tag)))
206
+ continue
207
+ if line.startswith("/backend "):
208
+ name = line.split(None, 1)[1].strip()
209
+ try:
210
+ agent.backend = get_backend(name)
211
+ print(pal.dim("(backend switched to %s)"
212
+ % agent.backend.describe()))
213
+ except ValueError as e:
214
+ print(pal.red(str(e)))
215
+ continue
216
+
217
+ turn += 1
218
+ print("%s %s" % (pal.gray("#%d" % turn), pal.accent("loom ▸")))
219
+ out = agent.reply(line)
220
+ if not out["text"]:
221
+ print(pal.dim("(no output)"))
222
+
223
+
224
+ if __name__ == "__main__":
225
+ main()
harness/loomspark_harness/loader.py ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Resolve + load a Loom Spark model for the harness.
2
+
3
+ Accepted --model values:
4
+ - a directory containing config.json + model.safetensors (HF export)
5
+ - a training checkpoint file ending in .pt (needs the loomspark repo next
6
+ to the harness for the architecture class)
7
+ - a HuggingFace hub id, e.g. TextileLabs/loom-spark
8
+
9
+ transformers v5 note (PROJECT.md known issue #3): build the tokenizer with
10
+ PreTrainedTokenizerFast(tokenizer_file=...); the
11
+ GPT2TokenizerFast(vocab_file, merges_file) path returns empty encodings.
12
+ """
13
+
14
+ import json
15
+ import os
16
+ import re
17
+
18
+ from . import DEFAULT_HUB_MODEL
19
+ from .tok_adapters import HFTokenizerAdapter, TokenizersLibAdapter
20
+
21
+ _HUB_ID_RE = re.compile(r"^[\w.-]+/[\w.-]+$")
22
+
23
+
24
+ def _candidate_paths(ref):
25
+ if ref:
26
+ yield ref
27
+ return
28
+ env = os.environ.get("LOOM_MODEL")
29
+ if env:
30
+ yield env
31
+ return
32
+ here = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
33
+ for cand in (
34
+ os.path.join(here, "export", "loom-spark-hf"),
35
+ os.path.join(os.getcwd(), "export", "loom-spark-hf"),
36
+ os.path.join(here, "..", "export", "loom-spark-hf"),
37
+ ):
38
+ if os.path.isdir(cand):
39
+ yield cand
40
+ return
41
+ yield DEFAULT_HUB_MODEL
42
+
43
+
44
+ def resolve_model(ref=None):
45
+ for cand in _candidate_paths(ref):
46
+ if os.path.isdir(cand) and \
47
+ os.path.exists(os.path.join(cand, "config.json")):
48
+ return cand
49
+ if os.path.isfile(cand) and cand.endswith(".pt"):
50
+ return cand
51
+ if _HUB_ID_RE.match(cand): # org/name on the HuggingFace hub
52
+ return cand
53
+ return DEFAULT_HUB_MODEL
54
+
55
+
56
+ def _load_hf(ref):
57
+ """Load from a local export dir OR a hub id (from_pretrained handles both)."""
58
+ from transformers import GPT2LMHeadModel, PreTrainedTokenizerFast
59
+
60
+ model = GPT2LMHeadModel.from_pretrained(ref)
61
+ tok = None
62
+ if os.path.isdir(ref):
63
+ tok = PreTrainedTokenizerFast(
64
+ tokenizer_file=os.path.join(ref, "tokenizer.json"))
65
+ else:
66
+ try:
67
+ from transformers import AutoTokenizer
68
+ tok = AutoTokenizer.from_pretrained(ref)
69
+ except Exception:
70
+ pass # transformers 4.x can't map the v5 TokenizersBackend class
71
+ if tok is None:
72
+ from huggingface_hub import hf_hub_download
73
+ tok = PreTrainedTokenizerFast(
74
+ tokenizer_file=hf_hub_download(repo_id=ref,
75
+ filename="tokenizer.json"))
76
+ tok.eos_token = "<|endoftext|>"
77
+ tok.pad_token = "<|endoftext|>"
78
+ block_size = int(model.config.n_positions)
79
+ return model, HFTokenizerAdapter(tok), block_size
80
+
81
+
82
+ def _load_checkpoint(pt_path):
83
+ import torch
84
+ from loomspark.model import LoomConfig, LoomGPT
85
+ from loomspark.tokenizer import load as load_training_tokenizer
86
+ import loomspark.config as train_cfg
87
+
88
+ ck = torch.load(pt_path, map_location="cpu", weights_only=False)
89
+ c = ck["config"]
90
+ model = LoomGPT(LoomConfig(
91
+ vocab_size=c["vocab_size"], block_size=c["block_size"],
92
+ n_layer=c["n_layer"], n_head=c["n_head"], n_embd=c["n_embd"],
93
+ dropout=0.0))
94
+ model.load_state_dict(ck["model"])
95
+ tok = TokenizersLibAdapter(load_training_tokenizer(train_cfg.TOKENIZER_DIR))
96
+ return model.eval(), tok, int(c["block_size"])
97
+
98
+
99
+ def load_model_and_tokenizer(ref=None):
100
+ resolved = resolve_model(ref)
101
+
102
+ import torch
103
+ torch.set_num_threads(max(1, os.cpu_count() or 4))
104
+
105
+ if os.path.isdir(resolved):
106
+ model, tok, block = _load_hf(resolved)
107
+ elif resolved.endswith(".pt"):
108
+ model, tok, block = _load_checkpoint(resolved)
109
+ else:
110
+ model, tok, block = _load_hf(resolved) # hub id via from_pretrained
111
+
112
+ model.eval()
113
+ return model, tok, block
harness/loomspark_harness/search/__init__.py ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Search backends for the harness. A backend turns a query into plain text
2
+ (or None when the internet cannot be reached — models are trained to fall
3
+ back gracefully on empty/error results)."""
4
+
5
+ from .base import SearchBackend, format_results, truncate_result
6
+ from .mock import MockSearch
7
+ from .duckduckgo import DuckDuckGoSearch
8
+
9
+ BACKENDS = {
10
+ "mock": MockSearch,
11
+ "duckduckgo": DuckDuckGoSearch,
12
+ }
13
+
14
+
15
+ def get_backend(name):
16
+ name = (name or "mock").lower()
17
+ if name not in BACKENDS:
18
+ raise ValueError("unknown backend %r; available: %s"
19
+ % (name, ", ".join(sorted(BACKENDS))))
20
+ return BACKENDS[name]()
harness/loomspark_harness/search/base.py ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Search backend interface."""
2
+
3
+ RESULT_MAX_CHARS = 600
4
+
5
+
6
+ class SearchBackend:
7
+ """Subclasses return a plain-text result for `query`, or None on failure.
8
+
9
+ None is a first-class outcome: the training corpus includes graceful
10
+ fallback responses to empty/error results (PROJECT.md §4), so the harness
11
+ feeds back an honest 'could not reach the internet' string rather than
12
+ pretending.
13
+ """
14
+
15
+ name = "base"
16
+ available = True
17
+
18
+ def search(self, query):
19
+ raise NotImplementedError
20
+
21
+ def describe(self):
22
+ return self.name
23
+
24
+
25
+ def format_results(items, max_items=4):
26
+ """items: list of (title, snippet, url) -> compact numbered text block."""
27
+ lines = []
28
+ for i, (title, snippet, url) in enumerate(items[:max_items], 1):
29
+ line = "%d. %s" % (i, title.strip())
30
+ sn = " ".join((snippet or "").split())
31
+ if sn:
32
+ if len(sn) > 200:
33
+ sn = sn[:197] + "..."
34
+ line += " — %s" % sn
35
+ lines.append(line)
36
+ return "\n".join(lines)
37
+
38
+
39
+ def truncate_result(text, limit=RESULT_MAX_CHARS):
40
+ text = " ".join(text.split())
41
+ if len(text) <= limit:
42
+ return text
43
+ cut = text[:limit]
44
+ if " " in cut[40:]:
45
+ cut = cut[:cut.rfind(" ", 40)]
46
+ return cut + "…"
harness/loomspark_harness/search/duckduckgo.py ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Real internet search with no API key: scrapes DuckDuckGo's lightweight
2
+ HTML endpoints (html.duckduckgo.com/html). Stdlib only — urllib + html.parser.
3
+
4
+ DuckDuckGo's markup changes occasionally; every failure path returns None so
5
+ the model falls back gracefully instead of crashing the chat.
6
+ """
7
+
8
+ import html as html_mod
9
+ import re
10
+ import time
11
+ import urllib.parse
12
+ import urllib.request
13
+
14
+ from .base import SearchBackend, format_results
15
+
16
+ _UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
17
+ "(KHTML, like Gecko) Chrome/120.0 Safari/537.36")
18
+ _ENDPOINT = "https://html.duckduckgo.com/html/?q="
19
+
20
+ _RESULT_RE = re.compile(
21
+ r'<a[^>]+class="[^"]*result__a[^"]*"[^>]+href="([^"]+)"[^>]*>(.*?)</a>',
22
+ re.S)
23
+ _SNIPPET_RE = re.compile(
24
+ r'<a[^>]+class="[^"]*result__snippet[^"]*"[^>]*>(.*?)</a>', re.S)
25
+ _TAG_RE = re.compile(r"<[^>]+>")
26
+
27
+
28
+ def _clean(fragment):
29
+ text = _TAG_RE.sub("", fragment)
30
+ text = html_mod.unescape(text)
31
+ return " ".join(text.split())
32
+
33
+
34
+ def _real_url(href):
35
+ if href.startswith("//"):
36
+ href = "https:" + href
37
+ if "/l/?" in href or "uddg=" in href:
38
+ try:
39
+ qs = urllib.parse.urlsplit(href).query
40
+ params = urllib.parse.parse_qs(qs)
41
+ if "uddg" in params:
42
+ return urllib.parse.unquote(params["uddg"][0])
43
+ except ValueError:
44
+ pass
45
+ return href
46
+
47
+
48
+ def _fetch(url, timeout=10):
49
+ req = urllib.request.Request(url, headers={
50
+ "User-Agent": _UA,
51
+ "Accept-Language": "en-US,en;q=0.9",
52
+ })
53
+ with urllib.request.urlopen(req, timeout=timeout) as resp:
54
+ charset = resp.headers.get_content_charset() or "utf-8"
55
+ return resp.read().decode(charset, errors="replace")
56
+
57
+
58
+ class DuckDuckGoSearch(SearchBackend):
59
+ name = "duckduckgo"
60
+ available = True
61
+
62
+ def __init__(self, max_results=4, timeout=10):
63
+ self.max_results = max_results
64
+ self.timeout = timeout
65
+ self._last_request = 0.0
66
+
67
+ def search(self, query):
68
+ try:
69
+ # be polite: >= 1s between requests
70
+ wait = 1.0 - (time.time() - self._last_request)
71
+ if wait > 0:
72
+ time.sleep(wait)
73
+ self._last_request = time.time()
74
+
75
+ url = _ENDPOINT + urllib.parse.quote_plus(query)
76
+ page = _fetch(url, self.timeout)
77
+
78
+ titles = [(m.group(1), m.group(2))
79
+ for m in _RESULT_RE.finditer(page)]
80
+ snippets = [m.group(1) for m in _SNIPPET_RE.finditer(page)]
81
+ if not titles:
82
+ return None
83
+
84
+ items = []
85
+ for i, (href, title_html) in enumerate(titles):
86
+ snippet = snippets[i] if i < len(snippets) else ""
87
+ items.append((_clean(title_html), _clean(snippet),
88
+ _real_url(href)))
89
+ items = [it for it in items if it[0]]
90
+ if not items:
91
+ return None
92
+ return format_results(items[:self.max_results])
93
+ except Exception:
94
+ return None
95
+
96
+
97
+ if __name__ == "__main__":
98
+ import sys
99
+ q = sys.argv[1] if len(sys.argv) > 1 else "who wrote the loom"
100
+ print(DuckDuckGoSearch().search(q) or "(no results / unreachable)")
harness/loomspark_harness/search/mock.py ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Deterministic canned search — offline demos, tests, and no-internet
2
+ machines. Matches a few curriculum-style fact domains plus an honest
3
+ empty-result fallback."""
4
+
5
+ from .base import SearchBackend, format_results
6
+
7
+ _KB = {
8
+ "titanic": [
9
+ ("RMS Titanic - Wikipedia",
10
+ "The Titanic sank on 15 April 1912 in the North Atlantic after "
11
+ "hitting an iceberg on her maiden voyage from Southampton to New York.",
12
+ "https://en.wikipedia.org/wiki/RMS_Titanic"),
13
+ ],
14
+ "solar system": [
15
+ ("Solar System - NASA Science",
16
+ "Our solar system has eight planets: Mercury, Venus, Earth, Mars, "
17
+ "Jupiter, Saturn, Uranus and Neptune. Earth is the third planet "
18
+ "from the Sun.",
19
+ "https://science.nasa.gov/solar-system"),
20
+ ("How many moons does Mars have? - ESA",
21
+ "Mars has two small moons, Phobos and Deimos.", "https://esa.int"),
22
+ ],
23
+ "mars": [
24
+ ("Mars - NASA Science",
25
+ "Mars is the fourth planet from the Sun. It has two moons, Phobos "
26
+ "and Deimos, and a day length of about 24.6 hours.",
27
+ "https://science.nasa.gov/mars"),
28
+ ],
29
+ "water": [
30
+ ("Water facts - USGS",
31
+ "Water boils at 100 degrees Celsius at sea level and freezes at 0 "
32
+ "degrees Celsius. About 71 percent of Earth's surface is water.",
33
+ "https://usgs.gov/water"),
34
+ ],
35
+ "weather": [
36
+ ("Weather report (demo)",
37
+ "This is the mock backend: plug DuckDuckGo or your own backend for "
38
+ "real weather. No live data in mock mode.", ""),
39
+ ],
40
+ }
41
+
42
+
43
+ class MockSearch(SearchBackend):
44
+ name = "mock"
45
+ available = True
46
+
47
+ def search(self, query):
48
+ q = query.lower()
49
+ for key, items in _KB.items():
50
+ if key in q:
51
+ return format_results(items)
52
+ return ("[mock] No results for %r. (Canned demo backend — switch to "
53
+ "--backend duckduckgo for real searches.)" % query)
harness/loomspark_harness/session.py ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Conversation state for a Loom Spark session.
2
+
3
+ Document shape (matches the training corpus, PROJECT.md §4):
4
+
5
+ <tools:on>
6
+ <user> hello there
7
+ <loom> Hey. I'm here...<|endoftext|>
8
+ <user> ...
9
+ <loom>
10
+
11
+ The mode header is set once per session; switching modes resets history
12
+ (same policy as loomspark/chat.py).
13
+ """
14
+
15
+ from . import TOOLS_ON, TOOLS_OFF, USER_TOK, LOOM_TOK, EOS_TOK
16
+
17
+
18
+ class Session:
19
+ def __init__(self, online=False):
20
+ self.online = online
21
+ self.turns = [] # list of (user_text, loom_text)
22
+
23
+ @property
24
+ def header(self):
25
+ return TOOLS_ON if self.online else TOOLS_OFF
26
+
27
+ def set_online(self, online):
28
+ if online != self.online:
29
+ self.online = online
30
+ self.reset()
31
+
32
+ def reset(self):
33
+ self.turns = []
34
+
35
+ # ---------------------------------------------------------------- build
36
+
37
+ def completed_block(self):
38
+ """Text of all finished turns."""
39
+ parts = []
40
+ for user_text, loom_text in self.turns:
41
+ parts.append("%s %s\n%s%s%s\n" % (
42
+ USER_TOK, user_text, LOOM_TOK, loom_text, EOS_TOK))
43
+ return "".join(parts)
44
+
45
+ def prompt_for(self, user_text):
46
+ """Full context ending mid-turn at <loom>, ready for generation."""
47
+ return "%s\n%s%s %s\n%s" % (
48
+ self.header, self.completed_block(), USER_TOK, user_text, LOOM_TOK)
49
+
50
+ def commit(self, user_text, loom_text):
51
+ self.turns.append((user_text, loom_text))
52
+
53
+ def trim(self, n_tokens, count_fn):
54
+ """Drop oldest turns until the full next-prompt fits n_tokens.
55
+
56
+ count_fn: str -> int token count (injected so Session stays
57
+ tokenizer-agnostic)."""
58
+ probe_user = self.turns[-1][0] if self.turns else ""
59
+ while self.turns:
60
+ if count_fn(self.prompt_for(probe_user)) <= n_tokens:
61
+ break
62
+ self.turns.pop(0)
harness/loomspark_harness/static/index.html ADDED
@@ -0,0 +1,250 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="utf-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1">
6
+ <link rel="icon" type="image/png" href="/logo.png">
7
+ <title>Loom Spark · Textile Labs</title>
8
+ <style>
9
+ :root {
10
+ --bg: #0f1117; --panel: #161a23; --panel2: #1c2130;
11
+ --line: #2a3042; --text: #e8e6df; --dim: #8b93a7;
12
+ --accent: #d98a4b; --accent2: #7aa2f7; --ok: #9ece6a; --err: #f7768e;
13
+ font-size: 16px;
14
+ }
15
+ * { box-sizing: border-box; margin: 0; padding: 0; }
16
+ body {
17
+ background: var(--bg); color: var(--text);
18
+ font-family: ui-sans-serif, system-ui, "Segoe UI", sans-serif;
19
+ height: 100vh; display: flex; overflow: hidden;
20
+ }
21
+ header {
22
+ position: fixed; top: 0; left: 0; right: 0; height: 52px; z-index: 10;
23
+ background: var(--panel); border-bottom: 1px solid var(--line);
24
+ display: flex; align-items: center; gap: 14px; padding: 0 18px;
25
+ }
26
+ .logo { font-weight: 700; letter-spacing: .4px; }
27
+ .logo small { color: var(--dim); font-weight: 400; margin-left: 8px; }
28
+ .spacer { flex: 1; }
29
+ .mode { display: flex; align-items: center; gap: 8px; color: var(--dim); }
30
+ .switch {
31
+ width: 44px; height: 24px; border-radius: 12px; background: var(--panel2);
32
+ border: 1px solid var(--line); cursor: pointer; position: relative;
33
+ }
34
+ .switch::after {
35
+ content: ""; position: absolute; top: 2px; left: 2px; width: 18px;
36
+ height: 18px; border-radius: 50%; background: var(--dim);
37
+ transition: all .15s ease;
38
+ }
39
+ .switch.on::after { left: 22px; background: var(--ok); }
40
+ .mode-label { min-width: 74px; text-align: right; }
41
+ main { flex: 1; display: flex; margin-top: 52px; min-width: 0; }
42
+ #chatcol { flex: 1; display: flex; flex-direction: column; min-width: 0; }
43
+ #feed {
44
+ width: 340px; background: var(--panel); border-left: 1px solid var(--line);
45
+ overflow-y: auto; padding: 14px; display: none;
46
+ }
47
+ #feed.show { display: block; }
48
+ #feed h3 {
49
+ font-size: .78rem; text-transform: uppercase; letter-spacing: .8px;
50
+ color: var(--dim); margin-bottom: 10px;
51
+ }
52
+ .lookup-card {
53
+ background: var(--panel2); border: 1px solid var(--line);
54
+ border-radius: 8px; padding: 10px 12px; margin-bottom: 10px;
55
+ font-size: .85rem;
56
+ }
57
+ .lookup-card .q { color: var(--accent2); word-break: break-word; }
58
+ .lookup-card .r { color: var(--dim); margin-top: 6px; white-space: pre-wrap; }
59
+ .lookup-card .fail { color: var(--err); margin-top: 6px; }
60
+ #log { flex: 1; overflow-y: auto; padding: 26px 10%; scroll-behavior: smooth; }
61
+ .msg { max-width: 720px; margin: 0 auto 18px; line-height: 1.55; }
62
+ .msg .who {
63
+ font-size: .75rem; letter-spacing: .8px; text-transform: uppercase;
64
+ margin-bottom: 4px;
65
+ }
66
+ .msg.user .who { color: var(--accent2); }
67
+ .msg.loom .who { color: var(--accent); }
68
+ .msg.user .body { background: var(--panel); border: 1px solid var(--line);
69
+ padding: 10px 14px; border-radius: 10px; }
70
+ .msg.loom .body { white-space: pre-wrap; }
71
+ .msg.loom.thinking .body::after {
72
+ content: "▍"; color: var(--accent); animation: blink 1s steps(2) infinite;
73
+ }
74
+ @keyframes blink { 50% { opacity: 0; } }
75
+ form {
76
+ display: flex; gap: 10px; padding: 16px 10% 20px; max-width: 900px;
77
+ margin: 0 auto; width: 100%;
78
+ }
79
+ input[type=text] {
80
+ flex: 1; background: var(--panel); border: 1px solid var(--line);
81
+ color: var(--text); border-radius: 10px; padding: 12px 14px;
82
+ font-size: 1rem; outline: none;
83
+ }
84
+ input[type=text]:focus { border-color: var(--accent); }
85
+ button {
86
+ background: var(--accent); border: 0; color: #14100c; font-weight: 700;
87
+ border-radius: 10px; padding: 0 22px; cursor: pointer; font-size: 1rem;
88
+ }
89
+ button:hover { filter: brightness(1.08); }
90
+ button:disabled { opacity: .5; cursor: default; }
91
+ .hint { max-width: 900px; margin: -12px auto 0; padding: 0 10%;
92
+ color: var(--dim); font-size: .78rem; text-align: center; }
93
+ @media (max-width: 900px) { #feed { display: none !important; } #log,
94
+ form { padding-left: 5%; padding-right: 5%; } }
95
+ </style>
96
+ </head>
97
+ <body>
98
+ <header>
99
+ <img src="/logo.png" alt="Textile Labs"
100
+ style="width:32px;height:32px;border-radius:50%;
101
+ border:1px solid var(--line);object-fit:cover;">
102
+ <div class="logo">Loom Spark <small>Textile Labs · harness v0.1.1</small></div>
103
+ <div class="spacer"></div>
104
+ <button id="reset" style="background:var(--panel2);color:var(--dim);
105
+ border:1px solid var(--line);padding:6px 14px;font-size:.85rem;">reset</button>
106
+ <div class="mode">
107
+ <span class="mode-label" id="modelabel">tools: on</span>
108
+ <div class="switch on" id="modeswitch" title="toggle internet access"></div>
109
+ </div>
110
+ </header>
111
+
112
+ <main>
113
+ <div id="chatcol">
114
+ <div id="log"></div>
115
+ <form id="form">
116
+ <input type="text" id="box" autocomplete="off"
117
+ placeholder="say something…">
118
+ <button id="send">send</button>
119
+ </form>
120
+ <div class="hint" id="hint"></div>
121
+ </div>
122
+ <aside id="feed"><h3>harness feed — internet lookups</h3><div id="cards"></div></aside>
123
+ </main>
124
+
125
+ <script>
126
+ const log = document.getElementById("log");
127
+ const box = document.getElementById("box");
128
+ const send = document.getElementById("send");
129
+ const feed = document.getElementById("feed");
130
+ const cards = document.getElementById("cards");
131
+ const sw = document.getElementById("modeswitch");
132
+ const modelabel = document.getElementById("modelabel");
133
+ const hint = document.getElementById("hint");
134
+ let online = true, busy = false;
135
+
136
+ function addMsg(cls, who) {
137
+ const m = document.createElement("div");
138
+ m.className = "msg " + cls;
139
+ const w = document.createElement("div"); w.className = "who";
140
+ w.textContent = who;
141
+ const b = document.createElement("div"); b.className = "body";
142
+ m.appendChild(w); m.appendChild(b); log.appendChild(m);
143
+ log.scrollTop = log.scrollHeight;
144
+ return b;
145
+ }
146
+ function scrollDown() { log.scrollTop = log.scrollHeight; }
147
+
148
+ function setOnline(v) {
149
+ online = v;
150
+ sw.classList.toggle("on", v);
151
+ modelabel.textContent = v ? "tools: on" : "tools: off";
152
+ }
153
+ sw.onclick = () => setOnline(!online);
154
+
155
+ document.getElementById("reset").onclick = async () => {
156
+ await fetch("/api/reset", {method: "POST"});
157
+ document.getElementById("cards").innerHTML = "";
158
+ log.innerHTML = "";
159
+ hint.textContent = "";
160
+ };
161
+
162
+ function addCard(query) {
163
+ const c = document.createElement("div");
164
+ c.className = "lookup-card";
165
+ const q = document.createElement("div"); q.className = "q";
166
+ q.textContent = "⌕ " + query;
167
+ c.appendChild(q);
168
+ cards.prepend(c);
169
+ feed.classList.add("show");
170
+ return c;
171
+ }
172
+
173
+ async function sendMsg() {
174
+ if (busy || !box.value.trim()) return;
175
+ busy = true; send.disabled = true;
176
+ addMsg("user", "you").textContent = box.value.trim();
177
+ const body = addMsg("loom", "loom spark");
178
+ body.parentElement.classList.add("thinking");
179
+ let card = null;
180
+
181
+ try {
182
+ const res = await fetch("/api/chat", {
183
+ method: "POST",
184
+ headers: {"Content-Type": "application/json"},
185
+ body: JSON.stringify({message: box.value.trim(), mode: online ? "online" : "offline"}),
186
+ });
187
+ box.value = "";
188
+ const reader = res.body.getReader();
189
+ const dec = new TextDecoder();
190
+ let buf = "";
191
+ while (true) {
192
+ const {done, value} = await reader.read();
193
+ if (done) break;
194
+ buf += dec.decode(value, {stream: true});
195
+ let idx;
196
+ while ((idx = buf.indexOf("\n")) >= 0) {
197
+ const line = buf.slice(0, idx).trim(); buf = buf.slice(idx + 1);
198
+ if (!line) continue;
199
+ let ev; try { ev = JSON.parse(line); } catch { continue; }
200
+
201
+ switch (ev.type) {
202
+ case "token":
203
+ body.textContent += ev.text; scrollDown(); break;
204
+ case "preamble":
205
+ body.textContent += ev.text; break;
206
+ case "lookup":
207
+ card = addCard(ev.query); break;
208
+ case "result":
209
+ if (card) {
210
+ const r = document.createElement("div");
211
+ r.className = ev.failed ? "fail" : "r";
212
+ r.textContent = ev.failed
213
+ ? "(search failed / empty)" : ev.text;
214
+ card.appendChild(r);
215
+ } break;
216
+ case "done":
217
+ body.textContent = ev.text; break;
218
+ case "final":
219
+ hint.textContent = ev.query
220
+ ? ("last lookup: " + ev.query) : "";
221
+ break;
222
+ case "error":
223
+ body.textContent = "⚠ " + ev.text; break;
224
+ }
225
+ scrollDown();
226
+ }
227
+ }
228
+ } catch (e) {
229
+ body.textContent = "⚠ connection to the local server failed";
230
+ }
231
+ body.parentElement.classList.remove("thinking");
232
+ busy = false; send.disabled = false; box.focus();
233
+ }
234
+
235
+ document.getElementById("form").addEventListener("submit", e => {
236
+ e.preventDefault(); sendMsg();
237
+ });
238
+ box.focus();
239
+
240
+ fetch("/api/info").then(r => r.json()).then(info => {
241
+ setOnline(!!info.online);
242
+ for (const t of (info.history || [])) {
243
+ addMsg("user", "you").textContent = t.user;
244
+ addMsg("loom", "loom spark").textContent = t.loom;
245
+ }
246
+ scrollDown();
247
+ }).catch(() => {});
248
+ </script>
249
+ </body>
250
+ </html>
harness/loomspark_harness/static/logo.png ADDED
harness/loomspark_harness/tok_adapters.py ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tokenizer adapters so the agent can run on either the training-repo
2
+ `tokenizers.Tokenizer` or a transformers fast tokenizer (HF hub / exported
3
+ folder), without depending on either at import time."""
4
+
5
+ from . import (EOS_TOK, USER_TOK, LOOM_TOK, TOOLS_ON, TOOLS_OFF,
6
+ LOOKUP_OPEN, LOOKUP_CLOSE, RESULT_OPEN, RESULT_CLOSE)
7
+
8
+ SPECIALS = [EOS_TOK, USER_TOK, LOOM_TOK, TOOLS_ON, TOOLS_OFF,
9
+ LOOKUP_OPEN, LOOKUP_CLOSE, RESULT_OPEN, RESULT_CLOSE]
10
+
11
+
12
+ class TokenizersLibAdapter:
13
+ """Wraps tokenizers.Tokenizer (training repo: data/tokenizer)."""
14
+
15
+ def __init__(self, tok):
16
+ self.tok = tok
17
+
18
+ def encode_ids(self, text):
19
+ return self.tok.encode(text, add_special_tokens=False).ids
20
+
21
+ def decode(self, ids):
22
+ return self.tok.decode(ids, skip_special_tokens=False)
23
+
24
+ def id_of(self, special):
25
+ tid = self.tok.token_to_id(special)
26
+ return None if tid is None else int(tid)
27
+
28
+
29
+ class HFTokenizerAdapter:
30
+ """Wraps a transformers fast tokenizer (v5-safe construction happens in
31
+ loader.py; here we only need encode/decode/token-id lookups)."""
32
+
33
+ def __init__(self, tok):
34
+ self.tok = tok
35
+
36
+ def encode_ids(self, text):
37
+ return self.tok(text, add_special_tokens=False)["input_ids"]
38
+
39
+ def decode(self, ids):
40
+ return self.tok.decode(ids, skip_special_tokens=False)
41
+
42
+ def id_of(self, special):
43
+ tid = self.tok.convert_tokens_to_ids(special)
44
+ if tid is None:
45
+ return None
46
+ # transformers returns unk id for missing tokens; guard against it
47
+ if getattr(self.tok, "unk_token_id", None) == tid and \
48
+ special not in SPECIALS:
49
+ return None
50
+ return int(tid)
51
+
52
+
53
+ def load_training_tokenizer(tokenizer_dir):
54
+ from tokenizers import Tokenizer
55
+ import os
56
+ return TokenizersLibAdapter(
57
+ Tokenizer.from_file(os.path.join(tokenizer_dir, "tokenizer.json")))
harness/loomspark_harness/web.py ADDED
@@ -0,0 +1,177 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """loom-web — local web GUI for the Loom Spark harness.
2
+
3
+ Stdlib only (ThreadingHTTPServer). Serves one self-contained HTML page and
4
+ streams harness events as newline-delimited JSON over chunked responses.
5
+
6
+ loom-web --port 7860 # then open http://localhost:7860
7
+ """
8
+
9
+ import argparse
10
+ import json
11
+ import os
12
+ import threading
13
+ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
14
+
15
+ from .agent import LoomAgent
16
+ from .loader import load_model_and_tokenizer, resolve_model
17
+ from .search import get_backend, BACKENDS
18
+
19
+ _STATIC = os.path.join(os.path.dirname(os.path.abspath(__file__)),
20
+ "static", "index.html")
21
+
22
+ _state = {"agent": None, "lock": threading.Lock(), "model_ref": None}
23
+
24
+
25
+ class Handler(BaseHTTPRequestHandler):
26
+ protocol_version = "HTTP/1.1"
27
+
28
+ def log_message(self, fmt, *args): # quiet
29
+ pass
30
+
31
+ # ------------------------------------------------------------- helpers
32
+
33
+ def _send(self, code, body, ctype):
34
+ data = body if isinstance(body, bytes) else body.encode("utf-8")
35
+ self.send_response(code)
36
+ self.send_header("Content-Type", ctype)
37
+ self.send_header("Content-Length", str(len(data)))
38
+ self.send_header("Cache-Control", "no-store")
39
+ self.end_headers()
40
+ try:
41
+ self.wfile.write(data)
42
+ except BrokenPipeError:
43
+ pass
44
+
45
+ def _sse(self, ev):
46
+ return json.dumps(ev, ensure_ascii=False) + "\n"
47
+
48
+ # -------------------------------------------------------------- routes
49
+
50
+ def do_GET(self):
51
+ path = self.path.split("?")[0]
52
+ if path in ("/", "/index.html"):
53
+ with open(_STATIC, "rb") as f:
54
+ self._send(200, f.read(), "text/html; charset=utf-8")
55
+ elif path == "/favicon.ico":
56
+ self._send(204, b"", "image/x-icon")
57
+ elif path == "/logo.png":
58
+ logo = os.path.join(os.path.dirname(_STATIC), "logo.png")
59
+ if os.path.exists(logo):
60
+ with open(logo, "rb") as f:
61
+ self._send(200, f.read(), "image/png")
62
+ else:
63
+ self._send(404, "no logo", "text/plain")
64
+ elif path == "/api/info":
65
+ agent = _state["agent"]
66
+ info = {
67
+ "model": _state["model_ref"],
68
+ "online": agent.online,
69
+ "backend": agent.backend.describe() if agent.backend else None,
70
+ "history": [
71
+ {"user": u, "loom": l} for u, l in agent.session.turns],
72
+ }
73
+ self._send(200, json.dumps(info), "application/json")
74
+ else:
75
+ self._send(404, "not found", "text/plain")
76
+
77
+ def do_POST(self):
78
+ if self.path == "/api/reset":
79
+ with _state["lock"]:
80
+ _state["agent"].reset()
81
+ self._send(200, '{"ok": true}', "application/json")
82
+ return
83
+ if self.path != "/api/chat":
84
+ self._send(404, "not found", "text/plain")
85
+ return
86
+
87
+ length = int(self.headers.get("Content-Length") or 0)
88
+ try:
89
+ req = json.loads(self.rfile.read(length) or b"{}")
90
+ except json.JSONDecodeError:
91
+ self._send(400, "bad json", "text/plain")
92
+ return
93
+
94
+ message = str(req.get("message", "")).strip()
95
+ mode = str(req.get("mode", "online"))
96
+ if not message:
97
+ self._send(400, "empty message", "text/plain")
98
+ return
99
+
100
+ agent = _state["agent"]
101
+ with _state["lock"]:
102
+ agent.set_online(mode == "online")
103
+
104
+ self.send_response(200)
105
+ self.send_header("Content-Type",
106
+ "application/x-ndjson; charset=utf-8")
107
+ self.send_header("Transfer-Encoding", "chunked")
108
+ self.send_header("Cache-Control", "no-store")
109
+ self.end_headers()
110
+
111
+ def on_event(ev):
112
+ try:
113
+ payload = self._sse(ev).encode("utf-8")
114
+ self.wfile.write(b"%x\r\n" % len(payload) + payload
115
+ + b"\r\n")
116
+ self.wfile.flush()
117
+ except (BrokenPipeError, ConnectionResetError):
118
+ raise RuntimeError("client gone")
119
+
120
+ try:
121
+ out = agent.reply(message)
122
+ tail = self._sse({"type": "final", "query": out["query"],
123
+ "result": out["result"],
124
+ "text": out["text"]}).encode("utf-8")
125
+ self.wfile.write(b"%x\r\n" % len(tail) + tail + b"\r\n")
126
+ self.wfile.write(b"0\r\n\r\n")
127
+ except RuntimeError:
128
+ pass
129
+ except Exception as e: # model blew up mid-turn; keep server alive
130
+ err = self._sse({"type": "error",
131
+ "text": str(e)}).encode("utf-8")
132
+ try:
133
+ self.wfile.write(b"%x\r\n" % len(err) + err + b"\r\n0\r\n\r\n")
134
+ except OSError:
135
+ pass
136
+
137
+
138
+ def main():
139
+ ap = argparse.ArgumentParser(
140
+ prog="loom-web",
141
+ description="Local chat GUI for Loom Spark with internet search.")
142
+ ap.add_argument("--model", default=None,
143
+ help="HF export dir, .pt checkpoint, or hub id")
144
+ ap.add_argument("--backend", default="duckduckgo",
145
+ choices=sorted(BACKENDS))
146
+ ap.add_argument("--port", type=int, default=7860)
147
+ ap.add_argument("--host", default="127.0.0.1")
148
+ ap.add_argument("--offline", action="store_true")
149
+ args = ap.parse_args()
150
+
151
+ model, tok, block = load_model_and_tokenizer(args.model)
152
+ backend = get_backend(args.backend)
153
+ _state["agent"] = LoomAgent(model, tok, backend=backend,
154
+ online=not args.offline, block_size=block)
155
+ _state["model_ref"] = resolve_model(args.model)
156
+
157
+ url = "http://%s:%d" % (args.host, args.port)
158
+ print("Loom Spark · Textile Labs — web harness v0.1")
159
+ print("model: %s" % _state["model_ref"])
160
+ print("backend: %s | mode: %s" % (
161
+ backend.describe(), "ONLINE" if not args.offline else "OFFLINE"))
162
+ print("opening %s (Ctrl-C to stop)" % url)
163
+ try:
164
+ import webbrowser
165
+ webbrowser.open(url)
166
+ except Exception:
167
+ pass
168
+
169
+ server = ThreadingHTTPServer((args.host, args.port), Handler)
170
+ try:
171
+ server.serve_forever()
172
+ except KeyboardInterrupt:
173
+ print("\nbye. someone small enjoyed that.")
174
+
175
+
176
+ if __name__ == "__main__":
177
+ main()
harness/pyproject.toml ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [build-system]
2
+ requires = ["setuptools>=68"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "loomspark-harness"
7
+ version = "0.1.1"
8
+ description = "Agent harness for Loom Spark: chat interface + internet search reflex (lookup/result protocol)"
9
+ readme = "README.md"
10
+ license = { text = "MIT" }
11
+ authors = [{ name = "Textile Labs" }]
12
+ requires-python = ">=3.9"
13
+ classifiers = [
14
+ "Programming Language :: Python :: 3",
15
+ "Programming Language :: Python :: 3.9",
16
+ "Programming Language :: Python :: 3.10",
17
+ "Programming Language :: Python :: 3.11",
18
+ "Programming Language :: Python :: 3.12",
19
+ "Programming Language :: Python :: 3.13",
20
+ "License :: OSI Approved :: MIT License",
21
+ "Operating System :: OS Independent",
22
+ ]
23
+ dependencies = [
24
+ "torch",
25
+ "transformers>=4.40",
26
+ "tokenizers",
27
+ ]
28
+
29
+ [project.optional-dependencies]
30
+ logo = ["pillow"]
31
+
32
+ [project.scripts]
33
+ loom-chat = "loomspark_harness.cli:main"
34
+ loom-web = "loomspark_harness.web:main"
35
+
36
+ [tool.setuptools.packages.find]
37
+ include = ["loomspark_harness*"]
38
+
39
+ [tool.setuptools.package-data]
40
+ loomspark_harness = ["static/*.html", "static/*.png"]
loom-spark-f32.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0838a0532e2301dc89eaa83c4d263f6f5c6af4e5e669dde5f4b70030231fb86
3
+ size 35610336
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d60b708001baecca341312d92af01927a779b8407048f11dff1d98e67a28c980
3
+ size 30238648
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<|endoftext|>",
4
+ "eos_token": "<|endoftext|>",
5
+ "model_max_length": 1000000000000000019884624838656,
6
+ "pad_token": "<|endoftext|>",
7
+ "tokenizer_class": "TokenizersBackend"
8
+ }