ereniko commited on
Commit
40a5fbc
·
unverified ·
1 Parent(s): ff322b9

v0.5.0: more local model options and a model picker

Browse files
Files changed (12) hide show
  1. README.md +22 -4
  2. css/app.css +19 -1
  3. index.html +26 -2
  4. js/agent.js +5 -2
  5. js/config.js +41 -9
  6. js/lazy-weights.js +22 -12
  7. js/llm-engine.js +15 -5
  8. js/llm-worker.js +5 -4
  9. js/llm.js +7 -3
  10. js/main.js +87 -13
  11. js/setup.js +23 -3
  12. sw.js +1 -1
README.md CHANGED
@@ -7,9 +7,13 @@ sdk: static
7
  app_file: index.html
8
  pinned: false
9
  license: apache-2.0
10
- short_description: LFM3-1.7B agent driving a real Linux VM, fully offline
11
  models:
12
  - LiquidAI/LFM3-1.7B-ONNX
 
 
 
 
13
  custom_headers:
14
  cross-origin-embedder-policy: require-corp
15
  cross-origin-opener-policy: same-origin
@@ -20,7 +24,7 @@ custom_headers:
20
 
21
  An AI agent with its own Linux computer, entirely inside one web page.
22
 
23
- - **Model:** [LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B) (q4f16 ONNX) running on your GPU through WebGPU with [transformers.js](https://github.com/huggingface/transformers.js). If your device can't run it, you can [use an API model instead](#use-an-api-model).
24
  - **Computer:** a real Debian (i386) machine running in the browser on [CheerpX](https://cheerpx.io), the engine behind [WebVM](https://webvm.io). It always runs on your device, whichever model you use.
25
  - **Agent harness:** the model calls `run_command`, `write_file` and `read_file` tools. Commands run in the same terminal you see, so you can watch and take over at any time.
26
  - **File viewer:** browse the machine's files and open code (syntax highlighted), text, Markdown, images, audio and video. HTML pages open as sandboxed live previews together with the stylesheets, scripts, images and data files they reference, and links between pages work.
@@ -28,9 +32,23 @@ An AI agent with its own Linux computer, entirely inside one web page.
28
 
29
  **Requirements for the local model:** desktop Chrome or Edge with WebGPU and `shader-f16` support, about 2.5 GB of free disk space, and 8 GB or more RAM. Open the app at its own URL (`https://<owner>-localwebagent.static.hf.space`), because the embedded view on huggingface.co can't enable the cross-origin isolation CheerpX needs.
30
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  ## Phones and low-memory devices
32
 
33
- The local model needs about 1.4 GB of graphics memory. Most phones can't give that much to a browser tab, so the app warns you on phones and suggests an API model instead. You can still try the local model. If it fails to load, the app explains why (out of graphics memory, no WebGPU support, or a failed download), keeps Linux running, and offers to switch to an API model or try again. The raw error is under "Technical details".
34
 
35
  To make the local model fit on more devices, the weights are kept on disk and read tensor by tensor while they are copied to the GPU, instead of being held in memory all at once. On phones and devices reporting less than 8 GB of RAM, the model also loads before Linux starts, so the two don't compete for memory.
36
 
@@ -59,6 +77,6 @@ The key stays only until you close the tab, unless you turn on "Remember key on
59
  ## Licenses
60
 
61
  - This app: Apache-2.0.
62
- - LFM3-1.7B: [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM3-1.7B-ONNX/blob/main/LICENSE). The weights are loaded directly from Liquid AI's repository, not re-hosted here.
63
  - CheerpX: proprietary, free for personal and open-source projects. It is **not** included in this Space. Each visitor's browser loads it from Leaning Technologies' CDN and caches it locally.
64
  - The Debian disk image (`disk/`) contains Debian packages under their respective free licenses.
 
7
  app_file: index.html
8
  pinned: false
9
  license: apache-2.0
10
+ short_description: Local AI agent driving a real Linux VM, fully offline
11
  models:
12
  - LiquidAI/LFM3-1.7B-ONNX
13
+ - onnx-community/Qwen3.5-0.8B-Text-ONNX
14
+ - onnx-community/Qwen3-1.7B-ONNX
15
+ - onnx-community/granite-4.0-1b-ONNX-web
16
+ - onnx-community/Qwen3-4B-ONNX
17
  custom_headers:
18
  cross-origin-embedder-policy: require-corp
19
  cross-origin-opener-policy: same-origin
 
24
 
25
  An AI agent with its own Linux computer, entirely inside one web page.
26
 
27
+ - **Model:** [LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B) by default, or one of [several other local models](#local-models), running on your GPU through WebGPU with [transformers.js](https://github.com/huggingface/transformers.js). If your device can't run them, you can [use an API model instead](#use-an-api-model).
28
  - **Computer:** a real Debian (i386) machine running in the browser on [CheerpX](https://cheerpx.io), the engine behind [WebVM](https://webvm.io). It always runs on your device, whichever model you use.
29
  - **Agent harness:** the model calls `run_command`, `write_file` and `read_file` tools. Commands run in the same terminal you see, so you can watch and take over at any time.
30
  - **File viewer:** browse the machine's files and open code (syntax highlighted), text, Markdown, images, audio and video. HTML pages open as sandboxed live previews together with the stylesheets, scripts, images and data files they reference, and links between pages work.
 
32
 
33
  **Requirements for the local model:** desktop Chrome or Edge with WebGPU and `shader-f16` support, about 2.5 GB of free disk space, and 8 GB or more RAM. Open the app at its own URL (`https://<owner>-localwebagent.static.hf.space`), because the embedded view on huggingface.co can't enable the cross-origin isolation CheerpX needs.
34
 
35
+ ## Local models
36
+
37
+ Pick one in **Settings → Model → Change**, or with "Choose a different local model" on the setup screen. All are 4-bit (q4f16) ONNX builds loaded straight from their Hugging Face repositories; switching downloads the new model once, and the model picker can delete models you no longer use.
38
+
39
+ | Model | Download | Context | Reasoning | Notes |
40
+ |---|---:|---:|:---:|---|
41
+ | [LFM3 1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B-ONNX) (default) | 1.38 GB | 256k | yes | Fast hybrid model by Liquid AI, good at tool use. |
42
+ | [Qwen3.5 0.8B](https://huggingface.co/onnx-community/Qwen3.5-0.8B-Text-ONNX) | 0.47 GB | 256k | yes | Smallest. Best chance on weak GPUs and phones; less capable. |
43
+ | [Qwen3 1.7B](https://huggingface.co/onnx-community/Qwen3-1.7B-ONNX) | 1.43 GB | 40k | yes | Similar size to LFM3, strong reasoning. |
44
+ | [Granite 4.0 1B](https://huggingface.co/onnx-community/granite-4.0-1b-ONNX-web) | 1.25 GB | 128k | no | IBM's hybrid model built for tool calling. |
45
+ | [Qwen3 4B](https://huggingface.co/onnx-community/Qwen3-4B-ONNX) | 2.83 GB | 40k | yes | Most capable, but slow. Needs a GPU with about 4 GB of memory. |
46
+
47
+ The context size setting is capped at each model's own window.
48
+
49
  ## Phones and low-memory devices
50
 
51
+ The default model needs about 1.4 GB of graphics memory. Most phones can't give that much to a browser tab, so the app warns you on phones and suggests an API model or the smaller Qwen3.5 0.8B instead. You can still try any local model. If it fails to load, the app explains why (out of graphics memory, no WebGPU support, or a failed download), keeps Linux running, and offers to switch to an API model, pick a smaller model, or try again. The raw error is under "Technical details".
52
 
53
  To make the local model fit on more devices, the weights are kept on disk and read tensor by tensor while they are copied to the GPU, instead of being held in memory all at once. On phones and devices reporting less than 8 GB of RAM, the model also loads before Linux starts, so the two don't compete for memory.
54
 
 
77
  ## Licenses
78
 
79
  - This app: Apache-2.0.
80
+ - LFM3-1.7B: [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM3-1.7B-ONNX/blob/main/LICENSE). Qwen3, Qwen3.5 and Granite 4.0: Apache-2.0. No weights are re-hosted here: your browser downloads them directly from the repositories linked above.
81
  - CheerpX: proprietary, free for personal and open-source projects. It is **not** included in this Space. Each visitor's browser loads it from Leaning Technologies' CDN and caches it locally.
82
  - The Debian disk image (`disk/`) contains Debian packages under their respective free licenses.
css/app.css CHANGED
@@ -497,7 +497,25 @@ dialog[open] { animation: pop .14s ease-out; }
497
  .field-hint { margin: -4px 0 8px; font-size: 12px; color: var(--text-3); }
498
  .note.ok { color: var(--ok); }
499
  .dlg-foot { display: flex; align-items: center; gap: 8px; padding: 12px 22px 18px; border-top: 1px solid var(--border); }
500
- #api-dlg { max-height: calc(100dvh - 32px); overflow-y: auto; }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
501
 
502
  /* ---------- narrow screens ---------- */
503
  @media (max-width: 900px) {
 
497
  .field-hint { margin: -4px 0 8px; font-size: 12px; color: var(--text-3); }
498
  .note.ok { color: var(--ok); }
499
  .dlg-foot { display: flex; align-items: center; gap: 8px; padding: 12px 22px 18px; border-top: 1px solid var(--border); }
500
+ #api-dlg, #model-dlg { max-height: calc(100dvh - 32px); overflow-y: auto; }
501
+ #model-dlg { width: min(560px, calc(100% - 32px)); }
502
+ .model-list { display: flex; flex-direction: column; gap: 8px; }
503
+ .model-opt {
504
+ display: flex; gap: 12px; align-items: flex-start; padding: 12px 14px; cursor: pointer;
505
+ border: 1px solid var(--border); border-radius: var(--radius); background: var(--surface);
506
+ }
507
+ .model-opt:hover { background: var(--surface-2); }
508
+ .model-opt:has(input:checked) { border-color: var(--primary); box-shadow: 0 0 0 1px var(--primary); }
509
+ .model-opt input { margin: 3px 0 0; accent-color: var(--primary); flex: none; }
510
+ .mo-main { display: flex; flex-direction: column; gap: 2px; min-width: 0; }
511
+ .mo-name { font-weight: 600; display: flex; flex-wrap: wrap; align-items: center; gap: 6px; }
512
+ .mo-tag { font-size: 10.5px; font-weight: 500; color: var(--text-2); border: 1px solid var(--border-strong); border-radius: 99px; padding: 0 7px; }
513
+ .mo-note { font-size: 12.5px; color: var(--text-2); }
514
+ .mo-meta { font-size: 11.5px; color: var(--text-3); }
515
+ .mo-state { font-size: 11.5px; color: var(--text-3); display: flex; gap: 10px; align-items: center; margin-top: 2px; }
516
+ .mo-dl { color: var(--ok); }
517
+ .linkish { border: 0; background: none; padding: 0; color: var(--bad); font-size: 11.5px; text-decoration: underline; cursor: pointer; }
518
+ #tog-thinking:disabled { opacity: .45; cursor: not-allowed; }
519
 
520
  /* ---------- narrow screens ---------- */
521
  @media (max-width: 900px) {
index.html CHANGED
@@ -135,7 +135,7 @@
135
  <svg><use href="#i-alert"/></svg>
136
  <div>
137
  <strong>Phones usually can't run the local model.</strong>
138
- It needs about 1.4 GB of graphics memory, more than most phones give a browser tab. Use an API model instead: the Linux machine still runs on your phone, only the model runs on the provider's servers. You can also install everything and try the local model anyway.
139
  </div>
140
  </div>
141
  <h2>Install for offline use</h2>
@@ -147,7 +147,7 @@
147
  <div class="section-label">Components</div>
148
  <div class="components">
149
  <div class="component" data-row="model" data-state="todo">
150
- <div class="c-main"><span class="c-name">LFM3-1.7B</span><span class="c-desc">Language model by Liquid AI, q4f16 ONNX</span></div>
151
  <div class="c-side"><span class="c-size mono" id="model-size"></span><span class="row-state">Not installed</span></div>
152
  <progress max="1" value="0"></progress>
153
  </div>
@@ -165,6 +165,7 @@
165
 
166
  <button class="btn primary block" id="install" disabled>Install</button>
167
  <button class="btn block secondary" id="setup-api" type="button">Use an API model instead</button>
 
168
  <p class="note" id="setup-note"></p>
169
  <p class="fine">
170
  The model is <a href="https://huggingface.co/LiquidAI/LFM3-1.7B" target="_blank" rel="noopener">LFM3-1.7B</a> (LFM Open License), loaded from Hugging Face.
@@ -223,6 +224,29 @@
223
  </form>
224
  </dialog>
225
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
226
  <dialog id="api-dlg" aria-labelledby="api-title">
227
  <form id="api-form" novalidate>
228
  <div class="dlg-head">
 
135
  <svg><use href="#i-alert"/></svg>
136
  <div>
137
  <strong>Phones usually can't run the local model.</strong>
138
+ It needs about 1.4 GB of graphics memory, more than most phones give a browser tab. Use an API model instead: the Linux machine still runs on your phone, only the model runs on the provider's servers. You can also pick the smaller Qwen3.5 0.8B (0.47 GB) and try it anyway.
139
  </div>
140
  </div>
141
  <h2>Install for offline use</h2>
 
147
  <div class="section-label">Components</div>
148
  <div class="components">
149
  <div class="component" data-row="model" data-state="todo">
150
+ <div class="c-main"><span class="c-name">LFM3 1.7B</span><span class="c-desc">Language model by Liquid AI, q4f16 ONNX</span></div>
151
  <div class="c-side"><span class="c-size mono" id="model-size"></span><span class="row-state">Not installed</span></div>
152
  <progress max="1" value="0"></progress>
153
  </div>
 
165
 
166
  <button class="btn primary block" id="install" disabled>Install</button>
167
  <button class="btn block secondary" id="setup-api" type="button">Use an API model instead</button>
168
+ <button class="btn block secondary" id="setup-model" type="button">Choose a different local model</button>
169
  <p class="note" id="setup-note"></p>
170
  <p class="fine">
171
  The model is <a href="https://huggingface.co/LiquidAI/LFM3-1.7B" target="_blank" rel="noopener">LFM3-1.7B</a> (LFM Open License), loaded from Hugging Face.
 
224
  </form>
225
  </dialog>
226
 
227
+ <dialog id="model-dlg" aria-labelledby="model-title">
228
+ <div class="dlg-head">
229
+ <h2 id="model-title">Choose a model</h2>
230
+ <button class="icon-btn" type="button" id="model-close" aria-label="Close"><svg><use href="#i-x"/></svg></button>
231
+ </div>
232
+ <div class="dlg-section">
233
+ <div class="section-label">On this device</div>
234
+ <p class="dlg-lede">Runs on your GPU with WebGPU and works offline once downloaded. Bigger models are more capable but slower and need more graphics memory.</p>
235
+ <div class="model-list" id="model-list"></div>
236
+ </div>
237
+ <div class="dlg-section">
238
+ <div class="section-label">Through an API</div>
239
+ <div class="row-setting">
240
+ <span><span class="rs-title">API model</span><span class="rs-desc" id="model-api-desc"></span></span>
241
+ <button type="button" class="btn" id="model-api">Set up</button>
242
+ </div>
243
+ </div>
244
+ <div class="dlg-foot">
245
+ <span class="spacer"></span>
246
+ <button type="button" class="btn primary" id="model-save">Use this model</button>
247
+ </div>
248
+ </dialog>
249
+
250
  <dialog id="api-dlg" aria-labelledby="api-title">
251
  <form id="api-form" novalidate>
252
  <div class="dlg-head">
js/agent.js CHANGED
@@ -53,7 +53,7 @@ export class Agent {
53
  for (let step = 1; step <= this.settings.maxSteps; step++) {
54
  if (this.stopRequested) break;
55
  this.#compact();
56
- const thinking = this.settings.thinking;
57
  const view = this.ui.startTurn(step);
58
  let raw = "";
59
  const res = await this.llm.generate(
@@ -81,7 +81,10 @@ export class Agent {
81
  }
82
  // Store exactly what was generated (plus the "<think>" the prompt ended with),
83
  // so the next prompt re-tokenizes to the same prefix and the KV cache is reused.
84
- this.history.push({ role: "assistant", content: (thinking ? "<think>" : "<think></think>") + text });
 
 
 
85
 
86
  const calls = parsed.calls.filter((c) => c.complete || !res.hitLimit);
87
  if (!calls.length) {
 
53
  for (let step = 1; step <= this.settings.maxSteps; step++) {
54
  if (this.stopRequested) break;
55
  this.#compact();
56
+ const thinking = this.settings.thinking && this.llm.supportsThinking !== false;
57
  const view = this.ui.startTurn(step);
58
  let raw = "";
59
  const res = await this.llm.generate(
 
81
  }
82
  // Store exactly what was generated (plus the "<think>" the prompt ended with),
83
  // so the next prompt re-tokenizes to the same prefix and the KV cache is reused.
84
+ // Some chat templates end the prompt with "<think>" (LFM3), others let the model
85
+ // write it (Qwen3); don't double it.
86
+ const prefix = /^\s*<think>/.test(text) ? "" : thinking ? "<think>" : "<think></think>";
87
+ this.history.push({ role: "assistant", content: prefix + text });
88
 
89
  const calls = parsed.calls.filter((c) => c.complete || !res.hitLimit);
90
  if (!calls.length) {
js/config.js CHANGED
@@ -1,17 +1,49 @@
1
  // Central configuration. Everything that pins a version or a location lives here.
2
- export const APP_VERSION = "0.4.1";
3
 
4
- // --- Language model (loaded from the Hugging Face Hub, cached by transformers.js) ---
5
- export const MODEL_ID = "LiquidAI/LFM3-1.7B-ONNX";
6
- export const MODEL_DTYPE = "q4f16";
 
7
  export const MODEL_DEVICE = "webgpu";
8
- // Files transformers.js fetches for this model; used to check the offline cache.
9
- export const MODEL_FILES = [
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  "config.json", "generation_config.json", "tokenizer.json", "tokenizer_config.json",
11
- `onnx/model_${MODEL_DTYPE}.onnx`, `onnx/model_${MODEL_DTYPE}.onnx_data`,
12
  ];
13
- export const MODEL_DOWNLOAD_BYTES = 1_383_000_000; // approx, for the setup screen
14
- export const hubUrl = (file) => `https://huggingface.co/${MODEL_ID}/resolve/main/${file}`;
15
 
16
  // --- CheerpX (WebVM engine). Proprietary: loaded from Leaning Technologies' CDN,
17
  // never redistributed by this Space. The visitor's browser caches it for offline use.
 
1
  // Central configuration. Everything that pins a version or a location lives here.
2
+ export const APP_VERSION = "0.5.0";
3
 
4
+ // --- Local language models (loaded from the Hugging Face Hub, cached in Cache Storage) ---
5
+ // Each runs on WebGPU through transformers.js. `chunks` is how many external weight files
6
+ // (model_<dtype>.onnx_data, _data_1, ...) the ONNX graph uses; 0 means the weights are inside
7
+ // the .onnx file itself. `bytes` is the approximate download, `contextMax` the model's window.
8
  export const MODEL_DEVICE = "webgpu";
9
+ export const LOCAL_MODELS = [
10
+ {
11
+ id: "lfm3-1.7b", label: "LFM3 1.7B", repo: "LiquidAI/LFM3-1.7B-ONNX", dtype: "q4f16", chunks: 1,
12
+ bytes: 1_383_000_000, contextMax: 262_144, thinking: true, by: "Liquid AI", license: "LFM Open License",
13
+ note: "The default. Fast hybrid model, good at tool use.",
14
+ },
15
+ {
16
+ id: "qwen3.5-0.8b", label: "Qwen3.5 0.8B", repo: "onnx-community/Qwen3.5-0.8B-Text-ONNX", dtype: "q4f16", chunks: 1,
17
+ bytes: 470_000_000, contextMax: 262_144, thinking: true, by: "Qwen", license: "Apache-2.0",
18
+ note: "Smallest download and GPU memory use. Best chance on weak GPUs and phones; less capable.",
19
+ },
20
+ {
21
+ id: "qwen3-1.7b", label: "Qwen3 1.7B", repo: "onnx-community/Qwen3-1.7B-ONNX", dtype: "q4f16", chunks: 0,
22
+ bytes: 1_426_000_000, contextMax: 40_960, thinking: true, by: "Qwen", license: "Apache-2.0",
23
+ note: "Similar size to LFM3, strong reasoning.",
24
+ },
25
+ {
26
+ id: "granite-4.0-1b", label: "Granite 4.0 1B", repo: "onnx-community/granite-4.0-1b-ONNX-web", dtype: "q4f16", chunks: 1,
27
+ bytes: 1_247_000_000, contextMax: 131_072, thinking: false, by: "IBM", license: "Apache-2.0",
28
+ note: "Hybrid model built for tool calling. No reasoning mode.",
29
+ },
30
+ {
31
+ id: "qwen3-4b", label: "Qwen3 4B", repo: "onnx-community/Qwen3-4B-ONNX", dtype: "q4f16", chunks: 2,
32
+ bytes: 2_833_000_000, contextMax: 40_960, thinking: true, by: "Qwen", license: "Apache-2.0",
33
+ note: "Most capable, but slow. Needs a desktop GPU with about 4 GB of memory.",
34
+ },
35
+ ];
36
+ export const DEFAULT_MODEL_ID = LOCAL_MODELS[0].id;
37
+ export const findModel = (id) => LOCAL_MODELS.find((m) => m.id === id) ?? LOCAL_MODELS[0];
38
+
39
+ export const dataFileNames = (m) =>
40
+ Array.from({ length: m.chunks }, (_, i) => `model_${m.dtype}.onnx_data${i === 0 ? "" : "_" + i}`);
41
+ // Files transformers.js fetches for a model; used to check the offline cache.
42
+ export const modelFiles = (m) => [
43
  "config.json", "generation_config.json", "tokenizer.json", "tokenizer_config.json",
44
+ `onnx/model_${m.dtype}.onnx`, ...dataFileNames(m).map((f) => "onnx/" + f),
45
  ];
46
+ export const hubUrl = (m, file) => `https://huggingface.co/${m.repo}/resolve/main/${file}`;
 
47
 
48
  // --- CheerpX (WebVM engine). Proprietary: loaded from Leaning Technologies' CDN,
49
  // never redistributed by this Space. The visitor's browser caches it for offline use.
js/lazy-weights.js CHANGED
@@ -53,16 +53,10 @@ async function download(cache, url, file, progress) {
53
  progress?.({ status: "done", file, loaded, total: total || loaded });
54
  }
55
 
56
- /**
57
- * Returns from_pretrained options that make onnxruntime read the external weights
58
- * lazily from Cache Storage, downloading them first if needed.
59
- */
60
- export async function lazyWeightsOptions(modelId, dtype, progress) {
61
- if (typeof FileReaderSync === "undefined" || !self.caches) throw new Error("lazy weights unsupported here");
62
- const name = `model_${dtype}.onnx_data`;
63
  const file = `onnx/${name}`;
64
  const url = `https://huggingface.co/${modelId}/resolve/main/${file}`;
65
- const cache = await caches.open(CACHE_NAME);
66
  let res = await cache.match(url);
67
  if (!res) {
68
  await download(cache, url, file, progress);
@@ -73,11 +67,27 @@ export async function lazyWeightsOptions(modelId, dtype, progress) {
73
  progress?.({ status: "done", file, loaded: size, total: size });
74
  }
75
  const blob = await res.blob(); // disk-backed in Chromium; not read into memory
76
- if (!blob.size) throw new Error("cached weights are empty");
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  return {
78
- // Stop transformers.js from fetching the .onnx_data itself...
79
  use_external_data_format: false,
80
- // ...and mount ours under the name the ONNX graph refers to.
81
- session_options: { externalData: [{ path: name, data: new BlobBytes(blob) }] },
82
  };
83
  }
 
53
  progress?.({ status: "done", file, loaded, total: total || loaded });
54
  }
55
 
56
+ /** Gets one external-data file as a disk-backed BlobBytes, downloading it first if needed. */
57
+ async function lazyFile(cache, modelId, name, progress) {
 
 
 
 
 
58
  const file = `onnx/${name}`;
59
  const url = `https://huggingface.co/${modelId}/resolve/main/${file}`;
 
60
  let res = await cache.match(url);
61
  if (!res) {
62
  await download(cache, url, file, progress);
 
67
  progress?.({ status: "done", file, loaded: size, total: size });
68
  }
69
  const blob = await res.blob(); // disk-backed in Chromium; not read into memory
70
+ if (!blob.size) throw new Error(`cached ${name} is empty`);
71
+ return new BlobBytes(blob);
72
+ }
73
+
74
+ /**
75
+ * Returns from_pretrained options that make onnxruntime read the external weights
76
+ * (model_<dtype>.onnx_data, _data_1, ...) lazily from Cache Storage, downloading them
77
+ * first if needed.
78
+ */
79
+ export async function lazyWeightsOptions(modelId, dtype, chunks, progress) {
80
+ if (typeof FileReaderSync === "undefined" || !self.caches) throw new Error("lazy weights unsupported here");
81
+ const cache = await caches.open(CACHE_NAME);
82
+ const externalData = [];
83
+ for (let i = 0; i < chunks; i++) {
84
+ const name = `model_${dtype}.onnx_data${i === 0 ? "" : "_" + i}`;
85
+ externalData.push({ path: name, data: await lazyFile(cache, modelId, name, progress) });
86
+ }
87
  return {
88
+ // Stop transformers.js from fetching the .onnx_data files itself...
89
  use_external_data_format: false,
90
+ // ...and mount ours under the names the ONNX graph refers to.
91
+ session_options: { externalData },
92
  };
93
  }
js/llm-engine.js CHANGED
@@ -15,6 +15,13 @@ const THINK_CLOSE = {
15
  budget: "\n\nI have thought about this enough. Time to act.\n</think>\n",
16
  };
17
 
 
 
 
 
 
 
 
18
  export function createEngine(tf) {
19
  const { AutoTokenizer, AutoModelForCausalLM, DynamicCache, Tensor, TextStreamer, InterruptableStoppingCriteria, StoppingCriteria } = tf;
20
  let tokenizer = null;
@@ -72,10 +79,13 @@ export function createEngine(tf) {
72
  model = await AutoModelForCausalLM.from_pretrained(modelId, { dtype, device, progress_callback, ...modelOptions });
73
  const te = tokenizer.encode("</think>", { add_special_tokens: false });
74
  thinkEndId = te.length === 1 ? te[0] : -1;
75
- // The LFM3 ONNX graph takes `logits_to_keep` (rank-1 int64). transformers.js only
76
- // fills `num_logits_to_keep`, so inject it on every forward pass.
77
- const forward = model.forward.bind(model);
78
- model.forward = (inputs) => forward({ ...inputs, logits_to_keep: inputs.logits_to_keep ?? int64([1], [1]) });
 
 
 
79
  onStatus?.("Warming up");
80
  await generate([{ role: "user", content: "Hi" }], [], { thinking: false, maxNewTokens: 1 });
81
  await reset();
@@ -158,7 +168,7 @@ export function createEngine(tf) {
158
  });
159
  const entries = {};
160
  for (const k in out) {
161
- if (k.startsWith("present")) entries[k.replace("present_conv", "past_conv").replace("present", "past_key_values")] = out[k];
162
  }
163
  cache.update(entries);
164
  out.logits?.dispose?.();
 
15
  budget: "\n\nI have thought about this enough. Time to act.\n</think>\n",
16
  };
17
 
18
+ // Maps a graph output (present_*) to the matching cache input, for every cache kind the
19
+ // supported architectures use: attention KV, conv (LFM), SSM (Granite), recurrent (Qwen3.5).
20
+ // Mirrors transformers.js' own mapping.
21
+ const pastName = (k) => k
22
+ .replace("present_ssm", "past_ssm").replace("present_conv", "past_conv")
23
+ .replace("present_recurrent", "past_recurrent").replace("present", "past_key_values");
24
+
25
  export function createEngine(tf) {
26
  const { AutoTokenizer, AutoModelForCausalLM, DynamicCache, Tensor, TextStreamer, InterruptableStoppingCriteria, StoppingCriteria } = tf;
27
  let tokenizer = null;
 
79
  model = await AutoModelForCausalLM.from_pretrained(modelId, { dtype, device, progress_callback, ...modelOptions });
80
  const te = tokenizer.encode("</think>", { add_special_tokens: false });
81
  thinkEndId = te.length === 1 ? te[0] : -1;
82
+ // Some graphs (LFM3) take `logits_to_keep` (rank-1 int64). transformers.js only
83
+ // fills `num_logits_to_keep`, so inject it on every forward pass for those.
84
+ const inputNames = model.sessions?.model?.inputNames ?? [];
85
+ if (inputNames.includes("logits_to_keep")) {
86
+ const forward = model.forward.bind(model);
87
+ model.forward = (inputs) => forward({ ...inputs, logits_to_keep: inputs.logits_to_keep ?? int64([1], [1]) });
88
+ }
89
  onStatus?.("Warming up");
90
  await generate([{ role: "user", content: "Hi" }], [], { thinking: false, maxNewTokens: 1 });
91
  await reset();
 
168
  });
169
  const entries = {};
170
  for (const k in out) {
171
+ if (k.startsWith("present")) entries[pastName(k)] = out[k];
172
  }
173
  cache.update(entries);
174
  out.logits?.dispose?.();
js/llm-worker.js CHANGED
@@ -27,7 +27,7 @@ self.onmessage = async ({ data }) => {
27
  }
28
  };
29
 
30
- async function load({ modelId, dtype, device }) {
31
  if (device === "webgpu") {
32
  const adapter = await navigator.gpu?.requestAdapter({ powerPreference: "high-performance" });
33
  if (!adapter) throw new Error("WebGPU is not available in this browser. Use a recent desktop Chrome or Edge.");
@@ -42,11 +42,12 @@ async function load({ modelId, dtype, device }) {
42
  post({ type: "progress", file: p.file, loaded: p.loaded ?? 0, total: p.total ?? 0, status: p.status });
43
  }
44
  };
45
- // Read the ~1.4 GB of weights from disk tensor by tensor instead of holding them all
46
- // in memory while they are copied to the GPU (see lazy-weights.js).
 
47
  let modelOptions;
48
  try {
49
- modelOptions = await lazyWeightsOptions(modelId, dtype, progress_callback);
50
  } catch (err) {
51
  console.warn("Low-memory weight loading unavailable, using the default path:", err);
52
  }
 
27
  }
28
  };
29
 
30
+ async function load({ modelId, dtype, chunks = 1, device }) {
31
  if (device === "webgpu") {
32
  const adapter = await navigator.gpu?.requestAdapter({ powerPreference: "high-performance" });
33
  if (!adapter) throw new Error("WebGPU is not available in this browser. Use a recent desktop Chrome or Edge.");
 
42
  post({ type: "progress", file: p.file, loaded: p.loaded ?? 0, total: p.total ?? 0, status: p.status });
43
  }
44
  };
45
+ // Read external weights from disk tensor by tensor instead of holding them all in
46
+ // memory while they are copied to the GPU (see lazy-weights.js). Models whose weights
47
+ // are inside the .onnx file itself (chunks = 0) load the default way.
48
  let modelOptions;
49
  try {
50
+ if (chunks > 0) modelOptions = await lazyWeightsOptions(modelId, dtype, chunks, progress_callback);
51
  } catch (err) {
52
  console.warn("Low-memory weight loading unavailable, using the default path:", err);
53
  }
js/llm.js CHANGED
@@ -1,8 +1,11 @@
1
  // Main-thread client for the model worker.
2
- import { MODEL_ID, MODEL_DTYPE, MODEL_DEVICE } from "./config.js";
3
 
4
  export class LLM {
5
- constructor() {
 
 
 
6
  this.worker = new Worker(new URL("./llm-worker.js", import.meta.url), { type: "module" });
7
  this.pending = new Map();
8
  this.nextId = 1;
@@ -17,7 +20,8 @@ export class LLM {
17
  load() {
18
  this.loading ??= new Promise((resolve, reject) => {
19
  this.pending.set("load", { resolve, reject });
20
- this.worker.postMessage({ type: "load", modelId: MODEL_ID, dtype: MODEL_DTYPE, device: MODEL_DEVICE });
 
21
  });
22
  return this.loading;
23
  }
 
1
  // Main-thread client for the model worker.
2
+ import { MODEL_DEVICE } from "./config.js";
3
 
4
  export class LLM {
5
+ /** model: an entry of LOCAL_MODELS (config.js). */
6
+ constructor(model) {
7
+ this.model = model;
8
+ this.supportsThinking = model.thinking;
9
  this.worker = new Worker(new URL("./llm-worker.js", import.meta.url), { type: "module" });
10
  this.pending = new Map();
11
  this.nextId = 1;
 
20
  load() {
21
  this.loading ??= new Promise((resolve, reject) => {
22
  this.pending.set("load", { resolve, reject });
23
+ const { repo, dtype, chunks } = this.model;
24
+ this.worker.postMessage({ type: "load", modelId: repo, dtype, chunks, device: MODEL_DEVICE });
25
  });
26
  return this.loading;
27
  }
js/main.js CHANGED
@@ -1,4 +1,4 @@
1
- import { DEFAULTS, MODEL_DOWNLOAD_BYTES, MODEL_ID, MODEL_DEVICE, APP_VERSION } from "./config.js";
2
  import * as setup from "./setup.js";
3
  import { LLM } from "./llm.js";
4
  import { RemoteLLM, API_PRESETS, loadApiKey, saveApiKey, apiKeyRemembered } from "./remote-llm.js";
@@ -58,12 +58,14 @@ const apiConfig = () => ({ baseUrl: settings.api.baseUrl, model: settings.api.mo
58
  const apiHost = () => { try { return new URL(settings.api.baseUrl).host; } catch { return settings.api?.baseUrl ?? ""; } };
59
 
60
  // The active context budget follows the backend.
 
 
61
  const syncContextBudget = () => {
62
- settings.contextBudgetTokens = usingApi() ? settings.contextApiTokens : settings.contextLocalTokens;
63
  };
64
  syncContextBudget();
65
 
66
- let llm = usingApi() ? new RemoteLLM(apiConfig()) : new LLM();
67
  let modelError = null; // why the local model failed to load, if it did
68
  const vm = new VM();
69
  let agent = null;
@@ -75,7 +77,10 @@ let files = null;
75
  async function init() {
76
  $("#version").textContent = "v" + APP_VERSION;
77
  setBackendLabels();
78
- $("#model-size").textContent = fmtBytes(MODEL_DOWNLOAD_BYTES);
 
 
 
79
  syncToggles();
80
 
81
  const checks = await environmentChecks();
@@ -85,7 +90,7 @@ async function init() {
85
  try { await setup.ensureServiceWorker(); }
86
  catch (err) { checks.sw = { ok: false, text: err.message }; renderChecks(checks); return showSetup(false); }
87
 
88
- const [cx, disk, model] = await Promise.all([setup.cheerpxStatus(), setup.installedDisk(), setup.modelStatus()]);
89
  diskManifest = disk;
90
  const installed = { cx: cx.ready, disk: !!disk, model: model.ready };
91
  for (const k of Object.keys(installed)) if (installed[k]) setRow(k, 1, "Installed");
@@ -205,7 +210,7 @@ function loadModel(onProgress) {
205
  files.set(p.file, { loaded: p.status === "done" ? (prev?.total ?? p.loaded) : p.loaded, total: p.total || prev?.total || 0 });
206
  let loaded = 0, total = 0;
207
  for (const f of files.values()) { loaded += f.loaded; total += f.total; }
208
- total = Math.max(total, MODEL_DOWNLOAD_BYTES);
209
  const frac = Math.min(0.99, loaded / total);
210
  onProgress?.(frac, `${fmtBytes(loaded)} of ${fmtBytes(total)}`);
211
  status("model", "busy", `Loading ${Math.round(frac * 100)}%`);
@@ -324,6 +329,8 @@ function showModelFailure(err) {
324
  `${friendlyModelError(err)} Linux is still running on this device. You can connect an API model instead and the agent will keep using this machine.`,
325
  [
326
  h("button", { class: "btn primary small", type: "button", onclick: () => openApiDialog() }, "Use an API model"),
 
 
327
  h("button", { class: "btn small", type: "button", onclick: () => { el.remove(); retryLocalModel(); } }, "Try again"),
328
  ],
329
  String(err?.message ?? err));
@@ -332,21 +339,25 @@ function showModelFailure(err) {
332
 
333
  function showMobileWarning() {
334
  callout("warn", "You're on a phone or tablet",
335
- "The local model needs about 1.4 GB of graphics memory, which most phones can't give a browser tab, so it will probably fail to load. Trying anyway. To skip the wait, use an API model: only the model moves to the provider, Linux keeps running here.",
336
- [h("button", { class: "btn primary small", type: "button", onclick: () => openApiDialog() }, "Use an API model")]);
 
 
 
 
337
  }
338
 
339
  function retryLocalModel() {
340
  modelError = null;
341
  llm.dispose?.();
342
- llm = new LLM();
343
  if (agent) agent.llm = llm;
344
  setComposerEnabled();
345
  loadModelOnce();
346
  }
347
 
348
  function setBackendLabels() {
349
- $("#model-name").textContent = usingApi() ? settings.api.model : MODEL_ID.split("/")[1].replace("-ONNX", "");
350
  $("#gpu-name").textContent = usingApi() ? "API" : MODEL_DEVICE === "webgpu" ? "WebGPU" : MODEL_DEVICE;
351
  }
352
 
@@ -544,7 +555,10 @@ function newConversation() {
544
 
545
  // ---------- composer toggles ----------
546
  function syncToggles() {
547
- $("#tog-thinking").setAttribute("aria-pressed", String(!!settings.thinking));
 
 
 
548
  $("#tog-confirm").setAttribute("aria-pressed", String(!!settings.confirmCommands));
549
  }
550
  $("#tog-thinking").addEventListener("click", () => { settings.thinking = !settings.thinking; saveSettings(); syncToggles(); });
@@ -634,7 +648,7 @@ for (const tab of document.querySelectorAll("[data-tab]")) tab.addEventListener(
634
 
635
  const dlg = $("#settings");
636
  $("#open-settings").addEventListener("click", () => {
637
- $("#set-model-desc").textContent = usingApi() ? `${settings.api.model} via ${apiHost()}` : "LFM3-1.7B on this device (WebGPU)";
638
  $("#set-thinking").checked = settings.thinking;
639
  $("#set-confirm").checked = settings.confirmCommands;
640
  $("#set-steps").value = settings.maxSteps;
@@ -662,7 +676,67 @@ dlg.addEventListener("close", () => {
662
  syncToggles();
663
  });
664
  dlg.addEventListener("click", (e) => { if (e.target === dlg) dlg.close(); }); // click on backdrop
665
- $("#set-model").addEventListener("click", () => { dlg.close(); openApiDialog(); });
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
666
 
667
  // ---------- API model dialog ----------
668
  const apiDlg = $("#api-dlg");
 
1
+ import { DEFAULTS, MODEL_DEVICE, APP_VERSION, LOCAL_MODELS, findModel } from "./config.js";
2
  import * as setup from "./setup.js";
3
  import { LLM } from "./llm.js";
4
  import { RemoteLLM, API_PRESETS, loadApiKey, saveApiKey, apiKeyRemembered } from "./remote-llm.js";
 
58
  const apiHost = () => { try { return new URL(settings.api.baseUrl).host; } catch { return settings.api?.baseUrl ?? ""; } };
59
 
60
  // The active context budget follows the backend.
61
+ const currentModel = () => findModel(settings.localModel);
62
+ const smallestModel = () => LOCAL_MODELS.reduce((a, b) => (b.bytes < a.bytes ? b : a));
63
  const syncContextBudget = () => {
64
+ settings.contextBudgetTokens = usingApi() ? settings.contextApiTokens : Math.min(settings.contextLocalTokens, currentModel().contextMax);
65
  };
66
  syncContextBudget();
67
 
68
+ let llm = usingApi() ? new RemoteLLM(apiConfig()) : new LLM(currentModel());
69
  let modelError = null; // why the local model failed to load, if it did
70
  const vm = new VM();
71
  let agent = null;
 
77
  async function init() {
78
  $("#version").textContent = "v" + APP_VERSION;
79
  setBackendLabels();
80
+ const m = currentModel();
81
+ $("#model-size").textContent = fmtBytes(m.bytes);
82
+ $(".c-name", $('[data-row="model"]')).textContent = m.label;
83
+ $(".c-desc", $('[data-row="model"]')).textContent = `Language model by ${m.by}, ${m.dtype} ONNX`;
84
  syncToggles();
85
 
86
  const checks = await environmentChecks();
 
90
  try { await setup.ensureServiceWorker(); }
91
  catch (err) { checks.sw = { ok: false, text: err.message }; renderChecks(checks); return showSetup(false); }
92
 
93
+ const [cx, disk, model] = await Promise.all([setup.cheerpxStatus(), setup.installedDisk(), setup.modelStatus(currentModel())]);
94
  diskManifest = disk;
95
  const installed = { cx: cx.ready, disk: !!disk, model: model.ready };
96
  for (const k of Object.keys(installed)) if (installed[k]) setRow(k, 1, "Installed");
 
210
  files.set(p.file, { loaded: p.status === "done" ? (prev?.total ?? p.loaded) : p.loaded, total: p.total || prev?.total || 0 });
211
  let loaded = 0, total = 0;
212
  for (const f of files.values()) { loaded += f.loaded; total += f.total; }
213
+ total = Math.max(total, currentModel().bytes);
214
  const frac = Math.min(0.99, loaded / total);
215
  onProgress?.(frac, `${fmtBytes(loaded)} of ${fmtBytes(total)}`);
216
  status("model", "busy", `Loading ${Math.round(frac * 100)}%`);
 
329
  `${friendlyModelError(err)} Linux is still running on this device. You can connect an API model instead and the agent will keep using this machine.`,
330
  [
331
  h("button", { class: "btn primary small", type: "button", onclick: () => openApiDialog() }, "Use an API model"),
332
+ currentModel() !== smallestModel()
333
+ ? h("button", { class: "btn small", type: "button", onclick: () => openModelDialog() }, "Choose a smaller model") : null,
334
  h("button", { class: "btn small", type: "button", onclick: () => { el.remove(); retryLocalModel(); } }, "Try again"),
335
  ],
336
  String(err?.message ?? err));
 
339
 
340
  function showMobileWarning() {
341
  callout("warn", "You're on a phone or tablet",
342
+ `${currentModel().label} needs about ${fmtBytes(currentModel().bytes)} of graphics memory, which most phones can't give a browser tab, so it will probably fail to load. Trying anyway. To skip the wait, use an API model (only the model moves to the provider, Linux keeps running here)${currentModel() !== smallestModel() ? `, or try the smaller ${smallestModel().label}` : ""}.`,
343
+ [
344
+ h("button", { class: "btn primary small", type: "button", onclick: () => openApiDialog() }, "Use an API model"),
345
+ currentModel() !== smallestModel()
346
+ ? h("button", { class: "btn small", type: "button", onclick: () => openModelDialog() }, "Choose a smaller model") : null,
347
+ ]);
348
  }
349
 
350
  function retryLocalModel() {
351
  modelError = null;
352
  llm.dispose?.();
353
+ llm = new LLM(currentModel());
354
  if (agent) agent.llm = llm;
355
  setComposerEnabled();
356
  loadModelOnce();
357
  }
358
 
359
  function setBackendLabels() {
360
+ $("#model-name").textContent = usingApi() ? settings.api.model : currentModel().label;
361
  $("#gpu-name").textContent = usingApi() ? "API" : MODEL_DEVICE === "webgpu" ? "WebGPU" : MODEL_DEVICE;
362
  }
363
 
 
555
 
556
  // ---------- composer toggles ----------
557
  function syncToggles() {
558
+ const canThink = usingApi() || currentModel().thinking;
559
+ $("#tog-thinking").disabled = !canThink;
560
+ $("#tog-thinking").title = canThink ? "Let the model reason before each step" : `${currentModel().label} has no reasoning mode`;
561
+ $("#tog-thinking").setAttribute("aria-pressed", String(!!settings.thinking && canThink));
562
  $("#tog-confirm").setAttribute("aria-pressed", String(!!settings.confirmCommands));
563
  }
564
  $("#tog-thinking").addEventListener("click", () => { settings.thinking = !settings.thinking; saveSettings(); syncToggles(); });
 
648
 
649
  const dlg = $("#settings");
650
  $("#open-settings").addEventListener("click", () => {
651
+ $("#set-model-desc").textContent = usingApi() ? `${settings.api.model} via ${apiHost()}` : `${currentModel().label} on this device (WebGPU)`;
652
  $("#set-thinking").checked = settings.thinking;
653
  $("#set-confirm").checked = settings.confirmCommands;
654
  $("#set-steps").value = settings.maxSteps;
 
676
  syncToggles();
677
  });
678
  dlg.addEventListener("click", (e) => { if (e.target === dlg) dlg.close(); }); // click on backdrop
679
+ $("#set-model").addEventListener("click", () => { dlg.close(); openModelDialog(); });
680
+
681
+ // ---------- model picker ----------
682
+ const modelDlg = $("#model-dlg");
683
+ let pickedModel = null;
684
+
685
+ async function openModelDialog() {
686
+ pickedModel = usingApi() ? null : currentModel().id;
687
+ $("#model-api-desc").textContent = usingApi() ? `In use: ${settings.api.model} via ${apiHost()}` : "OpenRouter, OpenAI, Hugging Face or any OpenAI-compatible API. Linux stays on this device.";
688
+ renderModelList();
689
+ modelDlg.showModal();
690
+ // Fill in what's downloaded (slow-ish: reads Cache Storage).
691
+ for (const m of LOCAL_MODELS) {
692
+ const bytes = await setup.modelCachedBytes(m).catch(() => 0);
693
+ const el = $(`[data-model="${m.id}"] .mo-state`, modelDlg);
694
+ if (!el) continue;
695
+ const loaded = !usingApi() && m.id === currentModel().id && llm.ready;
696
+ el.replaceChildren(bytes > 1e6
697
+ ? h("span", { class: "mo-dl" }, `Downloaded · ${fmtBytes(bytes)}`)
698
+ : h("span", {}, `${fmtBytes(m.bytes)} download`));
699
+ if (bytes > 1e6 && !loaded) {
700
+ el.append(h("button", { class: "linkish", type: "button", onclick: async (e) => {
701
+ e.preventDefault();
702
+ if (!confirm(`Delete the downloaded files of ${m.label}? It can be downloaded again later.`)) return;
703
+ await setup.removeModel(m);
704
+ el.replaceChildren(h("span", {}, `${fmtBytes(m.bytes)} download`));
705
+ } }, "Delete"));
706
+ }
707
+ }
708
+ }
709
+
710
+ function renderModelList() {
711
+ $("#model-list").replaceChildren(...LOCAL_MODELS.map((m) => h("label", { class: "model-opt", "data-model": m.id },
712
+ h("input", { type: "radio", name: "local-model", value: m.id, checked: m.id === pickedModel, onchange: () => { pickedModel = m.id; syncModelSave(); } }),
713
+ h("span", { class: "mo-main" },
714
+ h("span", { class: "mo-name" }, m.label, m.id === LOCAL_MODELS[0].id ? h("span", { class: "mo-tag" }, "Default") : null,
715
+ !m.thinking ? h("span", { class: "mo-tag" }, "No reasoning") : null),
716
+ h("span", { class: "mo-note" }, m.note),
717
+ h("span", { class: "mo-meta" }, `${m.by} · ${m.license} · up to ${Math.round(m.contextMax / 1024)}k context`),
718
+ h("span", { class: "mo-state" }, `${fmtBytes(m.bytes)} download`)))));
719
+ syncModelSave();
720
+ }
721
+
722
+ function syncModelSave() {
723
+ const same = !usingApi() && pickedModel === currentModel().id;
724
+ $("#model-save").disabled = !pickedModel || same;
725
+ $("#model-save").textContent = same ? "In use" : "Use this model";
726
+ }
727
+
728
+ $("#model-close").addEventListener("click", () => modelDlg.close());
729
+ modelDlg.addEventListener("click", (e) => { if (e.target === modelDlg) modelDlg.close(); });
730
+ $("#model-api").addEventListener("click", () => { modelDlg.close(); openApiDialog(); });
731
+ $("#model-save").addEventListener("click", () => {
732
+ if (!pickedModel) return;
733
+ if (agent?.running) return alert("Stop the agent before switching models.");
734
+ settings.localModel = pickedModel;
735
+ settings.backend = "local";
736
+ saveSettings();
737
+ location.reload(); // a fresh worker loads (and if needed downloads) the new model
738
+ });
739
+ $("#setup-model").addEventListener("click", () => openModelDialog());
740
 
741
  // ---------- API model dialog ----------
742
  const apiDlg = $("#api-dlg");
js/setup.js CHANGED
@@ -2,7 +2,7 @@
2
  // Cache Storage: the app shell and CheerpX (via sw.js), the disk image chunks
3
  // (stored decompressed by downloadDisk), and the model (by transformers.js).
4
  import {
5
- CX_BASE, CX_FILES, CACHE_CX, CACHE_DISK, CACHE_MODEL, DISK_BASE, MODEL_FILES, hubUrl,
6
  } from "./config.js";
7
 
8
  const scopeUrl = (p) => new URL(p, document.baseURI).href;
@@ -48,12 +48,32 @@ export async function installedDisk() {
48
  return res ? res.json() : null;
49
  }
50
 
51
- export async function modelStatus() {
52
  const cache = await caches.open(CACHE_MODEL);
53
- const have = await Promise.all(MODEL_FILES.map((f) => cache.match(hubUrl(f))));
54
  return { ready: have.every(Boolean) };
55
  }
56
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
  export async function fetchRemoteManifest() {
58
  const res = await fetch(DISK_BASE + "manifest.json", { cache: "no-store" });
59
  if (!res.ok) throw new Error(`disk manifest: HTTP ${res.status}`);
 
2
  // Cache Storage: the app shell and CheerpX (via sw.js), the disk image chunks
3
  // (stored decompressed by downloadDisk), and the model (by transformers.js).
4
  import {
5
+ CX_BASE, CX_FILES, CACHE_CX, CACHE_DISK, CACHE_MODEL, DISK_BASE, modelFiles, hubUrl,
6
  } from "./config.js";
7
 
8
  const scopeUrl = (p) => new URL(p, document.baseURI).href;
 
48
  return res ? res.json() : null;
49
  }
50
 
51
+ export async function modelStatus(model) {
52
  const cache = await caches.open(CACHE_MODEL);
53
+ const have = await Promise.all(modelFiles(model).map((f) => cache.match(hubUrl(model, f))));
54
  return { ready: have.every(Boolean) };
55
  }
56
 
57
+ /** Bytes this model's files take in Cache Storage (0 if not downloaded). */
58
+ export async function modelCachedBytes(model) {
59
+ const cache = await caches.open(CACHE_MODEL);
60
+ const prefix = `https://huggingface.co/${model.repo}/`;
61
+ let total = 0;
62
+ for (const req of await cache.keys()) {
63
+ if (!req.url.startsWith(prefix)) continue;
64
+ const res = await cache.match(req);
65
+ total += Number(res?.headers.get("Content-Length")) || 0;
66
+ }
67
+ return total;
68
+ }
69
+
70
+ /** Deletes one model's downloaded files. */
71
+ export async function removeModel(model) {
72
+ const cache = await caches.open(CACHE_MODEL);
73
+ const prefix = `https://huggingface.co/${model.repo}/`;
74
+ for (const req of await cache.keys()) if (req.url.startsWith(prefix)) await cache.delete(req);
75
+ }
76
+
77
  export async function fetchRemoteManifest() {
78
  const res = await fetch(DISK_BASE + "manifest.json", { cache: "no-store" });
79
  if (!res.ok) throw new Error(`disk manifest: HTTP ${res.status}`);
sw.js CHANGED
@@ -6,7 +6,7 @@
6
  // (__vm__/disk.ext2), backed by the chunks the setup screen stored in Cache Storage.
7
  // 4. Adds the cross-origin isolation headers (COOP/COEP) that SharedArrayBuffer,
8
  // CheerpX and multi-threaded WASM need, even when a host forgets to send them.
9
- const SHELL_VERSION = "lwa-shell-0.4.1";
10
  const CX_ORIGIN = "https://cxrtnc.leaningtech.com";
11
  const CX_CACHE_PREFIX = "lwa-cheerpx-";
12
  const DISK_CACHE = "lwa-disk";
 
6
  // (__vm__/disk.ext2), backed by the chunks the setup screen stored in Cache Storage.
7
  // 4. Adds the cross-origin isolation headers (COOP/COEP) that SharedArrayBuffer,
8
  // CheerpX and multi-threaded WASM need, even when a host forgets to send them.
9
+ const SHELL_VERSION = "lwa-shell-0.5.0";
10
  const CX_ORIGIN = "https://cxrtnc.leaningtech.com";
11
  const CX_CACHE_PREFIX = "lwa-cheerpx-";
12
  const DISK_CACHE = "lwa-disk";