NakliTechie commited on
Commit
eb85f90
Β·
1 Parent(s): 2a67fa4

Sync from GitHub 2026-10-22T19:10:03Z

Browse files
Files changed (1) hide show
  1. index.html +105 -1
index.html CHANGED
@@ -2688,6 +2688,7 @@
2688
  <li><strong>Gemma 4 26B-A4B &middot; SSD-streamed (experimental)</strong> (~14 GB) &mdash; text, with reasoning. Google's Gemma 4 26B-A4B mixture-of-experts (128 experts, 8 active per token) from its official 4-bit QAT GGUF (14.4 GB) on the same from-scratch WebGPU engine family: attention, the dense MLP, routers and KV cache stay on the GPU (~1.6 GB plus a 4 GB expert pool) while the 3,840 routed experts stream from on-device storage (OPFS) as the router picks them. The first load copies the model into this browser's storage once (~14 GB of disk; about 12 minutes on a fast connection). About 23 tokens/s on an M4 Pro with 24 GB, and prompts are read in chunks at about 55 tokens/s. Greedy output matches llama.cpp. Flip <strong>Show reasoning</strong> for its thinking.</li>
2689
  <li><strong>LFM2.5 24B-A2B &middot; SSD-streamed (experimental)</strong> (~13.6 GB) &mdash; text, reasoning-only. Liquid AI's LFM2.5-24B-A2B (released 2026-10-13): a 24B mixture-of-experts (64 experts, 4 active per token, about 2.4B parameters per token) with short-convolution and attention layers, from Liquid's own 4-bit GGUF. Short-convolution and attention layers, routers and KV cache stay on the GPU while the 2,432 routed experts stream from on-device storage (OPFS) as the router picks them. The first load copies the model into this browser's storage once (~13.6 GB of disk). About 16 tokens/s on an M4 Pro with 24 GB, and prompts are read in chunks at about 50 tokens/s. Greedy output matches llama.cpp (8 of 8 prompts). It always reasons before it answers, so every reply opens with its reasoning. It can use tools (calculator, time, memory, reminders): it called the right tool on 6 of 6 test prompts and answered 5 of 5 plain questions without one.</li>
2690
  <li><strong>Flint 40B-A2.5B &middot; SSD-streamed (experimental)</strong> (~26 GB) &mdash; text, coding, reasoning. Recursive's Flint (released 2026-10-21, Apache-2.0): a 39.8B mixture-of-experts coding model (256 experts, 6 active per token, about 2.5B parameters per token), from Recursive's own 4-bit (Q4_1) GGUF, whose card asks for 48 GB of memory under llama.cpp. Attention, routers, the token table and KV cache stay on the GPU while the 12,032 routed experts stream from on-device storage (OPFS) as the router picks them. The first load copies the model into this browser's storage once (~26 GB of disk). About 8.7 tokens/s on an M4 Pro with 24 GB, and prompts are read at about 27 tokens/s. Its output stays as close to llama.cpp's Metal backend as llama.cpp's own CPU backend does. It reasons before it answers, so replies open with its reasoning. Tools are off for now.</li>
 
2691
  <li><strong>Qwen3 4B &middot; FlashMap speculative (experimental)</strong> (~5.2 GB) &mdash; text, reasoning. Qwen's Qwen3-4B (Q8_0 GGUF, 4.3 GB) with FlashMap, a speculative-decoding drafter by Shen et al. (arXiv:2610.24365; 0.95 GB) that proposes 16 tokens in one step for the model to check in one pass. You get exactly the tokens the model would write on its own, sooner: on an M4 Pro about 2.1&ndash;2.2&times; on maths and 1.2&ndash;1.5&times; on code. Chat text drafts poorly, so there it falls back to plain decoding (about 0.7&ndash;1.1&times;). Both files are copied into this browser's storage once. Tools are off for now.</li>
2692
  </ul>
2693
  </details>
@@ -3005,6 +3006,7 @@
3005
  <option value="gemma4-26b-a4b-ssd">Gemma 4 26B-A4B &middot; SSD-streamed, experimental (~14 GB)</option>
3006
  <option value="lfm25-24b-a2b-ssd">LFM2.5 24B-A2B &middot; SSD-streamed, experimental (~13.6 GB)</option>
3007
  <option value="flint-40b-a2b-ssd">Flint 40B-A2.5B &middot; SSD-streamed, experimental, coding (~26 GB)</option>
 
3008
  <option value="qwen3-4b-flashmap">Qwen3 4B &middot; FlashMap speculative decoding, experimental (~5.2 GB)</option>
3009
  </select>
3010
  <!-- Model picker (Chunk L layer 2): renders #modelSelect's options grouped by what is
@@ -9395,6 +9397,21 @@ self.addEventListener('message', async (e) => {
9395
  contextSize: 8192,
9396
  genConfig: { max_new_tokens: 4096 },
9397
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9398
  // backend:'qwen3dense' β†’ Qwen3Dense (qwen35_dense.js) with FlashMap speculative decoding (flashmap_drafter.js):
9399
  // Qwen's own Q8_0 GGUF of Qwen3-4B (4.28 GB) plus Shen et al.'s flow-map drafter j-shen/FlashMap-Qwen3-4B (0.95 GB
9400
  // bf16, arXiv:2610.24365) in WGSL. Greedy output is the target's own; measured on an M4 Pro (2026-10-22, 8 prompts):
@@ -12047,6 +12064,20 @@ self.addEventListener('message', async (e) => {
12047
  add(repo, 'on disk (OPFS)', bytes, () => ssd.removeEntry(key, { recursive: true }));
12048
  }
12049
  } catch {}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12050
  // 4. OPFS (wllama)
12051
  try {
12052
  const dir = await (await navigator.storage.getDirectory()).getDirectoryHandle('cache');
@@ -17730,6 +17761,7 @@ self.addEventListener('message', async (e) => {
17730
  'gemma4moe-ssd': { engine: 'gemma4_moe_ssd.js', exportName: 'Gemma4MoeSsd', label: 'Gemma 4 26B-A4B (Q4_0)' },
17731
  'lfm2moe-ssd': { engine: 'lfm2_moe_ssd.js', exportName: 'Lfm2MoeSsd', label: 'LFM2.5 24B-A2B (Q4_0)' },
17732
  'flint-ssd': { engine: 'flint_moe_ssd.js', exportName: 'FlintMoeSsd', label: 'Flint 40B-A2.5B (Q4_1)' },
 
17733
  'qwen3dense': { engine: 'qwen35_dense.js', exportName: 'Qwen3Dense', label: 'Qwen3 4B (Q8_0) + FlashMap' },
17734
  'hyv3-ssd': { engine: 'hy_v3_moe_ssd.js', exportName: 'HyV3MoeSsd', label: 'Hy-Memory 30B-A3B (Q8_0)' }, // the memory writer only (WRITER_MODELS)
17735
  'qwen35dense': { engine: 'qwen35_dense.js', exportName: 'Qwen35Dense', label: 'Underdog Saluki 27B (IQ2-mix)' },
@@ -19346,7 +19378,7 @@ self.addEventListener('message', async (e) => {
19346
  const ENGINE_STORES = {
19347
  'transformers': ['transformers-cache'], 'wllama': ['wllama'], 'lfm2-webgpu': ['gguf-v1'],
19348
  'bonsai2-webgpu': ['gguf-cache-v1'], 'gemma4-webgpu': ['safetensors-cache-v1', 'bonsai-pipeline-v1'],
19349
- 'qwen35moe-ssd': ['on disk (OPFS)'], 'minicpmv-ssd': ['on disk (OPFS)'], 'gemma4moe-ssd': ['on disk (OPFS)'], 'lfm2moe-ssd': ['on disk (OPFS)'], 'flint-ssd': ['on disk (OPFS)'], 'qwen3dense': ['on disk (OPFS)'], 'qwen35dense': ['on disk (OPFS)'],
19350
  };
19351
  function parseSizeBytes(size) {
19352
  const m = /([\d.]+)\s*(GB|MB)/i.exec(String(size || ''));
@@ -23964,6 +23996,78 @@ self.addEventListener('message', async (e) => {
23964
  }
23965
  </script>
23966
  <!-- rolled-in modules: generated from src/ by scripts/roll-in.mjs. Edit src/, then run `node scripts/roll-in.mjs`. -->
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23967
  <script type="text/plain" data-module="errors.js">/* errors.js β€” coded errors for the store, ingest and the engines. A host branches on `code`, never on the
23968
  * message text and never on `instanceof` (two copies of this module, e.g. a CDN copy and a vendored one, make
23969
  * instanceof false for the same error). Check with isDiskformerError(e, code).
 
2688
  <li><strong>Gemma 4 26B-A4B &middot; SSD-streamed (experimental)</strong> (~14 GB) &mdash; text, with reasoning. Google's Gemma 4 26B-A4B mixture-of-experts (128 experts, 8 active per token) from its official 4-bit QAT GGUF (14.4 GB) on the same from-scratch WebGPU engine family: attention, the dense MLP, routers and KV cache stay on the GPU (~1.6 GB plus a 4 GB expert pool) while the 3,840 routed experts stream from on-device storage (OPFS) as the router picks them. The first load copies the model into this browser's storage once (~14 GB of disk; about 12 minutes on a fast connection). About 23 tokens/s on an M4 Pro with 24 GB, and prompts are read in chunks at about 55 tokens/s. Greedy output matches llama.cpp. Flip <strong>Show reasoning</strong> for its thinking.</li>
2689
  <li><strong>LFM2.5 24B-A2B &middot; SSD-streamed (experimental)</strong> (~13.6 GB) &mdash; text, reasoning-only. Liquid AI's LFM2.5-24B-A2B (released 2026-10-13): a 24B mixture-of-experts (64 experts, 4 active per token, about 2.4B parameters per token) with short-convolution and attention layers, from Liquid's own 4-bit GGUF. Short-convolution and attention layers, routers and KV cache stay on the GPU while the 2,432 routed experts stream from on-device storage (OPFS) as the router picks them. The first load copies the model into this browser's storage once (~13.6 GB of disk). About 16 tokens/s on an M4 Pro with 24 GB, and prompts are read in chunks at about 50 tokens/s. Greedy output matches llama.cpp (8 of 8 prompts). It always reasons before it answers, so every reply opens with its reasoning. It can use tools (calculator, time, memory, reminders): it called the right tool on 6 of 6 test prompts and answered 5 of 5 plain questions without one.</li>
2690
  <li><strong>Flint 40B-A2.5B &middot; SSD-streamed (experimental)</strong> (~26 GB) &mdash; text, coding, reasoning. Recursive's Flint (released 2026-10-21, Apache-2.0): a 39.8B mixture-of-experts coding model (256 experts, 6 active per token, about 2.5B parameters per token), from Recursive's own 4-bit (Q4_1) GGUF, whose card asks for 48 GB of memory under llama.cpp. Attention, routers, the token table and KV cache stay on the GPU while the 12,032 routed experts stream from on-device storage (OPFS) as the router picks them. The first load copies the model into this browser's storage once (~26 GB of disk). About 8.7 tokens/s on an M4 Pro with 24 GB, and prompts are read at about 27 tokens/s. Its output stays as close to llama.cpp's Metal backend as llama.cpp's own CPU backend does. It reasons before it answers, so replies open with its reasoning. Tools are off for now.</li>
2691
+ <li><strong>GLM-4.7-Flash 30B-A3B &middot; via anyformer (experimental)</strong> (~18 GB) &mdash; text, reasoning. Z.ai's GLM-4.7-Flash, a 30B mixture-of-experts with about 3B parameters per token, from unsloth's 4-bit GGUF (18.3 GB). LocalMind has no engine of its own for it: anyformer picks diskformer's llama.cpp path, which runs llama.cpp's WebGPU build in a worker and pages the experts in from on-device storage (OPFS). The first load copies the file into this browser's storage once. Flip <strong>Show reasoning</strong> for its thinking. Tools are off for now.</li>
2692
  <li><strong>Qwen3 4B &middot; FlashMap speculative (experimental)</strong> (~5.2 GB) &mdash; text, reasoning. Qwen's Qwen3-4B (Q8_0 GGUF, 4.3 GB) with FlashMap, a speculative-decoding drafter by Shen et al. (arXiv:2610.24365; 0.95 GB) that proposes 16 tokens in one step for the model to check in one pass. You get exactly the tokens the model would write on its own, sooner: on an M4 Pro about 2.1&ndash;2.2&times; on maths and 1.2&ndash;1.5&times; on code. Chat text drafts poorly, so there it falls back to plain decoding (about 0.7&ndash;1.1&times;). Both files are copied into this browser's storage once. Tools are off for now.</li>
2693
  </ul>
2694
  </details>
 
3006
  <option value="gemma4-26b-a4b-ssd">Gemma 4 26B-A4B &middot; SSD-streamed, experimental (~14 GB)</option>
3007
  <option value="lfm25-24b-a2b-ssd">LFM2.5 24B-A2B &middot; SSD-streamed, experimental (~13.6 GB)</option>
3008
  <option value="flint-40b-a2b-ssd">Flint 40B-A2.5B &middot; SSD-streamed, experimental, coding (~26 GB)</option>
3009
+ <option value="glm-4.7-flash-anyformer">GLM-4.7-Flash 30B-A3B &middot; via anyformer, experimental (~18 GB)</option>
3010
  <option value="qwen3-4b-flashmap">Qwen3 4B &middot; FlashMap speculative decoding, experimental (~5.2 GB)</option>
3011
  </select>
3012
  <!-- Model picker (Chunk L layer 2): renders #modelSelect's options grouped by what is
 
9397
  contextSize: 8192,
9398
  genConfig: { max_new_tokens: 4096 },
9399
  },
9400
+ // backend:'anyformer-glm47flash' β†’ loaded through anyformer (anyformer_models.js, jsDelivr tag v0.2.0), the first model
9401
+ // LocalMind runs with no engine of its own (convergence plan, 2026-10-22). anyformer sends it to diskformer's
9402
+ // llama.cpp path (llama.cpp WebGPU in a worker, experts paged from OPFS); unsloth's Q4_K_M GGUF, 18.3 GB.
9403
+ 'glm-4.7-flash-anyformer': {
9404
+ id: 'unsloth/GLM-4.7-Flash-GGUF',
9405
+ label: 'GLM-4.7-Flash 30B-A3B Β· via anyformer (experimental)',
9406
+ backend: 'anyformer-glm47flash',
9407
+ size: '~18 GB',
9408
+ type: 'causal',
9409
+ multimodal: false,
9410
+ agentCapable: false,
9411
+ thinking: true,
9412
+ contextSize: 4096,
9413
+ genConfig: { max_new_tokens: 2048 },
9414
+ },
9415
  // backend:'qwen3dense' β†’ Qwen3Dense (qwen35_dense.js) with FlashMap speculative decoding (flashmap_drafter.js):
9416
  // Qwen's own Q8_0 GGUF of Qwen3-4B (4.28 GB) plus Shen et al.'s flow-map drafter j-shen/FlashMap-Qwen3-4B (0.95 GB
9417
  // bf16, arXiv:2610.24365) in WGSL. Greedy output is the target's own; measured on an M4 Pro (2026-10-22, 8 prompts):
 
12064
  add(repo, 'on disk (OPFS)', bytes, () => ssd.removeEntry(key, { recursive: true }));
12065
  }
12066
  } catch {}
12067
+ // 3b. OPFS: GGUFs anyformer's llama.cpp path keeps (diskformer: gguf/<file name>), with anyformer's index
12068
+ // (anyformer-llama.json: file name β†’ repo) naming the model. A file the index does not name keeps its own name.
12069
+ try {
12070
+ const root = await navigator.storage.getDirectory();
12071
+ const dir = await root.getDirectoryHandle('gguf');
12072
+ let index = {};
12073
+ try { index = JSON.parse(await (await (await root.getFileHandle('anyformer-llama.json')).getFile()).text()) || {}; } catch {}
12074
+ for await (const [name, f] of dir.entries()) {
12075
+ if (f.kind !== 'file') continue;
12076
+ let bytes = 0;
12077
+ if (measure) { try { bytes = (await f.getFile()).size; } catch {} }
12078
+ add(index[name] || `gguf/${name}`, 'on disk (OPFS)', bytes, () => dir.removeEntry(name));
12079
+ }
12080
+ } catch {}
12081
  // 4. OPFS (wllama)
12082
  try {
12083
  const dir = await (await navigator.storage.getDirectory()).getDirectoryHandle('cache');
 
17761
  'gemma4moe-ssd': { engine: 'gemma4_moe_ssd.js', exportName: 'Gemma4MoeSsd', label: 'Gemma 4 26B-A4B (Q4_0)' },
17762
  'lfm2moe-ssd': { engine: 'lfm2_moe_ssd.js', exportName: 'Lfm2MoeSsd', label: 'LFM2.5 24B-A2B (Q4_0)' },
17763
  'flint-ssd': { engine: 'flint_moe_ssd.js', exportName: 'FlintMoeSsd', label: 'Flint 40B-A2.5B (Q4_1)' },
17764
+ 'anyformer-glm47flash': { engine: 'anyformer_models.js', exportName: 'Glm47Flash', label: 'GLM-4.7-Flash (Q4_K_M, via anyformer)' },
17765
  'qwen3dense': { engine: 'qwen35_dense.js', exportName: 'Qwen3Dense', label: 'Qwen3 4B (Q8_0) + FlashMap' },
17766
  'hyv3-ssd': { engine: 'hy_v3_moe_ssd.js', exportName: 'HyV3MoeSsd', label: 'Hy-Memory 30B-A3B (Q8_0)' }, // the memory writer only (WRITER_MODELS)
17767
  'qwen35dense': { engine: 'qwen35_dense.js', exportName: 'Qwen35Dense', label: 'Underdog Saluki 27B (IQ2-mix)' },
 
19378
  const ENGINE_STORES = {
19379
  'transformers': ['transformers-cache'], 'wllama': ['wllama'], 'lfm2-webgpu': ['gguf-v1'],
19380
  'bonsai2-webgpu': ['gguf-cache-v1'], 'gemma4-webgpu': ['safetensors-cache-v1', 'bonsai-pipeline-v1'],
19381
+ 'qwen35moe-ssd': ['on disk (OPFS)'], 'minicpmv-ssd': ['on disk (OPFS)'], 'gemma4moe-ssd': ['on disk (OPFS)'], 'lfm2moe-ssd': ['on disk (OPFS)'], 'flint-ssd': ['on disk (OPFS)'], 'anyformer-glm47flash': ['on disk (OPFS)'], 'qwen3dense': ['on disk (OPFS)'], 'qwen35dense': ['on disk (OPFS)'],
19382
  };
19383
  function parseSizeBytes(size) {
19384
  const m = /([\d.]+)\s*(GB|MB)/i.exec(String(size || ''));
 
23996
  }
23997
  </script>
23998
  <!-- rolled-in modules: generated from src/ by scripts/roll-in.mjs. Edit src/, then run `node scripts/roll-in.mjs`. -->
23999
+ <script type="text/plain" data-module="anyformer_models.js">/* anyformer_models.js β€” models LocalMind loads through anyformer (github.com/NakliTechie/anyformer.js), not through an
24000
+ * engine of its own: the convergence plan of 2026-10-22 (LocalMind β†’ anyformer β†’ transformers.js / diskformer).
24001
+ * anyformer picks the back end per model; LocalMind sees one call shape.
24002
+ *
24003
+ * AnyformerModel gives an anyformer session the engine interface LocalMind's SSD worker drives (moeSsdWorkerSrc):
24004
+ * static load(null, { maxCtx, onProgress, signal }), generate(messages, { maxNewTokens, signal }) streaming
24005
+ * { text }, reset(), dispose(), and the think markers. One subclass per model names its repo.
24006
+ *
24007
+ * anyformer loads from jsDelivr by GitHub tag until it is on npm. Through diskformer's llama.cpp path reasoning is off
24008
+ * unless generate() passes { thinking: true } (anyformer v0.2.1+), and the reply is text only (no token ids). With
24009
+ * reasoning on, the reply opens inside the thought; the class drops the literal <think> and </think> from the text
24010
+ * and marks the boundary with a chunk whose token is THINK_CLOSE (the worker's close event).
24011
+ */
24012
+ const THINK_CLOSE = -2;
24013
+
24014
+ export const ANYFORMER_URL = 'https://cdn.jsdelivr.net/gh/NakliTechie/anyformer.js@v0.2.2/index.js';
24015
+
24016
+ // anyformer's progress events β†’ the engine events the SSD worker posts: a download as 'weights', planning as 'init'.
24017
+ function toEngineProgress(e) {
24018
+ if (!e) return null;
24019
+ if (e.status === 'download') return { status: 'weights', loaded: e.loaded, total: e.total };
24020
+ if (e.status === 'load' || e.status === 'plan') return { status: 'init' };
24021
+ return null;
24022
+ }
24023
+
24024
+ export class AnyformerModel {
24025
+ static repo = null;
24026
+ static file = undefined; // a GGUF in the repo; undefined lets anyformer pick (the file diskformer checked)
24027
+ static budget = 4e9; // GPU bytes; the llama.cpp path plans its expert slots from it
24028
+
24029
+ static async load(_id, opts = {}) {
24030
+ const { load } = await import(ANYFORMER_URL);
24031
+ const session = await load(this.repo, {
24032
+ budget: opts.budget || this.budget, maxCtx: opts.maxCtx, signal: opts.signal, ...(this.file ? { file: this.file } : {}),
24033
+ onProgress: (e) => { const ev = toEngineProgress(e); if (ev && opts.onProgress) opts.onProgress(ev); },
24034
+ });
24035
+ return new this(session);
24036
+ }
24037
+
24038
+ constructor(session) { this.session = session; this.backend = session.backend; }
24039
+
24040
+ get thinkOpenTokenId() { return null; }
24041
+ get thinkCloseTokenId() { return THINK_CLOSE; }
24042
+ get thinkInPrompt() { return true; }
24043
+ get alwaysThinks() { return false; }
24044
+
24045
+ // The reply text without its <think> / </think> tags. When </think> first appears, the text up to it goes out, then
24046
+ // the same text again with token THINK_CLOSE (an empty delta), so the worker closes the thought with no text lost.
24047
+ async *generate(messages, { maxNewTokens = 1024, signal, enableThinking = false } = {}) {
24048
+ let closed = false;
24049
+ for await (const c of this.session.generate(messages, { maxNewTokens, signal, thinking: enableThinking })) {
24050
+ const raw = String(c.text || '').replace(/^\s*<think>\n?/, '');
24051
+ const at = raw.indexOf('</think>');
24052
+ // The worker posts no text on its close event, so the thought's last text goes out first.
24053
+ if (at >= 0 && !closed) { closed = true; yield { text: raw.slice(0, at) }; yield { text: raw.slice(0, at), token: THINK_CLOSE }; }
24054
+ const text = at >= 0 ? raw.slice(0, at) + raw.slice(at + '</think>'.length).replace(/^\n+/, '') : raw;
24055
+ yield { text, tokens: c.tokens, tokPerSec: c.tokPerSec };
24056
+ }
24057
+ }
24058
+
24059
+ reset() {}
24060
+ async dispose() { await this.session.dispose(); }
24061
+ }
24062
+
24063
+ // Z.ai's GLM-4.7-Flash (deepseek2 architecture, 30B-A3B): unsloth's Q4_K_M GGUF (18.3 GB), live-checked in LocalMind
24064
+ // 2026-10-23 (plan/2026-10-23-glm-anyformer). diskformer gated the architecture on this model's Q3_K_M (14.6 GB, 8/9),
24065
+ // which anyformer would pick by default; Q4_K_M keeps the higher-precision file.
24066
+ export class Glm47Flash extends AnyformerModel {
24067
+ static repo = 'unsloth/GLM-4.7-Flash-GGUF';
24068
+ static file = 'GLM-4.7-Flash-Q4_K_M.gguf';
24069
+ }
24070
+ </script>
24071
  <script type="text/plain" data-module="errors.js">/* errors.js β€” coded errors for the store, ingest and the engines. A host branches on `code`, never on the
24072
  * message text and never on `instanceof` (two copies of this module, e.g. a CDN copy and a vendored one, make
24073
  * instanceof false for the same error). Check with isDiskformerError(e, code).