File size: 5,292 Bytes
1944112 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>Everything WebGPU — Models</title>
<link rel="stylesheet" href="../ui.css" />
<link rel="stylesheet" href="manager.css" />
</head>
<body>
<div class="wrap">
<header class="row">
<span id="dot" class="dot"></span>
<h1 class="grow">Everything WebGPU</h1>
<span id="engineStatus" class="muted mono"></span>
<div class="bar" id="engineBar" hidden><i id="engineProgress"></i></div>
<div id="engineHint" class="muted hint" hidden></div>
</header>
<section id="gpu" class="banner" hidden></section>
<section id="api">
<h2>Engine API</h2>
<div class="idbar">
<code id="selfId" class="mono grow"></code>
<button id="copyId">Copy ID</button>
</div>
<p class="muted">
Any Firefox extension drives this engine by messaging that ID — <code>runtime.sendMessage</code> for
request/response, <code>runtime.connect</code> for streaming. Send raw requests: the engine schedules a
shared GPU, it does not author prompts, so prompts stay on your side and survive a model change.
</p>
<div class="tabs" role="tablist">
<button class="tab" data-tab="completion" role="tab">Completion</button>
<button class="tab" data-tab="translation" role="tab">Translation</button>
<button class="tab" data-tab="reformat" role="tab">Reformat</button>
</div>
<p id="apiWhy" class="muted hint"></p>
<div class="codewrap">
<button id="copyCode" class="copy">Copy</button>
<pre class="mono"><code id="apiSample"></code></pre>
</div>
<p class="muted hint">
One task holds one engine, so a whole <code>batch</code> occupies one and can never starve a completion.
A second engine is built only when a second task is waiting, and on this model it is worth ~1.06x — it
buys isolation, not speed. Full reference in <code>AI.md</code>.
</p>
</section>
<section>
<h2>Add a model</h2>
<div id="drop" tabindex="0">
<strong>Drop a compiled MLC model folder here</strong>
<p class="muted">
Files are copied straight into this extension's Cache Storage — nothing is downloaded.
The folder needs <code>mlc-chat-config.json</code>, <code>tensor-cache.json</code>
(or <code>ndarray-cache.json</code>), every <code>params_shard_*.bin</code>,
<code>tokenizer.json</code>, and exactly one <code>*-webgpu.wasm</code> model library.
</p>
<button id="pick">Choose folder…</button>
<input id="picker" type="file" webkitdirectory directory multiple hidden />
</div>
<div id="ingest" hidden>
<div class="bar"><i id="ingestBar"></i></div>
<div id="ingestStatus" class="muted mono"></div>
</div>
<div id="ingestError" class="banner error" hidden></div>
</section>
<section>
<h2>Local models</h2>
<table id="models">
<thead>
<tr><th>Model</th><th>Size</th><th>Shards</th><th>Library</th><th>Cache</th><th></th></tr>
</thead>
<tbody></tbody>
</table>
<p id="noModels" class="muted" hidden>Nothing registered yet.</p>
<p class="muted mono">Storage in use: <span id="quota">—</span></p>
</section>
<section>
<h2>Generation defaults</h2>
<div class="grid">
<label>Temperature <input id="temperature" type="number" min="0" max="2" step="0.05" /></label>
<label>Max tokens <input id="maxTokens" type="number" min="1" max="32768" step="1" /></label>
<label>Engines in pool <input id="engineCount" type="number" min="1" max="4" step="1" /></label>
<label>Decode steps per sync <input id="decodeSteps" type="number" min="1" max="32" step="1" /></label>
</div>
<p id="poolCost" class="muted mono"></p>
<p id="decodeCost" class="muted mono"></p>
<label class="block">System prompt
<textarea id="systemPrompt" rows="3" placeholder="(none)"></textarea>
</label>
<label class="block">External extension allowlist
<input id="allowedExternalIds" type="text" placeholder="empty = allow every installed extension" />
<span class="muted mono">Comma-separated extension IDs permitted to call the engine API.</span>
</label>
<div class="row">
<button id="save" class="primary">Save settings</button>
<span id="saved" class="muted" hidden>Saved.</span>
</div>
</section>
<section>
<h2>Firefox setup (macOS)</h2>
<p class="muted">WebGPU/Metal must be enabled in <code>about:config</code> before a model can load:</p>
<table class="flags">
<tbody>
<tr><td class="mono">dom.webgpu.enabled</td><td class="mono">true</td><td class="muted">Exposes <code>navigator.gpu</code>.</td></tr>
<tr><td class="mono">gfx.webgpu.ignore-blocklist</td><td class="mono">true</td><td class="muted">Only if your Mac's GPU is blocklisted.</td></tr>
<tr><td class="mono">dom.webgpu.service-workers.enabled</td><td class="mono">true</td><td class="muted">Harmless; needed on builds that gate worker contexts.</td></tr>
</tbody>
</table>
<p class="muted">Restart Firefox after changing these. Nightly on Apple Silicon is the best-tested combination.</p>
</section>
</div>
<script type="module" src="manager.js"></script>
</body>
</html>
|