Spaces:
Running
Running
Valeriy Selitskiy commited on
Commit ·
c3633b1
1
Parent(s): 67dce87
Release v1.2.0 live model telemetry
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .gitignore +0 -1
- README.md +13 -21
- THIRD_PARTY_NOTICES.md +0 -2
- dist/assets/index-BY7HNxCd.js +0 -0
- dist/assets/index-C-2tm91j.css +0 -1
- dist/assets/index-Dn6W9Rfz.js +0 -0
- dist/assets/index-DnMbdiUB.css +1 -0
- dist/assets/{worker-CUYMhoYj.js → worker-Bn87aL54.js} +0 -0
- dist/index.html +2 -2
- dist/manifest/models.json +1 -90
- dist/wasm/wllama-compat.wasm +2 -2
- dist/wasm/wllama.wasm +2 -2
- public/manifest/models.json +1 -90
- public/wasm/wllama-compat.wasm +2 -2
- public/wasm/wllama.wasm +2 -2
- scripts/verify-wllama-assets.mjs +9 -0
- src/app/App.test.ts +22 -0
- src/app/App.tsx +232 -60
- src/app/components/ArtifactPane.tsx +1 -1
- src/app/components/BonsaiShell.tsx +2 -4
- src/app/components/ChatSurface.tsx +39 -13
- src/app/components/ComposerActivity.tsx +205 -0
- src/app/components/InspectorPanel.tsx +23 -47
- src/app/components/ModelRail.tsx +1 -42
- src/app/styles.css +405 -296
- src/bench/report.test.ts +1 -62
- src/bench/report.ts +0 -20
- src/engine/blob-integrity.test.ts +45 -1
- src/engine/blob-integrity.ts +11 -0
- src/engine/client.test.ts +26 -0
- src/engine/client.ts +5 -1
- src/engine/device-gate.test.ts +1 -0
- src/engine/manifest.test.ts +0 -130
- src/engine/manifest.ts +0 -293
- src/engine/native-log.test.ts +80 -0
- src/engine/native-log.ts +80 -1
- src/engine/protocol.ts +35 -1
- src/engine/runtime-device-lost.test.ts +1 -0
- src/engine/runtime-download.test.ts +32 -1
- src/engine/runtime-generate.test.ts +208 -4
- src/engine/runtime-score-sequence.test.ts +1 -0
- src/engine/runtime.ts +295 -14
- src/lib/contracts.ts +27 -10
- src/vite-env.d.ts +6 -0
- vendor/wllama-bonsai/PATCH.diff +73 -6
- vendor/wllama-bonsai/SOURCE.json +10 -9
- vendor/wllama-bonsai/esm/index.cjs +0 -0
- vendor/wllama-bonsai/esm/index.js +0 -0
- vendor/wllama-bonsai/esm/index.min.js +0 -0
- vendor/wllama-bonsai/esm/index.min.js.map +0 -0
.gitignore
CHANGED
|
@@ -1,5 +1,4 @@
|
|
| 1 |
node_modules/
|
| 2 |
-
dist/
|
| 3 |
playwright-report/
|
| 4 |
test-results/
|
| 5 |
*.log
|
|
|
|
| 1 |
node_modules/
|
|
|
|
| 2 |
playwright-report/
|
| 3 |
test-results/
|
| 4 |
*.log
|
README.md
CHANGED
|
@@ -57,7 +57,12 @@ shard from byte zero; the UI does not claim resumable Range downloads.
|
|
| 57 |
`maxStorageBufferBindingSize`, `shader-f16`, browser quota, or all-WebGPU graph
|
| 58 |
requirement does not pass. Chrome on Apple Silicon is the validated release
|
| 59 |
path. Safari 26.5 on Apple Silicon has also passed a real foreground cold 27B
|
| 60 |
-
load through compat/Asyncify plus explicit OPFS eviction.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
validated on macOS: it selected compat, offered CPU-WASM for dense tiers, and
|
| 62 |
refused 27B without WebGPU; Firefox model generation was not exercised.
|
| 63 |
Windows Chrome/Edge on a discrete GPU and a physical low-limit iGPU laptop
|
|
@@ -85,7 +90,7 @@ validates the pinned native CPU prompt/reference fixture, then teacher-forces
|
|
| 85 |
that fixed 1,024-token sequence through raw token-ID prefixes. Schema v3 keeps
|
| 86 |
the forced reference logprob, natural top five, mean NLL, and perplexity
|
| 87 |
separate from the natural browser generation. This scoring method is not
|
| 88 |
-
available to normal chat
|
| 89 |
|
| 90 |
On the pinned Apple `metal-3` release path, the final JSPI runtime completed the
|
| 91 |
natural 1,024-token 27B run at 11.3 tok/s with one graph, zero CPU ops, and
|
|
@@ -128,18 +133,6 @@ first-download claim:
|
|
| 128 |
| 8B | WebGPU | 9,870 ms | 998 ms | 33.8 tok/s |
|
| 129 |
| 27B | WebGPU | 29,488 ms | 3,348 ms | 11.9 tok/s |
|
| 130 |
|
| 131 |
-
## DFlash research lane
|
| 132 |
-
|
| 133 |
-
DFlash is visible only as an inspection-only experiment. Its BF16 source and
|
| 134 |
-
Q8_0 native GGUF reference are pinned in the manifest. The strict D1 audit
|
| 135 |
-
passes all 39 shape, tokenizer, target-tap, conversion, revision, and evidence
|
| 136 |
-
checks, including a 32-token temperature-0 native Bonsai-27B parity smoke. The
|
| 137 |
-
custom WASM contains the DFlash llama.cpp architecture, but the browser load
|
| 138 |
-
API does not yet plumb `spec_type=draft-dflash`; the Q8
|
| 139 |
-
`fc.weight` also exceeds the 128 MiB baseline WebGPU binding limit. It remains
|
| 140 |
-
disabled and unavailable until a lower-bit candidate, browser parity,
|
| 141 |
-
acceptance, and positive speed gates all pass.
|
| 142 |
-
|
| 143 |
## Local tools and safety boundary
|
| 144 |
|
| 145 |
Tools are off by default. Memory writes/deletes require operator confirmation.
|
|
@@ -155,10 +148,10 @@ stay in the browser; only model/static asset downloads contact Hugging Face.
|
|
| 155 |
- Dawn native WebGPU validation: `18eb229ef5f707c1464cc581252e7603c73a3ef0`
|
| 156 |
- custom wllama source: `912c18b75d4358c1405a64646b8dbe43a205943b`
|
| 157 |
- custom nested llama.cpp: `00fa7cb284cbf133fc426733bd64238a3588a33e`
|
| 158 |
-
- custom patch set: `
|
| 159 |
-
- custom ESM: `
|
| 160 |
-
- JSPI WASM: `
|
| 161 |
-
- compat WASM: `
|
| 162 |
- compat worker: `0b667db536815fb9e5dc9ffbfbd6ba29affdda38a4ba892d5372ddb41ce9a8c8`
|
| 163 |
|
| 164 |
The vendored runtime exposes the validated token-embedding WebGPU placement
|
|
@@ -178,8 +171,7 @@ remains off and F16/F16 remains the release default.
|
|
| 178 |
## Licenses and attribution
|
| 179 |
|
| 180 |
The Bonsai GGUF tiers are Apache-2.0 model artifacts; their upstream notices
|
| 181 |
-
are published in `WaveCut/Bonsai-web-GGUF@v1.1.3/notices/`. The
|
| 182 |
-
|
| 183 |
-
same release. The shipped wllama/llama.cpp engines, React runtime, and
|
| 184 |
`@noble/hashes` are MIT-licensed. Exact runtime license texts are included in
|
| 185 |
`public/licenses/`; the consolidated inventory is `THIRD_PARTY_NOTICES.md`.
|
|
|
|
| 57 |
`maxStorageBufferBindingSize`, `shader-f16`, browser quota, or all-WebGPU graph
|
| 58 |
requirement does not pass. Chrome on Apple Silicon is the validated release
|
| 59 |
path. Safari 26.5 on Apple Silicon has also passed a real foreground cold 27B
|
| 60 |
+
load through compat/Asyncify plus explicit OPFS eviction. The normal compat
|
| 61 |
+
path caps prefill at `n_batch=32` and `n_ubatch=16` so Safari submits shorter
|
| 62 |
+
Metal command buffers; explicit benchmark tuning can still override the cap.
|
| 63 |
+
On Safari 26.5.2, the final 64-token gate completed at 1.2 prefill tok/s and
|
| 64 |
+
3.5 decode tok/s with one graph, zero CPU ops, and 65/65 GPU layers. Chrome's
|
| 65 |
+
JSPI path does not use this cap. Firefox gating was
|
| 66 |
validated on macOS: it selected compat, offered CPU-WASM for dense tiers, and
|
| 67 |
refused 27B without WebGPU; Firefox model generation was not exercised.
|
| 68 |
Windows Chrome/Edge on a discrete GPU and a physical low-limit iGPU laptop
|
|
|
|
| 90 |
that fixed 1,024-token sequence through raw token-ID prefixes. Schema v3 keeps
|
| 91 |
the forced reference logprob, natural top five, mean NLL, and perplexity
|
| 92 |
separate from the natural browser generation. This scoring method is not
|
| 93 |
+
available to normal chat.
|
| 94 |
|
| 95 |
On the pinned Apple `metal-3` release path, the final JSPI runtime completed the
|
| 96 |
natural 1,024-token 27B run at 11.3 tok/s with one graph, zero CPU ops, and
|
|
|
|
| 133 |
| 8B | WebGPU | 9,870 ms | 998 ms | 33.8 tok/s |
|
| 134 |
| 27B | WebGPU | 29,488 ms | 3,348 ms | 11.9 tok/s |
|
| 135 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
## Local tools and safety boundary
|
| 137 |
|
| 138 |
Tools are off by default. Memory writes/deletes require operator confirmation.
|
|
|
|
| 148 |
- Dawn native WebGPU validation: `18eb229ef5f707c1464cc581252e7603c73a3ef0`
|
| 149 |
- custom wllama source: `912c18b75d4358c1405a64646b8dbe43a205943b`
|
| 150 |
- custom nested llama.cpp: `00fa7cb284cbf133fc426733bd64238a3588a33e`
|
| 151 |
+
- custom patch set: `e94aef3f672211e72bab039385b3bcbeda46f0e3218db7a86a9b5468186ab491`
|
| 152 |
+
- custom ESM: `b7e883ebeed6aad1b4587d89c20cc4ee8e69e7a0e94e1d307dcad756d657a41a`
|
| 153 |
+
- JSPI WASM: `5a8686c10f7639653e7e1854d34f39c31ddba404de876a5ae15d36c42263ddc4`
|
| 154 |
+
- compat WASM: `c823684229149dd2386308978288e366f2ff209b231b78485507db8cb735bbdd`
|
| 155 |
- compat worker: `0b667db536815fb9e5dc9ffbfbd6ba29affdda38a4ba892d5372ddb41ce9a8c8`
|
| 156 |
|
| 157 |
The vendored runtime exposes the validated token-embedding WebGPU placement
|
|
|
|
| 171 |
## Licenses and attribution
|
| 172 |
|
| 173 |
The Bonsai GGUF tiers are Apache-2.0 model artifacts; their upstream notices
|
| 174 |
+
are published in `WaveCut/Bonsai-web-GGUF@v1.1.3/notices/`. The shipped
|
| 175 |
+
wllama/llama.cpp engines, React runtime, and
|
|
|
|
| 176 |
`@noble/hashes` are MIT-licensed. Exact runtime license texts are included in
|
| 177 |
`public/licenses/`; the consolidated inventory is `THIRD_PARTY_NOTICES.md`.
|
THIRD_PARTY_NOTICES.md
CHANGED
|
@@ -4,8 +4,6 @@ The distributed Space bundle includes:
|
|
| 4 |
|
| 5 |
- Bonsai 1.7B/4B/8B/27B GGUF model downloads, Apache-2.0, with per-tier
|
| 6 |
notices in `WaveCut/Bonsai-web-GGUF@v1.1.3/notices/`;
|
| 7 |
-
- the optional, runtime-disabled DFlash Q8_0 native reference, MIT,
|
| 8 |
-
Copyright (c) 2026 Z Lab, with `NOTICE-DFLASH.txt` in that model release;
|
| 9 |
- wllama at `912c18b75d4358c1405a64646b8dbe43a205943b`, MIT;
|
| 10 |
- llama.cpp/ggml at `00fa7cb284cbf133fc426733bd64238a3588a33e`, MIT;
|
| 11 |
- React and React DOM 19.2.4, MIT;
|
|
|
|
| 4 |
|
| 5 |
- Bonsai 1.7B/4B/8B/27B GGUF model downloads, Apache-2.0, with per-tier
|
| 6 |
notices in `WaveCut/Bonsai-web-GGUF@v1.1.3/notices/`;
|
|
|
|
|
|
|
| 7 |
- wllama at `912c18b75d4358c1405a64646b8dbe43a205943b`, MIT;
|
| 8 |
- llama.cpp/ggml at `00fa7cb284cbf133fc426733bd64238a3588a33e`, MIT;
|
| 9 |
- React and React DOM 19.2.4, MIT;
|
dist/assets/index-BY7HNxCd.js
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
dist/assets/index-C-2tm91j.css
DELETED
|
@@ -1 +0,0 @@
|
|
| 1 |
-
:root{--lightningcss-light: ;--lightningcss-dark:initial;color-scheme:dark;--ink:#111512;--ink-raised:#191e1a;--ink-soft:#252b25;--paper:#e8dfca;--paper-deep:#d8cdb4;--paper-ghost:#e8dfca9e;--phosphor:#b9ff45;--phosphor-dim:#b9ff4529;--moss:#6f8062;--rust:#c96f43;--line-light:#e8dfca24;--line-dark:#1115122e;--display:"Bodoni 72", Didot, "Iowan Old Style", Baskerville, serif;--body:"Avenir Next Condensed", "Gill Sans", "Trebuchet MS", sans-serif;--mono:"Berkeley Mono", Iosevka, "SFMono-Regular", Consolas, monospace;font-family:var(--body);font-synthesis:none;text-rendering:optimizelegibility}*{box-sizing:border-box}html{background:var(--ink)}body{background:linear-gradient(90deg, #1115120f 1px, transparent 1px) 0 0 / 22px 22px, linear-gradient(#1115120b 1px, transparent 1px) 0 0 / 22px 22px, var(--paper);min-width:320px;min-height:100vh;color:var(--paper);margin:0}button,textarea,select{font:inherit}button{color:inherit}button:focus-visible,textarea:focus-visible,select:focus-visible,summary:focus-visible{outline:2px solid var(--phosphor);outline-offset:3px}button:disabled{cursor:not-allowed}::selection{color:var(--ink);background:var(--phosphor)}.station-frame{isolation:isolate;min-height:100dvh;animation:.58s cubic-bezier(.16,1,.3,1) both station-in}.station-frame:before{z-index:20;pointer-events:none;content:"";opacity:.14;mix-blend-mode:soft-light;background-image:url("data:image/svg+xml,%3Csvg viewBox='0 0 180 180' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='3' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)' opacity='.3'/%3E%3C/svg%3E");position:fixed;inset:0}.station-header{background:var(--ink);border-bottom:1px solid var(--phosphor);z-index:10;grid-template-columns:minmax(330px,1fr) auto minmax(220px,1fr);align-items:stretch;min-height:76px;display:grid;position:relative;box-shadow:0 12px 38px #11151238}.brand-block{align-items:center;gap:14px;padding:12px 24px;display:flex}.brand-block h1{font:400 clamp(25px, 2vw, 34px) / .9 var(--display);letter-spacing:-.035em;margin:0}.brand-block h1 i{color:var(--phosphor);font-weight:400}.brand-mark{background:repeating-radial-gradient(at 52% 62%,#0000 0 4px,#b9ff456b 4.5px 5px);border:1px solid #b9ff457a;border-radius:50%;flex:0 0 42px;place-items:center;width:42px;height:42px;display:grid;transform:rotate(-11deg)}.brand-mark:before,.brand-mark:after,.brand-mark span:before,.brand-mark span:after{content:"";background:var(--phosphor);transform-origin:bottom;width:1px;position:absolute}.brand-mark:before{height:25px;transform:translate(-1px,4px)rotate(0)}.brand-mark:after{height:13px;transform:translate(4px,-2px)rotate(43deg)}.brand-mark span:before{height:11px;transform:translate(-5px,-1px)rotate(-47deg)}.brand-mark span:after{height:8px;transform:translate(3px,-9px)rotate(72deg)}.eyebrow{color:#e8dfca85;font:600 9px/1 var(--mono);letter-spacing:.18em;text-transform:uppercase;margin:0 0 5px}.eyebrow.ink{color:#11151285}.local-promise{border-inline:1px solid var(--line-light);background:linear-gradient(115deg,#b9ff4514,#0000 66%);align-items:center;gap:11px;min-width:360px;padding:0 28px;display:flex}.signal-dot,.live-label i,.sampling-label i{background:var(--phosphor);width:7px;height:7px;box-shadow:0 0 0 5px var(--phosphor-dim), 0 0 17px #b9ff459e;border-radius:50%;display:inline-block}.local-promise span:last-child{gap:3px;display:grid}.local-promise strong{color:var(--phosphor);font:600 10px/1 var(--mono);letter-spacing:.13em;text-transform:uppercase}.local-promise small{color:var(--paper-ghost);font:11px/1.2 var(--body)}.header-actions{justify-self:end;align-items:center;padding-right:18px;display:flex}.station-clock{color:var(--paper-ghost);font-family:var(--mono);justify-items:end;padding:0 18px;display:grid}.station-clock small{letter-spacing:.18em;font-size:8px}.station-clock span{letter-spacing:.09em;margin-top:4px;font-size:10px}.icon-button{cursor:pointer;background:0 0;border:1px solid #e8dfca3d;border-radius:50%;place-items:center;width:38px;height:38px;transition:color .16s,border-color .16s,transform .16s;display:grid}.icon-button:hover{color:var(--phosphor);border-color:var(--phosphor);transform:rotate(-7deg)}.icon-button svg{fill:none;stroke:currentColor;stroke-width:1.4px;width:18px}.workspace-grid{grid-template-columns:clamp(240px,18vw,286px) minmax(510px,1fr) clamp(310px,23vw,370px);grid-template-areas:"models chat inspector";min-height:calc(100dvh - 76px);display:grid}.model-rail,.inspector{background:var(--ink);min-width:0}.model-rail{border-right:1px solid var(--line-light);flex-direction:column;grid-area:models;padding:24px 16px 14px;animation:.52s cubic-bezier(.16,1,.3,1) .1s both rise-in;display:flex;position:relative;overflow:hidden}.model-rail:before{z-index:0;content:"";opacity:.18;background:repeating-radial-gradient(ellipse at 51% 47%, transparent 0 7px, var(--moss) 7.5px 8.5px);border-radius:43% 57% 52% 48%;width:265px;height:265px;position:absolute;bottom:44px;left:-103px;transform:rotate(18deg)}.rail-heading,.section-heading{z-index:1;align-items:flex-start;gap:11px;display:flex;position:relative}.rail-heading>span,.section-heading>span:first-child{color:var(--phosphor);font:500 9px/1 var(--mono);border-top:1px solid var(--phosphor);padding-top:5px}.rail-heading h2,.section-heading h2{font:400 22px/1 var(--display);letter-spacing:-.025em;margin:0}.model-list{z-index:1;gap:5px;margin-top:21px;display:grid;position:relative}.model-card{border:1px solid #0000;transition:border-color .18s,background .18s}.model-card.is-active{border-color:var(--phosphor);background:var(--phosphor-dim);box-shadow:inset 3px 0 0 var(--phosphor)}.model-select{text-align:left;width:100%;color:var(--paper);cursor:pointer;background:0 0;border:0;grid-template-columns:23px 1fr auto;gap:8px;padding:12px 8px 11px;display:grid}.model-card:not(.is-active) .model-select:hover{background:#e8dfca0e}.model-index{color:#e8dfca61;font:8px/1 var(--mono);padding-top:4px}.model-copy{gap:3px;min-width:0;display:grid}.model-title-line{justify-content:space-between;align-items:center;gap:7px;display:flex}.model-title-line strong{font:500 20px/1 var(--display)}.model-copy em{color:#e8dfca75;font:italic 10px/1.2 var(--display);text-overflow:ellipsis;white-space:nowrap;overflow:hidden}.model-copy>span:last-child{color:var(--paper-ghost);margin-top:3px;font-size:10px}.availability{font:600 7px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;border:1px solid var(--line-light);padding:2px 4px}.availability-loaded,.availability-loading,.availability-cached{color:var(--phosphor);border-color:#b9ff4552}.availability-remote,.availability-limited{color:var(--rust);border-color:#c96f4366}.model-footprint{color:#e8dfca63;font:8px/1 var(--mono);white-space:nowrap;align-self:end}.text-action{width:calc(100% - 30px);color:var(--rust);font:8px/1 var(--mono);text-align:left;cursor:pointer;background:0 0;border:0;border-top:1px dashed #c96f4359;margin:-3px 8px 9px 30px;padding:7px 0}.text-action span{float:right}.text-action.is-status{cursor:default;text-overflow:ellipsis;white-space:nowrap;display:block;overflow:hidden}.model-limit-reason{color:#e8dfca94;font:8px/1.35 var(--mono);margin:-4px 8px 9px 30px;display:block}.dflash-card{z-index:1;background:#111512b8;border:1px dashed #e8dfca3b;margin-top:auto;padding:13px 12px 11px;position:relative}.dflash-card.is-available{border-color:#b9ff457a}.dflash-heading{justify-content:space-between;gap:12px;display:flex}.dflash-heading h3{font:500 14px/1 var(--display);margin:7px 0 0}.experimental-stamp{color:var(--rust);font:700 7px/1 var(--mono);letter-spacing:.12em;text-transform:uppercase;border:1px solid;padding:3px 5px 2px;display:inline-block;transform:rotate(-2deg)}.switch{cursor:pointer;background:0 0;border:1px solid #e8dfca4d;border-radius:10px;width:35px;height:18px;padding:2px}.switch span{background:#e8dfca6b;border-radius:50%;width:12px;height:12px;transition:transform .17s,background .17s;display:block}.switch[aria-checked=true]{border-color:var(--phosphor)}.switch[aria-checked=true] span{background:var(--phosphor);transform:translate(17px)}.switch:disabled{opacity:.38}.dflash-card p{color:#e8dfca80;font:9px/1.45 var(--body);margin:9px 0 5px}.dflash-contract{color:#c96f43b8;font:7px/1.3 var(--mono);text-transform:uppercase;margin-bottom:11px;display:block}.dflash-axis{border-top:1px solid #e8dfca1f;justify-content:space-between;align-items:center;display:flex}.dflash-axis span{background:#e8dfca47;width:1px;height:5px;margin-top:-1px}.rail-footnote{z-index:1;color:#e8dfca4d;font:7px/1 var(--mono);letter-spacing:.03em;justify-content:space-between;gap:8px;margin-top:12px;display:flex;position:relative}.chat-surface{min-width:0;height:calc(100dvh - 76px);min-height:calc(100dvh - 76px);color:var(--ink);background:radial-gradient(ellipse at 82% 16%, transparent 0 77px, #1115120e 78px 79px, transparent 80px 89px, #1115120b 90px 91px, transparent 92px), linear-gradient(100deg, transparent 0 67%, #11151206 67.2% 67.35%, transparent 67.5%), var(--paper);grid-area:chat;grid-template-rows:auto 19px minmax(0,1fr) auto;animation:.56s cubic-bezier(.16,1,.3,1) .16s both rise-in;display:grid;position:relative;overflow:hidden}.chat-header{border-bottom:1px solid var(--line-dark);justify-content:space-between;align-items:center;gap:20px;min-height:82px;padding:17px clamp(21px,3vw,44px) 13px;display:flex}.chat-header h2{font:500 clamp(25px, 2.4vw, 37px)/1 var(--display);letter-spacing:-.045em;margin:0;display:inline}.botanical-subtitle{color:#11151275;font:italic 12px/1 var(--display);margin-left:10px}.chat-header-meta{color:#1115128c;font:8px/1 var(--mono);text-transform:uppercase;letter-spacing:.07em;grid-template-columns:repeat(2,auto);gap:5px 13px;display:grid}.header-load-button{color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 7px/1 var(--mono);letter-spacing:.06em;text-transform:uppercase;cursor:pointer;grid-column:span 2;justify-self:end;padding:6px 9px}.header-load-button:disabled{opacity:.42}.live-label{color:#3d551d;grid-column:span 2;justify-self:end;align-items:center;gap:7px;display:flex}.live-label i{background:#5d7e2e;width:5px;height:5px;box-shadow:0 0 0 3px #5a782b21}.live-label.is-dormant{color:#1115127a}.live-label.is-dormant i{background:#1115125c;box-shadow:0 0 0 3px #11151217}.runtime-error{z-index:4;color:var(--paper);border:1px solid var(--rust);background:#111512f5;justify-content:space-between;align-items:center;gap:16px;padding:11px 13px;display:flex;position:absolute;top:105px;left:clamp(21px,3vw,44px);right:clamp(21px,3vw,44px);box-shadow:4px 4px #c96f433d}.runtime-error div{gap:3px;display:grid}.runtime-error strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.runtime-error span{font:10px/1.35 var(--mono);overflow-wrap:anywhere}.runtime-error button{font:7px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:1px solid #e8dfca4d;flex:none;padding:5px 8px}.runtime-error button:disabled{cursor:wait;opacity:.5}.runtime-error.download-retry button{max-width:132px;line-height:1.25}.chat-ruler{color:#11151266;font:7px/1 var(--mono);background:repeating-linear-gradient(90deg,#0000 0 14px,#11151233 14px 15px) 0 100%/100% 4px no-repeat;border-bottom:1px solid #11151217;justify-content:space-between;padding-inline:clamp(21px,3vw,44px);display:flex}.transcript{scrollbar-color:#11151247 transparent;min-height:0;padding:20px clamp(21px,4vw,64px) 32px;overflow-y:auto}.transcript-date{color:#11151273;font:8px/1 var(--mono);letter-spacing:.1em;text-transform:uppercase;align-items:center;gap:10px;margin-bottom:22px;display:flex}.transcript-date:before,.transcript-date:after{content:"";background:#11151224;flex:1;height:1px}.transcript-empty{text-align:center;color:#11151280;max-width:430px;margin:12vh auto 0}.transcript-empty>span{color:#507324;font:42px/1 var(--display);display:block}.transcript-empty h3{color:var(--ink);font:400 25px/1 var(--display);margin:8px 0 6px}.transcript-empty p{margin:0;font-size:11px;line-height:1.5}.message{max-width:690px;margin-bottom:25px;animation:.43s cubic-bezier(.16,1,.3,1) both note-in}.message>header{color:#11151278;font:8px/1 var(--mono);text-transform:uppercase;letter-spacing:.07em;align-items:center;gap:8px;margin-bottom:8px;display:flex}.message>header strong{color:var(--ink)}.message>header time{margin-left:auto}.message-ordinal{color:#4d6728;min-width:22px}.message-content{font:15px/1.55 var(--body)}.message-content>p{margin:0}.markdown-content>:first-child{margin-top:0}.markdown-content>:last-child{margin-bottom:0}.markdown-content p,.markdown-content ul,.markdown-content ol,.markdown-content blockquote{margin:0 0 10px}.markdown-content ul,.markdown-content ol{padding-left:22px}.markdown-content h3,.markdown-content h4,.markdown-content h5{font-family:var(--display);margin:16px 0 7px;font-weight:500;line-height:1.1}.markdown-content blockquote{color:#111512ad;white-space:pre-wrap;border-left:2px solid #5073248c;padding-left:12px}.markdown-content code{font:.78em/1.4 var(--mono);background:#11151214;padding:1px 4px}.markdown-content pre{max-width:100%;color:var(--paper);background:var(--ink);border-left:2px solid var(--phosphor);margin:11px 0;padding:12px;position:relative;overflow:auto}.markdown-content pre code{white-space:pre;background:0 0;padding:0}.markdown-content a{color:#3f6417;text-underline-offset:2px;text-decoration-thickness:1px}.message-user .markdown-content code{background:#e8dfca1f}.reasoning-block{color:#1115129e;background:#11151209;border:1px dashed #1115123d;margin:0 0 11px;padding:8px 10px;font-size:11px}.reasoning-block summary{color:#4d6728;font:700 7px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;cursor:pointer}.reasoning-block[open] summary{margin-bottom:8px}.generation-placeholder{color:#11151273;margin:0;font-style:italic}.message-error{color:#8d321b;font:9px/1.4 var(--mono);margin:9px 0 0}.message.has-error{border-color:var(--rust)}.message-assistant{border-left:2px solid var(--ink);padding-left:22px}.message-assistant .message-content>p:first-letter{float:left;color:#38511d;font:42px/.75 var(--display);margin:1px 6px 0 0}.message-user{max-width:min(78%,610px);color:var(--paper);background:var(--ink);clip-path:polygon(0 0,calc(100% - 10px) 0,100% 10px,100% 100%,0 100%);margin-left:auto;padding:12px 15px 13px;box-shadow:5px 5px #1115121f}.message-user>header,.message-user>header strong{color:#e8dfca8f}.message-user .message-ordinal{color:var(--phosphor)}.message>footer{color:#1115126e;font:8px/1 var(--mono);border-top:1px dashed #11151230;gap:12px;margin-top:9px;padding-top:7px;display:flex}.stream-cursor{vertical-align:-2px;background:#507324;width:7px;height:14px;margin-left:4px;animation:.86s step-end infinite cursor-blink;display:inline-block}.tool-call{background:#1115120b;border:1px solid #1115122e;margin-top:13px}.tool-call summary{cursor:pointer;grid-template-columns:auto 1fr auto;align-items:center;gap:8px;padding:8px 10px;list-style:none;display:grid}.tool-call summary::-webkit-details-marker{display:none}.tool-call code{font:9px/1.4 var(--mono)}.tool-call summary>span:last-child{color:#1115127a;font:7px/1 var(--mono);text-transform:uppercase}.tool-state,.event-pip{background:var(--moss);border-radius:50%;width:6px;height:6px}.tool-state-complete,.event-complete{background:#55772c}.tool-state-running,.event-running{background:var(--phosphor);box-shadow:0 0 0 3px #5a782b26}.tool-state-error,.event-error{background:var(--rust)}.tool-call>div{border-top:1px solid #11151221;padding:9px 10px}.tool-call>div p{color:#11151294;margin:5px 0 0;font-size:10px}.composer{border-top:1px solid var(--line-dark);-webkit-backdrop-filter:blur(9px);backdrop-filter:blur(9px);background:#d8cdb4d1;padding:10px clamp(21px,3vw,44px) 18px;position:relative;box-shadow:0 -15px 38px #1115120f}.composer:before{content:"";background:repeating-linear-gradient(90deg, var(--ink) 0 1px, transparent 1px 12px);opacity:.23;height:5px;position:absolute;top:-5px;left:0;right:0}.composer-meta{color:#11151275;font:8px/1 var(--mono);justify-content:space-between;align-items:center;gap:12px;margin-bottom:7px;display:flex}.tool-toggle{color:#1115128a;font:700 8px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:0;align-items:center;gap:5px;padding:0;display:flex}.tool-toggle.is-on{color:#425f1f}.regenerate-button{color:#1115128a;font:700 8px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:0;margin-right:auto;padding:0}.regenerate-button:disabled{cursor:default;opacity:.45}.tool-toggle svg{fill:none;stroke:currentColor;stroke-width:1.3px;width:12px}.composer-field{border:1px solid var(--ink);background:#e8dfcad4;grid-template-columns:1fr auto;min-height:76px;display:grid;position:relative;box-shadow:4px 4px #1115121a}.composer-field label{color:#11151273;font:7px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase;position:absolute;top:7px;left:11px}.composer-field textarea{resize:none;min-width:0;color:var(--ink);font:14px/1.4 var(--body);background:0 0;border:0;outline:none;padding:23px 13px 9px}.composer-field textarea::placeholder{color:#11151257}.send-button{min-width:82px;color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 9px/1 var(--mono);text-transform:uppercase;cursor:pointer;justify-content:center;align-items:center;gap:7px;margin:7px;transition:transform .15s,box-shadow .15s,background .15s;display:flex}.send-button:hover:not(:disabled){box-shadow:3px 3px 0 var(--ink);transform:translate(-2px,-2px)}.send-button:disabled{opacity:.34}.send-button svg{fill:none;stroke:currentColor;stroke-width:1.4px;width:15px}.send-button.stop{color:var(--paper);background:var(--ink)}.send-button.stop span{background:var(--rust);width:7px;height:7px}.inspector{border-left:1px solid var(--line-light);scrollbar-color:#e8dfca2e transparent;grid-area:inspector;height:calc(100dvh - 76px);min-height:calc(100dvh - 76px);animation:.57s cubic-bezier(.16,1,.3,1) .22s both rise-in;overflow-y:auto}.inspector-section{border-bottom:1px solid var(--line-light);padding:20px 18px}.section-heading{align-items:center;margin-bottom:17px}.section-heading.compact{margin-bottom:14px}.section-heading>div{flex:1}.section-heading h2{font-size:20px}.sampling-label,.persistence{color:#e8dfca6b;font:7px/1 var(--mono);text-transform:uppercase;letter-spacing:.06em;align-items:center;gap:6px;display:flex}.sampling-label i{width:4px;height:4px}.speed-gauge{grid-template-columns:116px 1fr;align-items:center;gap:15px;display:grid}.gauge-ring{background:radial-gradient(circle, var(--ink) 0 51%, transparent 52%), conic-gradient(var(--phosphor) 0 var(--gauge), #e8dfca1c var(--gauge) 100%);border-radius:49% 51% 47% 53%;place-items:center;width:116px;height:116px;display:grid;transform:rotate(-9deg);box-shadow:inset 0 0 0 1px #e8dfca1a,0 0 30px #b9ff450e}.gauge-ring:after{content:"";border:1px dashed #e8dfca21;border-radius:45% 55% 52% 48%;width:128px;height:128px;position:absolute}.gauge-ring>div{z-index:1;justify-items:center;display:grid;position:relative;transform:rotate(9deg)}.gauge-ring strong{color:var(--phosphor);font:400 29px/1 var(--display)}.gauge-ring small{color:var(--paper-ghost);font:7px/1 var(--mono);text-transform:uppercase;margin-top:3px}.speed-gauge dl{margin:0}.speed-gauge dl div{border-bottom:1px solid var(--line-light);font:8px/1 var(--mono);justify-content:space-between;padding:7px 0;display:flex}.speed-gauge dt{color:#e8dfca6e}.speed-gauge dd{color:var(--paper);margin:0}.device-card{background:linear-gradient(90deg,#b9ff4514,#0000);border:1px solid #b9ff453b;grid-template-columns:auto 1fr auto;align-items:center;gap:10px;margin:19px 0 14px;padding:10px;display:grid}.device-icon{border:1px solid #b9ff4540;align-items:flex-end;gap:3px;width:28px;height:28px;padding:5px;display:flex}.device-icon i{background:var(--phosphor);width:4px}.device-icon i:first-child{opacity:.45;height:7px}.device-icon i:nth-child(2){height:15px}.device-icon i:nth-child(3){opacity:.7;height:11px}.device-card>div{gap:3px;display:grid}.device-card small,.memory-grid small,.meter small,.download-item small{color:#e8dfca6b;font:7px/1 var(--mono);text-transform:uppercase}.device-card strong{font:500 11px/1 var(--mono)}.device-card>span:last-child{color:var(--phosphor);font:600 13px/1 var(--mono);justify-items:end;display:grid}.device-card>span:last-child small{color:#e8dfca5e;margin-top:3px;font-size:6px}.meter{margin-top:13px}.meter>div:first-child{font:8px/1 var(--mono);justify-content:space-between;margin-bottom:6px;display:flex}.meter-track{border:1px solid #e8dfca36;height:5px;padding:1px}.meter-track>span{background:repeating-linear-gradient(90deg, var(--phosphor) 0 5px, transparent 5px 7px);height:100%;display:block}.memory-grid{background:var(--line-light);grid-template-columns:1fr 1fr;gap:1px;margin-top:14px;display:grid}.memory-grid>div{background:var(--ink);gap:6px;padding:10px;display:grid}.memory-grid strong{font:500 12px/1 var(--mono)}.event-log{gap:0;margin:0;padding:0;list-style:none;display:grid;position:relative}.event-log:before{content:"";background:var(--line-light);width:1px;position:absolute;top:5px;bottom:5px;left:3px}.event-log li{grid-template-columns:8px 64px 1fr;align-items:start;gap:8px;padding:7px 0;display:grid}.event-log li.event-empty{color:#e8dfca5c;font:8px/1.4 var(--mono);padding:5px 0;display:block}.event-pip{z-index:1;border:1px solid var(--ink);margin-top:2px}.event-log time{color:#e8dfca57;font:7px/1.2 var(--mono)}.event-log div{gap:3px;display:grid}.event-log strong{font:500 9px/1 var(--mono)}.event-log small{color:#e8dfca6b;font-size:8px}.persistence{border:1px solid var(--line-light);padding:3px 5px}.persistence.is-persistent{color:var(--phosphor);border-color:#b9ff4547}.download-item{border:1px solid var(--line-light);grid-template-columns:1fr auto;gap:8px;margin-top:16px;padding:11px 10px 10px;display:grid;position:relative}.download-item>div:first-child{gap:5px;display:grid}.download-item strong{font:500 10px/1 var(--mono)}.download-state{width:fit-content;color:var(--rust);font:700 6px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase}.state-downloading{color:var(--phosphor)}.download-percent{color:var(--phosphor);font:500 15px/1 var(--display)}.download-track{background:#e8dfca1a;grid-column:span 2;height:3px}.download-track span{background:var(--phosphor);height:100%;display:block;box-shadow:0 0 9px #b9ff4559}.artifact-close{border:1px solid var(--line-light);cursor:pointer;background:0 0;border-radius:50%;width:25px;height:25px}.artifact-empty{color:#e8dfca66;font:8px/1.5 var(--mono);margin:0}.artifact-pane{border:1px solid var(--line-light);background:#f8f5eb}.artifact-tabs{color:var(--paper);border-bottom:1px solid var(--line-light);background:var(--ink-raised);grid-template-columns:auto auto 1fr;align-items:center;display:grid}.artifact-tabs button{border:0;border-right:1px solid var(--line-light);font:7px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;padding:7px 9px}.artifact-tabs button[aria-selected=true]{color:var(--ink);background:var(--phosphor)}.artifact-tabs span{color:#e8dfca73;font:7px/1 var(--mono);text-align:right;text-overflow:ellipsis;white-space:nowrap;padding:0 8px;overflow:hidden}.artifact-pane iframe{background:#fff;border:0;width:100%;height:260px;display:block}.artifact-running{position:relative}.artifact-running>button{z-index:2;color:var(--paper);border:1px solid var(--rust);font:7px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:#111512e6;padding:5px 7px;position:absolute;top:7px;right:7px}.artifact-launch{min-height:180px;color:var(--ink);text-align:center;align-content:center;justify-items:center;gap:13px;padding:20px;display:grid}.artifact-launch p{color:#1115129e;max-width:250px;font:9px/1.5 var(--mono);margin:0}.artifact-launch button{color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 7px/1 var(--mono);text-transform:uppercase;cursor:pointer;padding:8px 10px}.artifact-code{height:260px;color:var(--paper);font:8px/1.45 var(--mono);white-space:pre-wrap;background:#0d100e;margin:0;padding:10px;overflow:auto}.settings-layer{z-index:50;-webkit-backdrop-filter:blur(4px);backdrop-filter:blur(4px);background:#080b09a3;justify-content:flex-end;animation:.16s both station-in;display:flex;position:fixed;inset:0}.settings-sheet{width:min(430px,100%);min-height:100%;color:var(--paper);border-left:1px solid var(--phosphor);background:repeating-radial-gradient(ellipse at 104% 101%, transparent 0 13px, #b9ff450e 13.5px 14.5px), var(--ink);padding:30px 27px;animation:.3s cubic-bezier(.16,1,.3,1) both sheet-in;overflow-y:auto;box-shadow:-24px 0 70px #00000052}.settings-sheet>header{border-bottom:1px solid var(--line-light);justify-content:space-between;align-items:flex-start;gap:20px;margin-bottom:37px;padding-bottom:20px;display:flex}.settings-sheet h2{font:400 31px/1 var(--display);margin:0}.sheet-close{border:1px solid var(--line-light);width:35px;height:35px;font:300 22px/1 var(--body);cursor:pointer;background:0 0;border-radius:50%}.sheet-close:hover{color:var(--phosphor);border-color:var(--phosphor)}.setting-group{border-bottom:1px solid var(--line-light);grid-template-columns:24px 1fr auto;align-items:center;gap:12px;padding:15px 0;display:grid}.setting-number{color:var(--phosphor);font:8px/1 var(--mono)}.setting-group>div,.range-setting>span:first-child{gap:5px;display:grid}.setting-group small,.range-setting small{color:#e8dfca6e;font:7px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.setting-group strong,.range-setting strong{font:500 13px/1 var(--mono)}.setting-locked{color:#e8dfca5c;font:7px/1 var(--mono);text-transform:uppercase}.setting-select select{max-width:112px;color:var(--paper);border:1px solid var(--line-light);background:var(--ink-raised);font:8px/1 var(--mono);border-radius:0;padding:6px 7px}.range-setting{gap:13px;margin-top:29px;display:grid}.range-setting>span:first-child{justify-content:space-between;align-items:end;display:flex}.range-setting input{appearance:none;background:linear-gradient(90deg, var(--phosphor) 0 35%, #e8dfca1c 35%);border:1px solid #e8dfca3d;border-radius:0;width:100%;height:4px;margin:0}.range-setting:first-of-type input{background:var(--phosphor)}.range-setting input::-webkit-slider-thumb{appearance:none;border:2px solid var(--ink);background:var(--phosphor);width:12px;height:18px;box-shadow:0 0 0 1px var(--phosphor);cursor:grab}.range-setting input::-moz-range-thumb{border:2px solid var(--ink);background:var(--phosphor);width:10px;height:16px;box-shadow:0 0 0 1px var(--phosphor);cursor:grab;border-radius:0}.range-ticks{pointer-events:none;justify-content:space-between;margin-top:-13px;display:flex}.range-limits{color:#e8dfca59;font:7px/1 var(--mono);justify-content:space-between;display:flex}.range-limits i{font-style:normal}.system-prompt-setting{gap:9px;margin-top:29px;display:grid}.system-prompt-setting>span{color:#e8dfca6e;font:7px/1 var(--mono);letter-spacing:.05em;text-transform:uppercase;justify-content:space-between;display:flex}.system-prompt-setting textarea{resize:vertical;color:var(--paper);border:1px solid var(--line-light);background:var(--ink-raised);font:10px/1.45 var(--mono);padding:10px}.settings-actions{grid-template-columns:1fr 1fr;gap:7px;margin-top:29px;display:grid}.settings-actions button{color:var(--paper);border:1px solid var(--line-light);font:7px/1.2 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;padding:9px}.settings-actions button:hover{color:var(--phosphor);border-color:var(--phosphor)}.settings-actions .danger-action{color:#e78a62;border-color:#c96f4373;grid-column:span 2}.boot-screen{min-height:100dvh;color:var(--paper);background:var(--ink);justify-content:center;align-items:center;gap:18px;padding:24px;display:flex}.boot-screen h1{color:var(--phosphor);font:400 34px/1 var(--display);margin:0}.boot-screen p:last-child{color:var(--paper-ghost);font:10px/1.5 var(--mono)}.range-ticks i{background:#e8dfca40;width:1px;height:5px}.settings-note{background:#b9ff450f;border:1px solid #b9ff4540;align-items:flex-start;gap:13px;margin-top:42px;padding:15px;display:flex}.settings-note .signal-dot{flex:0 0 7px;margin-top:6px}.settings-note p{color:#e8dfca94;margin:0;font-size:10px;line-height:1.5}.settings-note strong{color:var(--phosphor)}@keyframes station-in{0%{opacity:0}to{opacity:1}}@keyframes rise-in{0%{opacity:0;transform:translateY(11px)}to{opacity:1;transform:translateY(0)}}@keyframes note-in{0%{opacity:0;transform:translateY(7px)}to{opacity:1;transform:translateY(0)}}@keyframes cursor-blink{0%,52%{opacity:1}53%,to{opacity:0}}@keyframes sheet-in{0%{opacity:0;transform:translate(34px)}to{opacity:1;transform:translate(0)}}@media (width<=1279px){.station-header{grid-template-columns:minmax(320px,1fr) auto auto}.local-promise{min-width:auto}.local-promise small{display:none}.workspace-grid{grid-template-columns:260px minmax(500px,1fr);grid-template-areas:"models chat""inspector inspector"}.model-rail,.chat-surface{height:calc(100dvh - 76px);min-height:calc(100dvh - 76px)}.inspector{border-top:1px solid var(--phosphor);border-left:0;grid-template-columns:repeat(2,minmax(0,1fr));height:auto;min-height:auto;display:grid;overflow:visible}.inspector-section{border-right:1px solid var(--line-light);border-bottom:0}}@media (width<=820px){.station-header{min-height:70px;display:block}.brand-block{padding:11px 70px 11px 15px}.brand-mark{flex-basis:36px;width:36px;height:36px}.local-promise{border:0;border-top:1px solid var(--line-light);min-height:31px;padding:0 16px}.header-actions{position:absolute;top:10px;right:0}.station-clock{display:none}.workspace-grid{display:block}.model-rail,.chat-surface,.inspector{height:auto;min-height:auto}.model-rail{border-right:0;border-bottom:1px solid var(--phosphor);padding:18px 15px;display:block}.model-list{scroll-snap-type:x mandatory;gap:7px;padding-bottom:7px;display:flex;overflow-x:auto}.model-card{scroll-snap-align:start;min-width:205px}.dflash-card{margin-top:13px}.rail-footnote{display:none}.chat-surface{grid-template-rows:auto 19px minmax(450px,auto) auto;display:grid;overflow:visible}.transcript{padding-inline:22px;overflow:visible}.chat-header{padding:17px 19px}.chat-header-meta span:not(.live-label){display:none}.chat-header-meta .header-load-button{display:block}.live-label{grid-column:1}.composer{z-index:5;padding-inline:14px;position:sticky;bottom:0}.inspector{border-top:1px solid var(--phosphor);display:block}.inspector-section{border-right:0;border-bottom:1px solid var(--line-light)}}@media (width<=520px){.brand-block h1{font-size:24px}.brand-block .eyebrow,.local-promise strong{font-size:7px}.chat-header{align-items:flex-end}.botanical-subtitle{margin:4px 0 0;display:block}.message-user{max-width:91%}.composer-meta>span{display:none}.composer-field{grid-template-columns:1fr;padding-bottom:50px}.send-button{height:35px;position:absolute;bottom:0;right:0}.speed-gauge{grid-template-columns:106px 1fr}.gauge-ring{width:102px;height:102px}.gauge-ring:after{width:113px;height:113px}}@media (prefers-reduced-motion:reduce){*,:before,:after{scroll-behavior:auto!important;transition-duration:.01ms!important;animation-duration:.01ms!important;animation-iteration-count:1!important}}.bench-page{min-height:100dvh;color:var(--ink);background:var(--paper)}.bench-masthead{min-height:86px;color:var(--paper);border-bottom:1px solid var(--phosphor);background:var(--ink);grid-template-columns:minmax(310px,1fr) minmax(360px,1.2fr) auto;align-items:stretch;display:grid}.bench-masthead .brand-block h1{color:var(--paper)}.bench-masthead-note{border-inline:1px solid var(--line-light);background:linear-gradient(110deg,#b9ff4517,#0000 72%);flex-direction:column;justify-content:center;gap:5px;padding:14px 28px;display:flex}.bench-masthead-note strong{color:var(--phosphor);font:700 9px/1 var(--mono);letter-spacing:.11em;text-transform:uppercase}.bench-masthead-note span{color:#e8dfca8c;max-width:520px;font:10px/1.35 var(--body)}.bench-return{color:var(--paper);font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;align-items:center;gap:8px;padding:0 25px;text-decoration:none;display:flex}.bench-return:hover{color:var(--phosphor)}.bench-layout{grid-template-columns:minmax(275px,330px) minmax(0,1fr);min-height:calc(100dvh - 86px);display:grid}.bench-controls{color:var(--paper);border-right:1px solid var(--phosphor);background:repeating-radial-gradient(ellipse at -12% 102%, transparent 0 24px, #b9ff450b 25px 26px), var(--ink);padding:28px 22px}.bench-controls h2{font:400 30px/1 var(--display);letter-spacing:-.03em;margin:0}.bench-control-intro{color:#e8dfca80;margin:9px 0 22px;font-size:11px;line-height:1.45}.bench-controls form{gap:15px;display:grid}.bench-field{gap:6px;min-width:0;display:grid}.bench-field>span{color:#e8dfcac7;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.bench-field select,.bench-field input{width:100%;min-width:0;height:39px;color:var(--paper);background:var(--ink-raised);font:10px/1 var(--mono);border:1px solid #e8dfca38;border-radius:0;outline:0;padding:0 10px}.bench-field input::placeholder{color:#e8dfca40}.bench-field select:disabled,.bench-field input:disabled{opacity:.5}.bench-field small{color:#e8dfca59;font:8px/1.2 var(--mono)}.bench-field-pair{grid-template-columns:repeat(2,minmax(0,1fr));gap:10px;display:grid}.bench-experimental{background:#c96f430d;border:1px dashed #c96f437a;padding:11px}.bench-experimental summary{color:#e19a76;font:700 8px/1 var(--mono);letter-spacing:.07em;text-transform:uppercase;cursor:pointer}.bench-experimental>p{color:#e8dfca75;margin:10px 0;font-size:9px;line-height:1.4}.bench-locks{color:#e8dfca70;font:7px/1.35 var(--mono);gap:5px;margin:10px 0 0;display:grid}.bench-locks div{justify-content:space-between;gap:8px;display:flex}.bench-locks dt,.bench-locks dd{margin:0}.bench-locks dt{color:#e8dfcab8}.bench-gate{border-left:2px solid var(--rust);background:#c96f4314;gap:5px;padding:10px;display:grid}.bench-gate.pass{border-color:var(--phosphor);background:var(--phosphor-dim)}.bench-gate strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.06em;text-transform:uppercase}.bench-gate.pass strong{color:var(--phosphor)}.bench-gate span{color:#e8dfca80;font:8px/1.4 var(--mono);overflow-wrap:anywhere}.bench-run-actions{grid-template-columns:1fr auto;gap:8px;display:grid}.bench-run-button,.bench-stop-button,.bench-section-heading button{border:1px solid var(--ink);background:var(--phosphor);min-height:38px;font:800 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;cursor:pointer;border-radius:0;padding:0 13px}.bench-run-button:disabled{opacity:.35}.bench-stop-button{color:#ffb08a;border-color:var(--rust);background:0 0}.bench-workbench{background:linear-gradient(90deg, #1115120b 1px, transparent 1px) 0 0 / 22px 22px, linear-gradient(#1115120a 1px, transparent 1px) 0 0 / 22px 22px, var(--paper);min-width:0;padding:clamp(26px,4vw,58px)}.bench-hero{border-bottom:1px solid var(--line-dark);grid-template-columns:minmax(320px,1.3fr) minmax(260px,.7fr);align-items:end;gap:34px;padding-bottom:29px;display:grid}.bench-hero h2{max-width:690px;font:400 clamp(42px, 6vw, 82px)/.84 var(--display);letter-spacing:-.065em;margin:0}.bench-hero h2 em{color:#4f7125;font-weight:400}.bench-hero>p{color:#11151294;max-width:430px;margin:0;font-size:11px;line-height:1.55}.bench-progress{gap:8px;padding:16px 0;display:grid}.bench-progress>div{font:8px/1 var(--mono);letter-spacing:.05em;text-transform:uppercase;justify-content:space-between;gap:20px;display:flex}.bench-progress>div span{color:#11151273}.bench-progress-track{background:#11151221;height:3px}.bench-progress-track i{background:#527426;height:100%;transition:width .18s;display:block}.bench-error{color:var(--paper);border-left:3px solid var(--rust);background:var(--ink);gap:5px;margin-bottom:18px;padding:12px 14px;display:grid}.bench-error strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.bench-error span{font:9px/1.4 var(--mono);overflow-wrap:anywhere}.bench-core-section{margin-top:13px}.bench-section-heading{justify-content:space-between;align-items:end;gap:20px;margin-bottom:13px;display:flex}.bench-section-heading h3,.bench-output h3{font:500 24px/1 var(--display);margin:0}.bench-section-heading button{min-height:31px;color:var(--paper);border-color:var(--ink);background:var(--ink)}.bench-core-sample{border:1px solid var(--ink);background:var(--paper-deep);grid-template-columns:repeat(4,minmax(0,1fr));display:grid;position:relative;overflow:hidden}.bench-core-sample:before{content:"";pointer-events:none;opacity:.35;background:repeating-radial-gradient(#0000 0 34px,#1115122b 35px 36px);position:absolute;inset:-130% -5%}.bench-core-sample article{background:#e8dfca91;border-right:1px solid #11151242;flex-direction:column;justify-content:space-between;min-height:128px;padding:17px;display:flex;position:relative}.bench-core-sample article:nth-child(2n){background:#d8cdb499}.bench-core-sample article:last-child{color:var(--paper);background:#111512e8;border-right:0}.bench-core-sample article>span{font:800 8px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase}.bench-core-sample article strong{font:400 clamp(28px, 3vw, 43px)/1 var(--display);letter-spacing:-.04em;margin-block:auto}.bench-core-sample article small{color:#11151280;font:7px/1.2 var(--mono);text-transform:uppercase}.bench-core-sample article:last-child small{color:#e8dfca7a}.bench-detail-grid{grid-template-columns:repeat(2,minmax(0,1fr));gap:12px;margin-top:12px;display:grid}.bench-data-card{border:1px solid var(--line-dark);background:#e8dfcaad;padding:16px}.bench-runtime-card,.bench-tuning-card{grid-column:span 2}.bench-data-card dl{grid-template-columns:repeat(2,minmax(0,1fr));gap:12px 26px;margin:0;display:grid}.bench-runtime-card dl{grid-template-columns:minmax(180px,1.5fr) repeat(4,minmax(100px,.7fr))}.bench-tuning-card dl{grid-template-columns:repeat(4,minmax(0,1fr))}.bench-data-card dl div{border-top:1px solid #11151221;gap:4px;min-width:0;padding-top:8px;display:grid}.bench-data-card dt{color:#11151273;font:7px/1 var(--mono);letter-spacing:.07em;text-transform:uppercase}.bench-data-card dd{min-width:0;font:500 16px/1.2 var(--body);text-overflow:ellipsis;white-space:nowrap;margin:0;overflow:hidden}.bench-output{border:1px dashed #1115124d;grid-template-columns:repeat(2,minmax(0,1fr));gap:34px;margin-top:12px;padding:21px;display:grid}.bench-output>div:last-child{border-left:1px solid var(--line-dark);padding-left:34px}.bench-output p:not(.eyebrow){color:#11151299;white-space:pre-wrap;margin:8px 0 0;font-size:11px;line-height:1.5}@media (width<=1050px){.bench-masthead{grid-template-columns:minmax(310px,1fr) auto}.bench-masthead-note{display:none}.bench-layout{grid-template-columns:285px minmax(0,1fr)}.bench-core-sample{grid-template-columns:repeat(2,minmax(0,1fr))}.bench-core-sample article:nth-child(2){border-right:0}.bench-core-sample article:nth-child(-n+2){border-bottom:1px solid #11151242}.bench-runtime-card dl{grid-template-columns:repeat(2,minmax(0,1fr))}}@media (width<=760px){.bench-masthead{align-items:center;display:flex}.bench-masthead .brand-block{flex:1;min-width:0}.bench-return{padding:0 15px;font-size:0}.bench-return span{font-size:17px}.bench-layout{display:block}.bench-controls{border-right:0;border-bottom:1px solid var(--phosphor)}.bench-workbench{padding:28px 16px}.bench-hero,.bench-output{grid-template-columns:1fr;gap:18px}.bench-hero h2{font-size:clamp(42px,14vw,68px)}.bench-detail-grid{display:block}.bench-data-card+.bench-data-card{margin-top:10px}.bench-output>div:last-child{border-top:1px solid var(--line-dark);border-left:0;padding-top:18px;padding-left:0}}@media (width<=480px){.bench-masthead .brand-mark{display:none}.bench-core-sample,.bench-data-card dl,.bench-runtime-card dl,.bench-tuning-card dl{grid-template-columns:1fr}.bench-core-sample article{border-bottom:1px solid #11151242;border-right:0;min-height:105px}.bench-core-sample article:last-child{border-bottom:0}}
|
|
|
|
|
|
dist/assets/index-Dn6W9Rfz.js
DELETED
|
The diff for this file is too large to render.
See raw diff
|
|
|
dist/assets/index-DnMbdiUB.css
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
:root{--lightningcss-light: ;--lightningcss-dark:initial;color-scheme:dark;--ink:#111512;--ink-raised:#191e1a;--ink-soft:#252b25;--paper:#e8dfca;--paper-deep:#d8cdb4;--paper-ghost:#e8dfca9e;--phosphor:#b9ff45;--phosphor-dim:#b9ff4529;--moss:#6f8062;--rust:#c96f43;--line-light:#e8dfca24;--line-dark:#1115122e;--display:"Bodoni 72", Didot, "Iowan Old Style", Baskerville, serif;--body:"Avenir Next Condensed", "Gill Sans", "Trebuchet MS", sans-serif;--mono:"Berkeley Mono", Iosevka, "SFMono-Regular", Consolas, monospace;--type-utility:clamp(.6875rem, .15vw + .65rem, .8125rem);--type-caption:clamp(.75rem, .18vw + .7rem, .875rem);--type-body:clamp(1rem, .22vw + .95rem, 1.125rem);--type-heading:clamp(1.75rem, 1.15vw + 1.45rem, 2.5rem);font-family:var(--body);font-synthesis:none;text-rendering:optimizelegibility}*{box-sizing:border-box}html{background:var(--ink)}body{background:linear-gradient(90deg, #1115120f 1px, transparent 1px) 0 0 / 22px 22px, linear-gradient(#1115120b 1px, transparent 1px) 0 0 / 22px 22px, var(--paper);min-width:320px;min-height:100vh;color:var(--paper);font-size:var(--type-body);margin:0}button,textarea,select{font:inherit}button{color:inherit}button:focus-visible,textarea:focus-visible,select:focus-visible,summary:focus-visible{outline:2px solid var(--phosphor);outline-offset:3px}button:disabled{cursor:not-allowed}::selection{color:var(--ink);background:var(--phosphor)}.station-frame{isolation:isolate;min-height:100dvh;animation:.58s cubic-bezier(.16,1,.3,1) both station-in}.station-frame:before{z-index:20;pointer-events:none;content:"";opacity:.14;mix-blend-mode:soft-light;background-image:url("data:image/svg+xml,%3Csvg viewBox='0 0 180 180' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='3' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)' opacity='.3'/%3E%3C/svg%3E");position:fixed;inset:0}.station-header{background:var(--ink);border-bottom:1px solid var(--phosphor);z-index:10;grid-template-columns:minmax(330px,1fr) auto minmax(220px,1fr);align-items:stretch;min-height:76px;display:grid;position:relative;box-shadow:0 12px 38px #11151238}.brand-block{align-items:center;gap:14px;padding:12px 24px;display:flex}.brand-block h1{font:400 clamp(25px, 2vw, 34px) / .9 var(--display);letter-spacing:-.035em;margin:0}.brand-block h1 i{color:var(--phosphor);font-weight:400}.brand-mark{background:repeating-radial-gradient(at 52% 62%,#0000 0 4px,#b9ff456b 4.5px 5px);border:1px solid #b9ff457a;border-radius:50%;flex:0 0 42px;place-items:center;width:42px;height:42px;display:grid;transform:rotate(-11deg)}.brand-mark:before,.brand-mark:after,.brand-mark span:before,.brand-mark span:after{content:"";background:var(--phosphor);transform-origin:bottom;width:1px;position:absolute}.brand-mark:before{height:25px;transform:translate(-1px,4px)rotate(0)}.brand-mark:after{height:13px;transform:translate(4px,-2px)rotate(43deg)}.brand-mark span:before{height:11px;transform:translate(-5px,-1px)rotate(-47deg)}.brand-mark span:after{height:8px;transform:translate(3px,-9px)rotate(72deg)}.eyebrow{color:#e8dfca85;font:600 var(--type-utility)/1 var(--mono);letter-spacing:.18em;text-transform:uppercase;margin:0 0 5px}.eyebrow.ink{color:#11151285}.local-promise{border-inline:1px solid var(--line-light);background:linear-gradient(115deg,#b9ff4514,#0000 66%);align-items:center;gap:11px;min-width:360px;padding:0 28px;display:flex}.signal-dot,.live-label i,.sampling-label i{background:var(--phosphor);width:7px;height:7px;box-shadow:0 0 0 5px var(--phosphor-dim), 0 0 17px #b9ff459e;border-radius:50%;display:inline-block}.local-promise span:last-child{gap:3px;display:grid}.local-promise strong{color:var(--phosphor);font:600 var(--type-caption)/1 var(--mono);letter-spacing:.13em;text-transform:uppercase}.local-promise small{color:var(--paper-ghost);font:var(--type-caption)/1.25 var(--body)}.header-actions{justify-self:end;align-items:center;padding-right:18px;display:flex}.station-clock{color:var(--paper-ghost);font-family:var(--mono);justify-items:end;padding:0 18px;display:grid}.station-clock small{font-size:var(--type-utility);letter-spacing:.18em}.station-clock span{font-size:var(--type-caption);letter-spacing:.09em;margin-top:4px}.icon-button{cursor:pointer;background:0 0;border:1px solid #e8dfca3d;border-radius:50%;place-items:center;width:44px;height:44px;transition:color .16s,border-color .16s,transform .16s;display:grid}.icon-button:hover{color:var(--phosphor);border-color:var(--phosphor);transform:rotate(-7deg)}.icon-button svg{fill:none;stroke:currentColor;stroke-width:1.4px;width:18px}.workspace-grid{grid-template-columns:clamp(280px,19vw,340px) minmax(560px,1fr) clamp(340px,23vw,410px);grid-template-areas:"models chat inspector";min-height:calc(100dvh - 76px);display:grid}.model-rail,.inspector{background:var(--ink);min-width:0}.model-rail{border-right:1px solid var(--line-light);flex-direction:column;grid-area:models;padding:24px 16px 14px;animation:.52s cubic-bezier(.16,1,.3,1) .1s both rise-in;display:flex;position:relative;overflow:hidden}.model-rail:before{z-index:0;content:"";opacity:.18;background:repeating-radial-gradient(ellipse at 51% 47%, transparent 0 7px, var(--moss) 7.5px 8.5px);border-radius:43% 57% 52% 48%;width:265px;height:265px;position:absolute;bottom:44px;left:-103px;transform:rotate(18deg)}.rail-heading,.section-heading{z-index:1;align-items:flex-start;gap:11px;display:flex;position:relative}.rail-heading>span,.section-heading>span:first-child{color:var(--phosphor);font:500 var(--type-utility)/1 var(--mono);border-top:1px solid var(--phosphor);padding-top:5px}.rail-heading h2,.section-heading h2{font:400 26px/1 var(--display);letter-spacing:-.025em;margin:0}.model-list{z-index:1;gap:5px;margin-top:21px;display:grid;position:relative}.model-card{border:1px solid #0000;transition:border-color .18s,background .18s}.model-card.is-active{border-color:var(--phosphor);background:var(--phosphor-dim);box-shadow:inset 3px 0 0 var(--phosphor)}.model-select{text-align:left;width:100%;min-height:76px;color:var(--paper);cursor:pointer;background:0 0;border:0;grid-template-columns:23px 1fr auto;gap:8px;padding:13px 9px 12px;display:grid}.model-card:not(.is-active) .model-select:hover{background:#e8dfca0e}.model-index{color:#e8dfca61;font:var(--type-utility)/1 var(--mono);padding-top:4px}.model-copy{gap:3px;min-width:0;display:grid}.model-title-line{justify-content:space-between;align-items:center;gap:7px;display:flex}.model-title-line strong{font:500 24px/1 var(--display)}.model-copy em{color:#e8dfca75;font:italic var(--type-caption)/1.2 var(--display);text-overflow:ellipsis;white-space:nowrap;overflow:hidden}.model-copy>span:last-child{color:var(--paper-ghost);font-size:var(--type-caption);margin-top:3px}.availability{font:600 var(--type-utility)/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;border:1px solid var(--line-light);padding:2px 4px}.availability-loaded,.availability-loading,.availability-cached{color:var(--phosphor);border-color:#b9ff4552}.availability-remote,.availability-limited{color:var(--rust);border-color:#c96f4366}.model-footprint{color:#e8dfca63;font:var(--type-utility)/1 var(--mono);white-space:nowrap;align-self:end}.model-limit-reason{color:#e8dfca94;font:var(--type-utility)/1.35 var(--mono);margin:-4px 8px 9px 30px;display:block}.rail-footnote{z-index:1;color:#e8dfca4d;font:var(--type-utility)/1.2 var(--mono);letter-spacing:.03em;justify-content:space-between;gap:8px;margin-top:auto;display:flex;position:relative}.chat-surface{min-width:0;height:calc(100dvh - 76px);min-height:calc(100dvh - 76px);color:var(--ink);background:radial-gradient(ellipse at 82% 16%, transparent 0 77px, #1115120e 78px 79px, transparent 80px 89px, #1115120b 90px 91px, transparent 92px), linear-gradient(100deg, transparent 0 67%, #11151206 67.2% 67.35%, transparent 67.5%), var(--paper);grid-area:chat;grid-template-rows:auto 19px minmax(0,1fr) auto;animation:.56s cubic-bezier(.16,1,.3,1) .16s both rise-in;display:grid;position:relative;overflow:hidden}.chat-header{border-bottom:1px solid var(--line-dark);justify-content:space-between;align-items:center;gap:20px;min-height:82px;padding:17px clamp(21px,3vw,44px) 13px;display:flex}.chat-header h2{font:500 var(--type-heading)/1 var(--display);letter-spacing:-.045em;margin:0;display:inline}.botanical-subtitle{color:#11151275;font:italic var(--type-caption)/1 var(--display);margin-left:10px}.chat-header-meta{color:#1115128c;font:var(--type-utility)/1.2 var(--mono);text-transform:uppercase;letter-spacing:.07em;grid-template-columns:repeat(2,auto);gap:5px 13px;display:grid}.live-label{color:#3d551d;grid-column:span 2;justify-self:end;align-items:center;gap:7px;display:flex}.live-label i{background:#5d7e2e;width:5px;height:5px;box-shadow:0 0 0 3px #5a782b21}.live-label.is-dormant{color:#1115127a}.live-label.is-dormant i{background:#1115125c;box-shadow:0 0 0 3px #11151217}.runtime-error{z-index:4;color:var(--paper);border:1px solid var(--rust);background:#111512f5;justify-content:space-between;align-items:center;gap:16px;padding:11px 13px;display:flex;position:absolute;top:105px;left:clamp(21px,3vw,44px);right:clamp(21px,3vw,44px);box-shadow:4px 4px #c96f433d}.runtime-error div{gap:3px;display:grid}.runtime-error strong{color:#ff9a68;font:700 var(--type-utility)/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.runtime-error span{font:var(--type-caption)/1.35 var(--mono);overflow-wrap:anywhere}.runtime-error button{min-height:44px;font:var(--type-utility)/1.2 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:1px solid #e8dfca4d;flex:none;padding:5px 8px}.runtime-error button:disabled{cursor:wait;opacity:.5}.runtime-error.download-retry button{max-width:132px;line-height:1.25}.chat-ruler{color:#11151266;font:var(--type-utility)/1 var(--mono);background:repeating-linear-gradient(90deg,#0000 0 14px,#11151233 14px 15px) 0 100%/100% 4px no-repeat;border-bottom:1px solid #11151217;justify-content:space-between;padding-inline:clamp(21px,3vw,44px);display:flex}.transcript{scrollbar-color:#11151247 transparent;min-height:0;padding:20px clamp(21px,4vw,64px) 32px;overflow-y:auto}.transcript-date{color:#11151273;font:var(--type-utility)/1 var(--mono);letter-spacing:.1em;text-transform:uppercase;align-items:center;gap:10px;margin-bottom:22px;display:flex}.transcript-date:before,.transcript-date:after{content:"";background:#11151224;flex:1;height:1px}.transcript-empty{text-align:center;color:#11151280;max-width:430px;margin:12vh auto 0}.transcript-empty>span{color:#507324;font:42px/1 var(--display);display:block}.transcript-empty h3{color:var(--ink);font:400 25px/1 var(--display);margin:8px 0 6px}.transcript-empty p{font-size:var(--type-body);margin:0;line-height:1.5}.message{max-width:690px;margin-bottom:25px;animation:.43s cubic-bezier(.16,1,.3,1) both note-in}.message>header{color:#11151278;font:var(--type-utility)/1.2 var(--mono);text-transform:uppercase;letter-spacing:.07em;align-items:center;gap:8px;margin-bottom:8px;display:flex}.message>header strong{color:var(--ink)}.message>header time{margin-left:auto}.message-ordinal{color:#4d6728;min-width:22px}.message-content{font:var(--type-body)/1.58 var(--body)}.message-content>p{margin:0}.markdown-content>:first-child{margin-top:0}.markdown-content>:last-child{margin-bottom:0}.markdown-content p,.markdown-content ul,.markdown-content ol,.markdown-content blockquote{margin:0 0 10px}.markdown-content ul,.markdown-content ol{padding-left:22px}.markdown-content h3,.markdown-content h4,.markdown-content h5{font-family:var(--display);margin:16px 0 7px;font-weight:500;line-height:1.1}.markdown-content blockquote{color:#111512ad;white-space:pre-wrap;border-left:2px solid #5073248c;padding-left:12px}.markdown-content code{font:.78em/1.4 var(--mono);background:#11151214;padding:1px 4px}.markdown-content pre{max-width:100%;color:var(--paper);background:var(--ink);border-left:2px solid var(--phosphor);margin:11px 0;padding:12px;position:relative;overflow:auto}.markdown-content pre code{white-space:pre;background:0 0;padding:0}.markdown-content a{color:#3f6417;text-underline-offset:2px;text-decoration-thickness:1px}.message-user .markdown-content code{background:#e8dfca1f}.reasoning-block{color:#1115129e;font-size:var(--type-caption);background:#11151209;border:1px dashed #1115123d;margin:0 0 11px;padding:8px 10px}.reasoning-block summary{color:#4d6728;font:700 var(--type-utility)/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;cursor:pointer}.reasoning-block[open] summary{margin-bottom:8px}.generation-placeholder{color:#11151273;margin:0;font-style:italic}.message-error{color:#8d321b;font:var(--type-caption)/1.4 var(--mono);margin:9px 0 0}.message.has-error{border-color:var(--rust)}.message-assistant{border-left:2px solid var(--ink);padding-left:22px}.message-assistant .message-content>p:first-letter{float:left;color:#38511d;font:42px/.75 var(--display);margin:1px 6px 0 0}.message-user{max-width:min(78%,610px);color:var(--paper);background:var(--ink);clip-path:polygon(0 0,calc(100% - 10px) 0,100% 10px,100% 100%,0 100%);margin-left:auto;padding:12px 15px 13px;box-shadow:5px 5px #1115121f}.message-user>header,.message-user>header strong{color:#e8dfca8f}.message-user .message-ordinal{color:var(--phosphor)}.message>footer{color:#1115126e;font:var(--type-utility)/1.2 var(--mono);border-top:1px dashed #11151230;gap:12px;margin-top:9px;padding-top:7px;display:flex}.stream-cursor{vertical-align:-2px;background:#507324;width:7px;height:14px;margin-left:4px;animation:.86s step-end infinite cursor-blink;display:inline-block}.tool-call{background:#1115120b;border:1px solid #1115122e;margin-top:13px}.tool-call summary{cursor:pointer;grid-template-columns:auto 1fr auto;align-items:center;gap:8px;padding:8px 10px;list-style:none;display:grid}.tool-call summary::-webkit-details-marker{display:none}.tool-call code{font:var(--type-caption)/1.4 var(--mono)}.tool-call summary>span:last-child{color:#1115127a;font:var(--type-utility)/1 var(--mono);text-transform:uppercase}.tool-state,.event-pip{background:var(--moss);border-radius:50%;width:6px;height:6px}.tool-state-complete,.event-complete{background:#55772c}.tool-state-running,.event-running{background:var(--phosphor);box-shadow:0 0 0 3px #5a782b26}.tool-state-error,.event-error{background:var(--rust)}.tool-call>div{border-top:1px solid #11151221;padding:9px 10px}.tool-call>div p{color:#11151294;font-size:var(--type-caption);margin:5px 0 0}.composer{border-top:1px solid var(--line-dark);-webkit-backdrop-filter:blur(9px);backdrop-filter:blur(9px);background:#d8cdb4d1;padding:12px clamp(21px,3vw,44px) 18px;position:relative;box-shadow:0 -15px 38px #1115120f}.composer:before{content:"";background:repeating-linear-gradient(90deg, var(--ink) 0 1px, transparent 1px 12px);opacity:.23;height:5px;position:absolute;top:-5px;left:0;right:0}.composer-activity{height:164px;color:var(--paper);background:linear-gradient(105deg, #b9ff4512, transparent 58%), var(--ink);border:1px solid #111512d1;grid-template-rows:24px 26px 40px 34px;gap:6px;margin-bottom:10px;padding:11px 12px;display:grid;overflow:hidden;box-shadow:4px 4px #1115121f}.activity-stage-line{grid-template-columns:repeat(5,minmax(0,1fr));align-items:start;gap:6px;display:grid}.activity-stage-line span{color:#e8dfca57;min-width:0;font:700 var(--type-utility)/1 var(--mono);letter-spacing:.04em;text-overflow:ellipsis;text-transform:uppercase;white-space:nowrap;border-top:1px solid #e8dfca24;align-items:center;gap:5px;padding-top:5px;display:flex;overflow:hidden}.activity-stage-line i{border:1px solid;border-radius:50%;flex:0 0 6px;width:6px;height:6px}.activity-stage-line span.is-complete,.activity-stage-line span.is-current{color:var(--phosphor);border-color:#b9ff4585}.activity-stage-line span.is-complete i{background:var(--phosphor)}.activity-stage-line span.is-current i{box-shadow:0 0 0 4px var(--phosphor-dim)}.activity-status,.activity-status>div{align-items:center;display:flex}.activity-status{border-bottom:1px solid var(--line-light);min-width:0;font:var(--type-utility)/1 var(--mono);justify-content:space-between;gap:12px}.activity-status>div{gap:8px;min-width:0}.activity-status strong{color:var(--paper);letter-spacing:.04em;text-overflow:ellipsis;text-transform:uppercase;white-space:nowrap;font-weight:600;overflow:hidden}.activity-status>span{color:#e8dfca80;flex:none}.activity-pulse{background:#e8dfca59;border-radius:50%;flex:0 0 7px;width:7px;height:7px}.composer-activity.is-loading .activity-pulse,.composer-activity.is-inference .activity-pulse{background:var(--phosphor);box-shadow:0 0 0 4px var(--phosphor-dim);animation:1.15s ease-in-out infinite activity-pulse}.load-live-grid{grid-template-columns:minmax(0,1fr) minmax(0,1fr) auto;align-items:end;gap:12px;display:grid}.activity-track-block,.live-metric{gap:6px;min-width:0;display:grid}.activity-track-block>div,.live-metric>div{font:var(--type-utility)/1 var(--mono);justify-content:space-between;align-items:baseline;gap:8px;display:flex}.activity-track-block span,.live-metric span,.activity-memory span{color:#e8dfca75;text-transform:uppercase}.activity-track-block small,.live-metric small{color:#e8dfca80;font-size:inherit}.activity-memory{min-width:86px;font:var(--type-utility)/1 var(--mono);justify-items:end;gap:6px;display:grid}.activity-memory strong{color:var(--phosphor);font-size:var(--type-caption);font-weight:600}.activity-track{background:#e8dfca0e;border:1px solid #e8dfca2e;height:5px;overflow:hidden}.activity-track>span{background:repeating-linear-gradient(90deg, var(--phosphor) 0 7px, transparent 7px 9px);height:100%;transition:width .12s linear;display:block;box-shadow:0 0 10px #b9ff4557}.inference-live-grid{grid-template-columns:repeat(2,minmax(0,1fr));gap:12px;display:grid}.live-metric{border:1px solid var(--line-light);opacity:.64;padding:5px 8px 6px}.live-metric.is-active{opacity:1;background:#b9ff450f;border-color:#b9ff457a}.live-metric strong{color:var(--phosphor);font-size:var(--type-caption);margin-left:auto;font-weight:600}.shard-strip{grid-template-columns:repeat(8,minmax(0,1fr));align-items:end;gap:5px;min-width:0;display:grid}.shard-strip.is-empty{border-top:1px dashed var(--line-light);align-items:center;display:flex}.shard-placeholder{color:#e8dfca52;font:var(--type-utility)/1.2 var(--mono)}.shard-cell{opacity:.55;gap:5px;min-width:0;display:grid}.shard-cell.state-downloading,.shard-cell.state-verifying,.shard-cell.state-complete,.shard-cell.state-cached{opacity:1}.shard-cell>span{color:#e8dfca7a;font:var(--type-utility)/1 var(--mono);justify-content:space-between;gap:3px;display:flex}.shard-cell b{color:var(--paper);font-weight:600}.shard-cell small{font-size:inherit;text-overflow:ellipsis;overflow:hidden}.composer-meta{color:#11151275;min-height:24px;font:var(--type-utility)/1 var(--mono);justify-content:space-between;align-items:center;gap:12px;margin-bottom:7px;display:flex}.tool-toggle{color:#1115128a;min-height:32px;font:700 var(--type-utility)/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:0;align-items:center;gap:5px;padding:0 2px;display:flex}.tool-toggle.is-on{color:#425f1f}.regenerate-button{color:#1115128a;min-height:32px;font:700 var(--type-utility)/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:0;margin-right:auto;padding:0 2px}.regenerate-button:disabled{cursor:default;opacity:.45}.tool-toggle svg{fill:none;stroke:currentColor;stroke-width:1.3px;width:12px}.composer-field{border:1px solid var(--ink);background:#e8dfcad4;grid-template-columns:1fr auto;min-height:84px;display:grid;position:relative;box-shadow:4px 4px #1115121a}.composer-field label{color:#11151273;font:var(--type-utility)/1 var(--mono);letter-spacing:.09em;text-transform:uppercase;position:absolute;top:7px;left:11px}.composer-field textarea{resize:none;min-width:0;color:var(--ink);font:var(--type-body)/1.45 var(--body);background:0 0;border:0;outline:none;padding:23px 13px 9px}.composer-field textarea::placeholder{color:#11151257}.send-button{min-width:112px;min-height:52px;color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 var(--type-caption)/1 var(--mono);text-transform:uppercase;cursor:pointer;justify-content:center;align-items:center;gap:7px;margin:7px;transition:transform .15s,box-shadow .15s,background .15s;display:flex}.send-button:hover:not(:disabled){box-shadow:3px 3px 0 var(--ink);transform:translate(-2px,-2px)}.send-button:disabled{opacity:.34}.send-button svg{fill:none;stroke:currentColor;stroke-width:1.4px;width:15px}.send-button.stop{color:var(--paper);background:var(--ink)}.send-button.stop span{background:var(--rust);width:7px;height:7px}.send-button.load{background:var(--phosphor)}.inspector{border-left:1px solid var(--line-light);scrollbar-color:#e8dfca2e transparent;grid-area:inspector;height:calc(100dvh - 76px);min-height:calc(100dvh - 76px);animation:.57s cubic-bezier(.16,1,.3,1) .22s both rise-in;overflow-y:auto}.inspector-section{border-bottom:1px solid var(--line-light);padding:20px 18px}.section-heading{align-items:center;margin-bottom:17px}.section-heading.compact{margin-bottom:14px}.section-heading>div{flex:1}.section-heading h2{font-size:24px}.sampling-label{color:#e8dfca6b;font:var(--type-utility)/1 var(--mono);text-transform:uppercase;letter-spacing:.06em;align-items:center;gap:6px;display:flex}.sampling-label i{width:4px;height:4px}.speed-gauge{grid-template-columns:116px 1fr;align-items:center;gap:15px;display:grid}.gauge-ring{background:radial-gradient(circle, var(--ink) 0 51%, transparent 52%), conic-gradient(var(--phosphor) 0 var(--gauge), #e8dfca1c var(--gauge) 100%);border-radius:49% 51% 47% 53%;place-items:center;width:116px;height:116px;display:grid;transform:rotate(-9deg);box-shadow:inset 0 0 0 1px #e8dfca1a,0 0 30px #b9ff450e}.gauge-ring:after{content:"";border:1px dashed #e8dfca21;border-radius:45% 55% 52% 48%;width:128px;height:128px;position:absolute}.gauge-ring>div{z-index:1;justify-items:center;display:grid;position:relative;transform:rotate(9deg)}.gauge-ring strong{color:var(--phosphor);font:400 29px/1 var(--display)}.gauge-ring small{color:var(--paper-ghost);font:var(--type-utility)/1 var(--mono);text-transform:uppercase;margin-top:3px}.speed-gauge dl{margin:0}.speed-gauge dl div{border-bottom:1px solid var(--line-light);font:var(--type-utility)/1 var(--mono);justify-content:space-between;padding:7px 0;display:flex}.speed-gauge dt{color:#e8dfca6e}.speed-gauge dd{color:var(--paper);margin:0}.device-card{background:linear-gradient(90deg,#b9ff4514,#0000);border:1px solid #b9ff453b;grid-template-columns:auto 1fr auto;align-items:center;gap:10px;margin:19px 0 14px;padding:10px;display:grid}.device-icon{border:1px solid #b9ff4540;align-items:flex-end;gap:3px;width:28px;height:28px;padding:5px;display:flex}.device-icon i{background:var(--phosphor);width:4px}.device-icon i:first-child{opacity:.45;height:7px}.device-icon i:nth-child(2){height:15px}.device-icon i:nth-child(3){opacity:.7;height:11px}.device-card>div{gap:3px;display:grid}.device-card small,.memory-grid small,.meter small{color:#e8dfca6b;font:var(--type-utility)/1 var(--mono);text-transform:uppercase}.device-card strong{font:500 var(--type-caption)/1 var(--mono)}.device-card>span:last-child{color:var(--phosphor);font:600 15px/1 var(--mono);justify-items:end;display:grid}.device-card>span:last-child small{color:#e8dfca5e;font-size:var(--type-utility);margin-top:3px}.meter{margin-top:13px}.meter>div:first-child{font:var(--type-utility)/1 var(--mono);justify-content:space-between;margin-bottom:6px;display:flex}.meter-track{border:1px solid #e8dfca36;height:5px;padding:1px}.meter-track>span{background:repeating-linear-gradient(90deg, var(--phosphor) 0 5px, transparent 5px 7px);height:100%;display:block}.memory-grid{background:var(--line-light);grid-template-columns:1fr 1fr;gap:1px;margin-top:14px;display:grid}.memory-grid>div{background:var(--ink);gap:6px;padding:10px;display:grid}.memory-grid strong{font:500 var(--type-caption)/1 var(--mono)}.event-log{gap:0;margin:0;padding:0;list-style:none;display:grid;position:relative}.event-log:before{content:"";background:var(--line-light);width:1px;position:absolute;top:5px;bottom:5px;left:3px}.event-log li{grid-template-columns:8px 64px 1fr;align-items:start;gap:8px;padding:7px 0;display:grid}.event-log li.event-empty{color:#e8dfca5c;font:var(--type-utility)/1.4 var(--mono);padding:5px 0;display:block}.event-pip{z-index:1;border:1px solid var(--ink);margin-top:2px}.event-log time{color:#e8dfca57;font:var(--type-utility)/1.2 var(--mono)}.event-log div{gap:3px;display:grid}.event-log strong{font:500 var(--type-caption)/1 var(--mono)}.event-log small{color:#e8dfca6b;font-size:var(--type-utility)}.artifact-close{border:1px solid var(--line-light);cursor:pointer;background:0 0;border-radius:50%;width:25px;height:25px}.artifact-empty{color:#e8dfca66;font:var(--type-utility)/1.5 var(--mono);margin:0}.artifact-pane{border:1px solid var(--line-light);background:#f8f5eb}.artifact-tabs{color:var(--paper);border-bottom:1px solid var(--line-light);background:var(--ink-raised);grid-template-columns:auto auto 1fr;align-items:center;display:grid}.artifact-tabs button{border:0;border-right:1px solid var(--line-light);min-height:44px;font:var(--type-utility)/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;padding:7px 9px}.artifact-tabs button[aria-selected=true]{color:var(--ink);background:var(--phosphor)}.artifact-tabs span{color:#e8dfca73;font:var(--type-utility)/1 var(--mono);text-align:right;text-overflow:ellipsis;white-space:nowrap;padding:0 8px;overflow:hidden}.artifact-pane iframe{background:#fff;border:0;width:100%;height:260px;display:block}.artifact-running{position:relative}.artifact-running>button{z-index:2;color:var(--paper);border:1px solid var(--rust);min-height:44px;font:var(--type-utility)/1 var(--mono);text-transform:uppercase;cursor:pointer;background:#111512e6;padding:5px 7px;position:absolute;top:7px;right:7px}.artifact-launch{min-height:180px;color:var(--ink);text-align:center;align-content:center;justify-items:center;gap:13px;padding:20px;display:grid}.artifact-launch p{color:#1115129e;max-width:250px;font:var(--type-caption)/1.5 var(--mono);margin:0}.artifact-launch button{color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);min-height:44px;font:700 var(--type-utility)/1 var(--mono);text-transform:uppercase;cursor:pointer;padding:8px 10px}.artifact-code{height:260px;color:var(--paper);font:var(--type-utility)/1.5 var(--mono);white-space:pre-wrap;background:#0d100e;margin:0;padding:10px;overflow:auto}.settings-layer{z-index:50;-webkit-backdrop-filter:blur(4px);backdrop-filter:blur(4px);background:#080b09a3;justify-content:flex-end;animation:.16s both station-in;display:flex;position:fixed;inset:0}.settings-sheet{width:min(430px,100%);min-height:100%;color:var(--paper);border-left:1px solid var(--phosphor);background:repeating-radial-gradient(ellipse at 104% 101%, transparent 0 13px, #b9ff450e 13.5px 14.5px), var(--ink);padding:30px 27px;animation:.3s cubic-bezier(.16,1,.3,1) both sheet-in;overflow-y:auto;box-shadow:-24px 0 70px #00000052}.settings-sheet>header{border-bottom:1px solid var(--line-light);justify-content:space-between;align-items:flex-start;gap:20px;margin-bottom:37px;padding-bottom:20px;display:flex}.settings-sheet h2{font:400 31px/1 var(--display);margin:0}.sheet-close{border:1px solid var(--line-light);width:35px;height:35px;font:300 22px/1 var(--body);cursor:pointer;background:0 0;border-radius:50%}.sheet-close:hover{color:var(--phosphor);border-color:var(--phosphor)}.setting-group{border-bottom:1px solid var(--line-light);grid-template-columns:24px 1fr auto;align-items:center;gap:12px;padding:15px 0;display:grid}.setting-number{color:var(--phosphor);font:var(--type-utility)/1 var(--mono)}.setting-group>div,.range-setting>span:first-child{gap:5px;display:grid}.setting-group small,.range-setting small{color:#e8dfca6e;font:var(--type-utility)/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.setting-group strong,.range-setting strong{font:500 var(--type-caption)/1 var(--mono)}.setting-locked{color:#e8dfca5c;font:var(--type-utility)/1 var(--mono);text-transform:uppercase}.setting-select select{max-width:112px;color:var(--paper);border:1px solid var(--line-light);background:var(--ink-raised);min-height:44px;font:var(--type-utility)/1 var(--mono);border-radius:0;padding:6px 7px}.range-setting{gap:13px;margin-top:29px;display:grid}.range-setting>span:first-child{justify-content:space-between;align-items:end;display:flex}.range-setting input{appearance:none;background:linear-gradient(90deg, var(--phosphor) 0 35%, #e8dfca1c 35%);border:1px solid #e8dfca3d;border-radius:0;width:100%;height:4px;margin:0}.range-setting:first-of-type input{background:var(--phosphor)}.range-setting input::-webkit-slider-thumb{appearance:none;border:2px solid var(--ink);background:var(--phosphor);width:12px;height:18px;box-shadow:0 0 0 1px var(--phosphor);cursor:grab}.range-setting input::-moz-range-thumb{border:2px solid var(--ink);background:var(--phosphor);width:10px;height:16px;box-shadow:0 0 0 1px var(--phosphor);cursor:grab;border-radius:0}.range-ticks{pointer-events:none;justify-content:space-between;margin-top:-13px;display:flex}.range-limits{color:#e8dfca59;font:var(--type-utility)/1 var(--mono);justify-content:space-between;display:flex}.range-limits i{font-style:normal}.system-prompt-setting{gap:9px;margin-top:29px;display:grid}.system-prompt-setting>span{color:#e8dfca6e;font:var(--type-utility)/1 var(--mono);letter-spacing:.05em;text-transform:uppercase;justify-content:space-between;display:flex}.system-prompt-setting textarea{resize:vertical;color:var(--paper);border:1px solid var(--line-light);background:var(--ink-raised);font:var(--type-caption)/1.5 var(--mono);padding:10px}.settings-actions{grid-template-columns:1fr 1fr;gap:7px;margin-top:29px;display:grid}.settings-actions button{color:var(--paper);border:1px solid var(--line-light);min-height:44px;font:var(--type-utility)/1.2 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;padding:9px}.settings-actions button:hover{color:var(--phosphor);border-color:var(--phosphor)}.settings-actions .danger-action{color:#e78a62;border-color:#c96f4373;grid-column:span 2}.boot-screen{min-height:100dvh;color:var(--paper);background:var(--ink);justify-content:center;align-items:center;gap:18px;padding:24px;display:flex}.boot-screen h1{color:var(--phosphor);font:400 34px/1 var(--display);margin:0}.boot-screen p:last-child{color:var(--paper-ghost);font:var(--type-caption)/1.5 var(--mono)}.range-ticks i{background:#e8dfca40;width:1px;height:5px}.settings-note{background:#b9ff450f;border:1px solid #b9ff4540;align-items:flex-start;gap:13px;margin-top:42px;padding:15px;display:flex}.settings-note .signal-dot{flex:0 0 7px;margin-top:6px}.settings-note p{color:#e8dfca94;font-size:var(--type-caption);margin:0;line-height:1.5}.settings-note strong{color:var(--phosphor)}@keyframes station-in{0%{opacity:0}to{opacity:1}}@keyframes rise-in{0%{opacity:0;transform:translateY(11px)}to{opacity:1;transform:translateY(0)}}@keyframes note-in{0%{opacity:0;transform:translateY(7px)}to{opacity:1;transform:translateY(0)}}@keyframes cursor-blink{0%,52%{opacity:1}53%,to{opacity:0}}@keyframes activity-pulse{0%,to{opacity:.55;transform:scale(.82)}50%{opacity:1;transform:scale(1)}}@keyframes sheet-in{0%{opacity:0;transform:translate(34px)}to{opacity:1;transform:translate(0)}}@media (width<=1279px){.station-header{grid-template-columns:minmax(320px,1fr) auto auto}.local-promise{min-width:auto}.local-promise small{display:none}.workspace-grid{grid-template-columns:280px minmax(500px,1fr);grid-template-areas:"models chat""inspector inspector"}.model-rail,.chat-surface{height:calc(100dvh - 76px);min-height:calc(100dvh - 76px)}.inspector{border-top:1px solid var(--phosphor);border-left:0;grid-template-columns:repeat(2,minmax(0,1fr));height:auto;min-height:auto;display:grid;overflow:visible}.inspector-section{border-right:1px solid var(--line-light);border-bottom:0}}@media (width<=820px){.station-header{min-height:70px;display:block}.brand-block{padding:11px 70px 11px 15px}.brand-mark{flex-basis:36px;width:36px;height:36px}.local-promise{border:0;border-top:1px solid var(--line-light);min-height:31px;padding:0 16px}.header-actions{position:absolute;top:10px;right:0}.station-clock{display:none}.workspace-grid{display:block}.model-rail,.chat-surface,.inspector{height:auto;min-height:auto}.model-rail{border-right:0;border-bottom:1px solid var(--phosphor);padding:18px 15px;display:block}.model-list{scroll-snap-type:x mandatory;gap:7px;padding-bottom:7px;display:flex;overflow-x:auto}.model-card{scroll-snap-align:start;min-width:205px}.rail-footnote{display:none}.chat-surface{grid-template-rows:auto 19px minmax(240px,auto) auto;display:grid;overflow:visible}.transcript{padding-inline:22px;overflow:visible}.chat-header{padding:17px 19px}.chat-header-meta span:not(.live-label){display:none}.live-label{grid-column:1}.composer{z-index:5;padding-inline:14px;position:relative}.transcript-empty{margin-top:4vh}.inspector{border-top:1px solid var(--phosphor);display:block}.inspector-section{border-right:0;border-bottom:1px solid var(--line-light)}}@media (width<=520px){.brand-block h1{font-size:24px}.brand-block .eyebrow,.local-promise strong{font-size:var(--type-utility)}.chat-header{align-items:flex-end}.botanical-subtitle{margin:4px 0 0;display:block}.message-user{max-width:91%}.composer-meta>span{display:none}.composer-activity{grid-template-rows:24px 30px 44px 36px;height:176px;padding-inline:9px}.activity-stage-line{gap:3px}.activity-stage-line span{letter-spacing:0;gap:3px}.activity-stage-line i{display:none}.load-live-grid{gap:7px}.activity-memory{min-width:68px}.shard-strip{gap:3px}.shard-cell small{display:none}.composer-field{grid-template-columns:1fr;padding-bottom:62px}.send-button{height:52px;position:absolute;bottom:0;right:0}.speed-gauge{grid-template-columns:106px 1fr}.gauge-ring{width:102px;height:102px}.gauge-ring:after{width:113px;height:113px}}@media (prefers-reduced-motion:reduce){*,:before,:after{scroll-behavior:auto!important;transition-duration:.01ms!important;animation-duration:.01ms!important;animation-iteration-count:1!important}}.bench-page{min-height:100dvh;color:var(--ink);background:var(--paper)}.bench-masthead{min-height:86px;color:var(--paper);border-bottom:1px solid var(--phosphor);background:var(--ink);grid-template-columns:minmax(310px,1fr) minmax(360px,1.2fr) auto;align-items:stretch;display:grid}.bench-masthead .brand-block h1{color:var(--paper)}.bench-masthead-note{border-inline:1px solid var(--line-light);background:linear-gradient(110deg,#b9ff4517,#0000 72%);flex-direction:column;justify-content:center;gap:5px;padding:14px 28px;display:flex}.bench-masthead-note strong{color:var(--phosphor);font:700 9px/1 var(--mono);letter-spacing:.11em;text-transform:uppercase}.bench-masthead-note span{color:#e8dfca8c;max-width:520px;font:10px/1.35 var(--body)}.bench-return{color:var(--paper);font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;align-items:center;gap:8px;padding:0 25px;text-decoration:none;display:flex}.bench-return:hover{color:var(--phosphor)}.bench-layout{grid-template-columns:minmax(275px,330px) minmax(0,1fr);min-height:calc(100dvh - 86px);display:grid}.bench-controls{color:var(--paper);border-right:1px solid var(--phosphor);background:repeating-radial-gradient(ellipse at -12% 102%, transparent 0 24px, #b9ff450b 25px 26px), var(--ink);padding:28px 22px}.bench-controls h2{font:400 30px/1 var(--display);letter-spacing:-.03em;margin:0}.bench-control-intro{color:#e8dfca80;margin:9px 0 22px;font-size:11px;line-height:1.45}.bench-controls form{gap:15px;display:grid}.bench-field{gap:6px;min-width:0;display:grid}.bench-field>span{color:#e8dfcac7;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.bench-field select,.bench-field input{width:100%;min-width:0;height:39px;color:var(--paper);background:var(--ink-raised);font:10px/1 var(--mono);border:1px solid #e8dfca38;border-radius:0;outline:0;padding:0 10px}.bench-field input::placeholder{color:#e8dfca40}.bench-field select:disabled,.bench-field input:disabled{opacity:.5}.bench-field small{color:#e8dfca59;font:8px/1.2 var(--mono)}.bench-field-pair{grid-template-columns:repeat(2,minmax(0,1fr));gap:10px;display:grid}.bench-experimental{background:#c96f430d;border:1px dashed #c96f437a;padding:11px}.bench-experimental summary{color:#e19a76;font:700 8px/1 var(--mono);letter-spacing:.07em;text-transform:uppercase;cursor:pointer}.bench-experimental>p{color:#e8dfca75;margin:10px 0;font-size:9px;line-height:1.4}.bench-locks{color:#e8dfca70;font:7px/1.35 var(--mono);gap:5px;margin:10px 0 0;display:grid}.bench-locks div{justify-content:space-between;gap:8px;display:flex}.bench-locks dt,.bench-locks dd{margin:0}.bench-locks dt{color:#e8dfcab8}.bench-gate{border-left:2px solid var(--rust);background:#c96f4314;gap:5px;padding:10px;display:grid}.bench-gate.pass{border-color:var(--phosphor);background:var(--phosphor-dim)}.bench-gate strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.06em;text-transform:uppercase}.bench-gate.pass strong{color:var(--phosphor)}.bench-gate span{color:#e8dfca80;font:8px/1.4 var(--mono);overflow-wrap:anywhere}.bench-run-actions{grid-template-columns:1fr auto;gap:8px;display:grid}.bench-run-button,.bench-stop-button,.bench-section-heading button{border:1px solid var(--ink);background:var(--phosphor);min-height:38px;font:800 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;cursor:pointer;border-radius:0;padding:0 13px}.bench-run-button:disabled{opacity:.35}.bench-stop-button{color:#ffb08a;border-color:var(--rust);background:0 0}.bench-workbench{background:linear-gradient(90deg, #1115120b 1px, transparent 1px) 0 0 / 22px 22px, linear-gradient(#1115120a 1px, transparent 1px) 0 0 / 22px 22px, var(--paper);min-width:0;padding:clamp(26px,4vw,58px)}.bench-hero{border-bottom:1px solid var(--line-dark);grid-template-columns:minmax(320px,1.3fr) minmax(260px,.7fr);align-items:end;gap:34px;padding-bottom:29px;display:grid}.bench-hero h2{max-width:690px;font:400 clamp(42px, 6vw, 82px)/.84 var(--display);letter-spacing:-.065em;margin:0}.bench-hero h2 em{color:#4f7125;font-weight:400}.bench-hero>p{color:#11151294;max-width:430px;margin:0;font-size:11px;line-height:1.55}.bench-progress{gap:8px;padding:16px 0;display:grid}.bench-progress>div{font:8px/1 var(--mono);letter-spacing:.05em;text-transform:uppercase;justify-content:space-between;gap:20px;display:flex}.bench-progress>div span{color:#11151273}.bench-progress-track{background:#11151221;height:3px}.bench-progress-track i{background:#527426;height:100%;transition:width .18s;display:block}.bench-error{color:var(--paper);border-left:3px solid var(--rust);background:var(--ink);gap:5px;margin-bottom:18px;padding:12px 14px;display:grid}.bench-error strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.bench-error span{font:9px/1.4 var(--mono);overflow-wrap:anywhere}.bench-core-section{margin-top:13px}.bench-section-heading{justify-content:space-between;align-items:end;gap:20px;margin-bottom:13px;display:flex}.bench-section-heading h3,.bench-output h3{font:500 24px/1 var(--display);margin:0}.bench-section-heading button{min-height:31px;color:var(--paper);border-color:var(--ink);background:var(--ink)}.bench-core-sample{border:1px solid var(--ink);background:var(--paper-deep);grid-template-columns:repeat(4,minmax(0,1fr));display:grid;position:relative;overflow:hidden}.bench-core-sample:before{content:"";pointer-events:none;opacity:.35;background:repeating-radial-gradient(#0000 0 34px,#1115122b 35px 36px);position:absolute;inset:-130% -5%}.bench-core-sample article{background:#e8dfca91;border-right:1px solid #11151242;flex-direction:column;justify-content:space-between;min-height:128px;padding:17px;display:flex;position:relative}.bench-core-sample article:nth-child(2n){background:#d8cdb499}.bench-core-sample article:last-child{color:var(--paper);background:#111512e8;border-right:0}.bench-core-sample article>span{font:800 8px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase}.bench-core-sample article strong{font:400 clamp(28px, 3vw, 43px)/1 var(--display);letter-spacing:-.04em;margin-block:auto}.bench-core-sample article small{color:#11151280;font:7px/1.2 var(--mono);text-transform:uppercase}.bench-core-sample article:last-child small{color:#e8dfca7a}.bench-detail-grid{grid-template-columns:repeat(2,minmax(0,1fr));gap:12px;margin-top:12px;display:grid}.bench-data-card{border:1px solid var(--line-dark);background:#e8dfcaad;padding:16px}.bench-runtime-card,.bench-tuning-card{grid-column:span 2}.bench-data-card dl{grid-template-columns:repeat(2,minmax(0,1fr));gap:12px 26px;margin:0;display:grid}.bench-runtime-card dl{grid-template-columns:minmax(180px,1.5fr) repeat(4,minmax(100px,.7fr))}.bench-tuning-card dl{grid-template-columns:repeat(4,minmax(0,1fr))}.bench-data-card dl div{border-top:1px solid #11151221;gap:4px;min-width:0;padding-top:8px;display:grid}.bench-data-card dt{color:#11151273;font:7px/1 var(--mono);letter-spacing:.07em;text-transform:uppercase}.bench-data-card dd{min-width:0;font:500 16px/1.2 var(--body);text-overflow:ellipsis;white-space:nowrap;margin:0;overflow:hidden}.bench-output{border:1px dashed #1115124d;grid-template-columns:repeat(2,minmax(0,1fr));gap:34px;margin-top:12px;padding:21px;display:grid}.bench-output>div:last-child{border-left:1px solid var(--line-dark);padding-left:34px}.bench-output p:not(.eyebrow){color:#11151299;white-space:pre-wrap;margin:8px 0 0;font-size:11px;line-height:1.5}@media (width<=1050px){.bench-masthead{grid-template-columns:minmax(310px,1fr) auto}.bench-masthead-note{display:none}.bench-layout{grid-template-columns:285px minmax(0,1fr)}.bench-core-sample{grid-template-columns:repeat(2,minmax(0,1fr))}.bench-core-sample article:nth-child(2){border-right:0}.bench-core-sample article:nth-child(-n+2){border-bottom:1px solid #11151242}.bench-runtime-card dl{grid-template-columns:repeat(2,minmax(0,1fr))}}@media (width<=760px){.bench-masthead{align-items:center;display:flex}.bench-masthead .brand-block{flex:1;min-width:0}.bench-return{padding:0 15px;font-size:0}.bench-return span{font-size:17px}.bench-layout{display:block}.bench-controls{border-right:0;border-bottom:1px solid var(--phosphor)}.bench-workbench{padding:28px 16px}.bench-hero,.bench-output{grid-template-columns:1fr;gap:18px}.bench-hero h2{font-size:clamp(42px,14vw,68px)}.bench-detail-grid{display:block}.bench-data-card+.bench-data-card{margin-top:10px}.bench-output>div:last-child{border-top:1px solid var(--line-dark);border-left:0;padding-top:18px;padding-left:0}}@media (width<=480px){.bench-masthead .brand-mark{display:none}.bench-core-sample,.bench-data-card dl,.bench-runtime-card dl,.bench-tuning-card dl{grid-template-columns:1fr}.bench-core-sample article{border-bottom:1px solid #11151242;border-right:0;min-height:105px}.bench-core-sample article:last-child{border-bottom:0}}
|
dist/assets/{worker-CUYMhoYj.js → worker-Bn87aL54.js}
RENAMED
|
The diff for this file is too large to render.
See raw diff
|
|
|
dist/index.html
CHANGED
|
@@ -10,8 +10,8 @@
|
|
| 10 |
content="Bonsai local inference field station — private WebGPU language models in your browser."
|
| 11 |
/>
|
| 12 |
<title>Bonsai Field Station</title>
|
| 13 |
-
<script type="module" crossorigin src="/assets/index-
|
| 14 |
-
<link rel="stylesheet" crossorigin href="/assets/index-
|
| 15 |
</head>
|
| 16 |
<body>
|
| 17 |
<div id="root"></div>
|
|
|
|
| 10 |
content="Bonsai local inference field station — private WebGPU language models in your browser."
|
| 11 |
/>
|
| 12 |
<title>Bonsai Field Station</title>
|
| 13 |
+
<script type="module" crossorigin src="/assets/index-BY7HNxCd.js"></script>
|
| 14 |
+
<link rel="stylesheet" crossorigin href="/assets/index-DnMbdiUB.css">
|
| 15 |
</head>
|
| 16 |
<body>
|
| 17 |
<div id="root"></div>
|
dist/manifest/models.json
CHANGED
|
@@ -248,94 +248,5 @@
|
|
| 248 |
"requireSingleWebGPUGraph": true
|
| 249 |
}
|
| 250 |
}
|
| 251 |
-
]
|
| 252 |
-
"dflash": {
|
| 253 |
-
"status": "conditional",
|
| 254 |
-
"enabledByDefault": false,
|
| 255 |
-
"runtimeAvailable": false,
|
| 256 |
-
"source": {
|
| 257 |
-
"repo": "z-lab/Qwen3.6-27B-DFlash",
|
| 258 |
-
"revision": "0919688658996800f86b895034249700e9481106",
|
| 259 |
-
"file": "model.safetensors",
|
| 260 |
-
"bytes": 3460432504,
|
| 261 |
-
"sha256": "e0c050b34798d32728a164d2c3f1681746ff85c11945701b0205b654e2f1fdbe",
|
| 262 |
-
"format": "safetensors",
|
| 263 |
-
"dtype": "BF16",
|
| 264 |
-
"parameters": 1730213120,
|
| 265 |
-
"sourceOnly": true,
|
| 266 |
-
"license": "MIT"
|
| 267 |
-
},
|
| 268 |
-
"nativeReference": {
|
| 269 |
-
"path": "dflash/Qwen3.6-27B-DFlash-Q8_0.gguf",
|
| 270 |
-
"bytes": 1849481440,
|
| 271 |
-
"sha256": "80f04984d70c0989f4bc2c3b8ef34bc274226fce2723dc8a68e930ae46225fe9",
|
| 272 |
-
"format": "GGUF",
|
| 273 |
-
"quantization": "Q8_0",
|
| 274 |
-
"largestTensor": {
|
| 275 |
-
"name": "fc.weight",
|
| 276 |
-
"bytes": 139264000
|
| 277 |
-
},
|
| 278 |
-
"role": "native correctness reference only"
|
| 279 |
-
},
|
| 280 |
-
"browser": {
|
| 281 |
-
"enabled": false,
|
| 282 |
-
"architectureCompiled": true,
|
| 283 |
-
"specTypePlumbed": false,
|
| 284 |
-
"candidateQuantizationStatus": "q8-rejected-binding-limit"
|
| 285 |
-
},
|
| 286 |
-
"validation": {
|
| 287 |
-
"shape": "pass",
|
| 288 |
-
"tokenizer": "exact-match",
|
| 289 |
-
"targetTaps": [
|
| 290 |
-
2,
|
| 291 |
-
17,
|
| 292 |
-
32,
|
| 293 |
-
47,
|
| 294 |
-
62
|
| 295 |
-
],
|
| 296 |
-
"nativeGreedyParity": {
|
| 297 |
-
"status": "pass",
|
| 298 |
-
"tokens": 32,
|
| 299 |
-
"temperature": 0
|
| 300 |
-
},
|
| 301 |
-
"acceptanceBenchmark": "pending",
|
| 302 |
-
"browserSpeedGate": "pending"
|
| 303 |
-
},
|
| 304 |
-
"pairings": {
|
| 305 |
-
"1_7b": {
|
| 306 |
-
"support": "unsupported",
|
| 307 |
-
"reason": "No compatible DFlash drafter is locked and validated for the Bonsai 1.7B target."
|
| 308 |
-
},
|
| 309 |
-
"4b": {
|
| 310 |
-
"support": "unsupported",
|
| 311 |
-
"reason": "The pinned drafter is trained for Qwen3.6-27B (hidden size 5120, mask token 248070, and 64 target layers) and is not compatible with Bonsai 4B."
|
| 312 |
-
},
|
| 313 |
-
"8b": {
|
| 314 |
-
"support": "unsupported",
|
| 315 |
-
"reason": "The pinned drafter is trained for Qwen3.6-27B (hidden size 5120, mask token 248070, and 64 target layers) and is not compatible with Bonsai 8B."
|
| 316 |
-
},
|
| 317 |
-
"27b": {
|
| 318 |
-
"support": "conditional",
|
| 319 |
-
"compatibilityStatus": "native-greedy-parity-pass",
|
| 320 |
-
"runtimeAvailable": false,
|
| 321 |
-
"blockSize": 16,
|
| 322 |
-
"maskTokenId": 248070,
|
| 323 |
-
"targetLayerIdsZeroBased": [
|
| 324 |
-
1,
|
| 325 |
-
16,
|
| 326 |
-
31,
|
| 327 |
-
46,
|
| 328 |
-
61
|
| 329 |
-
],
|
| 330 |
-
"targetTapsOneBased": [
|
| 331 |
-
2,
|
| 332 |
-
17,
|
| 333 |
-
32,
|
| 334 |
-
47,
|
| 335 |
-
62
|
| 336 |
-
],
|
| 337 |
-
"reason": "The target/drafter contract and 32-token native greedy parity pass. Browser runtime stays disabled because spec_type=draft-dflash is not plumbed, the Q8 fc.weight exceeds the 128 MiB baseline binding limit, and acceptance/speed are unmeasured."
|
| 338 |
-
}
|
| 339 |
-
}
|
| 340 |
-
}
|
| 341 |
}
|
|
|
|
| 248 |
"requireSingleWebGPUGraph": true
|
| 249 |
}
|
| 250 |
}
|
| 251 |
+
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 252 |
}
|
dist/wasm/wllama-compat.wasm
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c823684229149dd2386308978288e366f2ff209b231b78485507db8cb735bbdd
|
| 3 |
+
size 14871898
|
dist/wasm/wllama.wasm
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5a8686c10f7639653e7e1854d34f39c31ddba404de876a5ae15d36c42263ddc4
|
| 3 |
+
size 8020809
|
public/manifest/models.json
CHANGED
|
@@ -248,94 +248,5 @@
|
|
| 248 |
"requireSingleWebGPUGraph": true
|
| 249 |
}
|
| 250 |
}
|
| 251 |
-
]
|
| 252 |
-
"dflash": {
|
| 253 |
-
"status": "conditional",
|
| 254 |
-
"enabledByDefault": false,
|
| 255 |
-
"runtimeAvailable": false,
|
| 256 |
-
"source": {
|
| 257 |
-
"repo": "z-lab/Qwen3.6-27B-DFlash",
|
| 258 |
-
"revision": "0919688658996800f86b895034249700e9481106",
|
| 259 |
-
"file": "model.safetensors",
|
| 260 |
-
"bytes": 3460432504,
|
| 261 |
-
"sha256": "e0c050b34798d32728a164d2c3f1681746ff85c11945701b0205b654e2f1fdbe",
|
| 262 |
-
"format": "safetensors",
|
| 263 |
-
"dtype": "BF16",
|
| 264 |
-
"parameters": 1730213120,
|
| 265 |
-
"sourceOnly": true,
|
| 266 |
-
"license": "MIT"
|
| 267 |
-
},
|
| 268 |
-
"nativeReference": {
|
| 269 |
-
"path": "dflash/Qwen3.6-27B-DFlash-Q8_0.gguf",
|
| 270 |
-
"bytes": 1849481440,
|
| 271 |
-
"sha256": "80f04984d70c0989f4bc2c3b8ef34bc274226fce2723dc8a68e930ae46225fe9",
|
| 272 |
-
"format": "GGUF",
|
| 273 |
-
"quantization": "Q8_0",
|
| 274 |
-
"largestTensor": {
|
| 275 |
-
"name": "fc.weight",
|
| 276 |
-
"bytes": 139264000
|
| 277 |
-
},
|
| 278 |
-
"role": "native correctness reference only"
|
| 279 |
-
},
|
| 280 |
-
"browser": {
|
| 281 |
-
"enabled": false,
|
| 282 |
-
"architectureCompiled": true,
|
| 283 |
-
"specTypePlumbed": false,
|
| 284 |
-
"candidateQuantizationStatus": "q8-rejected-binding-limit"
|
| 285 |
-
},
|
| 286 |
-
"validation": {
|
| 287 |
-
"shape": "pass",
|
| 288 |
-
"tokenizer": "exact-match",
|
| 289 |
-
"targetTaps": [
|
| 290 |
-
2,
|
| 291 |
-
17,
|
| 292 |
-
32,
|
| 293 |
-
47,
|
| 294 |
-
62
|
| 295 |
-
],
|
| 296 |
-
"nativeGreedyParity": {
|
| 297 |
-
"status": "pass",
|
| 298 |
-
"tokens": 32,
|
| 299 |
-
"temperature": 0
|
| 300 |
-
},
|
| 301 |
-
"acceptanceBenchmark": "pending",
|
| 302 |
-
"browserSpeedGate": "pending"
|
| 303 |
-
},
|
| 304 |
-
"pairings": {
|
| 305 |
-
"1_7b": {
|
| 306 |
-
"support": "unsupported",
|
| 307 |
-
"reason": "No compatible DFlash drafter is locked and validated for the Bonsai 1.7B target."
|
| 308 |
-
},
|
| 309 |
-
"4b": {
|
| 310 |
-
"support": "unsupported",
|
| 311 |
-
"reason": "The pinned drafter is trained for Qwen3.6-27B (hidden size 5120, mask token 248070, and 64 target layers) and is not compatible with Bonsai 4B."
|
| 312 |
-
},
|
| 313 |
-
"8b": {
|
| 314 |
-
"support": "unsupported",
|
| 315 |
-
"reason": "The pinned drafter is trained for Qwen3.6-27B (hidden size 5120, mask token 248070, and 64 target layers) and is not compatible with Bonsai 8B."
|
| 316 |
-
},
|
| 317 |
-
"27b": {
|
| 318 |
-
"support": "conditional",
|
| 319 |
-
"compatibilityStatus": "native-greedy-parity-pass",
|
| 320 |
-
"runtimeAvailable": false,
|
| 321 |
-
"blockSize": 16,
|
| 322 |
-
"maskTokenId": 248070,
|
| 323 |
-
"targetLayerIdsZeroBased": [
|
| 324 |
-
1,
|
| 325 |
-
16,
|
| 326 |
-
31,
|
| 327 |
-
46,
|
| 328 |
-
61
|
| 329 |
-
],
|
| 330 |
-
"targetTapsOneBased": [
|
| 331 |
-
2,
|
| 332 |
-
17,
|
| 333 |
-
32,
|
| 334 |
-
47,
|
| 335 |
-
62
|
| 336 |
-
],
|
| 337 |
-
"reason": "The target/drafter contract and 32-token native greedy parity pass. Browser runtime stays disabled because spec_type=draft-dflash is not plumbed, the Q8 fc.weight exceeds the 128 MiB baseline binding limit, and acceptance/speed are unmeasured."
|
| 338 |
-
}
|
| 339 |
-
}
|
| 340 |
-
}
|
| 341 |
}
|
|
|
|
| 248 |
"requireSingleWebGPUGraph": true
|
| 249 |
}
|
| 250 |
}
|
| 251 |
+
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 252 |
}
|
public/wasm/wllama-compat.wasm
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c823684229149dd2386308978288e366f2ff209b231b78485507db8cb735bbdd
|
| 3 |
+
size 14871898
|
public/wasm/wllama.wasm
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5a8686c10f7639653e7e1854d34f39c31ddba404de876a5ae15d36c42263ddc4
|
| 3 |
+
size 8020809
|
scripts/verify-wllama-assets.mjs
CHANGED
|
@@ -48,6 +48,15 @@ const oaiDeclarations = await readFile(resolve(root, 'vendor/wllama-bonsai/esm/t
|
|
| 48 |
if (!oaiDeclarations.includes('export interface ChatCompletionLogprob {\n id: number;')) {
|
| 49 |
throw new Error('Vendored Bonsai wllama declarations lack sampled token ids');
|
| 50 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
const moduleSource = await readFile(resolve(root, 'vendor/wllama-bonsai/esm/index.js'), 'utf8');
|
| 52 |
if (!moduleSource.includes('n_ubatch: params.n_ubatch')) {
|
| 53 |
throw new Error('Vendored Bonsai wllama module does not forward n_ubatch');
|
|
|
|
| 48 |
if (!oaiDeclarations.includes('export interface ChatCompletionLogprob {\n id: number;')) {
|
| 49 |
throw new Error('Vendored Bonsai wllama declarations lack sampled token ids');
|
| 50 |
}
|
| 51 |
+
if (!oaiDeclarations.includes('return_progress?: boolean;')) {
|
| 52 |
+
throw new Error('Vendored Bonsai wllama declarations lack prompt-progress opt-in');
|
| 53 |
+
}
|
| 54 |
+
if (!oaiDeclarations.includes('export interface PromptProgress')) {
|
| 55 |
+
throw new Error('Vendored Bonsai wllama declarations lack prompt-progress payloads');
|
| 56 |
+
}
|
| 57 |
+
if (!oaiDeclarations.includes('prompt_progress?: PromptProgress;')) {
|
| 58 |
+
throw new Error('Vendored Bonsai wllama declarations do not expose streamed prompt progress');
|
| 59 |
+
}
|
| 60 |
const moduleSource = await readFile(resolve(root, 'vendor/wllama-bonsai/esm/index.js'), 'utf8');
|
| 61 |
if (!moduleSource.includes('n_ubatch: params.n_ubatch')) {
|
| 62 |
throw new Error('Vendored Bonsai wllama module does not forward n_ubatch');
|
src/app/App.test.ts
CHANGED
|
@@ -2,6 +2,7 @@ import { describe, expect, it } from 'vitest';
|
|
| 2 |
import { EngineClientError } from '../engine';
|
| 3 |
import {
|
| 4 |
DEFAULT_MODEL_ID,
|
|
|
|
| 5 |
reconcileCompletionTokens,
|
| 6 |
reconcileTotalTokens,
|
| 7 |
reportsModelWeightProgress,
|
|
@@ -58,3 +59,24 @@ describe('App streamed completion telemetry', () => {
|
|
| 58 |
expect(reconcileTotalTokens(80, 5, 64)).toBe(80);
|
| 59 |
});
|
| 60 |
});
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
import { EngineClientError } from '../engine';
|
| 3 |
import {
|
| 4 |
DEFAULT_MODEL_ID,
|
| 5 |
+
publishBackendReport,
|
| 6 |
reconcileCompletionTokens,
|
| 7 |
reconcileTotalTokens,
|
| 8 |
reportsModelWeightProgress,
|
|
|
|
| 59 |
expect(reconcileTotalTokens(80, 5, 64)).toBe(80);
|
| 60 |
});
|
| 61 |
});
|
| 62 |
+
|
| 63 |
+
describe('App backend diagnostics', () => {
|
| 64 |
+
it('publishes the complete backend report for the browser gate without rendering it', () => {
|
| 65 |
+
const report = {
|
| 66 |
+
backends: ['WebGPU'],
|
| 67 |
+
nGraphSplits: 1,
|
| 68 |
+
opsOnCpu: 0,
|
| 69 |
+
layersGpu: { offloaded: 65, total: 65 },
|
| 70 |
+
flashAttention: false,
|
| 71 |
+
cacheTypeK: 'f16',
|
| 72 |
+
cacheTypeV: 'f16',
|
| 73 |
+
webgpuKvBufferBytes: 1024,
|
| 74 |
+
webgpuTrace: ['@@WEBGPU_TRACE@@completion_end id=1 steps=65 error=0'],
|
| 75 |
+
};
|
| 76 |
+
|
| 77 |
+
publishBackendReport(report);
|
| 78 |
+
|
| 79 |
+
expect(globalThis.__bonsaiBackendReport).toEqual(report);
|
| 80 |
+
delete globalThis.__bonsaiBackendReport;
|
| 81 |
+
});
|
| 82 |
+
});
|
src/app/App.tsx
CHANGED
|
@@ -6,6 +6,7 @@ import {
|
|
| 6 |
isShardDownloadFailureDetails,
|
| 7 |
loadModelManifestV2,
|
| 8 |
type EngineCapabilities,
|
|
|
|
| 9 |
type EngineChatMessage,
|
| 10 |
type EngineChatToolCall,
|
| 11 |
type EngineEvent,
|
|
@@ -18,7 +19,7 @@ import { AGENT_TOOLS, executeAgentTool, type ArtifactDocument } from '../lib/age
|
|
| 18 |
import type {
|
| 19 |
ChatMessage,
|
| 20 |
ComposerSubmission,
|
| 21 |
-
|
| 22 |
ModelId,
|
| 23 |
ModelOption,
|
| 24 |
RuntimeSettings,
|
|
@@ -100,6 +101,12 @@ const INITIAL_TELEMETRY: TelemetrySnapshot = {
|
|
| 100 |
contextUsed: 0,
|
| 101 |
contextLimit: 4096,
|
| 102 |
tokensPerSecond: 0,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
timeToFirstTokenMs: 0,
|
| 104 |
gpuLayers: '—',
|
| 105 |
graphSplits: null,
|
|
@@ -112,6 +119,18 @@ const INITIAL_STORAGE: StorageStatus = {
|
|
| 112 |
downloads: [],
|
| 113 |
};
|
| 114 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
let sharedClient: BrowserEngineClient | null = null;
|
| 116 |
|
| 117 |
function getEngineClient(): BrowserEngineClient {
|
|
@@ -189,6 +208,10 @@ export function reconcileTotalTokens(
|
|
| 189 |
return Math.max(totalCount, promptCount + completionTokens);
|
| 190 |
}
|
| 191 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 192 |
function appendSegment(current: string, next: string): string {
|
| 193 |
const trimmed = next.trim();
|
| 194 |
if (!trimmed) return current;
|
|
@@ -244,24 +267,6 @@ function fallbackToolCalls(text: string): EngineChatToolCall[] {
|
|
| 244 |
}));
|
| 245 |
}
|
| 246 |
|
| 247 |
-
function dflashState(manifest: ModelManifestV2 | null, tier: ModelTierId): DFlashState {
|
| 248 |
-
if (!manifest) {
|
| 249 |
-
return {
|
| 250 |
-
enabled: false,
|
| 251 |
-
available: false,
|
| 252 |
-
status: 'unsupported',
|
| 253 |
-
reason: 'No DFlash audit is present in the pinned manifest.',
|
| 254 |
-
};
|
| 255 |
-
}
|
| 256 |
-
const pairing = manifest.dflash.pairings[tier];
|
| 257 |
-
return {
|
| 258 |
-
enabled: false,
|
| 259 |
-
available: false,
|
| 260 |
-
status: pairing.support,
|
| 261 |
-
reason: pairing.reason,
|
| 262 |
-
};
|
| 263 |
-
}
|
| 264 |
-
|
| 265 |
export function App() {
|
| 266 |
const [client] = useState(getEngineClient);
|
| 267 |
const [manifest, setManifest] = useState<ModelManifestV2 | null>(null);
|
|
@@ -271,6 +276,7 @@ export function App() {
|
|
| 271 |
const [loadingModelId, setLoadingModelId] = useState<ModelId | null>(null);
|
| 272 |
const [messages, setMessages] = useState<ChatMessage[]>([]);
|
| 273 |
const [streamState, setStreamState] = useState<StreamState>('idle');
|
|
|
|
| 274 |
const [toolsEnabled, setToolsEnabled] = useState(false);
|
| 275 |
const [settings, setSettings] = useState<RuntimeSettings>(INITIAL_SETTINGS);
|
| 276 |
const [settingsOpen, setSettingsOpen] = useState(false);
|
|
@@ -284,6 +290,61 @@ export function App() {
|
|
| 284 |
const operationRef = useRef<AbortController | null>(null);
|
| 285 |
const loadedConfigurationRef = useRef<{ id: ModelId; backend: RequestedBackend; contextSize: number } | null>(null);
|
| 286 |
const historyReadyRef = useRef(false);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 287 |
|
| 288 |
const handleModelInvalidation = (runtimeError: unknown): boolean => {
|
| 289 |
if (!requiresModelReload(runtimeError)) return false;
|
|
@@ -295,6 +356,13 @@ export function App() {
|
|
| 295 |
modelMemoryBytes: 0,
|
| 296 |
kvMemoryBytes: 0,
|
| 297 |
contextUsed: 0,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 298 |
gpuLayers: '—',
|
| 299 |
graphSplits: null,
|
| 300 |
}));
|
|
@@ -306,6 +374,15 @@ export function App() {
|
|
| 306 |
};
|
| 307 |
const persistenceAttemptedRef = useRef(false);
|
| 308 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 309 |
useEffect(() => {
|
| 310 |
let cancelled = false;
|
| 311 |
const bootstrap = async () => {
|
|
@@ -413,14 +490,12 @@ export function App() {
|
|
| 413 |
capability: decor.capability,
|
| 414 |
contextLimit: model.contextLength,
|
| 415 |
defaultContext: model.defaultContext,
|
| 416 |
-
dflash: manifest.dflash.pairings[model.id].support,
|
| 417 |
...(limitReason ? { limitReason } : {}),
|
| 418 |
};
|
| 419 |
});
|
| 420 |
}, [capabilities, loadedModelId, loadingModelId, manifest, settings.backend, settings.contextSize, storage.downloads]);
|
| 421 |
|
| 422 |
const activeTier = UI_TO_TIER[activeModelId];
|
| 423 |
-
const dflash = useMemo(() => dflashState(manifest, activeTier), [activeTier, manifest]);
|
| 424 |
|
| 425 |
const refreshStorage = async (): Promise<void> => {
|
| 426 |
const estimate = await client.storageEstimate();
|
|
@@ -473,6 +548,15 @@ export function App() {
|
|
| 473 |
setError(null);
|
| 474 |
setShardRetry(null);
|
| 475 |
setBootStatus(`Preparing ${model.displayName}…`);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 476 |
try {
|
| 477 |
const result = await client.loadModel({
|
| 478 |
manifestUrl: manifestUrl(),
|
|
@@ -482,26 +566,51 @@ export function App() {
|
|
| 482 |
}, {
|
| 483 |
signal: controller.signal,
|
| 484 |
onProgress: (progress) => {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 485 |
if (!reportsModelWeightProgress(progress.phase)) {
|
| 486 |
setBootStatus('Reading pinned model manifest…');
|
| 487 |
return;
|
| 488 |
}
|
| 489 |
-
const
|
| 490 |
-
|
| 491 |
-
|
| 492 |
-
|
| 493 |
-
|
| 494 |
-
|
| 495 |
-
|
| 496 |
-
|
| 497 |
-
|
| 498 |
-
|
| 499 |
-
|
| 500 |
-
|
| 501 |
-
|
| 502 |
-
|
| 503 |
-
|
| 504 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 505 |
},
|
| 506 |
});
|
| 507 |
setLoadedModelId(modelId);
|
|
@@ -513,7 +622,8 @@ export function App() {
|
|
| 513 |
device: result.backend === 'webgpu'
|
| 514 |
? capabilities?.webgpu.name || 'WebGPU adapter'
|
| 515 |
: `CPU · ${capabilities?.hardwareConcurrency ?? navigator.hardwareConcurrency ?? 1} logical cores`,
|
| 516 |
-
modelMemoryBytes: model.downloadBytes,
|
|
|
|
| 517 |
contextUsed: 0,
|
| 518 |
contextLimit: result.context.size,
|
| 519 |
gpuLayers: result.backendReport.layersGpu
|
|
@@ -521,6 +631,17 @@ export function App() {
|
|
| 521 |
: result.backend === 'wasm' ? 'CPU' : 'unreported',
|
| 522 |
graphSplits: result.backendReport.nGraphSplits,
|
| 523 |
}));
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 524 |
setBootStatus(`${model.displayName} loaded · ${result.backend.toUpperCase()} · ${result.context.size.toLocaleString()} ctx`);
|
| 525 |
await refreshStorage();
|
| 526 |
} catch (loadError) {
|
|
@@ -539,6 +660,11 @@ export function App() {
|
|
| 539 |
error: errorMessage(loadError),
|
| 540 |
});
|
| 541 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 542 |
setStorage((current) => ({
|
| 543 |
...current,
|
| 544 |
downloads: operatorAborted
|
|
@@ -621,6 +747,22 @@ export function App() {
|
|
| 621 |
baseMessages: ChatMessage[],
|
| 622 |
) => {
|
| 623 |
if (operationRef.current || !manifest) return;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 624 |
const controller = new AbortController();
|
| 625 |
operationRef.current = controller;
|
| 626 |
const nonce = crypto.randomUUID();
|
|
@@ -645,37 +787,41 @@ export function App() {
|
|
| 645 |
setError(null);
|
| 646 |
|
| 647 |
const run = async () => {
|
| 648 |
-
const tier = UI_TO_TIER[activeModelId];
|
| 649 |
-
const model = manifest.models.find((candidate) => candidate.id === tier);
|
| 650 |
-
if (!model) throw new Error(`Model ${tier} is absent from the pinned manifest.`);
|
| 651 |
-
const requestedContext = targetContext(model);
|
| 652 |
-
const configured = loadedConfigurationRef.current;
|
| 653 |
-
const needsLoad = loadedModelId !== activeModelId
|
| 654 |
-
|| configured?.backend !== settings.backend
|
| 655 |
-
|| configured.contextSize !== requestedContext;
|
| 656 |
-
if (needsLoad) {
|
| 657 |
-
await requestPersistentStorage();
|
| 658 |
-
if (controller.signal.aborted) throw new DOMException('Model loading was aborted.', 'AbortError');
|
| 659 |
-
await loadModel(activeModelId, controller, requestedContext);
|
| 660 |
-
}
|
| 661 |
setStreamState('streaming');
|
| 662 |
-
setBootStatus(`Generating with ${
|
| 663 |
|
| 664 |
const engineMessages = buildEngineHistory(historySnapshot, settings.systemPrompt);
|
| 665 |
let visibleText = '';
|
| 666 |
let reasoningText = '';
|
| 667 |
let totalCompletionTokens = 0;
|
|
|
|
| 668 |
let lastTotalTokens = 0;
|
| 669 |
let tokensPerSecond = 0;
|
|
|
|
|
|
|
|
|
|
| 670 |
let firstTokenAt = 0;
|
| 671 |
const generationStartedAt = performance.now();
|
| 672 |
let toolRounds = 0;
|
| 673 |
let totalToolCalls = 0;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 674 |
|
| 675 |
while (true) {
|
| 676 |
let streamedText = '';
|
| 677 |
let streamedReasoning = '';
|
| 678 |
let streamedTokenEvents = 0;
|
|
|
|
| 679 |
const result = await client.generate({
|
| 680 |
messages: engineMessages,
|
| 681 |
maxTokens: settings.maxTokens,
|
|
@@ -686,6 +832,12 @@ export function App() {
|
|
| 686 |
cachePrompt: true,
|
| 687 |
}, {
|
| 688 |
signal: controller.signal,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 689 |
onToken: (delta, reasoningDelta) => {
|
| 690 |
if (!firstTokenAt && (delta || reasoningDelta)) firstTokenAt = performance.now();
|
| 691 |
if (delta || reasoningDelta) streamedTokenEvents += 1;
|
|
@@ -708,6 +860,8 @@ export function App() {
|
|
| 708 |
result.usage?.completionTokens,
|
| 709 |
streamedTokenEvents,
|
| 710 |
);
|
|
|
|
|
|
|
| 711 |
totalCompletionTokens += roundCompletionTokens;
|
| 712 |
lastTotalTokens = Math.max(
|
| 713 |
lastTotalTokens,
|
|
@@ -717,7 +871,14 @@ export function App() {
|
|
| 717 |
roundCompletionTokens,
|
| 718 |
),
|
| 719 |
);
|
| 720 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 721 |
const thought = parseThoughtStream(result.text);
|
| 722 |
const textual = parseToolCalls(thought.content);
|
| 723 |
if (textual.incomplete || textual.errors.length > 0) {
|
|
@@ -791,11 +952,14 @@ export function App() {
|
|
| 791 |
}
|
| 792 |
|
| 793 |
const generationElapsedSeconds = Math.max(0.001, (performance.now() - generationStartedAt) / 1000);
|
| 794 |
-
|
| 795 |
-
|
| 796 |
-
|
|
|
|
| 797 |
const ttft = firstTokenAt ? Math.round(firstTokenAt - generationStartedAt) : 0;
|
| 798 |
const report = await client.backendReport();
|
|
|
|
|
|
|
| 799 |
replaceMessage(setMessages, assistantId, (message) => ({
|
| 800 |
...message,
|
| 801 |
state: 'complete',
|
|
@@ -803,6 +967,7 @@ export function App() {
|
|
| 803 |
metrics: {
|
| 804 |
tokens: totalCompletionTokens,
|
| 805 |
tokensPerSecond,
|
|
|
|
| 806 |
timeToFirstTokenMs: ttft,
|
| 807 |
},
|
| 808 |
}));
|
|
@@ -810,6 +975,12 @@ export function App() {
|
|
| 810 |
...current,
|
| 811 |
contextUsed: Math.min(current.contextLimit, Math.max(lastTotalTokens, totalCompletionTokens)),
|
| 812 |
tokensPerSecond,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 813 |
timeToFirstTokenMs: ttft,
|
| 814 |
graphSplits: report.nGraphSplits,
|
| 815 |
gpuLayers: report.layersGpu
|
|
@@ -817,12 +988,13 @@ export function App() {
|
|
| 817 |
: current.gpuLayers,
|
| 818 |
}));
|
| 819 |
setStreamState('complete');
|
| 820 |
-
setBootStatus(`${
|
| 821 |
await refreshStorage();
|
| 822 |
};
|
| 823 |
|
| 824 |
void run()
|
| 825 |
.catch((runError) => {
|
|
|
|
| 826 |
if (handleModelInvalidation(runError)) {
|
| 827 |
const message = errorMessage(runError);
|
| 828 |
replaceMessage(setMessages, assistantId, (current) => ({
|
|
@@ -837,6 +1009,7 @@ export function App() {
|
|
| 837 |
if (isAborted(runError, controller.signal)) {
|
| 838 |
replaceMessage(setMessages, assistantId, (message) => ({ ...message, state: 'complete', isThinking: false }));
|
| 839 |
setStreamState('complete');
|
|
|
|
| 840 |
setBootStatus('Generation stopped by operator');
|
| 841 |
return;
|
| 842 |
}
|
|
@@ -849,6 +1022,7 @@ export function App() {
|
|
| 849 |
}));
|
| 850 |
setError(message);
|
| 851 |
setStreamState('error');
|
|
|
|
| 852 |
setBootStatus('Runtime stopped safely');
|
| 853 |
})
|
| 854 |
.finally(() => {
|
|
@@ -944,10 +1118,10 @@ export function App() {
|
|
| 944 |
loadedModelId={effectiveLoadedModelId}
|
| 945 |
messages={messages}
|
| 946 |
streamState={streamState}
|
|
|
|
| 947 |
telemetry={telemetry}
|
| 948 |
toolEvents={toolEvents}
|
| 949 |
storage={storage}
|
| 950 |
-
dflash={dflash}
|
| 951 |
artifact={artifact}
|
| 952 |
toolsEnabled={toolsEnabled}
|
| 953 |
settings={settings}
|
|
@@ -961,7 +1135,6 @@ export function App() {
|
|
| 961 |
onRegenerate={handleRegenerate}
|
| 962 |
onCancel={handleCancel}
|
| 963 |
onToolsEnabledChange={setToolsEnabled}
|
| 964 |
-
onDFlashChange={() => undefined}
|
| 965 |
onOpenSettings={() => { if (!operationRef.current) setSettingsOpen(true); }}
|
| 966 |
onCloseSettings={() => setSettingsOpen(false)}
|
| 967 |
onSettingsChange={(nextSettings) => { if (!operationRef.current) setSettings(nextSettings); }}
|
|
@@ -971,7 +1144,6 @@ export function App() {
|
|
| 971 |
onDismissError={() => setError(null)}
|
| 972 |
onRetryShard={handleLoadModel}
|
| 973 |
onCloseArtifact={() => setArtifact(null)}
|
| 974 |
-
onRequestDownload={handleLoadModel}
|
| 975 |
/>
|
| 976 |
);
|
| 977 |
}
|
|
|
|
| 6 |
isShardDownloadFailureDetails,
|
| 7 |
loadModelManifestV2,
|
| 8 |
type EngineCapabilities,
|
| 9 |
+
type BackendReport,
|
| 10 |
type EngineChatMessage,
|
| 11 |
type EngineChatToolCall,
|
| 12 |
type EngineEvent,
|
|
|
|
| 19 |
import type {
|
| 20 |
ChatMessage,
|
| 21 |
ComposerSubmission,
|
| 22 |
+
ModelLoadProgress,
|
| 23 |
ModelId,
|
| 24 |
ModelOption,
|
| 25 |
RuntimeSettings,
|
|
|
|
| 101 |
contextUsed: 0,
|
| 102 |
contextLimit: 4096,
|
| 103 |
tokensPerSecond: 0,
|
| 104 |
+
prefillTokensPerSecond: 0,
|
| 105 |
+
decodeTokensPerSecond: 0,
|
| 106 |
+
inferencePhase: 'idle',
|
| 107 |
+
promptProcessed: 0,
|
| 108 |
+
promptTotal: 0,
|
| 109 |
+
completionTokens: 0,
|
| 110 |
timeToFirstTokenMs: 0,
|
| 111 |
gpuLayers: '—',
|
| 112 |
graphSplits: null,
|
|
|
|
| 119 |
downloads: [],
|
| 120 |
};
|
| 121 |
|
| 122 |
+
const INITIAL_MODEL_LOAD_PROGRESS: ModelLoadProgress = {
|
| 123 |
+
modelId: null,
|
| 124 |
+
modelLabel: '',
|
| 125 |
+
stage: 'idle',
|
| 126 |
+
stageProgress: 0,
|
| 127 |
+
downloadedBytes: 0,
|
| 128 |
+
totalBytes: 0,
|
| 129 |
+
residentBytes: 0,
|
| 130 |
+
nativeStage: null,
|
| 131 |
+
shards: [],
|
| 132 |
+
};
|
| 133 |
+
|
| 134 |
let sharedClient: BrowserEngineClient | null = null;
|
| 135 |
|
| 136 |
function getEngineClient(): BrowserEngineClient {
|
|
|
|
| 208 |
return Math.max(totalCount, promptCount + completionTokens);
|
| 209 |
}
|
| 210 |
|
| 211 |
+
export function publishBackendReport(report: BackendReport): void {
|
| 212 |
+
globalThis.__bonsaiBackendReport = report;
|
| 213 |
+
}
|
| 214 |
+
|
| 215 |
function appendSegment(current: string, next: string): string {
|
| 216 |
const trimmed = next.trim();
|
| 217 |
if (!trimmed) return current;
|
|
|
|
| 267 |
}));
|
| 268 |
}
|
| 269 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 270 |
export function App() {
|
| 271 |
const [client] = useState(getEngineClient);
|
| 272 |
const [manifest, setManifest] = useState<ModelManifestV2 | null>(null);
|
|
|
|
| 276 |
const [loadingModelId, setLoadingModelId] = useState<ModelId | null>(null);
|
| 277 |
const [messages, setMessages] = useState<ChatMessage[]>([]);
|
| 278 |
const [streamState, setStreamState] = useState<StreamState>('idle');
|
| 279 |
+
const [modelLoadProgress, setModelLoadProgress] = useState<ModelLoadProgress>(INITIAL_MODEL_LOAD_PROGRESS);
|
| 280 |
const [toolsEnabled, setToolsEnabled] = useState(false);
|
| 281 |
const [settings, setSettings] = useState<RuntimeSettings>(INITIAL_SETTINGS);
|
| 282 |
const [settingsOpen, setSettingsOpen] = useState(false);
|
|
|
|
| 290 |
const operationRef = useRef<AbortController | null>(null);
|
| 291 |
const loadedConfigurationRef = useRef<{ id: ModelId; backend: RequestedBackend; contextSize: number } | null>(null);
|
| 292 |
const historyReadyRef = useRef(false);
|
| 293 |
+
const loadProgressFrameRef = useRef<number | null>(null);
|
| 294 |
+
const pendingLoadProgressRef = useRef<ModelLoadProgress | null>(null);
|
| 295 |
+
const modelLoadProgressRef = useRef<ModelLoadProgress>(INITIAL_MODEL_LOAD_PROGRESS);
|
| 296 |
+
const inferenceFrameRef = useRef<number | null>(null);
|
| 297 |
+
const pendingInferenceRef = useRef<Extract<EngineEvent, { event: 'generation' }> | null>(null);
|
| 298 |
+
|
| 299 |
+
const publishModelLoadProgress = (next: ModelLoadProgress): void => {
|
| 300 |
+
modelLoadProgressRef.current = next;
|
| 301 |
+
pendingLoadProgressRef.current = next;
|
| 302 |
+
if (loadProgressFrameRef.current !== null) return;
|
| 303 |
+
loadProgressFrameRef.current = window.requestAnimationFrame(() => {
|
| 304 |
+
loadProgressFrameRef.current = null;
|
| 305 |
+
const pending = pendingLoadProgressRef.current;
|
| 306 |
+
pendingLoadProgressRef.current = null;
|
| 307 |
+
if (pending) setModelLoadProgress(pending);
|
| 308 |
+
});
|
| 309 |
+
};
|
| 310 |
+
|
| 311 |
+
const publishInferenceProgress = (
|
| 312 |
+
next: Extract<EngineEvent, { event: 'generation' }>,
|
| 313 |
+
): void => {
|
| 314 |
+
pendingInferenceRef.current = next;
|
| 315 |
+
if (inferenceFrameRef.current !== null) return;
|
| 316 |
+
inferenceFrameRef.current = window.requestAnimationFrame(() => {
|
| 317 |
+
inferenceFrameRef.current = null;
|
| 318 |
+
const pending = pendingInferenceRef.current;
|
| 319 |
+
pendingInferenceRef.current = null;
|
| 320 |
+
if (!pending) return;
|
| 321 |
+
const liveRate = pending.phase === 'prefill'
|
| 322 |
+
? pending.promptTokensPerSecond
|
| 323 |
+
: pending.decodeTokensPerSecond;
|
| 324 |
+
setTelemetry((current) => ({
|
| 325 |
+
...current,
|
| 326 |
+
inferencePhase: pending.phase,
|
| 327 |
+
tokensPerSecond: liveRate,
|
| 328 |
+
prefillTokensPerSecond: pending.promptTokensPerSecond || current.prefillTokensPerSecond,
|
| 329 |
+
decodeTokensPerSecond: pending.decodeTokensPerSecond || current.decodeTokensPerSecond,
|
| 330 |
+
promptProcessed: pending.promptProcessed,
|
| 331 |
+
promptTotal: pending.promptTotal,
|
| 332 |
+
completionTokens: pending.completionTokens,
|
| 333 |
+
contextUsed: Math.min(
|
| 334 |
+
current.contextLimit,
|
| 335 |
+
Math.max(current.contextUsed, pending.promptProcessed + pending.completionTokens),
|
| 336 |
+
),
|
| 337 |
+
}));
|
| 338 |
+
});
|
| 339 |
+
};
|
| 340 |
+
|
| 341 |
+
const clearPendingInferenceProgress = (): void => {
|
| 342 |
+
if (inferenceFrameRef.current !== null) {
|
| 343 |
+
window.cancelAnimationFrame(inferenceFrameRef.current);
|
| 344 |
+
inferenceFrameRef.current = null;
|
| 345 |
+
}
|
| 346 |
+
pendingInferenceRef.current = null;
|
| 347 |
+
};
|
| 348 |
|
| 349 |
const handleModelInvalidation = (runtimeError: unknown): boolean => {
|
| 350 |
if (!requiresModelReload(runtimeError)) return false;
|
|
|
|
| 356 |
modelMemoryBytes: 0,
|
| 357 |
kvMemoryBytes: 0,
|
| 358 |
contextUsed: 0,
|
| 359 |
+
tokensPerSecond: 0,
|
| 360 |
+
prefillTokensPerSecond: 0,
|
| 361 |
+
decodeTokensPerSecond: 0,
|
| 362 |
+
inferencePhase: 'idle',
|
| 363 |
+
promptProcessed: 0,
|
| 364 |
+
promptTotal: 0,
|
| 365 |
+
completionTokens: 0,
|
| 366 |
gpuLayers: '—',
|
| 367 |
graphSplits: null,
|
| 368 |
}));
|
|
|
|
| 374 |
};
|
| 375 |
const persistenceAttemptedRef = useRef(false);
|
| 376 |
|
| 377 |
+
useEffect(() => () => {
|
| 378 |
+
if (loadProgressFrameRef.current !== null) {
|
| 379 |
+
window.cancelAnimationFrame(loadProgressFrameRef.current);
|
| 380 |
+
}
|
| 381 |
+
if (inferenceFrameRef.current !== null) {
|
| 382 |
+
window.cancelAnimationFrame(inferenceFrameRef.current);
|
| 383 |
+
}
|
| 384 |
+
}, []);
|
| 385 |
+
|
| 386 |
useEffect(() => {
|
| 387 |
let cancelled = false;
|
| 388 |
const bootstrap = async () => {
|
|
|
|
| 490 |
capability: decor.capability,
|
| 491 |
contextLimit: model.contextLength,
|
| 492 |
defaultContext: model.defaultContext,
|
|
|
|
| 493 |
...(limitReason ? { limitReason } : {}),
|
| 494 |
};
|
| 495 |
});
|
| 496 |
}, [capabilities, loadedModelId, loadingModelId, manifest, settings.backend, settings.contextSize, storage.downloads]);
|
| 497 |
|
| 498 |
const activeTier = UI_TO_TIER[activeModelId];
|
|
|
|
| 499 |
|
| 500 |
const refreshStorage = async (): Promise<void> => {
|
| 501 |
const estimate = await client.storageEstimate();
|
|
|
|
| 548 |
setError(null);
|
| 549 |
setShardRetry(null);
|
| 550 |
setBootStatus(`Preparing ${model.displayName}…`);
|
| 551 |
+
const initialProgress: ModelLoadProgress = {
|
| 552 |
+
...INITIAL_MODEL_LOAD_PROGRESS,
|
| 553 |
+
modelId,
|
| 554 |
+
modelLabel: model.displayName,
|
| 555 |
+
stage: 'manifest',
|
| 556 |
+
totalBytes: model.downloadBytes,
|
| 557 |
+
};
|
| 558 |
+
modelLoadProgressRef.current = initialProgress;
|
| 559 |
+
setModelLoadProgress(initialProgress);
|
| 560 |
try {
|
| 561 |
const result = await client.loadModel({
|
| 562 |
manifestUrl: manifestUrl(),
|
|
|
|
| 566 |
}, {
|
| 567 |
signal: controller.signal,
|
| 568 |
onProgress: (progress) => {
|
| 569 |
+
const received = Math.min(progress.loadedBytes, model.downloadBytes);
|
| 570 |
+
const shard = progress.shardIndex === null ? '' : ` · shard ${progress.shardIndex + 1}/${progress.shardCount}`;
|
| 571 |
+
const stageProgress = progress.stageProgress
|
| 572 |
+
?? (progress.totalBytes > 0 ? progress.loadedBytes / progress.totalBytes : 0);
|
| 573 |
+
publishModelLoadProgress({
|
| 574 |
+
modelId,
|
| 575 |
+
modelLabel: model.displayName,
|
| 576 |
+
stage: progress.phase,
|
| 577 |
+
stageProgress: Math.min(1, Math.max(0, stageProgress)),
|
| 578 |
+
downloadedBytes: received,
|
| 579 |
+
totalBytes: model.downloadBytes,
|
| 580 |
+
residentBytes: progress.residentBytes ?? 0,
|
| 581 |
+
nativeStage: progress.nativeStage,
|
| 582 |
+
shards: progress.shards.map((item) => ({ ...item })),
|
| 583 |
+
});
|
| 584 |
+
if (progress.residentBytes !== null) {
|
| 585 |
+
setTelemetry((current) => ({ ...current, modelMemoryBytes: progress.residentBytes ?? 0 }));
|
| 586 |
+
}
|
| 587 |
if (!reportsModelWeightProgress(progress.phase)) {
|
| 588 |
setBootStatus('Reading pinned model manifest…');
|
| 589 |
return;
|
| 590 |
}
|
| 591 |
+
const stageLabel = progress.phase === 'cache'
|
| 592 |
+
? 'Inspecting cache'
|
| 593 |
+
: progress.phase === 'verify'
|
| 594 |
+
? 'Verifying shards'
|
| 595 |
+
: progress.phase === 'load'
|
| 596 |
+
? 'Allocating model'
|
| 597 |
+
: 'Downloading model';
|
| 598 |
+
setBootStatus(`${stageLabel}${shard} · ${Math.round(stageProgress * 100)}%`);
|
| 599 |
+
if (progress.phase !== 'load') {
|
| 600 |
+
setStorage((current) => ({
|
| 601 |
+
...current,
|
| 602 |
+
downloads: [
|
| 603 |
+
...current.downloads.filter((download) => download.modelId !== modelId),
|
| 604 |
+
{
|
| 605 |
+
modelId,
|
| 606 |
+
label: model.displayName,
|
| 607 |
+
receivedBytes: received,
|
| 608 |
+
totalBytes: model.downloadBytes,
|
| 609 |
+
state: received >= model.downloadBytes ? 'complete' : 'downloading',
|
| 610 |
+
},
|
| 611 |
+
],
|
| 612 |
+
}));
|
| 613 |
+
}
|
| 614 |
},
|
| 615 |
});
|
| 616 |
setLoadedModelId(modelId);
|
|
|
|
| 622 |
device: result.backend === 'webgpu'
|
| 623 |
? capabilities?.webgpu.name || 'WebGPU adapter'
|
| 624 |
: `CPU · ${capabilities?.hardwareConcurrency ?? navigator.hardwareConcurrency ?? 1} logical cores`,
|
| 625 |
+
modelMemoryBytes: result.backendReport.modelBufferBytes ?? model.downloadBytes,
|
| 626 |
+
kvMemoryBytes: result.backendReport.webgpuKvBufferBytes ?? 0,
|
| 627 |
contextUsed: 0,
|
| 628 |
contextLimit: result.context.size,
|
| 629 |
gpuLayers: result.backendReport.layersGpu
|
|
|
|
| 631 |
: result.backend === 'wasm' ? 'CPU' : 'unreported',
|
| 632 |
graphSplits: result.backendReport.nGraphSplits,
|
| 633 |
}));
|
| 634 |
+
publishModelLoadProgress({
|
| 635 |
+
...modelLoadProgressRef.current,
|
| 636 |
+
modelId,
|
| 637 |
+
modelLabel: model.displayName,
|
| 638 |
+
stage: 'ready',
|
| 639 |
+
stageProgress: 1,
|
| 640 |
+
downloadedBytes: model.downloadBytes,
|
| 641 |
+
totalBytes: model.downloadBytes,
|
| 642 |
+
residentBytes: result.backendReport.allocatedBufferBytes ?? result.backendReport.modelBufferBytes ?? model.downloadBytes,
|
| 643 |
+
nativeStage: null,
|
| 644 |
+
});
|
| 645 |
setBootStatus(`${model.displayName} loaded · ${result.backend.toUpperCase()} · ${result.context.size.toLocaleString()} ctx`);
|
| 646 |
await refreshStorage();
|
| 647 |
} catch (loadError) {
|
|
|
|
| 660 |
error: errorMessage(loadError),
|
| 661 |
});
|
| 662 |
}
|
| 663 |
+
publishModelLoadProgress({
|
| 664 |
+
...modelLoadProgressRef.current,
|
| 665 |
+
stage: operatorAborted ? 'idle' : 'error',
|
| 666 |
+
nativeStage: null,
|
| 667 |
+
});
|
| 668 |
setStorage((current) => ({
|
| 669 |
...current,
|
| 670 |
downloads: operatorAborted
|
|
|
|
| 747 |
baseMessages: ChatMessage[],
|
| 748 |
) => {
|
| 749 |
if (operationRef.current || !manifest) return;
|
| 750 |
+
const tier = UI_TO_TIER[activeModelId];
|
| 751 |
+
const selectedModel = manifest.models.find((candidate) => candidate.id === tier);
|
| 752 |
+
if (!selectedModel) {
|
| 753 |
+
setError(`Model ${tier} is absent from the pinned manifest.`);
|
| 754 |
+
return;
|
| 755 |
+
}
|
| 756 |
+
const requestedContext = targetContext(selectedModel);
|
| 757 |
+
const configured = loadedConfigurationRef.current;
|
| 758 |
+
const needsLoad = loadedModelId !== activeModelId
|
| 759 |
+
|| configured?.backend !== settings.backend
|
| 760 |
+
|| configured.contextSize !== requestedContext;
|
| 761 |
+
if (needsLoad) {
|
| 762 |
+
setError(`Load ${selectedModel.displayName} before running this prompt.`);
|
| 763 |
+
setBootStatus('Explicit model load required');
|
| 764 |
+
return;
|
| 765 |
+
}
|
| 766 |
const controller = new AbortController();
|
| 767 |
operationRef.current = controller;
|
| 768 |
const nonce = crypto.randomUUID();
|
|
|
|
| 787 |
setError(null);
|
| 788 |
|
| 789 |
const run = async () => {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 790 |
setStreamState('streaming');
|
| 791 |
+
setBootStatus(`Generating with ${selectedModel.displayName}…`);
|
| 792 |
|
| 793 |
const engineMessages = buildEngineHistory(historySnapshot, settings.systemPrompt);
|
| 794 |
let visibleText = '';
|
| 795 |
let reasoningText = '';
|
| 796 |
let totalCompletionTokens = 0;
|
| 797 |
+
let totalPromptTokens = 0;
|
| 798 |
let lastTotalTokens = 0;
|
| 799 |
let tokensPerSecond = 0;
|
| 800 |
+
let prefillTokensPerSecond = 0;
|
| 801 |
+
let totalPromptSeconds = 0;
|
| 802 |
+
let totalDecodeSeconds = 0;
|
| 803 |
let firstTokenAt = 0;
|
| 804 |
const generationStartedAt = performance.now();
|
| 805 |
let toolRounds = 0;
|
| 806 |
let totalToolCalls = 0;
|
| 807 |
+
clearPendingInferenceProgress();
|
| 808 |
+
setTelemetry((current) => ({
|
| 809 |
+
...current,
|
| 810 |
+
inferencePhase: 'prefill',
|
| 811 |
+
tokensPerSecond: 0,
|
| 812 |
+
prefillTokensPerSecond: 0,
|
| 813 |
+
decodeTokensPerSecond: 0,
|
| 814 |
+
promptProcessed: 0,
|
| 815 |
+
promptTotal: 0,
|
| 816 |
+
completionTokens: 0,
|
| 817 |
+
timeToFirstTokenMs: 0,
|
| 818 |
+
}));
|
| 819 |
|
| 820 |
while (true) {
|
| 821 |
let streamedText = '';
|
| 822 |
let streamedReasoning = '';
|
| 823 |
let streamedTokenEvents = 0;
|
| 824 |
+
const completionOffset = totalCompletionTokens;
|
| 825 |
const result = await client.generate({
|
| 826 |
messages: engineMessages,
|
| 827 |
maxTokens: settings.maxTokens,
|
|
|
|
| 832 |
cachePrompt: true,
|
| 833 |
}, {
|
| 834 |
signal: controller.signal,
|
| 835 |
+
onGenerationProgress: (progress) => {
|
| 836 |
+
publishInferenceProgress({
|
| 837 |
+
...progress,
|
| 838 |
+
completionTokens: completionOffset + progress.completionTokens,
|
| 839 |
+
});
|
| 840 |
+
},
|
| 841 |
onToken: (delta, reasoningDelta) => {
|
| 842 |
if (!firstTokenAt && (delta || reasoningDelta)) firstTokenAt = performance.now();
|
| 843 |
if (delta || reasoningDelta) streamedTokenEvents += 1;
|
|
|
|
| 860 |
result.usage?.completionTokens,
|
| 861 |
streamedTokenEvents,
|
| 862 |
);
|
| 863 |
+
const roundPromptTokens = result.usage?.promptTokens ?? 0;
|
| 864 |
+
totalPromptTokens += roundPromptTokens;
|
| 865 |
totalCompletionTokens += roundCompletionTokens;
|
| 866 |
lastTotalTokens = Math.max(
|
| 867 |
lastTotalTokens,
|
|
|
|
| 871 |
roundCompletionTokens,
|
| 872 |
),
|
| 873 |
);
|
| 874 |
+
const roundPrefillRate = result.timings?.promptTokensPerSecond ?? 0;
|
| 875 |
+
const roundDecodeRate = result.timings?.predictedTokensPerSecond ?? 0;
|
| 876 |
+
if (roundPromptTokens > 0 && roundPrefillRate > 0) {
|
| 877 |
+
totalPromptSeconds += roundPromptTokens / roundPrefillRate;
|
| 878 |
+
}
|
| 879 |
+
if (roundCompletionTokens > 0 && roundDecodeRate > 0) {
|
| 880 |
+
totalDecodeSeconds += roundCompletionTokens / roundDecodeRate;
|
| 881 |
+
}
|
| 882 |
const thought = parseThoughtStream(result.text);
|
| 883 |
const textual = parseToolCalls(thought.content);
|
| 884 |
if (textual.incomplete || textual.errors.length > 0) {
|
|
|
|
| 952 |
}
|
| 953 |
|
| 954 |
const generationElapsedSeconds = Math.max(0.001, (performance.now() - generationStartedAt) / 1000);
|
| 955 |
+
tokensPerSecond = totalDecodeSeconds > 0
|
| 956 |
+
? totalCompletionTokens / totalDecodeSeconds
|
| 957 |
+
: totalCompletionTokens / generationElapsedSeconds;
|
| 958 |
+
prefillTokensPerSecond = totalPromptSeconds > 0 ? totalPromptTokens / totalPromptSeconds : 0;
|
| 959 |
const ttft = firstTokenAt ? Math.round(firstTokenAt - generationStartedAt) : 0;
|
| 960 |
const report = await client.backendReport();
|
| 961 |
+
publishBackendReport(report);
|
| 962 |
+
clearPendingInferenceProgress();
|
| 963 |
replaceMessage(setMessages, assistantId, (message) => ({
|
| 964 |
...message,
|
| 965 |
state: 'complete',
|
|
|
|
| 967 |
metrics: {
|
| 968 |
tokens: totalCompletionTokens,
|
| 969 |
tokensPerSecond,
|
| 970 |
+
promptTokensPerSecond: prefillTokensPerSecond,
|
| 971 |
timeToFirstTokenMs: ttft,
|
| 972 |
},
|
| 973 |
}));
|
|
|
|
| 975 |
...current,
|
| 976 |
contextUsed: Math.min(current.contextLimit, Math.max(lastTotalTokens, totalCompletionTokens)),
|
| 977 |
tokensPerSecond,
|
| 978 |
+
prefillTokensPerSecond,
|
| 979 |
+
decodeTokensPerSecond: tokensPerSecond,
|
| 980 |
+
inferencePhase: 'complete',
|
| 981 |
+
promptProcessed: totalPromptTokens,
|
| 982 |
+
promptTotal: totalPromptTokens,
|
| 983 |
+
completionTokens: totalCompletionTokens,
|
| 984 |
timeToFirstTokenMs: ttft,
|
| 985 |
graphSplits: report.nGraphSplits,
|
| 986 |
gpuLayers: report.layersGpu
|
|
|
|
| 988 |
: current.gpuLayers,
|
| 989 |
}));
|
| 990 |
setStreamState('complete');
|
| 991 |
+
setBootStatus(`${selectedModel.displayName} complete · ${tokensPerSecond.toFixed(1)} tok/s`);
|
| 992 |
await refreshStorage();
|
| 993 |
};
|
| 994 |
|
| 995 |
void run()
|
| 996 |
.catch((runError) => {
|
| 997 |
+
clearPendingInferenceProgress();
|
| 998 |
if (handleModelInvalidation(runError)) {
|
| 999 |
const message = errorMessage(runError);
|
| 1000 |
replaceMessage(setMessages, assistantId, (current) => ({
|
|
|
|
| 1009 |
if (isAborted(runError, controller.signal)) {
|
| 1010 |
replaceMessage(setMessages, assistantId, (message) => ({ ...message, state: 'complete', isThinking: false }));
|
| 1011 |
setStreamState('complete');
|
| 1012 |
+
setTelemetry((current) => ({ ...current, inferencePhase: 'idle' }));
|
| 1013 |
setBootStatus('Generation stopped by operator');
|
| 1014 |
return;
|
| 1015 |
}
|
|
|
|
| 1022 |
}));
|
| 1023 |
setError(message);
|
| 1024 |
setStreamState('error');
|
| 1025 |
+
setTelemetry((current) => ({ ...current, inferencePhase: 'idle' }));
|
| 1026 |
setBootStatus('Runtime stopped safely');
|
| 1027 |
})
|
| 1028 |
.finally(() => {
|
|
|
|
| 1118 |
loadedModelId={effectiveLoadedModelId}
|
| 1119 |
messages={messages}
|
| 1120 |
streamState={streamState}
|
| 1121 |
+
modelLoadProgress={modelLoadProgress}
|
| 1122 |
telemetry={telemetry}
|
| 1123 |
toolEvents={toolEvents}
|
| 1124 |
storage={storage}
|
|
|
|
| 1125 |
artifact={artifact}
|
| 1126 |
toolsEnabled={toolsEnabled}
|
| 1127 |
settings={settings}
|
|
|
|
| 1135 |
onRegenerate={handleRegenerate}
|
| 1136 |
onCancel={handleCancel}
|
| 1137 |
onToolsEnabledChange={setToolsEnabled}
|
|
|
|
| 1138 |
onOpenSettings={() => { if (!operationRef.current) setSettingsOpen(true); }}
|
| 1139 |
onCloseSettings={() => setSettingsOpen(false)}
|
| 1140 |
onSettingsChange={(nextSettings) => { if (!operationRef.current) setSettings(nextSettings); }}
|
|
|
|
| 1144 |
onDismissError={() => setError(null)}
|
| 1145 |
onRetryShard={handleLoadModel}
|
| 1146 |
onCloseArtifact={() => setArtifact(null)}
|
|
|
|
| 1147 |
/>
|
| 1148 |
);
|
| 1149 |
}
|
src/app/components/ArtifactPane.tsx
CHANGED
|
@@ -25,7 +25,7 @@ export function ArtifactPane({ artifact, onClose }: ArtifactPaneProps) {
|
|
| 25 |
return (
|
| 26 |
<section className="inspector-section artifact-section" aria-labelledby="artifact-title">
|
| 27 |
<div className="section-heading compact">
|
| 28 |
-
<span>
|
| 29 |
<div>
|
| 30 |
<p className="eyebrow">Isolated terrarium</p>
|
| 31 |
<h2 id="artifact-title">Artifact</h2>
|
|
|
|
| 25 |
return (
|
| 26 |
<section className="inspector-section artifact-section" aria-labelledby="artifact-title">
|
| 27 |
<div className="section-heading compact">
|
| 28 |
+
<span>04</span>
|
| 29 |
<div>
|
| 30 |
<p className="eyebrow">Isolated terrarium</p>
|
| 31 |
<h2 id="artifact-title">Artifact</h2>
|
src/app/components/BonsaiShell.tsx
CHANGED
|
@@ -20,17 +20,16 @@ export function BonsaiShell(props: BonsaiShellProps) {
|
|
| 20 |
<ModelRail
|
| 21 |
models={props.models}
|
| 22 |
activeModelId={props.activeModelId}
|
| 23 |
-
dflash={props.dflash}
|
| 24 |
busy={busy}
|
| 25 |
onModelSelect={props.onModelSelect}
|
| 26 |
-
onDFlashChange={props.onDFlashChange}
|
| 27 |
-
onRequestDownload={props.onRequestDownload}
|
| 28 |
/>
|
| 29 |
<ChatSurface
|
| 30 |
activeModel={activeModel}
|
| 31 |
loadedModelId={props.loadedModelId}
|
| 32 |
messages={props.messages}
|
| 33 |
streamState={props.streamState}
|
|
|
|
|
|
|
| 34 |
toolsEnabled={props.toolsEnabled}
|
| 35 |
runtimeStatus={props.runtimeStatus}
|
| 36 |
error={props.error}
|
|
@@ -46,7 +45,6 @@ export function BonsaiShell(props: BonsaiShellProps) {
|
|
| 46 |
<InspectorPanel
|
| 47 |
telemetry={props.telemetry}
|
| 48 |
toolEvents={props.toolEvents}
|
| 49 |
-
storage={props.storage}
|
| 50 |
artifact={props.artifact}
|
| 51 |
onCloseArtifact={props.onCloseArtifact}
|
| 52 |
/>
|
|
|
|
| 20 |
<ModelRail
|
| 21 |
models={props.models}
|
| 22 |
activeModelId={props.activeModelId}
|
|
|
|
| 23 |
busy={busy}
|
| 24 |
onModelSelect={props.onModelSelect}
|
|
|
|
|
|
|
| 25 |
/>
|
| 26 |
<ChatSurface
|
| 27 |
activeModel={activeModel}
|
| 28 |
loadedModelId={props.loadedModelId}
|
| 29 |
messages={props.messages}
|
| 30 |
streamState={props.streamState}
|
| 31 |
+
modelLoadProgress={props.modelLoadProgress}
|
| 32 |
+
telemetry={props.telemetry}
|
| 33 |
toolsEnabled={props.toolsEnabled}
|
| 34 |
runtimeStatus={props.runtimeStatus}
|
| 35 |
error={props.error}
|
|
|
|
| 45 |
<InspectorPanel
|
| 46 |
telemetry={props.telemetry}
|
| 47 |
toolEvents={props.toolEvents}
|
|
|
|
| 48 |
artifact={props.artifact}
|
| 49 |
onCloseArtifact={props.onCloseArtifact}
|
| 50 |
/>
|
src/app/components/ChatSurface.tsx
CHANGED
|
@@ -2,12 +2,15 @@ import { useMemo, useState, type FormEvent, type KeyboardEvent } from 'react';
|
|
| 2 |
import type {
|
| 3 |
ChatMessage,
|
| 4 |
ComposerSubmission,
|
|
|
|
| 5 |
ModelId,
|
| 6 |
ModelOption,
|
| 7 |
ShardRetryNotice,
|
| 8 |
StreamState,
|
|
|
|
| 9 |
ToolRun,
|
| 10 |
} from '../../lib/contracts';
|
|
|
|
| 11 |
import { MarkdownContent } from './MarkdownContent';
|
| 12 |
|
| 13 |
interface ChatSurfaceProps {
|
|
@@ -15,6 +18,8 @@ interface ChatSurfaceProps {
|
|
| 15 |
loadedModelId: ModelId | null;
|
| 16 |
messages: ChatMessage[];
|
| 17 |
streamState: StreamState;
|
|
|
|
|
|
|
| 18 |
toolsEnabled: boolean;
|
| 19 |
runtimeStatus: string;
|
| 20 |
error: string | null;
|
|
@@ -33,6 +38,8 @@ export function ChatSurface({
|
|
| 33 |
loadedModelId,
|
| 34 |
messages,
|
| 35 |
streamState,
|
|
|
|
|
|
|
| 36 |
toolsEnabled,
|
| 37 |
runtimeStatus,
|
| 38 |
error,
|
|
@@ -46,7 +53,9 @@ export function ChatSurface({
|
|
| 46 |
onToolsEnabledChange,
|
| 47 |
}: ChatSurfaceProps) {
|
| 48 |
const [prompt, setPrompt] = useState('');
|
| 49 |
-
const
|
|
|
|
|
|
|
| 50 |
const isLoaded = loadedModelId === activeModel.id;
|
| 51 |
const logDate = useMemo(() => new Intl.DateTimeFormat('en-GB', {
|
| 52 |
day: '2-digit', month: 'long', year: 'numeric',
|
|
@@ -54,7 +63,7 @@ export function ChatSurface({
|
|
| 54 |
|
| 55 |
const submit = () => {
|
| 56 |
const trimmed = prompt.trim();
|
| 57 |
-
if (!trimmed || isBusy) return;
|
| 58 |
onSend({ prompt: trimmed, toolsEnabled });
|
| 59 |
setPrompt('');
|
| 60 |
};
|
|
@@ -83,11 +92,6 @@ export function ChatSurface({
|
|
| 83 |
<span>{activeModel.capability} canopy</span>
|
| 84 |
<span>{Math.round(activeModel.contextLimit / 1024)}K context</span>
|
| 85 |
<span className={`live-label${isLoaded ? '' : ' is-dormant'}`}><i /> {isLoaded ? 'local graph' : 'not loaded'}</span>
|
| 86 |
-
{!isLoaded && (
|
| 87 |
-
<button className="header-load-button" type="button" disabled={isBusy} onClick={() => onLoadModel(activeModel.id)}>
|
| 88 |
-
Load locally
|
| 89 |
-
</button>
|
| 90 |
-
)}
|
| 91 |
</div>
|
| 92 |
</header>
|
| 93 |
|
|
@@ -129,6 +133,13 @@ export function ChatSurface({
|
|
| 129 |
</section>
|
| 130 |
|
| 131 |
<form className="composer" onSubmit={handleSubmit}>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 132 |
<div className="composer-meta">
|
| 133 |
<button
|
| 134 |
className={`tool-toggle${toolsEnabled ? ' is-on' : ''}`}
|
|
@@ -139,7 +150,7 @@ export function ChatSurface({
|
|
| 139 |
<ToolIcon /> Tools {toolsEnabled ? 'armed' : 'off'}
|
| 140 |
</button>
|
| 141 |
{messages.some((message) => message.role === 'user') && (
|
| 142 |
-
<button className="regenerate-button" type="button" disabled={isBusy} onClick={onRegenerate}>
|
| 143 |
Regenerate last
|
| 144 |
</button>
|
| 145 |
)}
|
|
@@ -152,17 +163,21 @@ export function ChatSurface({
|
|
| 152 |
value={prompt}
|
| 153 |
onChange={(event) => setPrompt(event.target.value)}
|
| 154 |
onKeyDown={handleKeyDown}
|
| 155 |
-
placeholder={isLoaded ? 'Describe an observation or assign a tool…' : '
|
| 156 |
rows={2}
|
| 157 |
-
disabled={
|
| 158 |
/>
|
| 159 |
{isBusy ? (
|
| 160 |
<button className="send-button stop" type="button" onClick={onCancel}>
|
| 161 |
-
<span aria-hidden="true" /> Stop
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
</button>
|
| 163 |
) : (
|
| 164 |
<button className="send-button" type="submit" disabled={!prompt.trim()}>
|
| 165 |
-
|
| 166 |
</button>
|
| 167 |
)}
|
| 168 |
</div>
|
|
@@ -196,7 +211,10 @@ function Message({ message, ordinal }: { message: ChatMessage; ordinal: number }
|
|
| 196 |
{message.metrics && (
|
| 197 |
<footer>
|
| 198 |
<span>{message.metrics.tokens} tok</span>
|
| 199 |
-
|
|
|
|
|
|
|
|
|
|
| 200 |
<span>TTFT {message.metrics.timeToFirstTokenMs} ms</span>
|
| 201 |
</footer>
|
| 202 |
)}
|
|
@@ -235,3 +253,11 @@ function ArrowIcon() {
|
|
| 235 |
</svg>
|
| 236 |
);
|
| 237 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
import type {
|
| 3 |
ChatMessage,
|
| 4 |
ComposerSubmission,
|
| 5 |
+
ModelLoadProgress,
|
| 6 |
ModelId,
|
| 7 |
ModelOption,
|
| 8 |
ShardRetryNotice,
|
| 9 |
StreamState,
|
| 10 |
+
TelemetrySnapshot,
|
| 11 |
ToolRun,
|
| 12 |
} from '../../lib/contracts';
|
| 13 |
+
import { ComposerActivity } from './ComposerActivity';
|
| 14 |
import { MarkdownContent } from './MarkdownContent';
|
| 15 |
|
| 16 |
interface ChatSurfaceProps {
|
|
|
|
| 18 |
loadedModelId: ModelId | null;
|
| 19 |
messages: ChatMessage[];
|
| 20 |
streamState: StreamState;
|
| 21 |
+
modelLoadProgress: ModelLoadProgress;
|
| 22 |
+
telemetry: TelemetrySnapshot;
|
| 23 |
toolsEnabled: boolean;
|
| 24 |
runtimeStatus: string;
|
| 25 |
error: string | null;
|
|
|
|
| 38 |
loadedModelId,
|
| 39 |
messages,
|
| 40 |
streamState,
|
| 41 |
+
modelLoadProgress,
|
| 42 |
+
telemetry,
|
| 43 |
toolsEnabled,
|
| 44 |
runtimeStatus,
|
| 45 |
error,
|
|
|
|
| 53 |
onToolsEnabledChange,
|
| 54 |
}: ChatSurfaceProps) {
|
| 55 |
const [prompt, setPrompt] = useState('');
|
| 56 |
+
const isLoading = streamState === 'loading';
|
| 57 |
+
const isStreaming = streamState === 'streaming';
|
| 58 |
+
const isBusy = isStreaming || isLoading;
|
| 59 |
const isLoaded = loadedModelId === activeModel.id;
|
| 60 |
const logDate = useMemo(() => new Intl.DateTimeFormat('en-GB', {
|
| 61 |
day: '2-digit', month: 'long', year: 'numeric',
|
|
|
|
| 63 |
|
| 64 |
const submit = () => {
|
| 65 |
const trimmed = prompt.trim();
|
| 66 |
+
if (!trimmed || isBusy || !isLoaded) return;
|
| 67 |
onSend({ prompt: trimmed, toolsEnabled });
|
| 68 |
setPrompt('');
|
| 69 |
};
|
|
|
|
| 92 |
<span>{activeModel.capability} canopy</span>
|
| 93 |
<span>{Math.round(activeModel.contextLimit / 1024)}K context</span>
|
| 94 |
<span className={`live-label${isLoaded ? '' : ' is-dormant'}`}><i /> {isLoaded ? 'local graph' : 'not loaded'}</span>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
</div>
|
| 96 |
</header>
|
| 97 |
|
|
|
|
| 133 |
</section>
|
| 134 |
|
| 135 |
<form className="composer" onSubmit={handleSubmit}>
|
| 136 |
+
<ComposerActivity
|
| 137 |
+
modelId={activeModel.id}
|
| 138 |
+
isLoaded={isLoaded}
|
| 139 |
+
streamState={streamState}
|
| 140 |
+
progress={modelLoadProgress}
|
| 141 |
+
telemetry={telemetry}
|
| 142 |
+
/>
|
| 143 |
<div className="composer-meta">
|
| 144 |
<button
|
| 145 |
className={`tool-toggle${toolsEnabled ? ' is-on' : ''}`}
|
|
|
|
| 150 |
<ToolIcon /> Tools {toolsEnabled ? 'armed' : 'off'}
|
| 151 |
</button>
|
| 152 |
{messages.some((message) => message.role === 'user') && (
|
| 153 |
+
<button className="regenerate-button" type="button" disabled={isBusy || !isLoaded} onClick={onRegenerate}>
|
| 154 |
Regenerate last
|
| 155 |
</button>
|
| 156 |
)}
|
|
|
|
| 163 |
value={prompt}
|
| 164 |
onChange={(event) => setPrompt(event.target.value)}
|
| 165 |
onKeyDown={handleKeyDown}
|
| 166 |
+
placeholder={isLoaded ? 'Describe an observation or assign a tool…' : 'Write while the model loads, then run locally…'}
|
| 167 |
rows={2}
|
| 168 |
+
disabled={isStreaming}
|
| 169 |
/>
|
| 170 |
{isBusy ? (
|
| 171 |
<button className="send-button stop" type="button" onClick={onCancel}>
|
| 172 |
+
<span aria-hidden="true" /> {isLoading ? 'Stop load' : 'Stop'}
|
| 173 |
+
</button>
|
| 174 |
+
) : !isLoaded ? (
|
| 175 |
+
<button className="send-button load" type="button" onClick={() => onLoadModel(activeModel.id)}>
|
| 176 |
+
Load model <DownloadIcon />
|
| 177 |
</button>
|
| 178 |
) : (
|
| 179 |
<button className="send-button" type="submit" disabled={!prompt.trim()}>
|
| 180 |
+
Run <ArrowIcon />
|
| 181 |
</button>
|
| 182 |
)}
|
| 183 |
</div>
|
|
|
|
| 211 |
{message.metrics && (
|
| 212 |
<footer>
|
| 213 |
<span>{message.metrics.tokens} tok</span>
|
| 214 |
+
{message.metrics.promptTokensPerSecond !== undefined && (
|
| 215 |
+
<span>Prefill {message.metrics.promptTokensPerSecond.toFixed(1)} tok/s</span>
|
| 216 |
+
)}
|
| 217 |
+
<span>Decode {message.metrics.tokensPerSecond.toFixed(1)} tok/s</span>
|
| 218 |
<span>TTFT {message.metrics.timeToFirstTokenMs} ms</span>
|
| 219 |
</footer>
|
| 220 |
)}
|
|
|
|
| 253 |
</svg>
|
| 254 |
);
|
| 255 |
}
|
| 256 |
+
|
| 257 |
+
function DownloadIcon() {
|
| 258 |
+
return (
|
| 259 |
+
<svg viewBox="0 0 20 20" aria-hidden="true">
|
| 260 |
+
<path d="M10 2v11m-4-4 4 4 4-4M3 17h14" />
|
| 261 |
+
</svg>
|
| 262 |
+
);
|
| 263 |
+
}
|
src/app/components/ComposerActivity.tsx
ADDED
|
@@ -0,0 +1,205 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import type {
|
| 2 |
+
ModelId,
|
| 3 |
+
ModelLoadProgress,
|
| 4 |
+
StreamState,
|
| 5 |
+
TelemetrySnapshot,
|
| 6 |
+
} from '../../lib/contracts';
|
| 7 |
+
import { formatBytes, formatPercent } from '../../lib/format';
|
| 8 |
+
|
| 9 |
+
interface ComposerActivityProps {
|
| 10 |
+
modelId: ModelId;
|
| 11 |
+
isLoaded: boolean;
|
| 12 |
+
streamState: StreamState;
|
| 13 |
+
progress: ModelLoadProgress;
|
| 14 |
+
telemetry: TelemetrySnapshot;
|
| 15 |
+
}
|
| 16 |
+
|
| 17 |
+
const LOAD_STAGES = [
|
| 18 |
+
['manifest', 'Manifest'],
|
| 19 |
+
['cache', 'Cache'],
|
| 20 |
+
['download', 'Download'],
|
| 21 |
+
['verify', 'Verify'],
|
| 22 |
+
['load', 'Allocate'],
|
| 23 |
+
] as const;
|
| 24 |
+
|
| 25 |
+
type LoadStage = typeof LOAD_STAGES[number][0];
|
| 26 |
+
|
| 27 |
+
function ratio(value: number, total: number): number {
|
| 28 |
+
return total > 0 ? Math.min(1, Math.max(0, value / total)) : 0;
|
| 29 |
+
}
|
| 30 |
+
|
| 31 |
+
function humanizeStage(value: string | null): string {
|
| 32 |
+
if (!value) return '';
|
| 33 |
+
return value.replaceAll('_', ' ').replace(/\b\w/g, (letter) => letter.toUpperCase());
|
| 34 |
+
}
|
| 35 |
+
|
| 36 |
+
export function ComposerActivity({
|
| 37 |
+
modelId,
|
| 38 |
+
isLoaded,
|
| 39 |
+
streamState,
|
| 40 |
+
progress,
|
| 41 |
+
telemetry,
|
| 42 |
+
}: ComposerActivityProps) {
|
| 43 |
+
const tracksModel = progress.modelId === modelId;
|
| 44 |
+
const isLoading = streamState === 'loading' && tracksModel;
|
| 45 |
+
const isInferring = streamState === 'streaming'
|
| 46 |
+
|| telemetry.inferencePhase === 'prefill'
|
| 47 |
+
|| telemetry.inferencePhase === 'decode';
|
| 48 |
+
const activeStage = tracksModel && LOAD_STAGES.some(([stage]) => stage === progress.stage)
|
| 49 |
+
? progress.stage as LoadStage
|
| 50 |
+
: null;
|
| 51 |
+
const downloadRatio = tracksModel ? ratio(progress.downloadedBytes, progress.totalBytes) : 0;
|
| 52 |
+
const verifiedBytes = displayShardsTotal(progress.shards, 'verifiedBytes');
|
| 53 |
+
const verifiedRatio = tracksModel ? ratio(verifiedBytes, progress.totalBytes) : 0;
|
| 54 |
+
const prefillRatio = ratio(telemetry.promptProcessed, telemetry.promptTotal);
|
| 55 |
+
const displayShards = tracksModel ? progress.shards : [];
|
| 56 |
+
const pipelineProgress = !tracksModel
|
| 57 |
+
? 0
|
| 58 |
+
: progress.stage === 'ready'
|
| 59 |
+
? 1
|
| 60 |
+
: progress.stage === 'load'
|
| 61 |
+
? 0.85 + Math.min(1, Math.max(0, progress.stageProgress)) * 0.15
|
| 62 |
+
: progress.stage === 'manifest'
|
| 63 |
+
? Math.min(1, Math.max(0, progress.stageProgress)) * 0.05
|
| 64 |
+
: Math.min(0.85, 0.05 + downloadRatio * 0.55 + verifiedRatio * 0.25);
|
| 65 |
+
|
| 66 |
+
const status = isLoading
|
| 67 |
+
? progress.nativeStage
|
| 68 |
+
? humanizeStage(progress.nativeStage)
|
| 69 |
+
: LOAD_STAGES.find(([stage]) => stage === activeStage)?.[1] ?? 'Preparing model'
|
| 70 |
+
: isInferring
|
| 71 |
+
? telemetry.inferencePhase === 'prefill' ? 'Prefilling prompt' : 'Generating response'
|
| 72 |
+
: progress.stage === 'error' && tracksModel
|
| 73 |
+
? 'Model load stopped'
|
| 74 |
+
: isLoaded ? 'Model ready in this tab' : 'Model is not loaded';
|
| 75 |
+
|
| 76 |
+
return (
|
| 77 |
+
<section className={`composer-activity${isLoading ? ' is-loading' : ''}${isInferring ? ' is-inference' : ''}`} aria-label="Runtime activity">
|
| 78 |
+
<div className="activity-stage-line" aria-hidden="true">
|
| 79 |
+
{LOAD_STAGES.map(([stage, label]) => {
|
| 80 |
+
const complete = isLoaded || progress.stage === 'ready' || (
|
| 81 |
+
stage === 'manifest' ? progress.stage !== 'idle' && progress.stage !== 'manifest'
|
| 82 |
+
: stage === 'cache' ? progress.shards.length > 0 && progress.stage !== 'manifest'
|
| 83 |
+
: stage === 'download' ? downloadRatio >= 1
|
| 84 |
+
: stage === 'verify' ? progress.shards.length > 0 && verifiedRatio >= 1
|
| 85 |
+
: false
|
| 86 |
+
);
|
| 87 |
+
const current = isLoading && activeStage === stage;
|
| 88 |
+
return (
|
| 89 |
+
<span className={`${complete ? 'is-complete' : ''}${current ? ' is-current' : ''}`} key={stage}>
|
| 90 |
+
<i /> {label}
|
| 91 |
+
</span>
|
| 92 |
+
);
|
| 93 |
+
})}
|
| 94 |
+
</div>
|
| 95 |
+
|
| 96 |
+
<div className="activity-status" aria-live="polite" aria-atomic="true">
|
| 97 |
+
<div>
|
| 98 |
+
<span className="activity-pulse" aria-hidden="true" />
|
| 99 |
+
<strong>{status}</strong>
|
| 100 |
+
</div>
|
| 101 |
+
{isLoading && (
|
| 102 |
+
<span>{formatPercent(pipelineProgress)} · {formatBytes(progress.downloadedBytes)} / {formatBytes(progress.totalBytes)}</span>
|
| 103 |
+
)}
|
| 104 |
+
{isInferring && telemetry.inferencePhase === 'prefill' && (
|
| 105 |
+
<span>{telemetry.promptProcessed.toLocaleString()} / {telemetry.promptTotal.toLocaleString()} tokens</span>
|
| 106 |
+
)}
|
| 107 |
+
{isInferring && telemetry.inferencePhase === 'decode' && (
|
| 108 |
+
<span>{telemetry.completionTokens.toLocaleString()} generated tokens</span>
|
| 109 |
+
)}
|
| 110 |
+
{!isLoading && !isInferring && isLoaded && progress.residentBytes > 0 && (
|
| 111 |
+
<span>{formatBytes(progress.residentBytes)} allocated</span>
|
| 112 |
+
)}
|
| 113 |
+
</div>
|
| 114 |
+
|
| 115 |
+
{isInferring ? (
|
| 116 |
+
<div className="inference-live-grid">
|
| 117 |
+
<LiveMetric
|
| 118 |
+
label="Prefill"
|
| 119 |
+
value={`${telemetry.prefillTokensPerSecond.toFixed(1)} tok/s`}
|
| 120 |
+
detail={telemetry.promptTotal > 0 ? formatPercent(prefillRatio) : 'waiting'}
|
| 121 |
+
progress={prefillRatio}
|
| 122 |
+
active={telemetry.inferencePhase === 'prefill'}
|
| 123 |
+
/>
|
| 124 |
+
<LiveMetric
|
| 125 |
+
label="Decode"
|
| 126 |
+
value={`${telemetry.decodeTokensPerSecond.toFixed(1)} tok/s`}
|
| 127 |
+
detail={`${telemetry.completionTokens.toLocaleString()} tok`}
|
| 128 |
+
progress={Math.min(1, telemetry.decodeTokensPerSecond / 20)}
|
| 129 |
+
active={telemetry.inferencePhase === 'decode'}
|
| 130 |
+
/>
|
| 131 |
+
</div>
|
| 132 |
+
) : (
|
| 133 |
+
<div className="load-live-grid">
|
| 134 |
+
<div className="activity-track-block">
|
| 135 |
+
<div><span>Model transfer</span><small>{formatPercent(downloadRatio)}</small></div>
|
| 136 |
+
<ProgressTrack label="Model transfer" value={downloadRatio} />
|
| 137 |
+
</div>
|
| 138 |
+
<div className="activity-track-block">
|
| 139 |
+
<div><span>Load pipeline</span><small>{formatPercent(pipelineProgress)}</small></div>
|
| 140 |
+
<ProgressTrack label="Overall model load pipeline" value={pipelineProgress} />
|
| 141 |
+
</div>
|
| 142 |
+
<div className="activity-memory">
|
| 143 |
+
<span>Resident</span>
|
| 144 |
+
<strong>{formatBytes(tracksModel ? progress.residentBytes : 0)}</strong>
|
| 145 |
+
</div>
|
| 146 |
+
</div>
|
| 147 |
+
)}
|
| 148 |
+
|
| 149 |
+
<div className={`shard-strip${displayShards.length === 0 ? ' is-empty' : ''}`} aria-label="Model shard progress">
|
| 150 |
+
{displayShards.length === 0
|
| 151 |
+
? <span className="shard-placeholder">Shard detail appears during cache inspection and transfer.</span>
|
| 152 |
+
: displayShards.map((shard) => {
|
| 153 |
+
const shardRatio = shard.state === 'complete' || shard.state === 'cached'
|
| 154 |
+
? 1
|
| 155 |
+
: shard.state === 'verifying'
|
| 156 |
+
? ratio(shard.verifiedBytes, shard.totalBytes)
|
| 157 |
+
: ratio(shard.loadedBytes, shard.totalBytes);
|
| 158 |
+
return (
|
| 159 |
+
<div className={`shard-cell state-${shard.state}`} key={`${shard.index}-${shard.path}`} title={shard.path}>
|
| 160 |
+
<span><b>S{shard.index + 1}</b><small>{formatPercent(shardRatio)}</small></span>
|
| 161 |
+
<ProgressTrack label={`Shard ${shard.index + 1} ${shard.state}`} value={shardRatio} />
|
| 162 |
+
</div>
|
| 163 |
+
);
|
| 164 |
+
})}
|
| 165 |
+
</div>
|
| 166 |
+
</section>
|
| 167 |
+
);
|
| 168 |
+
}
|
| 169 |
+
|
| 170 |
+
function displayShardsTotal(
|
| 171 |
+
shards: ModelLoadProgress['shards'],
|
| 172 |
+
field: 'loadedBytes' | 'verifiedBytes',
|
| 173 |
+
): number {
|
| 174 |
+
return shards.reduce((total, shard) => total + shard[field], 0);
|
| 175 |
+
}
|
| 176 |
+
|
| 177 |
+
function LiveMetric({
|
| 178 |
+
label,
|
| 179 |
+
value,
|
| 180 |
+
detail,
|
| 181 |
+
progress,
|
| 182 |
+
active,
|
| 183 |
+
}: {
|
| 184 |
+
label: string;
|
| 185 |
+
value: string;
|
| 186 |
+
detail: string;
|
| 187 |
+
progress: number;
|
| 188 |
+
active: boolean;
|
| 189 |
+
}) {
|
| 190 |
+
return (
|
| 191 |
+
<div className={`live-metric${active ? ' is-active' : ''}`}>
|
| 192 |
+
<div><span>{label}</span><strong>{value}</strong><small>{detail}</small></div>
|
| 193 |
+
<ProgressTrack label={`${label} live progress`} value={progress} />
|
| 194 |
+
</div>
|
| 195 |
+
);
|
| 196 |
+
}
|
| 197 |
+
|
| 198 |
+
function ProgressTrack({ label, value }: { label: string; value: number }) {
|
| 199 |
+
const percentage = Math.round(Math.min(1, Math.max(0, value)) * 100);
|
| 200 |
+
return (
|
| 201 |
+
<div className="activity-track" role="progressbar" aria-label={label} aria-valuemin={0} aria-valuemax={100} aria-valuenow={percentage}>
|
| 202 |
+
<span style={{ width: `${percentage}%` }} />
|
| 203 |
+
</div>
|
| 204 |
+
);
|
| 205 |
+
}
|
src/app/components/InspectorPanel.tsx
CHANGED
|
@@ -1,20 +1,26 @@
|
|
| 1 |
import type { ArtifactDocument } from '../../lib/agent-tools';
|
| 2 |
-
import type {
|
| 3 |
-
import { formatBytes
|
| 4 |
import { ArtifactPane } from './ArtifactPane';
|
| 5 |
|
| 6 |
interface InspectorPanelProps {
|
| 7 |
telemetry: TelemetrySnapshot;
|
| 8 |
toolEvents: ToolEvent[];
|
| 9 |
-
storage: StorageStatus;
|
| 10 |
artifact: ArtifactDocument | null;
|
| 11 |
onCloseArtifact: () => void;
|
| 12 |
}
|
| 13 |
|
| 14 |
-
export function InspectorPanel({ telemetry, toolEvents,
|
| 15 |
const contextRatio = telemetry.contextLimit > 0 ? telemetry.contextUsed / telemetry.contextLimit : 0;
|
| 16 |
-
const
|
| 17 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
return (
|
| 20 |
<aside className="inspector" aria-label="Runtime inspector">
|
|
@@ -25,18 +31,24 @@ export function InspectorPanel({ telemetry, toolEvents, storage, artifact, onClo
|
|
| 25 |
<p className="eyebrow">Live cambium</p>
|
| 26 |
<h2 id="telemetry-title">Telemetry</h2>
|
| 27 |
</div>
|
| 28 |
-
<span className=
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
</div>
|
| 30 |
|
| 31 |
<div className="speed-gauge">
|
| 32 |
<div className="gauge-ring" style={{ '--gauge': `${Math.round(speedRatio * 100)}%` } as React.CSSProperties}>
|
| 33 |
<div>
|
| 34 |
-
<strong>{
|
| 35 |
-
<small>
|
| 36 |
</div>
|
| 37 |
</div>
|
| 38 |
<dl>
|
| 39 |
<div><dt>Backend</dt><dd>{telemetry.backend}</dd></div>
|
|
|
|
|
|
|
| 40 |
<div><dt>First token</dt><dd>{telemetry.timeToFirstTokenMs} ms</dd></div>
|
| 41 |
<div><dt>Graph splits</dt><dd>{telemetry.graphSplits ?? '—'}</dd></div>
|
| 42 |
</dl>
|
|
@@ -53,8 +65,8 @@ export function InspectorPanel({ telemetry, toolEvents, storage, artifact, onClo
|
|
| 53 |
|
| 54 |
<Meter label="Context rings" value={contextRatio} detail={`${telemetry.contextUsed.toLocaleString()} / ${telemetry.contextLimit.toLocaleString()}`} />
|
| 55 |
<div className="memory-grid">
|
| 56 |
-
<div><small>Model
|
| 57 |
-
<div><small>KV
|
| 58 |
</div>
|
| 59 |
</section>
|
| 60 |
|
|
@@ -78,42 +90,6 @@ export function InspectorPanel({ telemetry, toolEvents, storage, artifact, onClo
|
|
| 78 |
</ol>
|
| 79 |
</section>
|
| 80 |
|
| 81 |
-
<section className="inspector-section storage-section" aria-labelledby="storage-title">
|
| 82 |
-
<div className="section-heading compact">
|
| 83 |
-
<span>04</span>
|
| 84 |
-
<div>
|
| 85 |
-
<p className="eyebrow">Root cellar</p>
|
| 86 |
-
<h2 id="storage-title">Downloads</h2>
|
| 87 |
-
</div>
|
| 88 |
-
<span className={`persistence${storage.persistent ? ' is-persistent' : ''}`}>
|
| 89 |
-
{storage.persistent ? 'persistent' : 'session'}
|
| 90 |
-
</span>
|
| 91 |
-
</div>
|
| 92 |
-
|
| 93 |
-
<Meter
|
| 94 |
-
label="Browser storage"
|
| 95 |
-
value={storageRatio}
|
| 96 |
-
detail={`${formatBytes(storage.usedBytes)} of ${formatBytes(storage.quotaBytes)}`}
|
| 97 |
-
/>
|
| 98 |
-
|
| 99 |
-
{storage.downloads.map((download) => {
|
| 100 |
-
const ratio = download.receivedBytes / download.totalBytes;
|
| 101 |
-
return (
|
| 102 |
-
<div className="download-item" key={download.modelId}>
|
| 103 |
-
<div>
|
| 104 |
-
<span className={`download-state state-${download.state}`}>{download.state}</span>
|
| 105 |
-
<strong>{download.label}</strong>
|
| 106 |
-
<small>{formatBytes(download.receivedBytes)} / {formatBytes(download.totalBytes)}</small>
|
| 107 |
-
</div>
|
| 108 |
-
<span className="download-percent">{formatPercent(ratio)}</span>
|
| 109 |
-
<div className="download-track" aria-label={`${download.label} download progress`} role="progressbar" aria-valuemin={0} aria-valuemax={100} aria-valuenow={Math.round(ratio * 100)}>
|
| 110 |
-
<span style={{ width: formatPercent(ratio) }} />
|
| 111 |
-
</div>
|
| 112 |
-
</div>
|
| 113 |
-
);
|
| 114 |
-
})}
|
| 115 |
-
</section>
|
| 116 |
-
|
| 117 |
<ArtifactPane artifact={artifact} onClose={onCloseArtifact} />
|
| 118 |
</aside>
|
| 119 |
);
|
|
|
|
| 1 |
import type { ArtifactDocument } from '../../lib/agent-tools';
|
| 2 |
+
import type { TelemetrySnapshot, ToolEvent } from '../../lib/contracts';
|
| 3 |
+
import { formatBytes } from '../../lib/format';
|
| 4 |
import { ArtifactPane } from './ArtifactPane';
|
| 5 |
|
| 6 |
interface InspectorPanelProps {
|
| 7 |
telemetry: TelemetrySnapshot;
|
| 8 |
toolEvents: ToolEvent[];
|
|
|
|
| 9 |
artifact: ArtifactDocument | null;
|
| 10 |
onCloseArtifact: () => void;
|
| 11 |
}
|
| 12 |
|
| 13 |
+
export function InspectorPanel({ telemetry, toolEvents, artifact, onCloseArtifact }: InspectorPanelProps) {
|
| 14 |
const contextRatio = telemetry.contextLimit > 0 ? telemetry.contextUsed / telemetry.contextLimit : 0;
|
| 15 |
+
const activeRate = telemetry.inferencePhase === 'prefill'
|
| 16 |
+
? telemetry.prefillTokensPerSecond
|
| 17 |
+
: telemetry.decodeTokensPerSecond || telemetry.tokensPerSecond;
|
| 18 |
+
const speedRatio = Math.min(1, Math.max(0, activeRate / 50));
|
| 19 |
+
const gaugeLabel = telemetry.inferencePhase === 'prefill'
|
| 20 |
+
? 'prefill tok / sec'
|
| 21 |
+
: telemetry.inferencePhase === 'decode'
|
| 22 |
+
? 'decode tok / sec'
|
| 23 |
+
: 'average tok / sec';
|
| 24 |
|
| 25 |
return (
|
| 26 |
<aside className="inspector" aria-label="Runtime inspector">
|
|
|
|
| 31 |
<p className="eyebrow">Live cambium</p>
|
| 32 |
<h2 id="telemetry-title">Telemetry</h2>
|
| 33 |
</div>
|
| 34 |
+
<span className={`sampling-label phase-${telemetry.inferencePhase}`}><i /> {
|
| 35 |
+
telemetry.inferencePhase === 'idle'
|
| 36 |
+
? 'ready'
|
| 37 |
+
: telemetry.inferencePhase === 'complete' ? 'average' : 'live'
|
| 38 |
+
}</span>
|
| 39 |
</div>
|
| 40 |
|
| 41 |
<div className="speed-gauge">
|
| 42 |
<div className="gauge-ring" style={{ '--gauge': `${Math.round(speedRatio * 100)}%` } as React.CSSProperties}>
|
| 43 |
<div>
|
| 44 |
+
<strong>{activeRate.toFixed(1)}</strong>
|
| 45 |
+
<small>{gaugeLabel}</small>
|
| 46 |
</div>
|
| 47 |
</div>
|
| 48 |
<dl>
|
| 49 |
<div><dt>Backend</dt><dd>{telemetry.backend}</dd></div>
|
| 50 |
+
<div><dt>Prefill</dt><dd>{telemetry.prefillTokensPerSecond.toFixed(1)} tok/s</dd></div>
|
| 51 |
+
<div><dt>Decode</dt><dd>{telemetry.decodeTokensPerSecond.toFixed(1)} tok/s</dd></div>
|
| 52 |
<div><dt>First token</dt><dd>{telemetry.timeToFirstTokenMs} ms</dd></div>
|
| 53 |
<div><dt>Graph splits</dt><dd>{telemetry.graphSplits ?? '—'}</dd></div>
|
| 54 |
</dl>
|
|
|
|
| 65 |
|
| 66 |
<Meter label="Context rings" value={contextRatio} detail={`${telemetry.contextUsed.toLocaleString()} / ${telemetry.contextLimit.toLocaleString()}`} />
|
| 67 |
<div className="memory-grid">
|
| 68 |
+
<div><small>Model allocation</small><strong>{formatBytes(telemetry.modelMemoryBytes)}</strong></div>
|
| 69 |
+
<div><small>KV allocation</small><strong>{formatBytes(telemetry.kvMemoryBytes)}</strong></div>
|
| 70 |
</div>
|
| 71 |
</section>
|
| 72 |
|
|
|
|
| 90 |
</ol>
|
| 91 |
</section>
|
| 92 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
<ArtifactPane artifact={artifact} onClose={onCloseArtifact} />
|
| 94 |
</aside>
|
| 95 |
);
|
src/app/components/ModelRail.tsx
CHANGED
|
@@ -1,13 +1,10 @@
|
|
| 1 |
-
import type {
|
| 2 |
|
| 3 |
interface ModelRailProps {
|
| 4 |
models: ModelOption[];
|
| 5 |
activeModelId: ModelId;
|
| 6 |
-
dflash: DFlashState;
|
| 7 |
busy: boolean;
|
| 8 |
onModelSelect: (modelId: ModelId) => void;
|
| 9 |
-
onDFlashChange: (enabled: boolean) => void;
|
| 10 |
-
onRequestDownload: (modelId: ModelId) => void;
|
| 11 |
}
|
| 12 |
|
| 13 |
const availabilityLabels: Record<ModelOption['availability'], string> = {
|
|
@@ -21,11 +18,8 @@ const availabilityLabels: Record<ModelOption['availability'], string> = {
|
|
| 21 |
export function ModelRail({
|
| 22 |
models,
|
| 23 |
activeModelId,
|
| 24 |
-
dflash,
|
| 25 |
busy,
|
| 26 |
onModelSelect,
|
| 27 |
-
onDFlashChange,
|
| 28 |
-
onRequestDownload,
|
| 29 |
}: ModelRailProps) {
|
| 30 |
return (
|
| 31 |
<aside className="model-rail" aria-labelledby="model-rail-title">
|
|
@@ -62,47 +56,12 @@ export function ModelRail({
|
|
| 62 |
</span>
|
| 63 |
<span className="model-footprint">{model.footprint}</span>
|
| 64 |
</button>
|
| 65 |
-
{(model.availability === 'remote' || model.availability === 'cached' || model.availability === 'limited') && (
|
| 66 |
-
<button className="text-action" type="button" disabled={busy} onClick={() => onRequestDownload(model.id)}>
|
| 67 |
-
{model.availability === 'cached' ? 'Load cached specimen' : model.availability === 'limited' ? 'Try device gate' : 'Download & load'} <span aria-hidden="true">↘</span>
|
| 68 |
-
</button>
|
| 69 |
-
)}
|
| 70 |
-
{model.availability === 'loading' && <span className="text-action is-status">Preparing local graph…</span>}
|
| 71 |
{model.availability === 'limited' && <small className="model-limit-reason">{model.limitReason ?? 'Device gate unavailable'}</small>}
|
| 72 |
</article>
|
| 73 |
);
|
| 74 |
})}
|
| 75 |
</div>
|
| 76 |
|
| 77 |
-
<section className="dflash-card" aria-labelledby="dflash-title">
|
| 78 |
-
<div className="dflash-heading">
|
| 79 |
-
<div>
|
| 80 |
-
<span className="experimental-stamp">Inspection only</span>
|
| 81 |
-
<h3 id="dflash-title">DFlash research lane</h3>
|
| 82 |
-
</div>
|
| 83 |
-
<button
|
| 84 |
-
className="switch"
|
| 85 |
-
type="button"
|
| 86 |
-
role="switch"
|
| 87 |
-
aria-checked={dflash.enabled}
|
| 88 |
-
disabled={!dflash.available}
|
| 89 |
-
onClick={() => onDFlashChange(!dflash.enabled)}
|
| 90 |
-
aria-label="Toggle DFlash speculative decoding"
|
| 91 |
-
>
|
| 92 |
-
<span />
|
| 93 |
-
</button>
|
| 94 |
-
</div>
|
| 95 |
-
<p>{dflash.reason}</p>
|
| 96 |
-
<small className="dflash-contract">{dflash.status} · default-off D0–D3 gate</small>
|
| 97 |
-
<div className="dflash-axis" aria-hidden="true">
|
| 98 |
-
<span />
|
| 99 |
-
<span />
|
| 100 |
-
<span />
|
| 101 |
-
<span />
|
| 102 |
-
<span />
|
| 103 |
-
</div>
|
| 104 |
-
</section>
|
| 105 |
-
|
| 106 |
<footer className="rail-footnote">
|
| 107 |
<span>GGUF · Q1_0</span>
|
| 108 |
<span>Manifest v2 · commit pinned</span>
|
|
|
|
| 1 |
+
import type { ModelId, ModelOption } from '../../lib/contracts';
|
| 2 |
|
| 3 |
interface ModelRailProps {
|
| 4 |
models: ModelOption[];
|
| 5 |
activeModelId: ModelId;
|
|
|
|
| 6 |
busy: boolean;
|
| 7 |
onModelSelect: (modelId: ModelId) => void;
|
|
|
|
|
|
|
| 8 |
}
|
| 9 |
|
| 10 |
const availabilityLabels: Record<ModelOption['availability'], string> = {
|
|
|
|
| 18 |
export function ModelRail({
|
| 19 |
models,
|
| 20 |
activeModelId,
|
|
|
|
| 21 |
busy,
|
| 22 |
onModelSelect,
|
|
|
|
|
|
|
| 23 |
}: ModelRailProps) {
|
| 24 |
return (
|
| 25 |
<aside className="model-rail" aria-labelledby="model-rail-title">
|
|
|
|
| 56 |
</span>
|
| 57 |
<span className="model-footprint">{model.footprint}</span>
|
| 58 |
</button>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
{model.availability === 'limited' && <small className="model-limit-reason">{model.limitReason ?? 'Device gate unavailable'}</small>}
|
| 60 |
</article>
|
| 61 |
);
|
| 62 |
})}
|
| 63 |
</div>
|
| 64 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
<footer className="rail-footnote">
|
| 66 |
<span>GGUF · Q1_0</span>
|
| 67 |
<span>Manifest v2 · commit pinned</span>
|
src/app/styles.css
CHANGED
|
@@ -15,6 +15,10 @@
|
|
| 15 |
--display: "Bodoni 72", Didot, "Iowan Old Style", Baskerville, serif;
|
| 16 |
--body: "Avenir Next Condensed", "Gill Sans", "Trebuchet MS", sans-serif;
|
| 17 |
--mono: "Berkeley Mono", Iosevka, "SFMono-Regular", Consolas, monospace;
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
font-family: var(--body);
|
| 19 |
font-synthesis: none;
|
| 20 |
text-rendering: optimizeLegibility;
|
|
@@ -37,6 +41,7 @@ body {
|
|
| 37 |
linear-gradient(rgba(17, 21, 18, 0.045) 1px, transparent 1px) 0 0 / 22px 22px,
|
| 38 |
var(--paper);
|
| 39 |
color: var(--paper);
|
|
|
|
| 40 |
}
|
| 41 |
|
| 42 |
button,
|
|
@@ -159,7 +164,7 @@ button:disabled {
|
|
| 159 |
.eyebrow {
|
| 160 |
margin: 0 0 5px;
|
| 161 |
color: rgba(232, 223, 202, 0.52);
|
| 162 |
-
font: 600
|
| 163 |
letter-spacing: 0.18em;
|
| 164 |
text-transform: uppercase;
|
| 165 |
}
|
|
@@ -196,14 +201,14 @@ button:disabled {
|
|
| 196 |
|
| 197 |
.local-promise strong {
|
| 198 |
color: var(--phosphor);
|
| 199 |
-
font: 600
|
| 200 |
letter-spacing: 0.13em;
|
| 201 |
text-transform: uppercase;
|
| 202 |
}
|
| 203 |
|
| 204 |
.local-promise small {
|
| 205 |
color: var(--paper-ghost);
|
| 206 |
-
font:
|
| 207 |
}
|
| 208 |
|
| 209 |
.header-actions {
|
|
@@ -222,19 +227,19 @@ button:disabled {
|
|
| 222 |
}
|
| 223 |
|
| 224 |
.station-clock small {
|
| 225 |
-
font-size:
|
| 226 |
letter-spacing: 0.18em;
|
| 227 |
}
|
| 228 |
|
| 229 |
.station-clock span {
|
| 230 |
margin-top: 4px;
|
| 231 |
-
font-size:
|
| 232 |
letter-spacing: 0.09em;
|
| 233 |
}
|
| 234 |
|
| 235 |
.icon-button {
|
| 236 |
-
width:
|
| 237 |
-
height:
|
| 238 |
display: grid;
|
| 239 |
place-items: center;
|
| 240 |
border: 1px solid rgba(232, 223, 202, 0.24);
|
|
@@ -260,7 +265,7 @@ button:disabled {
|
|
| 260 |
.workspace-grid {
|
| 261 |
min-height: calc(100dvh - 76px);
|
| 262 |
display: grid;
|
| 263 |
-
grid-template-columns: clamp(
|
| 264 |
grid-template-areas: "models chat inspector";
|
| 265 |
}
|
| 266 |
|
|
@@ -307,7 +312,7 @@ button:disabled {
|
|
| 307 |
.rail-heading > span,
|
| 308 |
.section-heading > span:first-child {
|
| 309 |
color: var(--phosphor);
|
| 310 |
-
font: 500
|
| 311 |
border-top: 1px solid var(--phosphor);
|
| 312 |
padding-top: 5px;
|
| 313 |
}
|
|
@@ -315,7 +320,7 @@ button:disabled {
|
|
| 315 |
.rail-heading h2,
|
| 316 |
.section-heading h2 {
|
| 317 |
margin: 0;
|
| 318 |
-
font: 400
|
| 319 |
letter-spacing: -0.025em;
|
| 320 |
}
|
| 321 |
|
|
@@ -343,7 +348,8 @@ button:disabled {
|
|
| 343 |
display: grid;
|
| 344 |
grid-template-columns: 23px 1fr auto;
|
| 345 |
gap: 8px;
|
| 346 |
-
|
|
|
|
| 347 |
text-align: left;
|
| 348 |
color: var(--paper);
|
| 349 |
border: 0;
|
|
@@ -357,7 +363,7 @@ button:disabled {
|
|
| 357 |
|
| 358 |
.model-index {
|
| 359 |
color: rgba(232, 223, 202, 0.38);
|
| 360 |
-
font:
|
| 361 |
padding-top: 4px;
|
| 362 |
}
|
| 363 |
|
|
@@ -375,13 +381,13 @@ button:disabled {
|
|
| 375 |
}
|
| 376 |
|
| 377 |
.model-title-line strong {
|
| 378 |
-
font: 500
|
| 379 |
}
|
| 380 |
|
| 381 |
.model-copy em {
|
| 382 |
overflow: hidden;
|
| 383 |
color: rgba(232, 223, 202, 0.46);
|
| 384 |
-
font: italic
|
| 385 |
text-overflow: ellipsis;
|
| 386 |
white-space: nowrap;
|
| 387 |
}
|
|
@@ -389,12 +395,12 @@ button:disabled {
|
|
| 389 |
.model-copy > span:last-child {
|
| 390 |
margin-top: 3px;
|
| 391 |
color: var(--paper-ghost);
|
| 392 |
-
font-size:
|
| 393 |
}
|
| 394 |
|
| 395 |
.availability {
|
| 396 |
padding: 2px 4px;
|
| 397 |
-
font: 600
|
| 398 |
letter-spacing: 0.08em;
|
| 399 |
text-transform: uppercase;
|
| 400 |
border: 1px solid var(--line-light);
|
|
@@ -416,32 +422,7 @@ button:disabled {
|
|
| 416 |
.model-footprint {
|
| 417 |
align-self: end;
|
| 418 |
color: rgba(232, 223, 202, 0.39);
|
| 419 |
-
font:
|
| 420 |
-
white-space: nowrap;
|
| 421 |
-
}
|
| 422 |
-
|
| 423 |
-
.text-action {
|
| 424 |
-
width: calc(100% - 30px);
|
| 425 |
-
margin: -3px 8px 9px 30px;
|
| 426 |
-
padding: 7px 0;
|
| 427 |
-
color: var(--rust);
|
| 428 |
-
border: 0;
|
| 429 |
-
border-top: 1px dashed rgba(201, 111, 67, 0.35);
|
| 430 |
-
background: transparent;
|
| 431 |
-
font: 8px/1 var(--mono);
|
| 432 |
-
text-align: left;
|
| 433 |
-
cursor: pointer;
|
| 434 |
-
}
|
| 435 |
-
|
| 436 |
-
.text-action span {
|
| 437 |
-
float: right;
|
| 438 |
-
}
|
| 439 |
-
|
| 440 |
-
.text-action.is-status {
|
| 441 |
-
display: block;
|
| 442 |
-
cursor: default;
|
| 443 |
-
overflow: hidden;
|
| 444 |
-
text-overflow: ellipsis;
|
| 445 |
white-space: nowrap;
|
| 446 |
}
|
| 447 |
|
|
@@ -449,102 +430,7 @@ button:disabled {
|
|
| 449 |
display: block;
|
| 450 |
margin: -4px 8px 9px 30px;
|
| 451 |
color: rgba(232, 223, 202, 0.58);
|
| 452 |
-
font:
|
| 453 |
-
}
|
| 454 |
-
|
| 455 |
-
.dflash-card {
|
| 456 |
-
position: relative;
|
| 457 |
-
z-index: 1;
|
| 458 |
-
margin-top: auto;
|
| 459 |
-
padding: 13px 12px 11px;
|
| 460 |
-
border: 1px dashed rgba(232, 223, 202, 0.23);
|
| 461 |
-
background: rgba(17, 21, 18, 0.72);
|
| 462 |
-
}
|
| 463 |
-
|
| 464 |
-
.dflash-card.is-available {
|
| 465 |
-
border-color: rgba(185, 255, 69, 0.48);
|
| 466 |
-
}
|
| 467 |
-
|
| 468 |
-
.dflash-heading {
|
| 469 |
-
display: flex;
|
| 470 |
-
justify-content: space-between;
|
| 471 |
-
gap: 12px;
|
| 472 |
-
}
|
| 473 |
-
|
| 474 |
-
.dflash-heading h3 {
|
| 475 |
-
margin: 7px 0 0;
|
| 476 |
-
font: 500 14px/1 var(--display);
|
| 477 |
-
}
|
| 478 |
-
|
| 479 |
-
.experimental-stamp {
|
| 480 |
-
display: inline-block;
|
| 481 |
-
color: var(--rust);
|
| 482 |
-
border: 1px solid currentColor;
|
| 483 |
-
padding: 3px 5px 2px;
|
| 484 |
-
font: 700 7px/1 var(--mono);
|
| 485 |
-
letter-spacing: 0.12em;
|
| 486 |
-
text-transform: uppercase;
|
| 487 |
-
transform: rotate(-2deg);
|
| 488 |
-
}
|
| 489 |
-
|
| 490 |
-
.switch {
|
| 491 |
-
width: 35px;
|
| 492 |
-
height: 18px;
|
| 493 |
-
padding: 2px;
|
| 494 |
-
border: 1px solid rgba(232, 223, 202, 0.3);
|
| 495 |
-
border-radius: 10px;
|
| 496 |
-
background: transparent;
|
| 497 |
-
cursor: pointer;
|
| 498 |
-
}
|
| 499 |
-
|
| 500 |
-
.switch span {
|
| 501 |
-
display: block;
|
| 502 |
-
width: 12px;
|
| 503 |
-
height: 12px;
|
| 504 |
-
border-radius: 50%;
|
| 505 |
-
background: rgba(232, 223, 202, 0.42);
|
| 506 |
-
transition: transform 170ms ease, background 170ms ease;
|
| 507 |
-
}
|
| 508 |
-
|
| 509 |
-
.switch[aria-checked="true"] {
|
| 510 |
-
border-color: var(--phosphor);
|
| 511 |
-
}
|
| 512 |
-
|
| 513 |
-
.switch[aria-checked="true"] span {
|
| 514 |
-
background: var(--phosphor);
|
| 515 |
-
transform: translateX(17px);
|
| 516 |
-
}
|
| 517 |
-
|
| 518 |
-
.switch:disabled {
|
| 519 |
-
opacity: 0.38;
|
| 520 |
-
}
|
| 521 |
-
|
| 522 |
-
.dflash-card p {
|
| 523 |
-
margin: 9px 0 5px;
|
| 524 |
-
color: rgba(232, 223, 202, 0.5);
|
| 525 |
-
font: 9px/1.45 var(--body);
|
| 526 |
-
}
|
| 527 |
-
|
| 528 |
-
.dflash-contract {
|
| 529 |
-
display: block;
|
| 530 |
-
margin-bottom: 11px;
|
| 531 |
-
color: rgba(201, 111, 67, 0.72);
|
| 532 |
-
font: 7px/1.3 var(--mono);
|
| 533 |
-
text-transform: uppercase;
|
| 534 |
-
}
|
| 535 |
-
|
| 536 |
-
.dflash-axis {
|
| 537 |
-
display: flex;
|
| 538 |
-
align-items: center;
|
| 539 |
-
justify-content: space-between;
|
| 540 |
-
border-top: 1px solid rgba(232, 223, 202, 0.12);
|
| 541 |
-
}
|
| 542 |
-
|
| 543 |
-
.dflash-axis span {
|
| 544 |
-
width: 1px;
|
| 545 |
-
height: 5px;
|
| 546 |
-
margin-top: -1px;
|
| 547 |
-
background: rgba(232, 223, 202, 0.28);
|
| 548 |
}
|
| 549 |
|
| 550 |
.rail-footnote {
|
|
@@ -553,9 +439,9 @@ button:disabled {
|
|
| 553 |
display: flex;
|
| 554 |
justify-content: space-between;
|
| 555 |
gap: 8px;
|
| 556 |
-
margin-top:
|
| 557 |
color: rgba(232, 223, 202, 0.3);
|
| 558 |
-
font:
|
| 559 |
letter-spacing: 0.03em;
|
| 560 |
}
|
| 561 |
|
|
@@ -589,14 +475,14 @@ button:disabled {
|
|
| 589 |
.chat-header h2 {
|
| 590 |
display: inline;
|
| 591 |
margin: 0;
|
| 592 |
-
font: 500
|
| 593 |
letter-spacing: -0.045em;
|
| 594 |
}
|
| 595 |
|
| 596 |
.botanical-subtitle {
|
| 597 |
margin-left: 10px;
|
| 598 |
color: rgba(17, 21, 18, 0.46);
|
| 599 |
-
font: italic
|
| 600 |
}
|
| 601 |
|
| 602 |
.chat-header-meta {
|
|
@@ -604,28 +490,11 @@ button:disabled {
|
|
| 604 |
grid-template-columns: repeat(2, auto);
|
| 605 |
gap: 5px 13px;
|
| 606 |
color: rgba(17, 21, 18, 0.55);
|
| 607 |
-
font:
|
| 608 |
text-transform: uppercase;
|
| 609 |
letter-spacing: 0.07em;
|
| 610 |
}
|
| 611 |
|
| 612 |
-
.header-load-button {
|
| 613 |
-
grid-column: span 2;
|
| 614 |
-
justify-self: end;
|
| 615 |
-
padding: 6px 9px;
|
| 616 |
-
color: var(--ink);
|
| 617 |
-
border: 1px solid var(--ink);
|
| 618 |
-
background: var(--phosphor);
|
| 619 |
-
font: 700 7px/1 var(--mono);
|
| 620 |
-
letter-spacing: 0.06em;
|
| 621 |
-
text-transform: uppercase;
|
| 622 |
-
cursor: pointer;
|
| 623 |
-
}
|
| 624 |
-
|
| 625 |
-
.header-load-button:disabled {
|
| 626 |
-
opacity: 0.42;
|
| 627 |
-
}
|
| 628 |
-
|
| 629 |
.live-label {
|
| 630 |
grid-column: span 2;
|
| 631 |
justify-self: end;
|
|
@@ -675,13 +544,13 @@ button:disabled {
|
|
| 675 |
|
| 676 |
.runtime-error strong {
|
| 677 |
color: #ff9a68;
|
| 678 |
-
font: 700
|
| 679 |
letter-spacing: 0.08em;
|
| 680 |
text-transform: uppercase;
|
| 681 |
}
|
| 682 |
|
| 683 |
.runtime-error span {
|
| 684 |
-
font:
|
| 685 |
overflow-wrap: anywhere;
|
| 686 |
}
|
| 687 |
|
|
@@ -690,7 +559,8 @@ button:disabled {
|
|
| 690 |
padding: 5px 8px;
|
| 691 |
border: 1px solid rgba(232, 223, 202, 0.3);
|
| 692 |
background: transparent;
|
| 693 |
-
|
|
|
|
| 694 |
text-transform: uppercase;
|
| 695 |
cursor: pointer;
|
| 696 |
}
|
|
@@ -712,7 +582,7 @@ button:disabled {
|
|
| 712 |
color: rgba(17, 21, 18, 0.4);
|
| 713 |
border-bottom: 1px solid rgba(17, 21, 18, 0.09);
|
| 714 |
background: repeating-linear-gradient(90deg, transparent 0 14px, rgba(17, 21, 18, 0.2) 14px 15px) left bottom / 100% 4px no-repeat;
|
| 715 |
-
font:
|
| 716 |
}
|
| 717 |
|
| 718 |
.transcript {
|
|
@@ -728,7 +598,7 @@ button:disabled {
|
|
| 728 |
gap: 10px;
|
| 729 |
margin-bottom: 22px;
|
| 730 |
color: rgba(17, 21, 18, 0.45);
|
| 731 |
-
font:
|
| 732 |
letter-spacing: 0.1em;
|
| 733 |
text-transform: uppercase;
|
| 734 |
}
|
|
@@ -762,7 +632,7 @@ button:disabled {
|
|
| 762 |
|
| 763 |
.transcript-empty p {
|
| 764 |
margin: 0;
|
| 765 |
-
font-size:
|
| 766 |
line-height: 1.5;
|
| 767 |
}
|
| 768 |
|
|
@@ -778,7 +648,7 @@ button:disabled {
|
|
| 778 |
gap: 8px;
|
| 779 |
margin-bottom: 8px;
|
| 780 |
color: rgba(17, 21, 18, 0.47);
|
| 781 |
-
font:
|
| 782 |
text-transform: uppercase;
|
| 783 |
letter-spacing: 0.07em;
|
| 784 |
}
|
|
@@ -797,7 +667,7 @@ button:disabled {
|
|
| 797 |
}
|
| 798 |
|
| 799 |
.message-content {
|
| 800 |
-
font:
|
| 801 |
}
|
| 802 |
|
| 803 |
.message-content > p {
|
|
@@ -879,12 +749,12 @@ button:disabled {
|
|
| 879 |
color: rgba(17, 21, 18, 0.62);
|
| 880 |
border: 1px dashed rgba(17, 21, 18, 0.24);
|
| 881 |
background: rgba(17, 21, 18, 0.035);
|
| 882 |
-
font-size:
|
| 883 |
}
|
| 884 |
|
| 885 |
.reasoning-block summary {
|
| 886 |
color: #4d6728;
|
| 887 |
-
font: 700
|
| 888 |
letter-spacing: 0.08em;
|
| 889 |
text-transform: uppercase;
|
| 890 |
cursor: pointer;
|
|
@@ -903,7 +773,7 @@ button:disabled {
|
|
| 903 |
.message-error {
|
| 904 |
margin: 9px 0 0;
|
| 905 |
color: #8d321b;
|
| 906 |
-
font:
|
| 907 |
}
|
| 908 |
|
| 909 |
.message.has-error {
|
|
@@ -948,7 +818,7 @@ button:disabled {
|
|
| 948 |
padding-top: 7px;
|
| 949 |
color: rgba(17, 21, 18, 0.43);
|
| 950 |
border-top: 1px dashed rgba(17, 21, 18, 0.19);
|
| 951 |
-
font:
|
| 952 |
}
|
| 953 |
|
| 954 |
.stream-cursor {
|
|
@@ -982,12 +852,12 @@ button:disabled {
|
|
| 982 |
}
|
| 983 |
|
| 984 |
.tool-call code {
|
| 985 |
-
font:
|
| 986 |
}
|
| 987 |
|
| 988 |
.tool-call summary > span:last-child {
|
| 989 |
color: rgba(17, 21, 18, 0.48);
|
| 990 |
-
font:
|
| 991 |
text-transform: uppercase;
|
| 992 |
}
|
| 993 |
|
|
@@ -1023,12 +893,12 @@ button:disabled {
|
|
| 1023 |
.tool-call > div p {
|
| 1024 |
margin: 5px 0 0;
|
| 1025 |
color: rgba(17, 21, 18, 0.58);
|
| 1026 |
-
font-size:
|
| 1027 |
}
|
| 1028 |
|
| 1029 |
.composer {
|
| 1030 |
position: relative;
|
| 1031 |
-
padding:
|
| 1032 |
border-top: 1px solid var(--line-dark);
|
| 1033 |
background: rgba(216, 205, 180, 0.82);
|
| 1034 |
box-shadow: 0 -15px 38px rgba(17, 21, 18, 0.06);
|
|
@@ -1046,6 +916,258 @@ button:disabled {
|
|
| 1046 |
opacity: 0.23;
|
| 1047 |
}
|
| 1048 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1049 |
.composer-meta {
|
| 1050 |
display: flex;
|
| 1051 |
align-items: center;
|
|
@@ -1053,18 +1175,20 @@ button:disabled {
|
|
| 1053 |
gap: 12px;
|
| 1054 |
margin-bottom: 7px;
|
| 1055 |
color: rgba(17, 21, 18, 0.46);
|
| 1056 |
-
|
|
|
|
| 1057 |
}
|
| 1058 |
|
| 1059 |
.tool-toggle {
|
| 1060 |
display: flex;
|
| 1061 |
align-items: center;
|
| 1062 |
gap: 5px;
|
| 1063 |
-
|
|
|
|
| 1064 |
color: rgba(17, 21, 18, 0.54);
|
| 1065 |
border: 0;
|
| 1066 |
background: transparent;
|
| 1067 |
-
font: 700
|
| 1068 |
text-transform: uppercase;
|
| 1069 |
cursor: pointer;
|
| 1070 |
}
|
|
@@ -1075,11 +1199,12 @@ button:disabled {
|
|
| 1075 |
|
| 1076 |
.regenerate-button {
|
| 1077 |
margin-right: auto;
|
| 1078 |
-
|
|
|
|
| 1079 |
color: rgba(17, 21, 18, 0.54);
|
| 1080 |
border: 0;
|
| 1081 |
background: transparent;
|
| 1082 |
-
font: 700
|
| 1083 |
text-transform: uppercase;
|
| 1084 |
cursor: pointer;
|
| 1085 |
}
|
|
@@ -1100,7 +1225,7 @@ button:disabled {
|
|
| 1100 |
display: grid;
|
| 1101 |
grid-template-columns: 1fr auto;
|
| 1102 |
position: relative;
|
| 1103 |
-
min-height:
|
| 1104 |
border: 1px solid var(--ink);
|
| 1105 |
background: rgba(232, 223, 202, 0.83);
|
| 1106 |
box-shadow: 4px 4px 0 rgba(17, 21, 18, 0.1);
|
|
@@ -1111,7 +1236,7 @@ button:disabled {
|
|
| 1111 |
top: 7px;
|
| 1112 |
left: 11px;
|
| 1113 |
color: rgba(17, 21, 18, 0.45);
|
| 1114 |
-
font:
|
| 1115 |
letter-spacing: 0.09em;
|
| 1116 |
text-transform: uppercase;
|
| 1117 |
}
|
|
@@ -1124,7 +1249,7 @@ button:disabled {
|
|
| 1124 |
border: 0;
|
| 1125 |
outline: none;
|
| 1126 |
background: transparent;
|
| 1127 |
-
font:
|
| 1128 |
}
|
| 1129 |
|
| 1130 |
.composer-field textarea::placeholder {
|
|
@@ -1132,7 +1257,8 @@ button:disabled {
|
|
| 1132 |
}
|
| 1133 |
|
| 1134 |
.send-button {
|
| 1135 |
-
min-width:
|
|
|
|
| 1136 |
display: flex;
|
| 1137 |
align-items: center;
|
| 1138 |
justify-content: center;
|
|
@@ -1141,7 +1267,7 @@ button:disabled {
|
|
| 1141 |
color: var(--ink);
|
| 1142 |
border: 1px solid var(--ink);
|
| 1143 |
background: var(--phosphor);
|
| 1144 |
-
font: 700
|
| 1145 |
text-transform: uppercase;
|
| 1146 |
cursor: pointer;
|
| 1147 |
transition: transform 150ms ease, box-shadow 150ms ease, background 150ms ease;
|
|
@@ -1174,6 +1300,10 @@ button:disabled {
|
|
| 1174 |
background: var(--rust);
|
| 1175 |
}
|
| 1176 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1177 |
.inspector {
|
| 1178 |
grid-area: inspector;
|
| 1179 |
min-height: calc(100dvh - 76px);
|
|
@@ -1203,16 +1333,15 @@ button:disabled {
|
|
| 1203 |
}
|
| 1204 |
|
| 1205 |
.section-heading h2 {
|
| 1206 |
-
font-size:
|
| 1207 |
}
|
| 1208 |
|
| 1209 |
-
.sampling-label
|
| 1210 |
-
.persistence {
|
| 1211 |
display: flex;
|
| 1212 |
align-items: center;
|
| 1213 |
gap: 6px;
|
| 1214 |
color: rgba(232, 223, 202, 0.42);
|
| 1215 |
-
font:
|
| 1216 |
text-transform: uppercase;
|
| 1217 |
letter-spacing: 0.06em;
|
| 1218 |
}
|
|
@@ -1267,7 +1396,7 @@ button:disabled {
|
|
| 1267 |
.gauge-ring small {
|
| 1268 |
margin-top: 3px;
|
| 1269 |
color: var(--paper-ghost);
|
| 1270 |
-
font:
|
| 1271 |
text-transform: uppercase;
|
| 1272 |
}
|
| 1273 |
|
|
@@ -1280,7 +1409,7 @@ button:disabled {
|
|
| 1280 |
justify-content: space-between;
|
| 1281 |
padding: 7px 0;
|
| 1282 |
border-bottom: 1px solid var(--line-light);
|
| 1283 |
-
font:
|
| 1284 |
}
|
| 1285 |
|
| 1286 |
.speed-gauge dt {
|
|
@@ -1329,28 +1458,27 @@ button:disabled {
|
|
| 1329 |
|
| 1330 |
.device-card small,
|
| 1331 |
.memory-grid small,
|
| 1332 |
-
.meter small
|
| 1333 |
-
.download-item small {
|
| 1334 |
color: rgba(232, 223, 202, 0.42);
|
| 1335 |
-
font:
|
| 1336 |
text-transform: uppercase;
|
| 1337 |
}
|
| 1338 |
|
| 1339 |
.device-card strong {
|
| 1340 |
-
font: 500
|
| 1341 |
}
|
| 1342 |
|
| 1343 |
.device-card > span:last-child {
|
| 1344 |
display: grid;
|
| 1345 |
justify-items: end;
|
| 1346 |
color: var(--phosphor);
|
| 1347 |
-
font: 600
|
| 1348 |
}
|
| 1349 |
|
| 1350 |
.device-card > span:last-child small {
|
| 1351 |
margin-top: 3px;
|
| 1352 |
color: rgba(232, 223, 202, 0.37);
|
| 1353 |
-
font-size:
|
| 1354 |
}
|
| 1355 |
|
| 1356 |
.meter {
|
|
@@ -1361,7 +1489,7 @@ button:disabled {
|
|
| 1361 |
display: flex;
|
| 1362 |
justify-content: space-between;
|
| 1363 |
margin-bottom: 6px;
|
| 1364 |
-
font:
|
| 1365 |
}
|
| 1366 |
|
| 1367 |
.meter-track {
|
|
@@ -1392,7 +1520,7 @@ button:disabled {
|
|
| 1392 |
}
|
| 1393 |
|
| 1394 |
.memory-grid strong {
|
| 1395 |
-
font: 500
|
| 1396 |
}
|
| 1397 |
|
| 1398 |
.event-log {
|
|
@@ -1426,7 +1554,7 @@ button:disabled {
|
|
| 1426 |
display: block;
|
| 1427 |
padding: 5px 0;
|
| 1428 |
color: rgba(232, 223, 202, 0.36);
|
| 1429 |
-
font:
|
| 1430 |
}
|
| 1431 |
|
| 1432 |
.event-pip {
|
|
@@ -1437,7 +1565,7 @@ button:disabled {
|
|
| 1437 |
|
| 1438 |
.event-log time {
|
| 1439 |
color: rgba(232, 223, 202, 0.34);
|
| 1440 |
-
font:
|
| 1441 |
}
|
| 1442 |
|
| 1443 |
.event-log div {
|
|
@@ -1446,71 +1574,12 @@ button:disabled {
|
|
| 1446 |
}
|
| 1447 |
|
| 1448 |
.event-log strong {
|
| 1449 |
-
font: 500
|
| 1450 |
}
|
| 1451 |
|
| 1452 |
.event-log small {
|
| 1453 |
color: rgba(232, 223, 202, 0.42);
|
| 1454 |
-
font-size:
|
| 1455 |
-
}
|
| 1456 |
-
|
| 1457 |
-
.persistence {
|
| 1458 |
-
padding: 3px 5px;
|
| 1459 |
-
border: 1px solid var(--line-light);
|
| 1460 |
-
}
|
| 1461 |
-
|
| 1462 |
-
.persistence.is-persistent {
|
| 1463 |
-
color: var(--phosphor);
|
| 1464 |
-
border-color: rgba(185, 255, 69, 0.28);
|
| 1465 |
-
}
|
| 1466 |
-
|
| 1467 |
-
.download-item {
|
| 1468 |
-
position: relative;
|
| 1469 |
-
display: grid;
|
| 1470 |
-
grid-template-columns: 1fr auto;
|
| 1471 |
-
gap: 8px;
|
| 1472 |
-
margin-top: 16px;
|
| 1473 |
-
padding: 11px 10px 10px;
|
| 1474 |
-
border: 1px solid var(--line-light);
|
| 1475 |
-
}
|
| 1476 |
-
|
| 1477 |
-
.download-item > div:first-child {
|
| 1478 |
-
display: grid;
|
| 1479 |
-
gap: 5px;
|
| 1480 |
-
}
|
| 1481 |
-
|
| 1482 |
-
.download-item strong {
|
| 1483 |
-
font: 500 10px/1 var(--mono);
|
| 1484 |
-
}
|
| 1485 |
-
|
| 1486 |
-
.download-state {
|
| 1487 |
-
width: fit-content;
|
| 1488 |
-
color: var(--rust);
|
| 1489 |
-
font: 700 6px/1 var(--mono);
|
| 1490 |
-
letter-spacing: 0.09em;
|
| 1491 |
-
text-transform: uppercase;
|
| 1492 |
-
}
|
| 1493 |
-
|
| 1494 |
-
.state-downloading {
|
| 1495 |
-
color: var(--phosphor);
|
| 1496 |
-
}
|
| 1497 |
-
|
| 1498 |
-
.download-percent {
|
| 1499 |
-
color: var(--phosphor);
|
| 1500 |
-
font: 500 15px/1 var(--display);
|
| 1501 |
-
}
|
| 1502 |
-
|
| 1503 |
-
.download-track {
|
| 1504 |
-
grid-column: span 2;
|
| 1505 |
-
height: 3px;
|
| 1506 |
-
background: rgba(232, 223, 202, 0.1);
|
| 1507 |
-
}
|
| 1508 |
-
|
| 1509 |
-
.download-track span {
|
| 1510 |
-
display: block;
|
| 1511 |
-
height: 100%;
|
| 1512 |
-
background: var(--phosphor);
|
| 1513 |
-
box-shadow: 0 0 9px rgba(185, 255, 69, 0.35);
|
| 1514 |
}
|
| 1515 |
|
| 1516 |
.artifact-close {
|
|
@@ -1525,7 +1594,7 @@ button:disabled {
|
|
| 1525 |
.artifact-empty {
|
| 1526 |
margin: 0;
|
| 1527 |
color: rgba(232, 223, 202, 0.4);
|
| 1528 |
-
font:
|
| 1529 |
}
|
| 1530 |
|
| 1531 |
.artifact-pane {
|
|
@@ -1547,7 +1616,8 @@ button:disabled {
|
|
| 1547 |
border: 0;
|
| 1548 |
border-right: 1px solid var(--line-light);
|
| 1549 |
background: transparent;
|
| 1550 |
-
|
|
|
|
| 1551 |
text-transform: uppercase;
|
| 1552 |
cursor: pointer;
|
| 1553 |
}
|
|
@@ -1561,7 +1631,7 @@ button:disabled {
|
|
| 1561 |
overflow: hidden;
|
| 1562 |
padding: 0 8px;
|
| 1563 |
color: rgba(232, 223, 202, 0.45);
|
| 1564 |
-
font:
|
| 1565 |
text-align: right;
|
| 1566 |
text-overflow: ellipsis;
|
| 1567 |
white-space: nowrap;
|
|
@@ -1588,7 +1658,8 @@ button:disabled {
|
|
| 1588 |
color: var(--paper);
|
| 1589 |
border: 1px solid var(--rust);
|
| 1590 |
background: rgba(17, 21, 18, 0.9);
|
| 1591 |
-
|
|
|
|
| 1592 |
text-transform: uppercase;
|
| 1593 |
cursor: pointer;
|
| 1594 |
}
|
|
@@ -1608,7 +1679,7 @@ button:disabled {
|
|
| 1608 |
max-width: 250px;
|
| 1609 |
margin: 0;
|
| 1610 |
color: rgba(17, 21, 18, 0.62);
|
| 1611 |
-
font:
|
| 1612 |
}
|
| 1613 |
|
| 1614 |
.artifact-launch button {
|
|
@@ -1616,7 +1687,8 @@ button:disabled {
|
|
| 1616 |
color: var(--ink);
|
| 1617 |
border: 1px solid var(--ink);
|
| 1618 |
background: var(--phosphor);
|
| 1619 |
-
|
|
|
|
| 1620 |
text-transform: uppercase;
|
| 1621 |
cursor: pointer;
|
| 1622 |
}
|
|
@@ -1628,7 +1700,7 @@ button:disabled {
|
|
| 1628 |
padding: 10px;
|
| 1629 |
color: var(--paper);
|
| 1630 |
background: #0d100e;
|
| 1631 |
-
font:
|
| 1632 |
white-space: pre-wrap;
|
| 1633 |
}
|
| 1634 |
|
|
@@ -1698,7 +1770,7 @@ button:disabled {
|
|
| 1698 |
|
| 1699 |
.setting-number {
|
| 1700 |
color: var(--phosphor);
|
| 1701 |
-
font:
|
| 1702 |
}
|
| 1703 |
|
| 1704 |
.setting-group > div,
|
|
@@ -1710,19 +1782,19 @@ button:disabled {
|
|
| 1710 |
.setting-group small,
|
| 1711 |
.range-setting small {
|
| 1712 |
color: rgba(232, 223, 202, 0.43);
|
| 1713 |
-
font:
|
| 1714 |
letter-spacing: 0.08em;
|
| 1715 |
text-transform: uppercase;
|
| 1716 |
}
|
| 1717 |
|
| 1718 |
.setting-group strong,
|
| 1719 |
.range-setting strong {
|
| 1720 |
-
font: 500
|
| 1721 |
}
|
| 1722 |
|
| 1723 |
.setting-locked {
|
| 1724 |
color: rgba(232, 223, 202, 0.36);
|
| 1725 |
-
font:
|
| 1726 |
text-transform: uppercase;
|
| 1727 |
}
|
| 1728 |
|
|
@@ -1733,7 +1805,8 @@ button:disabled {
|
|
| 1733 |
border: 1px solid var(--line-light);
|
| 1734 |
border-radius: 0;
|
| 1735 |
background: var(--ink-raised);
|
| 1736 |
-
|
|
|
|
| 1737 |
}
|
| 1738 |
|
| 1739 |
.range-setting {
|
|
@@ -1793,7 +1866,7 @@ button:disabled {
|
|
| 1793 |
display: flex;
|
| 1794 |
justify-content: space-between;
|
| 1795 |
color: rgba(232, 223, 202, 0.35);
|
| 1796 |
-
font:
|
| 1797 |
}
|
| 1798 |
|
| 1799 |
.range-limits i {
|
|
@@ -1810,7 +1883,7 @@ button:disabled {
|
|
| 1810 |
display: flex;
|
| 1811 |
justify-content: space-between;
|
| 1812 |
color: rgba(232, 223, 202, 0.43);
|
| 1813 |
-
font:
|
| 1814 |
letter-spacing: 0.05em;
|
| 1815 |
text-transform: uppercase;
|
| 1816 |
}
|
|
@@ -1821,7 +1894,7 @@ button:disabled {
|
|
| 1821 |
color: var(--paper);
|
| 1822 |
border: 1px solid var(--line-light);
|
| 1823 |
background: var(--ink-raised);
|
| 1824 |
-
font:
|
| 1825 |
}
|
| 1826 |
|
| 1827 |
.settings-actions {
|
|
@@ -1836,7 +1909,8 @@ button:disabled {
|
|
| 1836 |
color: var(--paper);
|
| 1837 |
border: 1px solid var(--line-light);
|
| 1838 |
background: transparent;
|
| 1839 |
-
|
|
|
|
| 1840 |
text-transform: uppercase;
|
| 1841 |
cursor: pointer;
|
| 1842 |
}
|
|
@@ -1871,7 +1945,7 @@ button:disabled {
|
|
| 1871 |
|
| 1872 |
.boot-screen p:last-child {
|
| 1873 |
color: var(--paper-ghost);
|
| 1874 |
-
font:
|
| 1875 |
}
|
| 1876 |
|
| 1877 |
.range-ticks i {
|
|
@@ -1898,7 +1972,7 @@ button:disabled {
|
|
| 1898 |
.settings-note p {
|
| 1899 |
margin: 0;
|
| 1900 |
color: rgba(232, 223, 202, 0.58);
|
| 1901 |
-
font-size:
|
| 1902 |
line-height: 1.5;
|
| 1903 |
}
|
| 1904 |
|
|
@@ -1926,6 +2000,11 @@ button:disabled {
|
|
| 1926 |
53%, 100% { opacity: 0; }
|
| 1927 |
}
|
| 1928 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1929 |
@keyframes sheet-in {
|
| 1930 |
from { opacity: 0; transform: translateX(34px); }
|
| 1931 |
to { opacity: 1; transform: translateX(0); }
|
|
@@ -1945,7 +2024,7 @@ button:disabled {
|
|
| 1945 |
}
|
| 1946 |
|
| 1947 |
.workspace-grid {
|
| 1948 |
-
grid-template-columns:
|
| 1949 |
grid-template-areas:
|
| 1950 |
"models chat"
|
| 1951 |
"inspector inspector";
|
|
@@ -2038,17 +2117,13 @@ button:disabled {
|
|
| 2038 |
scroll-snap-align: start;
|
| 2039 |
}
|
| 2040 |
|
| 2041 |
-
.dflash-card {
|
| 2042 |
-
margin-top: 13px;
|
| 2043 |
-
}
|
| 2044 |
-
|
| 2045 |
.rail-footnote {
|
| 2046 |
display: none;
|
| 2047 |
}
|
| 2048 |
|
| 2049 |
.chat-surface {
|
| 2050 |
display: grid;
|
| 2051 |
-
grid-template-rows: auto 19px minmax(
|
| 2052 |
overflow: visible;
|
| 2053 |
}
|
| 2054 |
|
|
@@ -2065,21 +2140,20 @@ button:disabled {
|
|
| 2065 |
display: none;
|
| 2066 |
}
|
| 2067 |
|
| 2068 |
-
.chat-header-meta .header-load-button {
|
| 2069 |
-
display: block;
|
| 2070 |
-
}
|
| 2071 |
-
|
| 2072 |
.live-label {
|
| 2073 |
grid-column: 1;
|
| 2074 |
}
|
| 2075 |
|
| 2076 |
.composer {
|
| 2077 |
-
position:
|
| 2078 |
-
bottom: 0;
|
| 2079 |
z-index: 5;
|
| 2080 |
padding-inline: 14px;
|
| 2081 |
}
|
| 2082 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2083 |
.inspector {
|
| 2084 |
display: block;
|
| 2085 |
border-top: 1px solid var(--phosphor);
|
|
@@ -2098,7 +2172,7 @@ button:disabled {
|
|
| 2098 |
|
| 2099 |
.brand-block .eyebrow,
|
| 2100 |
.local-promise strong {
|
| 2101 |
-
font-size:
|
| 2102 |
}
|
| 2103 |
|
| 2104 |
.chat-header {
|
|
@@ -2118,16 +2192,51 @@ button:disabled {
|
|
| 2118 |
display: none;
|
| 2119 |
}
|
| 2120 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2121 |
.composer-field {
|
| 2122 |
grid-template-columns: 1fr;
|
| 2123 |
-
padding-bottom:
|
| 2124 |
}
|
| 2125 |
|
| 2126 |
.send-button {
|
| 2127 |
position: absolute;
|
| 2128 |
right: 0;
|
| 2129 |
bottom: 0;
|
| 2130 |
-
height:
|
| 2131 |
}
|
| 2132 |
|
| 2133 |
.speed-gauge {
|
|
|
|
| 15 |
--display: "Bodoni 72", Didot, "Iowan Old Style", Baskerville, serif;
|
| 16 |
--body: "Avenir Next Condensed", "Gill Sans", "Trebuchet MS", sans-serif;
|
| 17 |
--mono: "Berkeley Mono", Iosevka, "SFMono-Regular", Consolas, monospace;
|
| 18 |
+
--type-utility: clamp(0.6875rem, 0.15vw + 0.65rem, 0.8125rem);
|
| 19 |
+
--type-caption: clamp(0.75rem, 0.18vw + 0.7rem, 0.875rem);
|
| 20 |
+
--type-body: clamp(1rem, 0.22vw + 0.95rem, 1.125rem);
|
| 21 |
+
--type-heading: clamp(1.75rem, 1.15vw + 1.45rem, 2.5rem);
|
| 22 |
font-family: var(--body);
|
| 23 |
font-synthesis: none;
|
| 24 |
text-rendering: optimizeLegibility;
|
|
|
|
| 41 |
linear-gradient(rgba(17, 21, 18, 0.045) 1px, transparent 1px) 0 0 / 22px 22px,
|
| 42 |
var(--paper);
|
| 43 |
color: var(--paper);
|
| 44 |
+
font-size: var(--type-body);
|
| 45 |
}
|
| 46 |
|
| 47 |
button,
|
|
|
|
| 164 |
.eyebrow {
|
| 165 |
margin: 0 0 5px;
|
| 166 |
color: rgba(232, 223, 202, 0.52);
|
| 167 |
+
font: 600 var(--type-utility)/1 var(--mono);
|
| 168 |
letter-spacing: 0.18em;
|
| 169 |
text-transform: uppercase;
|
| 170 |
}
|
|
|
|
| 201 |
|
| 202 |
.local-promise strong {
|
| 203 |
color: var(--phosphor);
|
| 204 |
+
font: 600 var(--type-caption)/1 var(--mono);
|
| 205 |
letter-spacing: 0.13em;
|
| 206 |
text-transform: uppercase;
|
| 207 |
}
|
| 208 |
|
| 209 |
.local-promise small {
|
| 210 |
color: var(--paper-ghost);
|
| 211 |
+
font: var(--type-caption)/1.25 var(--body);
|
| 212 |
}
|
| 213 |
|
| 214 |
.header-actions {
|
|
|
|
| 227 |
}
|
| 228 |
|
| 229 |
.station-clock small {
|
| 230 |
+
font-size: var(--type-utility);
|
| 231 |
letter-spacing: 0.18em;
|
| 232 |
}
|
| 233 |
|
| 234 |
.station-clock span {
|
| 235 |
margin-top: 4px;
|
| 236 |
+
font-size: var(--type-caption);
|
| 237 |
letter-spacing: 0.09em;
|
| 238 |
}
|
| 239 |
|
| 240 |
.icon-button {
|
| 241 |
+
width: 44px;
|
| 242 |
+
height: 44px;
|
| 243 |
display: grid;
|
| 244 |
place-items: center;
|
| 245 |
border: 1px solid rgba(232, 223, 202, 0.24);
|
|
|
|
| 265 |
.workspace-grid {
|
| 266 |
min-height: calc(100dvh - 76px);
|
| 267 |
display: grid;
|
| 268 |
+
grid-template-columns: clamp(280px, 19vw, 340px) minmax(560px, 1fr) clamp(340px, 23vw, 410px);
|
| 269 |
grid-template-areas: "models chat inspector";
|
| 270 |
}
|
| 271 |
|
|
|
|
| 312 |
.rail-heading > span,
|
| 313 |
.section-heading > span:first-child {
|
| 314 |
color: var(--phosphor);
|
| 315 |
+
font: 500 var(--type-utility)/1 var(--mono);
|
| 316 |
border-top: 1px solid var(--phosphor);
|
| 317 |
padding-top: 5px;
|
| 318 |
}
|
|
|
|
| 320 |
.rail-heading h2,
|
| 321 |
.section-heading h2 {
|
| 322 |
margin: 0;
|
| 323 |
+
font: 400 26px/1 var(--display);
|
| 324 |
letter-spacing: -0.025em;
|
| 325 |
}
|
| 326 |
|
|
|
|
| 348 |
display: grid;
|
| 349 |
grid-template-columns: 23px 1fr auto;
|
| 350 |
gap: 8px;
|
| 351 |
+
min-height: 76px;
|
| 352 |
+
padding: 13px 9px 12px;
|
| 353 |
text-align: left;
|
| 354 |
color: var(--paper);
|
| 355 |
border: 0;
|
|
|
|
| 363 |
|
| 364 |
.model-index {
|
| 365 |
color: rgba(232, 223, 202, 0.38);
|
| 366 |
+
font: var(--type-utility)/1 var(--mono);
|
| 367 |
padding-top: 4px;
|
| 368 |
}
|
| 369 |
|
|
|
|
| 381 |
}
|
| 382 |
|
| 383 |
.model-title-line strong {
|
| 384 |
+
font: 500 24px/1 var(--display);
|
| 385 |
}
|
| 386 |
|
| 387 |
.model-copy em {
|
| 388 |
overflow: hidden;
|
| 389 |
color: rgba(232, 223, 202, 0.46);
|
| 390 |
+
font: italic var(--type-caption)/1.2 var(--display);
|
| 391 |
text-overflow: ellipsis;
|
| 392 |
white-space: nowrap;
|
| 393 |
}
|
|
|
|
| 395 |
.model-copy > span:last-child {
|
| 396 |
margin-top: 3px;
|
| 397 |
color: var(--paper-ghost);
|
| 398 |
+
font-size: var(--type-caption);
|
| 399 |
}
|
| 400 |
|
| 401 |
.availability {
|
| 402 |
padding: 2px 4px;
|
| 403 |
+
font: 600 var(--type-utility)/1 var(--mono);
|
| 404 |
letter-spacing: 0.08em;
|
| 405 |
text-transform: uppercase;
|
| 406 |
border: 1px solid var(--line-light);
|
|
|
|
| 422 |
.model-footprint {
|
| 423 |
align-self: end;
|
| 424 |
color: rgba(232, 223, 202, 0.39);
|
| 425 |
+
font: var(--type-utility)/1 var(--mono);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 426 |
white-space: nowrap;
|
| 427 |
}
|
| 428 |
|
|
|
|
| 430 |
display: block;
|
| 431 |
margin: -4px 8px 9px 30px;
|
| 432 |
color: rgba(232, 223, 202, 0.58);
|
| 433 |
+
font: var(--type-utility)/1.35 var(--mono);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 434 |
}
|
| 435 |
|
| 436 |
.rail-footnote {
|
|
|
|
| 439 |
display: flex;
|
| 440 |
justify-content: space-between;
|
| 441 |
gap: 8px;
|
| 442 |
+
margin-top: auto;
|
| 443 |
color: rgba(232, 223, 202, 0.3);
|
| 444 |
+
font: var(--type-utility)/1.2 var(--mono);
|
| 445 |
letter-spacing: 0.03em;
|
| 446 |
}
|
| 447 |
|
|
|
|
| 475 |
.chat-header h2 {
|
| 476 |
display: inline;
|
| 477 |
margin: 0;
|
| 478 |
+
font: 500 var(--type-heading)/1 var(--display);
|
| 479 |
letter-spacing: -0.045em;
|
| 480 |
}
|
| 481 |
|
| 482 |
.botanical-subtitle {
|
| 483 |
margin-left: 10px;
|
| 484 |
color: rgba(17, 21, 18, 0.46);
|
| 485 |
+
font: italic var(--type-caption)/1 var(--display);
|
| 486 |
}
|
| 487 |
|
| 488 |
.chat-header-meta {
|
|
|
|
| 490 |
grid-template-columns: repeat(2, auto);
|
| 491 |
gap: 5px 13px;
|
| 492 |
color: rgba(17, 21, 18, 0.55);
|
| 493 |
+
font: var(--type-utility)/1.2 var(--mono);
|
| 494 |
text-transform: uppercase;
|
| 495 |
letter-spacing: 0.07em;
|
| 496 |
}
|
| 497 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 498 |
.live-label {
|
| 499 |
grid-column: span 2;
|
| 500 |
justify-self: end;
|
|
|
|
| 544 |
|
| 545 |
.runtime-error strong {
|
| 546 |
color: #ff9a68;
|
| 547 |
+
font: 700 var(--type-utility)/1 var(--mono);
|
| 548 |
letter-spacing: 0.08em;
|
| 549 |
text-transform: uppercase;
|
| 550 |
}
|
| 551 |
|
| 552 |
.runtime-error span {
|
| 553 |
+
font: var(--type-caption)/1.35 var(--mono);
|
| 554 |
overflow-wrap: anywhere;
|
| 555 |
}
|
| 556 |
|
|
|
|
| 559 |
padding: 5px 8px;
|
| 560 |
border: 1px solid rgba(232, 223, 202, 0.3);
|
| 561 |
background: transparent;
|
| 562 |
+
min-height: 44px;
|
| 563 |
+
font: var(--type-utility)/1.2 var(--mono);
|
| 564 |
text-transform: uppercase;
|
| 565 |
cursor: pointer;
|
| 566 |
}
|
|
|
|
| 582 |
color: rgba(17, 21, 18, 0.4);
|
| 583 |
border-bottom: 1px solid rgba(17, 21, 18, 0.09);
|
| 584 |
background: repeating-linear-gradient(90deg, transparent 0 14px, rgba(17, 21, 18, 0.2) 14px 15px) left bottom / 100% 4px no-repeat;
|
| 585 |
+
font: var(--type-utility)/1 var(--mono);
|
| 586 |
}
|
| 587 |
|
| 588 |
.transcript {
|
|
|
|
| 598 |
gap: 10px;
|
| 599 |
margin-bottom: 22px;
|
| 600 |
color: rgba(17, 21, 18, 0.45);
|
| 601 |
+
font: var(--type-utility)/1 var(--mono);
|
| 602 |
letter-spacing: 0.1em;
|
| 603 |
text-transform: uppercase;
|
| 604 |
}
|
|
|
|
| 632 |
|
| 633 |
.transcript-empty p {
|
| 634 |
margin: 0;
|
| 635 |
+
font-size: var(--type-body);
|
| 636 |
line-height: 1.5;
|
| 637 |
}
|
| 638 |
|
|
|
|
| 648 |
gap: 8px;
|
| 649 |
margin-bottom: 8px;
|
| 650 |
color: rgba(17, 21, 18, 0.47);
|
| 651 |
+
font: var(--type-utility)/1.2 var(--mono);
|
| 652 |
text-transform: uppercase;
|
| 653 |
letter-spacing: 0.07em;
|
| 654 |
}
|
|
|
|
| 667 |
}
|
| 668 |
|
| 669 |
.message-content {
|
| 670 |
+
font: var(--type-body)/1.58 var(--body);
|
| 671 |
}
|
| 672 |
|
| 673 |
.message-content > p {
|
|
|
|
| 749 |
color: rgba(17, 21, 18, 0.62);
|
| 750 |
border: 1px dashed rgba(17, 21, 18, 0.24);
|
| 751 |
background: rgba(17, 21, 18, 0.035);
|
| 752 |
+
font-size: var(--type-caption);
|
| 753 |
}
|
| 754 |
|
| 755 |
.reasoning-block summary {
|
| 756 |
color: #4d6728;
|
| 757 |
+
font: 700 var(--type-utility)/1 var(--mono);
|
| 758 |
letter-spacing: 0.08em;
|
| 759 |
text-transform: uppercase;
|
| 760 |
cursor: pointer;
|
|
|
|
| 773 |
.message-error {
|
| 774 |
margin: 9px 0 0;
|
| 775 |
color: #8d321b;
|
| 776 |
+
font: var(--type-caption)/1.4 var(--mono);
|
| 777 |
}
|
| 778 |
|
| 779 |
.message.has-error {
|
|
|
|
| 818 |
padding-top: 7px;
|
| 819 |
color: rgba(17, 21, 18, 0.43);
|
| 820 |
border-top: 1px dashed rgba(17, 21, 18, 0.19);
|
| 821 |
+
font: var(--type-utility)/1.2 var(--mono);
|
| 822 |
}
|
| 823 |
|
| 824 |
.stream-cursor {
|
|
|
|
| 852 |
}
|
| 853 |
|
| 854 |
.tool-call code {
|
| 855 |
+
font: var(--type-caption)/1.4 var(--mono);
|
| 856 |
}
|
| 857 |
|
| 858 |
.tool-call summary > span:last-child {
|
| 859 |
color: rgba(17, 21, 18, 0.48);
|
| 860 |
+
font: var(--type-utility)/1 var(--mono);
|
| 861 |
text-transform: uppercase;
|
| 862 |
}
|
| 863 |
|
|
|
|
| 893 |
.tool-call > div p {
|
| 894 |
margin: 5px 0 0;
|
| 895 |
color: rgba(17, 21, 18, 0.58);
|
| 896 |
+
font-size: var(--type-caption);
|
| 897 |
}
|
| 898 |
|
| 899 |
.composer {
|
| 900 |
position: relative;
|
| 901 |
+
padding: 12px clamp(21px, 3vw, 44px) 18px;
|
| 902 |
border-top: 1px solid var(--line-dark);
|
| 903 |
background: rgba(216, 205, 180, 0.82);
|
| 904 |
box-shadow: 0 -15px 38px rgba(17, 21, 18, 0.06);
|
|
|
|
| 916 |
opacity: 0.23;
|
| 917 |
}
|
| 918 |
|
| 919 |
+
.composer-activity {
|
| 920 |
+
height: 164px;
|
| 921 |
+
display: grid;
|
| 922 |
+
grid-template-rows: 24px 26px 40px 34px;
|
| 923 |
+
gap: 6px;
|
| 924 |
+
margin-bottom: 10px;
|
| 925 |
+
padding: 11px 12px;
|
| 926 |
+
overflow: hidden;
|
| 927 |
+
color: var(--paper);
|
| 928 |
+
border: 1px solid rgba(17, 21, 18, 0.82);
|
| 929 |
+
background:
|
| 930 |
+
linear-gradient(105deg, rgba(185, 255, 69, 0.07), transparent 58%),
|
| 931 |
+
var(--ink);
|
| 932 |
+
box-shadow: 4px 4px 0 rgba(17, 21, 18, 0.12);
|
| 933 |
+
}
|
| 934 |
+
|
| 935 |
+
.activity-stage-line {
|
| 936 |
+
display: grid;
|
| 937 |
+
grid-template-columns: repeat(5, minmax(0, 1fr));
|
| 938 |
+
gap: 6px;
|
| 939 |
+
align-items: start;
|
| 940 |
+
}
|
| 941 |
+
|
| 942 |
+
.activity-stage-line span {
|
| 943 |
+
min-width: 0;
|
| 944 |
+
display: flex;
|
| 945 |
+
align-items: center;
|
| 946 |
+
gap: 5px;
|
| 947 |
+
padding-top: 5px;
|
| 948 |
+
overflow: hidden;
|
| 949 |
+
color: rgba(232, 223, 202, 0.34);
|
| 950 |
+
border-top: 1px solid rgba(232, 223, 202, 0.14);
|
| 951 |
+
font: 700 var(--type-utility)/1 var(--mono);
|
| 952 |
+
letter-spacing: 0.04em;
|
| 953 |
+
text-overflow: ellipsis;
|
| 954 |
+
text-transform: uppercase;
|
| 955 |
+
white-space: nowrap;
|
| 956 |
+
}
|
| 957 |
+
|
| 958 |
+
.activity-stage-line i {
|
| 959 |
+
width: 6px;
|
| 960 |
+
height: 6px;
|
| 961 |
+
flex: 0 0 6px;
|
| 962 |
+
border: 1px solid currentColor;
|
| 963 |
+
border-radius: 50%;
|
| 964 |
+
}
|
| 965 |
+
|
| 966 |
+
.activity-stage-line span.is-complete,
|
| 967 |
+
.activity-stage-line span.is-current {
|
| 968 |
+
color: var(--phosphor);
|
| 969 |
+
border-color: rgba(185, 255, 69, 0.52);
|
| 970 |
+
}
|
| 971 |
+
|
| 972 |
+
.activity-stage-line span.is-complete i {
|
| 973 |
+
background: var(--phosphor);
|
| 974 |
+
}
|
| 975 |
+
|
| 976 |
+
.activity-stage-line span.is-current i {
|
| 977 |
+
box-shadow: 0 0 0 4px var(--phosphor-dim);
|
| 978 |
+
}
|
| 979 |
+
|
| 980 |
+
.activity-status,
|
| 981 |
+
.activity-status > div {
|
| 982 |
+
display: flex;
|
| 983 |
+
align-items: center;
|
| 984 |
+
}
|
| 985 |
+
|
| 986 |
+
.activity-status {
|
| 987 |
+
min-width: 0;
|
| 988 |
+
justify-content: space-between;
|
| 989 |
+
gap: 12px;
|
| 990 |
+
border-bottom: 1px solid var(--line-light);
|
| 991 |
+
font: var(--type-utility)/1 var(--mono);
|
| 992 |
+
}
|
| 993 |
+
|
| 994 |
+
.activity-status > div {
|
| 995 |
+
min-width: 0;
|
| 996 |
+
gap: 8px;
|
| 997 |
+
}
|
| 998 |
+
|
| 999 |
+
.activity-status strong {
|
| 1000 |
+
overflow: hidden;
|
| 1001 |
+
color: var(--paper);
|
| 1002 |
+
font-weight: 600;
|
| 1003 |
+
letter-spacing: 0.04em;
|
| 1004 |
+
text-overflow: ellipsis;
|
| 1005 |
+
text-transform: uppercase;
|
| 1006 |
+
white-space: nowrap;
|
| 1007 |
+
}
|
| 1008 |
+
|
| 1009 |
+
.activity-status > span {
|
| 1010 |
+
flex: 0 0 auto;
|
| 1011 |
+
color: rgba(232, 223, 202, 0.5);
|
| 1012 |
+
}
|
| 1013 |
+
|
| 1014 |
+
.activity-pulse {
|
| 1015 |
+
width: 7px;
|
| 1016 |
+
height: 7px;
|
| 1017 |
+
flex: 0 0 7px;
|
| 1018 |
+
border-radius: 50%;
|
| 1019 |
+
background: rgba(232, 223, 202, 0.35);
|
| 1020 |
+
}
|
| 1021 |
+
|
| 1022 |
+
.composer-activity.is-loading .activity-pulse,
|
| 1023 |
+
.composer-activity.is-inference .activity-pulse {
|
| 1024 |
+
background: var(--phosphor);
|
| 1025 |
+
box-shadow: 0 0 0 4px var(--phosphor-dim);
|
| 1026 |
+
animation: activity-pulse 1.15s ease-in-out infinite;
|
| 1027 |
+
}
|
| 1028 |
+
|
| 1029 |
+
.load-live-grid {
|
| 1030 |
+
display: grid;
|
| 1031 |
+
grid-template-columns: minmax(0, 1fr) minmax(0, 1fr) auto;
|
| 1032 |
+
gap: 12px;
|
| 1033 |
+
align-items: end;
|
| 1034 |
+
}
|
| 1035 |
+
|
| 1036 |
+
.activity-track-block,
|
| 1037 |
+
.live-metric {
|
| 1038 |
+
min-width: 0;
|
| 1039 |
+
display: grid;
|
| 1040 |
+
gap: 6px;
|
| 1041 |
+
}
|
| 1042 |
+
|
| 1043 |
+
.activity-track-block > div,
|
| 1044 |
+
.live-metric > div {
|
| 1045 |
+
display: flex;
|
| 1046 |
+
align-items: baseline;
|
| 1047 |
+
justify-content: space-between;
|
| 1048 |
+
gap: 8px;
|
| 1049 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1050 |
+
}
|
| 1051 |
+
|
| 1052 |
+
.activity-track-block span,
|
| 1053 |
+
.live-metric span,
|
| 1054 |
+
.activity-memory span {
|
| 1055 |
+
color: rgba(232, 223, 202, 0.46);
|
| 1056 |
+
text-transform: uppercase;
|
| 1057 |
+
}
|
| 1058 |
+
|
| 1059 |
+
.activity-track-block small,
|
| 1060 |
+
.live-metric small {
|
| 1061 |
+
color: rgba(232, 223, 202, 0.5);
|
| 1062 |
+
font-size: inherit;
|
| 1063 |
+
}
|
| 1064 |
+
|
| 1065 |
+
.activity-memory {
|
| 1066 |
+
min-width: 86px;
|
| 1067 |
+
display: grid;
|
| 1068 |
+
gap: 6px;
|
| 1069 |
+
justify-items: end;
|
| 1070 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1071 |
+
}
|
| 1072 |
+
|
| 1073 |
+
.activity-memory strong {
|
| 1074 |
+
color: var(--phosphor);
|
| 1075 |
+
font-size: var(--type-caption);
|
| 1076 |
+
font-weight: 600;
|
| 1077 |
+
}
|
| 1078 |
+
|
| 1079 |
+
.activity-track {
|
| 1080 |
+
height: 5px;
|
| 1081 |
+
overflow: hidden;
|
| 1082 |
+
border: 1px solid rgba(232, 223, 202, 0.18);
|
| 1083 |
+
background: rgba(232, 223, 202, 0.055);
|
| 1084 |
+
}
|
| 1085 |
+
|
| 1086 |
+
.activity-track > span {
|
| 1087 |
+
display: block;
|
| 1088 |
+
height: 100%;
|
| 1089 |
+
background: repeating-linear-gradient(90deg, var(--phosphor) 0 7px, transparent 7px 9px);
|
| 1090 |
+
box-shadow: 0 0 10px rgba(185, 255, 69, 0.34);
|
| 1091 |
+
transition: width 120ms linear;
|
| 1092 |
+
}
|
| 1093 |
+
|
| 1094 |
+
.inference-live-grid {
|
| 1095 |
+
display: grid;
|
| 1096 |
+
grid-template-columns: repeat(2, minmax(0, 1fr));
|
| 1097 |
+
gap: 12px;
|
| 1098 |
+
}
|
| 1099 |
+
|
| 1100 |
+
.live-metric {
|
| 1101 |
+
padding: 5px 8px 6px;
|
| 1102 |
+
border: 1px solid var(--line-light);
|
| 1103 |
+
opacity: 0.64;
|
| 1104 |
+
}
|
| 1105 |
+
|
| 1106 |
+
.live-metric.is-active {
|
| 1107 |
+
border-color: rgba(185, 255, 69, 0.48);
|
| 1108 |
+
background: rgba(185, 255, 69, 0.06);
|
| 1109 |
+
opacity: 1;
|
| 1110 |
+
}
|
| 1111 |
+
|
| 1112 |
+
.live-metric strong {
|
| 1113 |
+
margin-left: auto;
|
| 1114 |
+
color: var(--phosphor);
|
| 1115 |
+
font-size: var(--type-caption);
|
| 1116 |
+
font-weight: 600;
|
| 1117 |
+
}
|
| 1118 |
+
|
| 1119 |
+
.shard-strip {
|
| 1120 |
+
min-width: 0;
|
| 1121 |
+
display: grid;
|
| 1122 |
+
grid-template-columns: repeat(8, minmax(0, 1fr));
|
| 1123 |
+
gap: 5px;
|
| 1124 |
+
align-items: end;
|
| 1125 |
+
}
|
| 1126 |
+
|
| 1127 |
+
.shard-strip.is-empty {
|
| 1128 |
+
display: flex;
|
| 1129 |
+
align-items: center;
|
| 1130 |
+
border-top: 1px dashed var(--line-light);
|
| 1131 |
+
}
|
| 1132 |
+
|
| 1133 |
+
.shard-placeholder {
|
| 1134 |
+
color: rgba(232, 223, 202, 0.32);
|
| 1135 |
+
font: var(--type-utility)/1.2 var(--mono);
|
| 1136 |
+
}
|
| 1137 |
+
|
| 1138 |
+
.shard-cell {
|
| 1139 |
+
min-width: 0;
|
| 1140 |
+
display: grid;
|
| 1141 |
+
gap: 5px;
|
| 1142 |
+
opacity: 0.55;
|
| 1143 |
+
}
|
| 1144 |
+
|
| 1145 |
+
.shard-cell.state-downloading,
|
| 1146 |
+
.shard-cell.state-verifying,
|
| 1147 |
+
.shard-cell.state-complete,
|
| 1148 |
+
.shard-cell.state-cached {
|
| 1149 |
+
opacity: 1;
|
| 1150 |
+
}
|
| 1151 |
+
|
| 1152 |
+
.shard-cell > span {
|
| 1153 |
+
display: flex;
|
| 1154 |
+
justify-content: space-between;
|
| 1155 |
+
gap: 3px;
|
| 1156 |
+
color: rgba(232, 223, 202, 0.48);
|
| 1157 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1158 |
+
}
|
| 1159 |
+
|
| 1160 |
+
.shard-cell b {
|
| 1161 |
+
color: var(--paper);
|
| 1162 |
+
font-weight: 600;
|
| 1163 |
+
}
|
| 1164 |
+
|
| 1165 |
+
.shard-cell small {
|
| 1166 |
+
overflow: hidden;
|
| 1167 |
+
font-size: inherit;
|
| 1168 |
+
text-overflow: ellipsis;
|
| 1169 |
+
}
|
| 1170 |
+
|
| 1171 |
.composer-meta {
|
| 1172 |
display: flex;
|
| 1173 |
align-items: center;
|
|
|
|
| 1175 |
gap: 12px;
|
| 1176 |
margin-bottom: 7px;
|
| 1177 |
color: rgba(17, 21, 18, 0.46);
|
| 1178 |
+
min-height: 24px;
|
| 1179 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1180 |
}
|
| 1181 |
|
| 1182 |
.tool-toggle {
|
| 1183 |
display: flex;
|
| 1184 |
align-items: center;
|
| 1185 |
gap: 5px;
|
| 1186 |
+
min-height: 32px;
|
| 1187 |
+
padding: 0 2px;
|
| 1188 |
color: rgba(17, 21, 18, 0.54);
|
| 1189 |
border: 0;
|
| 1190 |
background: transparent;
|
| 1191 |
+
font: 700 var(--type-utility)/1 var(--mono);
|
| 1192 |
text-transform: uppercase;
|
| 1193 |
cursor: pointer;
|
| 1194 |
}
|
|
|
|
| 1199 |
|
| 1200 |
.regenerate-button {
|
| 1201 |
margin-right: auto;
|
| 1202 |
+
min-height: 32px;
|
| 1203 |
+
padding: 0 2px;
|
| 1204 |
color: rgba(17, 21, 18, 0.54);
|
| 1205 |
border: 0;
|
| 1206 |
background: transparent;
|
| 1207 |
+
font: 700 var(--type-utility)/1 var(--mono);
|
| 1208 |
text-transform: uppercase;
|
| 1209 |
cursor: pointer;
|
| 1210 |
}
|
|
|
|
| 1225 |
display: grid;
|
| 1226 |
grid-template-columns: 1fr auto;
|
| 1227 |
position: relative;
|
| 1228 |
+
min-height: 84px;
|
| 1229 |
border: 1px solid var(--ink);
|
| 1230 |
background: rgba(232, 223, 202, 0.83);
|
| 1231 |
box-shadow: 4px 4px 0 rgba(17, 21, 18, 0.1);
|
|
|
|
| 1236 |
top: 7px;
|
| 1237 |
left: 11px;
|
| 1238 |
color: rgba(17, 21, 18, 0.45);
|
| 1239 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1240 |
letter-spacing: 0.09em;
|
| 1241 |
text-transform: uppercase;
|
| 1242 |
}
|
|
|
|
| 1249 |
border: 0;
|
| 1250 |
outline: none;
|
| 1251 |
background: transparent;
|
| 1252 |
+
font: var(--type-body)/1.45 var(--body);
|
| 1253 |
}
|
| 1254 |
|
| 1255 |
.composer-field textarea::placeholder {
|
|
|
|
| 1257 |
}
|
| 1258 |
|
| 1259 |
.send-button {
|
| 1260 |
+
min-width: 112px;
|
| 1261 |
+
min-height: 52px;
|
| 1262 |
display: flex;
|
| 1263 |
align-items: center;
|
| 1264 |
justify-content: center;
|
|
|
|
| 1267 |
color: var(--ink);
|
| 1268 |
border: 1px solid var(--ink);
|
| 1269 |
background: var(--phosphor);
|
| 1270 |
+
font: 700 var(--type-caption)/1 var(--mono);
|
| 1271 |
text-transform: uppercase;
|
| 1272 |
cursor: pointer;
|
| 1273 |
transition: transform 150ms ease, box-shadow 150ms ease, background 150ms ease;
|
|
|
|
| 1300 |
background: var(--rust);
|
| 1301 |
}
|
| 1302 |
|
| 1303 |
+
.send-button.load {
|
| 1304 |
+
background: var(--phosphor);
|
| 1305 |
+
}
|
| 1306 |
+
|
| 1307 |
.inspector {
|
| 1308 |
grid-area: inspector;
|
| 1309 |
min-height: calc(100dvh - 76px);
|
|
|
|
| 1333 |
}
|
| 1334 |
|
| 1335 |
.section-heading h2 {
|
| 1336 |
+
font-size: 24px;
|
| 1337 |
}
|
| 1338 |
|
| 1339 |
+
.sampling-label {
|
|
|
|
| 1340 |
display: flex;
|
| 1341 |
align-items: center;
|
| 1342 |
gap: 6px;
|
| 1343 |
color: rgba(232, 223, 202, 0.42);
|
| 1344 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1345 |
text-transform: uppercase;
|
| 1346 |
letter-spacing: 0.06em;
|
| 1347 |
}
|
|
|
|
| 1396 |
.gauge-ring small {
|
| 1397 |
margin-top: 3px;
|
| 1398 |
color: var(--paper-ghost);
|
| 1399 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1400 |
text-transform: uppercase;
|
| 1401 |
}
|
| 1402 |
|
|
|
|
| 1409 |
justify-content: space-between;
|
| 1410 |
padding: 7px 0;
|
| 1411 |
border-bottom: 1px solid var(--line-light);
|
| 1412 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1413 |
}
|
| 1414 |
|
| 1415 |
.speed-gauge dt {
|
|
|
|
| 1458 |
|
| 1459 |
.device-card small,
|
| 1460 |
.memory-grid small,
|
| 1461 |
+
.meter small {
|
|
|
|
| 1462 |
color: rgba(232, 223, 202, 0.42);
|
| 1463 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1464 |
text-transform: uppercase;
|
| 1465 |
}
|
| 1466 |
|
| 1467 |
.device-card strong {
|
| 1468 |
+
font: 500 var(--type-caption)/1 var(--mono);
|
| 1469 |
}
|
| 1470 |
|
| 1471 |
.device-card > span:last-child {
|
| 1472 |
display: grid;
|
| 1473 |
justify-items: end;
|
| 1474 |
color: var(--phosphor);
|
| 1475 |
+
font: 600 15px/1 var(--mono);
|
| 1476 |
}
|
| 1477 |
|
| 1478 |
.device-card > span:last-child small {
|
| 1479 |
margin-top: 3px;
|
| 1480 |
color: rgba(232, 223, 202, 0.37);
|
| 1481 |
+
font-size: var(--type-utility);
|
| 1482 |
}
|
| 1483 |
|
| 1484 |
.meter {
|
|
|
|
| 1489 |
display: flex;
|
| 1490 |
justify-content: space-between;
|
| 1491 |
margin-bottom: 6px;
|
| 1492 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1493 |
}
|
| 1494 |
|
| 1495 |
.meter-track {
|
|
|
|
| 1520 |
}
|
| 1521 |
|
| 1522 |
.memory-grid strong {
|
| 1523 |
+
font: 500 var(--type-caption)/1 var(--mono);
|
| 1524 |
}
|
| 1525 |
|
| 1526 |
.event-log {
|
|
|
|
| 1554 |
display: block;
|
| 1555 |
padding: 5px 0;
|
| 1556 |
color: rgba(232, 223, 202, 0.36);
|
| 1557 |
+
font: var(--type-utility)/1.4 var(--mono);
|
| 1558 |
}
|
| 1559 |
|
| 1560 |
.event-pip {
|
|
|
|
| 1565 |
|
| 1566 |
.event-log time {
|
| 1567 |
color: rgba(232, 223, 202, 0.34);
|
| 1568 |
+
font: var(--type-utility)/1.2 var(--mono);
|
| 1569 |
}
|
| 1570 |
|
| 1571 |
.event-log div {
|
|
|
|
| 1574 |
}
|
| 1575 |
|
| 1576 |
.event-log strong {
|
| 1577 |
+
font: 500 var(--type-caption)/1 var(--mono);
|
| 1578 |
}
|
| 1579 |
|
| 1580 |
.event-log small {
|
| 1581 |
color: rgba(232, 223, 202, 0.42);
|
| 1582 |
+
font-size: var(--type-utility);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1583 |
}
|
| 1584 |
|
| 1585 |
.artifact-close {
|
|
|
|
| 1594 |
.artifact-empty {
|
| 1595 |
margin: 0;
|
| 1596 |
color: rgba(232, 223, 202, 0.4);
|
| 1597 |
+
font: var(--type-utility)/1.5 var(--mono);
|
| 1598 |
}
|
| 1599 |
|
| 1600 |
.artifact-pane {
|
|
|
|
| 1616 |
border: 0;
|
| 1617 |
border-right: 1px solid var(--line-light);
|
| 1618 |
background: transparent;
|
| 1619 |
+
min-height: 44px;
|
| 1620 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1621 |
text-transform: uppercase;
|
| 1622 |
cursor: pointer;
|
| 1623 |
}
|
|
|
|
| 1631 |
overflow: hidden;
|
| 1632 |
padding: 0 8px;
|
| 1633 |
color: rgba(232, 223, 202, 0.45);
|
| 1634 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1635 |
text-align: right;
|
| 1636 |
text-overflow: ellipsis;
|
| 1637 |
white-space: nowrap;
|
|
|
|
| 1658 |
color: var(--paper);
|
| 1659 |
border: 1px solid var(--rust);
|
| 1660 |
background: rgba(17, 21, 18, 0.9);
|
| 1661 |
+
min-height: 44px;
|
| 1662 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1663 |
text-transform: uppercase;
|
| 1664 |
cursor: pointer;
|
| 1665 |
}
|
|
|
|
| 1679 |
max-width: 250px;
|
| 1680 |
margin: 0;
|
| 1681 |
color: rgba(17, 21, 18, 0.62);
|
| 1682 |
+
font: var(--type-caption)/1.5 var(--mono);
|
| 1683 |
}
|
| 1684 |
|
| 1685 |
.artifact-launch button {
|
|
|
|
| 1687 |
color: var(--ink);
|
| 1688 |
border: 1px solid var(--ink);
|
| 1689 |
background: var(--phosphor);
|
| 1690 |
+
min-height: 44px;
|
| 1691 |
+
font: 700 var(--type-utility)/1 var(--mono);
|
| 1692 |
text-transform: uppercase;
|
| 1693 |
cursor: pointer;
|
| 1694 |
}
|
|
|
|
| 1700 |
padding: 10px;
|
| 1701 |
color: var(--paper);
|
| 1702 |
background: #0d100e;
|
| 1703 |
+
font: var(--type-utility)/1.5 var(--mono);
|
| 1704 |
white-space: pre-wrap;
|
| 1705 |
}
|
| 1706 |
|
|
|
|
| 1770 |
|
| 1771 |
.setting-number {
|
| 1772 |
color: var(--phosphor);
|
| 1773 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1774 |
}
|
| 1775 |
|
| 1776 |
.setting-group > div,
|
|
|
|
| 1782 |
.setting-group small,
|
| 1783 |
.range-setting small {
|
| 1784 |
color: rgba(232, 223, 202, 0.43);
|
| 1785 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1786 |
letter-spacing: 0.08em;
|
| 1787 |
text-transform: uppercase;
|
| 1788 |
}
|
| 1789 |
|
| 1790 |
.setting-group strong,
|
| 1791 |
.range-setting strong {
|
| 1792 |
+
font: 500 var(--type-caption)/1 var(--mono);
|
| 1793 |
}
|
| 1794 |
|
| 1795 |
.setting-locked {
|
| 1796 |
color: rgba(232, 223, 202, 0.36);
|
| 1797 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1798 |
text-transform: uppercase;
|
| 1799 |
}
|
| 1800 |
|
|
|
|
| 1805 |
border: 1px solid var(--line-light);
|
| 1806 |
border-radius: 0;
|
| 1807 |
background: var(--ink-raised);
|
| 1808 |
+
min-height: 44px;
|
| 1809 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1810 |
}
|
| 1811 |
|
| 1812 |
.range-setting {
|
|
|
|
| 1866 |
display: flex;
|
| 1867 |
justify-content: space-between;
|
| 1868 |
color: rgba(232, 223, 202, 0.35);
|
| 1869 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1870 |
}
|
| 1871 |
|
| 1872 |
.range-limits i {
|
|
|
|
| 1883 |
display: flex;
|
| 1884 |
justify-content: space-between;
|
| 1885 |
color: rgba(232, 223, 202, 0.43);
|
| 1886 |
+
font: var(--type-utility)/1 var(--mono);
|
| 1887 |
letter-spacing: 0.05em;
|
| 1888 |
text-transform: uppercase;
|
| 1889 |
}
|
|
|
|
| 1894 |
color: var(--paper);
|
| 1895 |
border: 1px solid var(--line-light);
|
| 1896 |
background: var(--ink-raised);
|
| 1897 |
+
font: var(--type-caption)/1.5 var(--mono);
|
| 1898 |
}
|
| 1899 |
|
| 1900 |
.settings-actions {
|
|
|
|
| 1909 |
color: var(--paper);
|
| 1910 |
border: 1px solid var(--line-light);
|
| 1911 |
background: transparent;
|
| 1912 |
+
min-height: 44px;
|
| 1913 |
+
font: var(--type-utility)/1.2 var(--mono);
|
| 1914 |
text-transform: uppercase;
|
| 1915 |
cursor: pointer;
|
| 1916 |
}
|
|
|
|
| 1945 |
|
| 1946 |
.boot-screen p:last-child {
|
| 1947 |
color: var(--paper-ghost);
|
| 1948 |
+
font: var(--type-caption)/1.5 var(--mono);
|
| 1949 |
}
|
| 1950 |
|
| 1951 |
.range-ticks i {
|
|
|
|
| 1972 |
.settings-note p {
|
| 1973 |
margin: 0;
|
| 1974 |
color: rgba(232, 223, 202, 0.58);
|
| 1975 |
+
font-size: var(--type-caption);
|
| 1976 |
line-height: 1.5;
|
| 1977 |
}
|
| 1978 |
|
|
|
|
| 2000 |
53%, 100% { opacity: 0; }
|
| 2001 |
}
|
| 2002 |
|
| 2003 |
+
@keyframes activity-pulse {
|
| 2004 |
+
0%, 100% { opacity: 0.55; transform: scale(0.82); }
|
| 2005 |
+
50% { opacity: 1; transform: scale(1); }
|
| 2006 |
+
}
|
| 2007 |
+
|
| 2008 |
@keyframes sheet-in {
|
| 2009 |
from { opacity: 0; transform: translateX(34px); }
|
| 2010 |
to { opacity: 1; transform: translateX(0); }
|
|
|
|
| 2024 |
}
|
| 2025 |
|
| 2026 |
.workspace-grid {
|
| 2027 |
+
grid-template-columns: 280px minmax(500px, 1fr);
|
| 2028 |
grid-template-areas:
|
| 2029 |
"models chat"
|
| 2030 |
"inspector inspector";
|
|
|
|
| 2117 |
scroll-snap-align: start;
|
| 2118 |
}
|
| 2119 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2120 |
.rail-footnote {
|
| 2121 |
display: none;
|
| 2122 |
}
|
| 2123 |
|
| 2124 |
.chat-surface {
|
| 2125 |
display: grid;
|
| 2126 |
+
grid-template-rows: auto 19px minmax(240px, auto) auto;
|
| 2127 |
overflow: visible;
|
| 2128 |
}
|
| 2129 |
|
|
|
|
| 2140 |
display: none;
|
| 2141 |
}
|
| 2142 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2143 |
.live-label {
|
| 2144 |
grid-column: 1;
|
| 2145 |
}
|
| 2146 |
|
| 2147 |
.composer {
|
| 2148 |
+
position: relative;
|
|
|
|
| 2149 |
z-index: 5;
|
| 2150 |
padding-inline: 14px;
|
| 2151 |
}
|
| 2152 |
|
| 2153 |
+
.transcript-empty {
|
| 2154 |
+
margin-top: 4vh;
|
| 2155 |
+
}
|
| 2156 |
+
|
| 2157 |
.inspector {
|
| 2158 |
display: block;
|
| 2159 |
border-top: 1px solid var(--phosphor);
|
|
|
|
| 2172 |
|
| 2173 |
.brand-block .eyebrow,
|
| 2174 |
.local-promise strong {
|
| 2175 |
+
font-size: var(--type-utility);
|
| 2176 |
}
|
| 2177 |
|
| 2178 |
.chat-header {
|
|
|
|
| 2192 |
display: none;
|
| 2193 |
}
|
| 2194 |
|
| 2195 |
+
.composer-activity {
|
| 2196 |
+
height: 176px;
|
| 2197 |
+
grid-template-rows: 24px 30px 44px 36px;
|
| 2198 |
+
padding-inline: 9px;
|
| 2199 |
+
}
|
| 2200 |
+
|
| 2201 |
+
.activity-stage-line {
|
| 2202 |
+
gap: 3px;
|
| 2203 |
+
}
|
| 2204 |
+
|
| 2205 |
+
.activity-stage-line span {
|
| 2206 |
+
gap: 3px;
|
| 2207 |
+
letter-spacing: 0;
|
| 2208 |
+
}
|
| 2209 |
+
|
| 2210 |
+
.activity-stage-line i {
|
| 2211 |
+
display: none;
|
| 2212 |
+
}
|
| 2213 |
+
|
| 2214 |
+
.load-live-grid {
|
| 2215 |
+
gap: 7px;
|
| 2216 |
+
}
|
| 2217 |
+
|
| 2218 |
+
.activity-memory {
|
| 2219 |
+
min-width: 68px;
|
| 2220 |
+
}
|
| 2221 |
+
|
| 2222 |
+
.shard-strip {
|
| 2223 |
+
gap: 3px;
|
| 2224 |
+
}
|
| 2225 |
+
|
| 2226 |
+
.shard-cell small {
|
| 2227 |
+
display: none;
|
| 2228 |
+
}
|
| 2229 |
+
|
| 2230 |
.composer-field {
|
| 2231 |
grid-template-columns: 1fr;
|
| 2232 |
+
padding-bottom: 62px;
|
| 2233 |
}
|
| 2234 |
|
| 2235 |
.send-button {
|
| 2236 |
position: absolute;
|
| 2237 |
right: 0;
|
| 2238 |
bottom: 0;
|
| 2239 |
+
height: 52px;
|
| 2240 |
}
|
| 2241 |
|
| 2242 |
.speed-gauge {
|
src/bench/report.test.ts
CHANGED
|
@@ -23,17 +23,6 @@ const JSPI_WASM_SHA256 = 'd'.repeat(64);
|
|
| 23 |
const COMPAT_WASM_SHA256 = 'c'.repeat(64);
|
| 24 |
const COMPAT_WORKER_SHA256 = 'b'.repeat(64);
|
| 25 |
|
| 26 |
-
const dflash27bPairing = {
|
| 27 |
-
support: 'conditional',
|
| 28 |
-
compatibilityStatus: 'native-greedy-parity-pass',
|
| 29 |
-
runtimeAvailable: false,
|
| 30 |
-
blockSize: 16,
|
| 31 |
-
maskTokenId: 248_070,
|
| 32 |
-
targetLayerIdsZeroBased: [1, 16, 31, 46, 61],
|
| 33 |
-
targetTapsOneBased: [2, 17, 32, 47, 62],
|
| 34 |
-
reason: 'Native parity passes; browser activation remains gated.',
|
| 35 |
-
} as const;
|
| 36 |
-
|
| 37 |
const model = {
|
| 38 |
id: '8b',
|
| 39 |
displayName: 'Bonsai 8B',
|
|
@@ -58,15 +47,6 @@ const manifest = {
|
|
| 58 |
id: 'WaveCut/Bonsai-web-GGUF',
|
| 59 |
revision: '112ea7a1a6229bde132b176b9a72477a7ecfde64',
|
| 60 |
},
|
| 61 |
-
dflash: {
|
| 62 |
-
status: 'conditional',
|
| 63 |
-
pairings: {
|
| 64 |
-
'1_7b': { support: 'unsupported', reason: 'No compatible 1.7B drafter is locked.' },
|
| 65 |
-
'4b': { support: 'unsupported', reason: 'The 27B drafter is incompatible with Bonsai 4B.' },
|
| 66 |
-
'8b': { support: 'unsupported', reason: 'The 27B drafter is incompatible with Bonsai 8B.' },
|
| 67 |
-
'27b': dflash27bPairing,
|
| 68 |
-
},
|
| 69 |
-
},
|
| 70 |
} as unknown as ModelManifestV2;
|
| 71 |
|
| 72 |
const capabilities = {
|
|
@@ -112,6 +92,7 @@ const backendReport = {
|
|
| 112 |
cacheTypeK: 'f16',
|
| 113 |
cacheTypeV: 'f16',
|
| 114 |
webgpuKvBufferBytes: 56 * 1024 ** 2,
|
|
|
|
| 115 |
} satisfies BackendReport;
|
| 116 |
|
| 117 |
const loadResult = {
|
|
@@ -495,15 +476,6 @@ describe('benchmark report', () => {
|
|
| 495 |
operatorConfigurable: false,
|
| 496 |
status: 'fixed-release-default-alternatives-unvalidated',
|
| 497 |
},
|
| 498 |
-
dflash: {
|
| 499 |
-
enabled: false,
|
| 500 |
-
runtimeAvailable: false,
|
| 501 |
-
globalStatus: 'conditional',
|
| 502 |
-
pairing: {
|
| 503 |
-
support: 'unsupported',
|
| 504 |
-
reason: 'The 27B drafter is incompatible with Bonsai 8B.',
|
| 505 |
-
},
|
| 506 |
-
},
|
| 507 |
});
|
| 508 |
});
|
| 509 |
|
|
@@ -582,39 +554,6 @@ describe('benchmark report', () => {
|
|
| 582 |
expect(report.generation.rawReasoningText).toBe('reasoning-only fixture');
|
| 583 |
});
|
| 584 |
|
| 585 |
-
it('exports the selected model DFlash pairing for unsupported 1.7B and conditional 27B', () => {
|
| 586 |
-
const reportFor = (modelId: ManifestModelV2['id']) => buildBenchmarkReport(observation({
|
| 587 |
-
model: {
|
| 588 |
-
...model,
|
| 589 |
-
id: modelId,
|
| 590 |
-
displayName: `Bonsai ${modelId}`,
|
| 591 |
-
} as ManifestModelV2,
|
| 592 |
-
loadResult: { ...loadResult, modelId },
|
| 593 |
-
}));
|
| 594 |
-
|
| 595 |
-
expect(reportFor('1_7b').releasePolicy.dflash).toEqual({
|
| 596 |
-
enabled: false,
|
| 597 |
-
runtimeAvailable: false,
|
| 598 |
-
globalStatus: 'conditional',
|
| 599 |
-
pairing: {
|
| 600 |
-
support: 'unsupported',
|
| 601 |
-
reason: 'No compatible 1.7B drafter is locked.',
|
| 602 |
-
},
|
| 603 |
-
});
|
| 604 |
-
|
| 605 |
-
const report27b = reportFor('27b');
|
| 606 |
-
expect(report27b.releasePolicy.dflash).toEqual({
|
| 607 |
-
enabled: false,
|
| 608 |
-
runtimeAvailable: false,
|
| 609 |
-
globalStatus: 'conditional',
|
| 610 |
-
pairing: dflash27bPairing,
|
| 611 |
-
});
|
| 612 |
-
const exportedPairing = report27b.releasePolicy.dflash.pairing;
|
| 613 |
-
expect(exportedPairing.support).toBe('conditional');
|
| 614 |
-
if (exportedPairing.support !== 'conditional') throw new Error('Expected the conditional 27B pairing.');
|
| 615 |
-
expect(exportedPairing.targetLayerIdsZeroBased).not.toBe(dflash27bPairing.targetLayerIdsZeroBased);
|
| 616 |
-
});
|
| 617 |
-
|
| 618 |
it('uses the exact token-id trace when final engine usage is present but incomplete', () => {
|
| 619 |
const report = buildBenchmarkReport(observation({
|
| 620 |
streamedTokenEvents: 64,
|
|
|
|
| 23 |
const COMPAT_WASM_SHA256 = 'c'.repeat(64);
|
| 24 |
const COMPAT_WORKER_SHA256 = 'b'.repeat(64);
|
| 25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
const model = {
|
| 27 |
id: '8b',
|
| 28 |
displayName: 'Bonsai 8B',
|
|
|
|
| 47 |
id: 'WaveCut/Bonsai-web-GGUF',
|
| 48 |
revision: '112ea7a1a6229bde132b176b9a72477a7ecfde64',
|
| 49 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
} as unknown as ModelManifestV2;
|
| 51 |
|
| 52 |
const capabilities = {
|
|
|
|
| 92 |
cacheTypeK: 'f16',
|
| 93 |
cacheTypeV: 'f16',
|
| 94 |
webgpuKvBufferBytes: 56 * 1024 ** 2,
|
| 95 |
+
webgpuTrace: [],
|
| 96 |
} satisfies BackendReport;
|
| 97 |
|
| 98 |
const loadResult = {
|
|
|
|
| 476 |
operatorConfigurable: false,
|
| 477 |
status: 'fixed-release-default-alternatives-unvalidated',
|
| 478 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 479 |
});
|
| 480 |
});
|
| 481 |
|
|
|
|
| 554 |
expect(report.generation.rawReasoningText).toBe('reasoning-only fixture');
|
| 555 |
});
|
| 556 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 557 |
it('uses the exact token-id trace when final engine usage is present but incomplete', () => {
|
| 558 |
const report = buildBenchmarkReport(observation({
|
| 559 |
streamedTokenEvents: 64,
|
src/bench/report.ts
CHANGED
|
@@ -203,12 +203,6 @@ export interface BenchmarkReport {
|
|
| 203 |
operatorConfigurable: false;
|
| 204 |
status: 'fixed-release-default-alternatives-unvalidated';
|
| 205 |
};
|
| 206 |
-
dflash: {
|
| 207 |
-
enabled: false;
|
| 208 |
-
runtimeAvailable: false;
|
| 209 |
-
globalStatus: ModelManifestV2['dflash']['status'];
|
| 210 |
-
pairing: ModelManifestV2['dflash']['pairings'][ManifestModelV2['id']];
|
| 211 |
-
};
|
| 212 |
};
|
| 213 |
}
|
| 214 |
|
|
@@ -571,8 +565,6 @@ export function buildBenchmarkReport(observation: BenchmarkObservation): Benchma
|
|
| 571 |
const device = loadResult.backend === 'webgpu'
|
| 572 |
? capabilities.webgpu.name || 'WebGPU adapter'
|
| 573 |
: `CPU · ${capabilities.hardwareConcurrency} logical cores`;
|
| 574 |
-
const dflashPairing = observation.manifest.dflash.pairings[observation.model.id];
|
| 575 |
-
|
| 576 |
return {
|
| 577 |
schemaVersion: 3,
|
| 578 |
kind: 'bonsai-browser-benchmark',
|
|
@@ -694,18 +686,6 @@ export function buildBenchmarkReport(observation: BenchmarkObservation): Benchma
|
|
| 694 |
operatorConfigurable: false,
|
| 695 |
status: 'fixed-release-default-alternatives-unvalidated',
|
| 696 |
},
|
| 697 |
-
dflash: {
|
| 698 |
-
enabled: false,
|
| 699 |
-
runtimeAvailable: false,
|
| 700 |
-
globalStatus: observation.manifest.dflash.status,
|
| 701 |
-
pairing: dflashPairing.support === 'conditional'
|
| 702 |
-
? {
|
| 703 |
-
...dflashPairing,
|
| 704 |
-
targetLayerIdsZeroBased: [...dflashPairing.targetLayerIdsZeroBased],
|
| 705 |
-
targetTapsOneBased: [...dflashPairing.targetTapsOneBased],
|
| 706 |
-
}
|
| 707 |
-
: { ...dflashPairing },
|
| 708 |
-
},
|
| 709 |
},
|
| 710 |
};
|
| 711 |
}
|
|
|
|
| 203 |
operatorConfigurable: false;
|
| 204 |
status: 'fixed-release-default-alternatives-unvalidated';
|
| 205 |
};
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 206 |
};
|
| 207 |
}
|
| 208 |
|
|
|
|
| 565 |
const device = loadResult.backend === 'webgpu'
|
| 566 |
? capabilities.webgpu.name || 'WebGPU adapter'
|
| 567 |
: `CPU · ${capabilities.hardwareConcurrency} logical cores`;
|
|
|
|
|
|
|
| 568 |
return {
|
| 569 |
schemaVersion: 3,
|
| 570 |
kind: 'bonsai-browser-benchmark',
|
|
|
|
| 686 |
operatorConfigurable: false,
|
| 687 |
status: 'fixed-release-default-alternatives-unvalidated',
|
| 688 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 689 |
},
|
| 690 |
};
|
| 691 |
}
|
src/engine/blob-integrity.test.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
import { describe, expect, it } from 'vitest';
|
| 2 |
import { verifyBlobSha256 } from './blob-integrity';
|
| 3 |
|
| 4 |
describe('verifyBlobSha256', () => {
|
|
@@ -23,6 +23,50 @@ describe('verifyBlobSha256', () => {
|
|
| 23 |
expect(corrupted.actualSha256).not.toBe(expected.actualSha256);
|
| 24 |
});
|
| 25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
it('stops hashing when the caller aborts', async () => {
|
| 27 |
const controller = new AbortController();
|
| 28 |
controller.abort();
|
|
|
|
| 1 |
+
import { describe, expect, it, vi } from 'vitest';
|
| 2 |
import { verifyBlobSha256 } from './blob-integrity';
|
| 3 |
|
| 4 |
describe('verifyBlobSha256', () => {
|
|
|
|
| 23 |
expect(corrupted.actualSha256).not.toBe(expected.actualSha256);
|
| 24 |
});
|
| 25 |
|
| 26 |
+
it('reports monotonic byte progress while hashing a shard', async () => {
|
| 27 |
+
const data = new Uint8Array(256 * 1024);
|
| 28 |
+
data.fill(0x61);
|
| 29 |
+
const blob = new Blob([data]);
|
| 30 |
+
Object.defineProperty(blob, 'stream', {
|
| 31 |
+
value: () => {
|
| 32 |
+
let offset = 0;
|
| 33 |
+
return new ReadableStream<Uint8Array>({
|
| 34 |
+
pull(controller) {
|
| 35 |
+
if (offset >= data.byteLength) {
|
| 36 |
+
controller.close();
|
| 37 |
+
return;
|
| 38 |
+
}
|
| 39 |
+
const nextOffset = Math.min(data.byteLength, offset + (64 * 1024));
|
| 40 |
+
controller.enqueue(data.subarray(offset, nextOffset));
|
| 41 |
+
offset = nextOffset;
|
| 42 |
+
},
|
| 43 |
+
});
|
| 44 |
+
},
|
| 45 |
+
});
|
| 46 |
+
const progress: Array<{ loadedBytes: number; totalBytes: number }> = [];
|
| 47 |
+
let timestamp = 0;
|
| 48 |
+
const now = vi.spyOn(performance, 'now').mockImplementation(() => {
|
| 49 |
+
timestamp += 101;
|
| 50 |
+
return timestamp;
|
| 51 |
+
});
|
| 52 |
+
|
| 53 |
+
try {
|
| 54 |
+
await verifyBlobSha256(blob, '0'.repeat(64), undefined, (loadedBytes, totalBytes) => {
|
| 55 |
+
progress.push({ loadedBytes, totalBytes });
|
| 56 |
+
});
|
| 57 |
+
} finally {
|
| 58 |
+
now.mockRestore();
|
| 59 |
+
}
|
| 60 |
+
|
| 61 |
+
expect(progress[0]).toEqual({ loadedBytes: 0, totalBytes: blob.size });
|
| 62 |
+
expect(progress.at(-1)).toEqual({ loadedBytes: blob.size, totalBytes: blob.size });
|
| 63 |
+
expect(progress.some(({ loadedBytes }) => loadedBytes > 0 && loadedBytes < blob.size)).toBe(true);
|
| 64 |
+
expect(progress.every(({ totalBytes }) => totalBytes === blob.size)).toBe(true);
|
| 65 |
+
expect(progress.every((sample, index) => (
|
| 66 |
+
index === 0 || sample.loadedBytes >= (progress[index - 1]?.loadedBytes ?? 0)
|
| 67 |
+
))).toBe(true);
|
| 68 |
+
});
|
| 69 |
+
|
| 70 |
it('stops hashing when the caller aborts', async () => {
|
| 71 |
const controller = new AbortController();
|
| 72 |
controller.abort();
|
src/engine/blob-integrity.ts
CHANGED
|
@@ -11,6 +11,7 @@ export async function verifyBlobSha256(
|
|
| 11 |
blob: Blob,
|
| 12 |
expectedSha256: string,
|
| 13 |
signal?: AbortSignal,
|
|
|
|
| 14 |
): Promise<BlobIntegrityResult> {
|
| 15 |
if (signal) {
|
| 16 |
throwIfAborted(signal);
|
|
@@ -21,6 +22,9 @@ export async function verifyBlobSha256(
|
|
| 21 |
void reader.cancel(abortError()).catch(() => undefined);
|
| 22 |
};
|
| 23 |
signal?.addEventListener('abort', cancelRead, { once: true });
|
|
|
|
|
|
|
|
|
|
| 24 |
try {
|
| 25 |
while (true) {
|
| 26 |
if (signal) {
|
|
@@ -34,7 +38,14 @@ export async function verifyBlobSha256(
|
|
| 34 |
break;
|
| 35 |
}
|
| 36 |
hasher.update(value);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
}
|
|
|
|
| 38 |
const actualSha256 = bytesToHex(hasher.digest());
|
| 39 |
return {
|
| 40 |
actualSha256,
|
|
|
|
| 11 |
blob: Blob,
|
| 12 |
expectedSha256: string,
|
| 13 |
signal?: AbortSignal,
|
| 14 |
+
onProgress?: (loadedBytes: number, totalBytes: number) => void,
|
| 15 |
): Promise<BlobIntegrityResult> {
|
| 16 |
if (signal) {
|
| 17 |
throwIfAborted(signal);
|
|
|
|
| 22 |
void reader.cancel(abortError()).catch(() => undefined);
|
| 23 |
};
|
| 24 |
signal?.addEventListener('abort', cancelRead, { once: true });
|
| 25 |
+
let loadedBytes = 0;
|
| 26 |
+
let lastProgressAt = 0;
|
| 27 |
+
onProgress?.(0, blob.size);
|
| 28 |
try {
|
| 29 |
while (true) {
|
| 30 |
if (signal) {
|
|
|
|
| 38 |
break;
|
| 39 |
}
|
| 40 |
hasher.update(value);
|
| 41 |
+
loadedBytes += value.byteLength;
|
| 42 |
+
const now = performance.now();
|
| 43 |
+
if (now - lastProgressAt >= 100) {
|
| 44 |
+
lastProgressAt = now;
|
| 45 |
+
onProgress?.(Math.min(loadedBytes, blob.size), blob.size);
|
| 46 |
+
}
|
| 47 |
}
|
| 48 |
+
onProgress?.(blob.size, blob.size);
|
| 49 |
const actualSha256 = bytesToHex(hasher.digest());
|
| 50 |
return {
|
| 51 |
actualSha256,
|
src/engine/client.test.ts
CHANGED
|
@@ -63,6 +63,10 @@ describe('BrowserEngineClient worker lifecycle', () => {
|
|
| 63 |
shardIndex: null,
|
| 64 |
shardCount: 1,
|
| 65 |
shardPath: null,
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
});
|
| 67 |
|
| 68 |
controller.abort();
|
|
@@ -118,6 +122,17 @@ describe('BrowserEngineClient worker lifecycle', () => {
|
|
| 118 |
shardIndex: 0,
|
| 119 |
shardCount: 1,
|
| 120 |
shardPath: 'model.gguf',
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 121 |
});
|
| 122 |
|
| 123 |
controller.abort();
|
|
@@ -156,6 +171,17 @@ describe('BrowserEngineClient worker lifecycle', () => {
|
|
| 156 |
shardIndex: 0,
|
| 157 |
shardCount: 1,
|
| 158 |
shardPath: 'model.gguf',
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 159 |
});
|
| 160 |
|
| 161 |
controller.abort();
|
|
|
|
| 63 |
shardIndex: null,
|
| 64 |
shardCount: 1,
|
| 65 |
shardPath: null,
|
| 66 |
+
stageProgress: 0,
|
| 67 |
+
nativeStage: null,
|
| 68 |
+
residentBytes: null,
|
| 69 |
+
shards: [],
|
| 70 |
});
|
| 71 |
|
| 72 |
controller.abort();
|
|
|
|
| 122 |
shardIndex: 0,
|
| 123 |
shardCount: 1,
|
| 124 |
shardPath: 'model.gguf',
|
| 125 |
+
stageProgress: null,
|
| 126 |
+
nativeStage: null,
|
| 127 |
+
residentBytes: null,
|
| 128 |
+
shards: [{
|
| 129 |
+
index: 0,
|
| 130 |
+
path: 'model.gguf',
|
| 131 |
+
loadedBytes: 1,
|
| 132 |
+
verifiedBytes: 0,
|
| 133 |
+
totalBytes: 100,
|
| 134 |
+
state: 'downloading',
|
| 135 |
+
}],
|
| 136 |
});
|
| 137 |
|
| 138 |
controller.abort();
|
|
|
|
| 171 |
shardIndex: 0,
|
| 172 |
shardCount: 1,
|
| 173 |
shardPath: 'model.gguf',
|
| 174 |
+
stageProgress: null,
|
| 175 |
+
nativeStage: null,
|
| 176 |
+
residentBytes: null,
|
| 177 |
+
shards: [{
|
| 178 |
+
index: 0,
|
| 179 |
+
path: 'model.gguf',
|
| 180 |
+
loadedBytes: 50,
|
| 181 |
+
verifiedBytes: 0,
|
| 182 |
+
totalBytes: 100,
|
| 183 |
+
state: 'downloading',
|
| 184 |
+
}],
|
| 185 |
});
|
| 186 |
|
| 187 |
controller.abort();
|
src/engine/client.ts
CHANGED
|
@@ -14,12 +14,14 @@ import type {
|
|
| 14 |
|
| 15 |
type ProgressEvent = Extract<EngineEvent, { event: 'progress' }>;
|
| 16 |
type TokenEvent = Extract<EngineEvent, { event: 'token' }>;
|
|
|
|
| 17 |
|
| 18 |
export interface EngineRequestOptions {
|
| 19 |
requestId?: string;
|
| 20 |
signal?: AbortSignal;
|
| 21 |
onProgress?: (event: ProgressEvent) => void;
|
| 22 |
onToken?: (text: string, reasoningDelta?: string) => void;
|
|
|
|
| 23 |
}
|
| 24 |
|
| 25 |
interface PendingRequest {
|
|
@@ -244,8 +246,10 @@ export class BrowserEngineClient {
|
|
| 244 |
return;
|
| 245 |
}
|
| 246 |
pending.options.onProgress?.(message);
|
| 247 |
-
} else {
|
| 248 |
pending.options.onToken?.(message.text, message.reasoningDelta);
|
|
|
|
|
|
|
| 249 |
}
|
| 250 |
return;
|
| 251 |
}
|
|
|
|
| 14 |
|
| 15 |
type ProgressEvent = Extract<EngineEvent, { event: 'progress' }>;
|
| 16 |
type TokenEvent = Extract<EngineEvent, { event: 'token' }>;
|
| 17 |
+
type GenerationEvent = Extract<EngineEvent, { event: 'generation' }>;
|
| 18 |
|
| 19 |
export interface EngineRequestOptions {
|
| 20 |
requestId?: string;
|
| 21 |
signal?: AbortSignal;
|
| 22 |
onProgress?: (event: ProgressEvent) => void;
|
| 23 |
onToken?: (text: string, reasoningDelta?: string) => void;
|
| 24 |
+
onGenerationProgress?: (event: GenerationEvent) => void;
|
| 25 |
}
|
| 26 |
|
| 27 |
interface PendingRequest {
|
|
|
|
| 246 |
return;
|
| 247 |
}
|
| 248 |
pending.options.onProgress?.(message);
|
| 249 |
+
} else if (message.event === 'token') {
|
| 250 |
pending.options.onToken?.(message.text, message.reasoningDelta);
|
| 251 |
+
} else {
|
| 252 |
+
pending.options.onGenerationProgress?.(message);
|
| 253 |
}
|
| 254 |
return;
|
| 255 |
}
|
src/engine/device-gate.test.ts
CHANGED
|
@@ -222,6 +222,7 @@ describe('assertBackendPolicy', () => {
|
|
| 222 |
cacheTypeK: 'f16',
|
| 223 |
cacheTypeV: 'f16',
|
| 224 |
webgpuKvBufferBytes: 56 * 1024 ** 2,
|
|
|
|
| 225 |
};
|
| 226 |
|
| 227 |
it('accepts a proven single WebGPU graph', () => {
|
|
|
|
| 222 |
cacheTypeK: 'f16',
|
| 223 |
cacheTypeV: 'f16',
|
| 224 |
webgpuKvBufferBytes: 56 * 1024 ** 2,
|
| 225 |
+
webgpuTrace: [],
|
| 226 |
};
|
| 227 |
|
| 228 |
it('accepts a proven single WebGPU graph', () => {
|
src/engine/manifest.test.ts
CHANGED
|
@@ -50,70 +50,11 @@ function rawModel(id: ModelTierId) {
|
|
| 50 |
};
|
| 51 |
}
|
| 52 |
|
| 53 |
-
function rawDFlash() {
|
| 54 |
-
return {
|
| 55 |
-
status: 'conditional',
|
| 56 |
-
enabledByDefault: false,
|
| 57 |
-
runtimeAvailable: false,
|
| 58 |
-
source: {
|
| 59 |
-
repo: 'z-lab/Qwen3.6-27B-DFlash',
|
| 60 |
-
revision: REVISION,
|
| 61 |
-
file: 'model.safetensors',
|
| 62 |
-
bytes: 3_460_432_504,
|
| 63 |
-
sha256: SHA,
|
| 64 |
-
format: 'safetensors',
|
| 65 |
-
dtype: 'BF16',
|
| 66 |
-
parameters: 1_730_213_120,
|
| 67 |
-
sourceOnly: true,
|
| 68 |
-
license: 'MIT',
|
| 69 |
-
},
|
| 70 |
-
nativeReference: {
|
| 71 |
-
path: 'dflash/Qwen3.6-27B-DFlash-Q8_0.gguf',
|
| 72 |
-
bytes: 1_849_481_440,
|
| 73 |
-
sha256: SHA,
|
| 74 |
-
format: 'GGUF',
|
| 75 |
-
quantization: 'Q8_0',
|
| 76 |
-
largestTensor: { name: 'fc.weight', bytes: 139_264_000 },
|
| 77 |
-
role: 'native correctness reference only',
|
| 78 |
-
},
|
| 79 |
-
browser: {
|
| 80 |
-
enabled: false,
|
| 81 |
-
architectureCompiled: true,
|
| 82 |
-
specTypePlumbed: false,
|
| 83 |
-
candidateQuantizationStatus: 'q8-rejected-binding-limit',
|
| 84 |
-
},
|
| 85 |
-
validation: {
|
| 86 |
-
shape: 'pass',
|
| 87 |
-
tokenizer: 'exact-match',
|
| 88 |
-
targetTaps: [2, 17, 32, 47, 62],
|
| 89 |
-
nativeGreedyParity: { status: 'pass', tokens: 32, temperature: 0 },
|
| 90 |
-
acceptanceBenchmark: 'pending',
|
| 91 |
-
browserSpeedGate: 'pending',
|
| 92 |
-
},
|
| 93 |
-
pairings: {
|
| 94 |
-
'1_7b': { support: 'unsupported', reason: 'No compatible drafter.' },
|
| 95 |
-
'4b': { support: 'unsupported', reason: 'Tokenizer mismatch.' },
|
| 96 |
-
'8b': { support: 'unsupported', reason: 'Tokenizer mismatch.' },
|
| 97 |
-
'27b': {
|
| 98 |
-
support: 'conditional',
|
| 99 |
-
compatibilityStatus: 'native-greedy-parity-pass',
|
| 100 |
-
runtimeAvailable: false,
|
| 101 |
-
blockSize: 16,
|
| 102 |
-
maskTokenId: 248070,
|
| 103 |
-
targetLayerIdsZeroBased: [1, 16, 31, 46, 61],
|
| 104 |
-
targetTapsOneBased: [2, 17, 32, 47, 62],
|
| 105 |
-
reason: 'Native parity passes; browser activation remains gated.',
|
| 106 |
-
},
|
| 107 |
-
},
|
| 108 |
-
};
|
| 109 |
-
}
|
| 110 |
-
|
| 111 |
function rawManifest() {
|
| 112 |
return {
|
| 113 |
schemaVersion: 2,
|
| 114 |
repository: { id: 'WaveCut/Bonsai-web-GGUF', revision: REVISION },
|
| 115 |
models: (['1_7b', '4b', '8b', '27b'] as const).map(rawModel),
|
| 116 |
-
dflash: rawDFlash(),
|
| 117 |
};
|
| 118 |
}
|
| 119 |
|
|
@@ -128,77 +69,6 @@ describe('model manifest v2', () => {
|
|
| 128 |
]);
|
| 129 |
});
|
| 130 |
|
| 131 |
-
it('parses the typed, default-off DFlash evidence contract', () => {
|
| 132 |
-
const dflash = parseModelManifestV2(rawManifest()).dflash;
|
| 133 |
-
|
| 134 |
-
expect(dflash.status).toBe('conditional');
|
| 135 |
-
expect(dflash.runtimeAvailable).toBe(false);
|
| 136 |
-
expect(dflash.nativeReference.largestTensor).toEqual({ name: 'fc.weight', bytes: 139_264_000 });
|
| 137 |
-
expect(dflash.validation.nativeGreedyParity).toEqual({ status: 'pass', tokens: 32, temperature: 0 });
|
| 138 |
-
expect(dflash.pairings['27b'].support).toBe('conditional');
|
| 139 |
-
expect(dflash.pairings['8b'].support).toBe('unsupported');
|
| 140 |
-
});
|
| 141 |
-
|
| 142 |
-
it('requires the DFlash add-on contract', () => {
|
| 143 |
-
const { dflash: _dflash, ...withoutDFlash } = rawManifest();
|
| 144 |
-
|
| 145 |
-
expect(() => parseModelManifestV2(withoutDFlash)).toThrow(/dflash must be an object/);
|
| 146 |
-
});
|
| 147 |
-
|
| 148 |
-
const failClosedDFlashCases: Array<[
|
| 149 |
-
string,
|
| 150 |
-
(value: ReturnType<typeof rawManifest>) => void,
|
| 151 |
-
RegExp,
|
| 152 |
-
]> = [
|
| 153 |
-
[
|
| 154 |
-
'runtime activation before browser gates pass',
|
| 155 |
-
(value) => { value.dflash.runtimeAvailable = true; },
|
| 156 |
-
/runtimeAvailable must remain false/,
|
| 157 |
-
],
|
| 158 |
-
[
|
| 159 |
-
'speculative type plumbing claimed active',
|
| 160 |
-
(value) => { value.dflash.browser.specTypePlumbed = true; },
|
| 161 |
-
/specTypePlumbed must remain false/,
|
| 162 |
-
],
|
| 163 |
-
[
|
| 164 |
-
'invalid source evidence hash',
|
| 165 |
-
(value) => { value.dflash.source.sha256 = 'invalid'; },
|
| 166 |
-
/source\.sha256 must be lowercase SHA-256/,
|
| 167 |
-
],
|
| 168 |
-
[
|
| 169 |
-
'invalid native reference byte count',
|
| 170 |
-
(value) => { value.dflash.nativeReference.bytes = 0; },
|
| 171 |
-
/nativeReference\.bytes must be a positive integer/,
|
| 172 |
-
],
|
| 173 |
-
[
|
| 174 |
-
'Q8 tensor no longer explaining the binding-limit rejection',
|
| 175 |
-
(value) => { value.dflash.nativeReference.largestTensor.bytes = 128 * 1024 * 1024; },
|
| 176 |
-
/must exceed the rejected 128 MiB baseline limit/,
|
| 177 |
-
],
|
| 178 |
-
[
|
| 179 |
-
'27B pairing no longer conditional',
|
| 180 |
-
(value) => { value.dflash.pairings['27b'].support = 'unsupported'; },
|
| 181 |
-
/pairings\.27b\.support must equal conditional/,
|
| 182 |
-
],
|
| 183 |
-
[
|
| 184 |
-
'acceptance benchmark claimed complete',
|
| 185 |
-
(value) => { value.dflash.validation.acceptanceBenchmark = 'pass'; },
|
| 186 |
-
/acceptanceBenchmark must remain pending/,
|
| 187 |
-
],
|
| 188 |
-
[
|
| 189 |
-
'validation taps disagree with the 27B pairing',
|
| 190 |
-
(value) => { value.dflash.validation.targetTaps[0] = 3; },
|
| 191 |
-
/validation\.targetTaps must match the 27b pairing/,
|
| 192 |
-
],
|
| 193 |
-
];
|
| 194 |
-
|
| 195 |
-
it.each(failClosedDFlashCases)('rejects DFlash when %s', (_label, mutate, expected) => {
|
| 196 |
-
const value = rawManifest();
|
| 197 |
-
mutate(value);
|
| 198 |
-
|
| 199 |
-
expect(() => parseModelManifestV2(value)).toThrow(expected);
|
| 200 |
-
});
|
| 201 |
-
|
| 202 |
it('rejects out-of-order shards', () => {
|
| 203 |
const value = rawManifest();
|
| 204 |
const model = value.models.find((candidate) => candidate.id === '27b')!;
|
|
|
|
| 50 |
};
|
| 51 |
}
|
| 52 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
function rawManifest() {
|
| 54 |
return {
|
| 55 |
schemaVersion: 2,
|
| 56 |
repository: { id: 'WaveCut/Bonsai-web-GGUF', revision: REVISION },
|
| 57 |
models: (['1_7b', '4b', '8b', '27b'] as const).map(rawModel),
|
|
|
|
| 58 |
};
|
| 59 |
}
|
| 60 |
|
|
|
|
| 69 |
]);
|
| 70 |
});
|
| 71 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
it('rejects out-of-order shards', () => {
|
| 73 |
const value = rawManifest();
|
| 74 |
const model = value.models.find((candidate) => candidate.id === '27b')!;
|
src/engine/manifest.ts
CHANGED
|
@@ -47,80 +47,6 @@ export interface ManifestModelV2 {
|
|
| 47 |
};
|
| 48 |
}
|
| 49 |
|
| 50 |
-
export interface ManifestDFlashSource {
|
| 51 |
-
repo: string;
|
| 52 |
-
revision: string;
|
| 53 |
-
file: string;
|
| 54 |
-
bytes: number;
|
| 55 |
-
sha256: string;
|
| 56 |
-
format: 'safetensors';
|
| 57 |
-
dtype: 'BF16';
|
| 58 |
-
parameters: number;
|
| 59 |
-
sourceOnly: true;
|
| 60 |
-
license: 'MIT';
|
| 61 |
-
}
|
| 62 |
-
|
| 63 |
-
export interface ManifestDFlashNativeReference {
|
| 64 |
-
path: string;
|
| 65 |
-
bytes: number;
|
| 66 |
-
sha256: string;
|
| 67 |
-
format: 'GGUF';
|
| 68 |
-
quantization: 'Q8_0';
|
| 69 |
-
largestTensor: {
|
| 70 |
-
name: string;
|
| 71 |
-
bytes: number;
|
| 72 |
-
};
|
| 73 |
-
role: 'native correctness reference only';
|
| 74 |
-
}
|
| 75 |
-
|
| 76 |
-
export interface ManifestDFlashUnsupportedPairing {
|
| 77 |
-
support: 'unsupported';
|
| 78 |
-
reason: string;
|
| 79 |
-
}
|
| 80 |
-
|
| 81 |
-
export interface ManifestDFlashConditionalPairing {
|
| 82 |
-
support: 'conditional';
|
| 83 |
-
compatibilityStatus: 'native-greedy-parity-pass';
|
| 84 |
-
runtimeAvailable: false;
|
| 85 |
-
blockSize: number;
|
| 86 |
-
maskTokenId: number;
|
| 87 |
-
targetLayerIdsZeroBased: number[];
|
| 88 |
-
targetTapsOneBased: number[];
|
| 89 |
-
reason: string;
|
| 90 |
-
}
|
| 91 |
-
|
| 92 |
-
export interface ManifestDFlash {
|
| 93 |
-
status: 'conditional';
|
| 94 |
-
enabledByDefault: false;
|
| 95 |
-
runtimeAvailable: false;
|
| 96 |
-
source: ManifestDFlashSource;
|
| 97 |
-
nativeReference: ManifestDFlashNativeReference;
|
| 98 |
-
browser: {
|
| 99 |
-
enabled: false;
|
| 100 |
-
architectureCompiled: true;
|
| 101 |
-
specTypePlumbed: false;
|
| 102 |
-
candidateQuantizationStatus: 'q8-rejected-binding-limit';
|
| 103 |
-
};
|
| 104 |
-
validation: {
|
| 105 |
-
shape: 'pass';
|
| 106 |
-
tokenizer: 'exact-match';
|
| 107 |
-
targetTaps: number[];
|
| 108 |
-
nativeGreedyParity: {
|
| 109 |
-
status: 'pass';
|
| 110 |
-
tokens: number;
|
| 111 |
-
temperature: 0;
|
| 112 |
-
};
|
| 113 |
-
acceptanceBenchmark: 'pending';
|
| 114 |
-
browserSpeedGate: 'pending';
|
| 115 |
-
};
|
| 116 |
-
pairings: {
|
| 117 |
-
'1_7b': ManifestDFlashUnsupportedPairing;
|
| 118 |
-
'4b': ManifestDFlashUnsupportedPairing;
|
| 119 |
-
'8b': ManifestDFlashUnsupportedPairing;
|
| 120 |
-
'27b': ManifestDFlashConditionalPairing;
|
| 121 |
-
};
|
| 122 |
-
}
|
| 123 |
-
|
| 124 |
export interface ModelManifestV2 {
|
| 125 |
schemaVersion: 2;
|
| 126 |
repository: {
|
|
@@ -128,14 +54,12 @@ export interface ModelManifestV2 {
|
|
| 128 |
revision: string | null;
|
| 129 |
};
|
| 130 |
models: ManifestModelV2[];
|
| 131 |
-
dflash: ManifestDFlash;
|
| 132 |
}
|
| 133 |
|
| 134 |
const MODEL_IDS = new Set<ModelTierId>(['1_7b', '4b', '8b', '27b']);
|
| 135 |
const SHA256 = /^[0-9a-f]{64}$/;
|
| 136 |
const REVISION = /^[0-9a-f]{40}$/;
|
| 137 |
const SPLIT_FILE = /-(\d{5})-of-(\d{5})\.gguf$/i;
|
| 138 |
-
const BASELINE_WEBGPU_STORAGE_BUFFER_BYTES = 128 * 1024 * 1024;
|
| 139 |
|
| 140 |
function isRecord(value: unknown): value is Record<string, unknown> {
|
| 141 |
return typeof value === 'object' && value !== null && !Array.isArray(value);
|
|
@@ -188,25 +112,6 @@ function requirePath(value: unknown, field: string): string {
|
|
| 188 |
return path;
|
| 189 |
}
|
| 190 |
|
| 191 |
-
function requireSafetensorsPath(value: unknown, field: string): string {
|
| 192 |
-
const path = requireString(value, field);
|
| 193 |
-
assert(!path.startsWith('/'), `${field} must be repository-relative`);
|
| 194 |
-
assert(!path.split('/').some((part) => part === '' || part === '.' || part === '..'), `${field} contains an unsafe segment`);
|
| 195 |
-
assert(path.toLowerCase().endsWith('.safetensors'), `${field} must point to a safetensors file`);
|
| 196 |
-
return path;
|
| 197 |
-
}
|
| 198 |
-
|
| 199 |
-
function requireIntegerArray(value: unknown, field: string, minimum: number): number[] {
|
| 200 |
-
assert(Array.isArray(value) && value.length > 0, `${field} must be a non-empty array`);
|
| 201 |
-
return value.map((item, index) => {
|
| 202 |
-
assert(
|
| 203 |
-
Number.isSafeInteger(item) && Number(item) >= minimum,
|
| 204 |
-
`${field}[${index}] must be an integer greater than or equal to ${minimum}`,
|
| 205 |
-
);
|
| 206 |
-
return Number(item);
|
| 207 |
-
});
|
| 208 |
-
}
|
| 209 |
-
|
| 210 |
function parseSource(value: unknown, field: string): ManifestSourceModel {
|
| 211 |
assert(isRecord(value), `${field} must be an object`);
|
| 212 |
return {
|
|
@@ -262,201 +167,6 @@ function parseChatTemplate(value: unknown, field: string): ManifestChatTemplate
|
|
| 262 |
};
|
| 263 |
}
|
| 264 |
|
| 265 |
-
function parseDFlashUnsupportedPairing(
|
| 266 |
-
value: unknown,
|
| 267 |
-
field: string,
|
| 268 |
-
): ManifestDFlashUnsupportedPairing {
|
| 269 |
-
assert(isRecord(value), `${field} must be an object`);
|
| 270 |
-
assert(value.support === 'unsupported', `${field}.support must equal unsupported`);
|
| 271 |
-
return {
|
| 272 |
-
support: 'unsupported',
|
| 273 |
-
reason: requireString(value.reason, `${field}.reason`),
|
| 274 |
-
};
|
| 275 |
-
}
|
| 276 |
-
|
| 277 |
-
function parseDFlash(value: unknown): ManifestDFlash {
|
| 278 |
-
const field = 'dflash';
|
| 279 |
-
assert(isRecord(value), `${field} must be an object`);
|
| 280 |
-
assert(value.status === 'conditional', `${field}.status must equal conditional`);
|
| 281 |
-
assert(value.enabledByDefault === false, `${field}.enabledByDefault must remain false`);
|
| 282 |
-
assert(value.runtimeAvailable === false, `${field}.runtimeAvailable must remain false`);
|
| 283 |
-
|
| 284 |
-
assert(isRecord(value.source), `${field}.source must be an object`);
|
| 285 |
-
const source = value.source;
|
| 286 |
-
assert(source.format === 'safetensors', `${field}.source.format must equal safetensors`);
|
| 287 |
-
assert(source.dtype === 'BF16', `${field}.source.dtype must equal BF16`);
|
| 288 |
-
assert(source.sourceOnly === true, `${field}.source.sourceOnly must remain true`);
|
| 289 |
-
assert(source.license === 'MIT', `${field}.source.license must equal MIT`);
|
| 290 |
-
|
| 291 |
-
assert(isRecord(value.nativeReference), `${field}.nativeReference must be an object`);
|
| 292 |
-
const nativeReference = value.nativeReference;
|
| 293 |
-
assert(nativeReference.format === 'GGUF', `${field}.nativeReference.format must equal GGUF`);
|
| 294 |
-
assert(nativeReference.quantization === 'Q8_0', `${field}.nativeReference.quantization must equal Q8_0`);
|
| 295 |
-
assert(
|
| 296 |
-
nativeReference.role === 'native correctness reference only',
|
| 297 |
-
`${field}.nativeReference.role must remain native correctness reference only`,
|
| 298 |
-
);
|
| 299 |
-
assert(
|
| 300 |
-
isRecord(nativeReference.largestTensor),
|
| 301 |
-
`${field}.nativeReference.largestTensor must be an object`,
|
| 302 |
-
);
|
| 303 |
-
const largestTensor = {
|
| 304 |
-
name: requireString(nativeReference.largestTensor.name, `${field}.nativeReference.largestTensor.name`),
|
| 305 |
-
bytes: requirePositiveInteger(nativeReference.largestTensor.bytes, `${field}.nativeReference.largestTensor.bytes`),
|
| 306 |
-
};
|
| 307 |
-
const nativeReferenceBytes = requirePositiveInteger(
|
| 308 |
-
nativeReference.bytes,
|
| 309 |
-
`${field}.nativeReference.bytes`,
|
| 310 |
-
);
|
| 311 |
-
assert(
|
| 312 |
-
largestTensor.bytes <= nativeReferenceBytes,
|
| 313 |
-
`${field}.nativeReference.largestTensor.bytes cannot exceed nativeReference.bytes`,
|
| 314 |
-
);
|
| 315 |
-
assert(
|
| 316 |
-
largestTensor.bytes > BASELINE_WEBGPU_STORAGE_BUFFER_BYTES,
|
| 317 |
-
`${field}.nativeReference.largestTensor.bytes must exceed the rejected 128 MiB baseline limit`,
|
| 318 |
-
);
|
| 319 |
-
|
| 320 |
-
assert(isRecord(value.browser), `${field}.browser must be an object`);
|
| 321 |
-
const browser = value.browser;
|
| 322 |
-
assert(browser.enabled === false, `${field}.browser.enabled must remain false`);
|
| 323 |
-
assert(browser.architectureCompiled === true, `${field}.browser.architectureCompiled must equal true`);
|
| 324 |
-
assert(browser.specTypePlumbed === false, `${field}.browser.specTypePlumbed must remain false`);
|
| 325 |
-
assert(
|
| 326 |
-
browser.candidateQuantizationStatus === 'q8-rejected-binding-limit',
|
| 327 |
-
`${field}.browser.candidateQuantizationStatus must equal q8-rejected-binding-limit`,
|
| 328 |
-
);
|
| 329 |
-
|
| 330 |
-
assert(isRecord(value.validation), `${field}.validation must be an object`);
|
| 331 |
-
const validation = value.validation;
|
| 332 |
-
assert(validation.shape === 'pass', `${field}.validation.shape must equal pass`);
|
| 333 |
-
assert(validation.tokenizer === 'exact-match', `${field}.validation.tokenizer must equal exact-match`);
|
| 334 |
-
assert(
|
| 335 |
-
validation.acceptanceBenchmark === 'pending',
|
| 336 |
-
`${field}.validation.acceptanceBenchmark must remain pending`,
|
| 337 |
-
);
|
| 338 |
-
assert(validation.browserSpeedGate === 'pending', `${field}.validation.browserSpeedGate must remain pending`);
|
| 339 |
-
assert(
|
| 340 |
-
isRecord(validation.nativeGreedyParity),
|
| 341 |
-
`${field}.validation.nativeGreedyParity must be an object`,
|
| 342 |
-
);
|
| 343 |
-
assert(
|
| 344 |
-
validation.nativeGreedyParity.status === 'pass',
|
| 345 |
-
`${field}.validation.nativeGreedyParity.status must equal pass`,
|
| 346 |
-
);
|
| 347 |
-
assert(
|
| 348 |
-
validation.nativeGreedyParity.temperature === 0,
|
| 349 |
-
`${field}.validation.nativeGreedyParity.temperature must equal 0`,
|
| 350 |
-
);
|
| 351 |
-
const targetTaps = requireIntegerArray(validation.targetTaps, `${field}.validation.targetTaps`, 1);
|
| 352 |
-
assert(
|
| 353 |
-
new Set(targetTaps).size === targetTaps.length,
|
| 354 |
-
`${field}.validation.targetTaps must not contain duplicates`,
|
| 355 |
-
);
|
| 356 |
-
|
| 357 |
-
assert(isRecord(value.pairings), `${field}.pairings must be an object`);
|
| 358 |
-
const pairings = value.pairings;
|
| 359 |
-
assert(
|
| 360 |
-
Object.keys(pairings).length === MODEL_IDS.size
|
| 361 |
-
&& Object.keys(pairings).every((id) => MODEL_IDS.has(id as ModelTierId)),
|
| 362 |
-
`${field}.pairings must contain exactly the four Bonsai tiers`,
|
| 363 |
-
);
|
| 364 |
-
const unsupported1_7b = parseDFlashUnsupportedPairing(pairings['1_7b'], `${field}.pairings.1_7b`);
|
| 365 |
-
const unsupported4b = parseDFlashUnsupportedPairing(pairings['4b'], `${field}.pairings.4b`);
|
| 366 |
-
const unsupported8b = parseDFlashUnsupportedPairing(pairings['8b'], `${field}.pairings.8b`);
|
| 367 |
-
assert(isRecord(pairings['27b']), `${field}.pairings.27b must be an object`);
|
| 368 |
-
const pairing27b = pairings['27b'];
|
| 369 |
-
assert(pairing27b.support === 'conditional', `${field}.pairings.27b.support must equal conditional`);
|
| 370 |
-
assert(
|
| 371 |
-
pairing27b.compatibilityStatus === 'native-greedy-parity-pass',
|
| 372 |
-
`${field}.pairings.27b.compatibilityStatus must equal native-greedy-parity-pass`,
|
| 373 |
-
);
|
| 374 |
-
assert(pairing27b.runtimeAvailable === false, `${field}.pairings.27b.runtimeAvailable must remain false`);
|
| 375 |
-
const targetLayerIdsZeroBased = requireIntegerArray(
|
| 376 |
-
pairing27b.targetLayerIdsZeroBased,
|
| 377 |
-
`${field}.pairings.27b.targetLayerIdsZeroBased`,
|
| 378 |
-
0,
|
| 379 |
-
);
|
| 380 |
-
const targetTapsOneBased = requireIntegerArray(
|
| 381 |
-
pairing27b.targetTapsOneBased,
|
| 382 |
-
`${field}.pairings.27b.targetTapsOneBased`,
|
| 383 |
-
1,
|
| 384 |
-
);
|
| 385 |
-
assert(
|
| 386 |
-
targetLayerIdsZeroBased.length === targetTapsOneBased.length
|
| 387 |
-
&& targetTapsOneBased.every((tap, index) => tap === targetLayerIdsZeroBased[index]! + 1),
|
| 388 |
-
`${field}.pairings.27b target layers and taps must describe the same one-based positions`,
|
| 389 |
-
);
|
| 390 |
-
assert(
|
| 391 |
-
targetTaps.length === targetTapsOneBased.length
|
| 392 |
-
&& targetTaps.every((tap, index) => tap === targetTapsOneBased[index]),
|
| 393 |
-
`${field}.validation.targetTaps must match the 27b pairing`,
|
| 394 |
-
);
|
| 395 |
-
|
| 396 |
-
return {
|
| 397 |
-
status: 'conditional',
|
| 398 |
-
enabledByDefault: false,
|
| 399 |
-
runtimeAvailable: false,
|
| 400 |
-
source: {
|
| 401 |
-
repo: requireRepo(source.repo, `${field}.source.repo`),
|
| 402 |
-
revision: requireRevision(source.revision, `${field}.source.revision`),
|
| 403 |
-
file: requireSafetensorsPath(source.file, `${field}.source.file`),
|
| 404 |
-
bytes: requirePositiveInteger(source.bytes, `${field}.source.bytes`),
|
| 405 |
-
sha256: requireSha256(source.sha256, `${field}.source.sha256`),
|
| 406 |
-
format: 'safetensors',
|
| 407 |
-
dtype: 'BF16',
|
| 408 |
-
parameters: requirePositiveInteger(source.parameters, `${field}.source.parameters`),
|
| 409 |
-
sourceOnly: true,
|
| 410 |
-
license: 'MIT',
|
| 411 |
-
},
|
| 412 |
-
nativeReference: {
|
| 413 |
-
path: requirePath(nativeReference.path, `${field}.nativeReference.path`),
|
| 414 |
-
bytes: nativeReferenceBytes,
|
| 415 |
-
sha256: requireSha256(nativeReference.sha256, `${field}.nativeReference.sha256`),
|
| 416 |
-
format: 'GGUF',
|
| 417 |
-
quantization: 'Q8_0',
|
| 418 |
-
largestTensor,
|
| 419 |
-
role: 'native correctness reference only',
|
| 420 |
-
},
|
| 421 |
-
browser: {
|
| 422 |
-
enabled: false,
|
| 423 |
-
architectureCompiled: true,
|
| 424 |
-
specTypePlumbed: false,
|
| 425 |
-
candidateQuantizationStatus: 'q8-rejected-binding-limit',
|
| 426 |
-
},
|
| 427 |
-
validation: {
|
| 428 |
-
shape: 'pass',
|
| 429 |
-
tokenizer: 'exact-match',
|
| 430 |
-
targetTaps,
|
| 431 |
-
nativeGreedyParity: {
|
| 432 |
-
status: 'pass',
|
| 433 |
-
tokens: requirePositiveInteger(
|
| 434 |
-
validation.nativeGreedyParity.tokens,
|
| 435 |
-
`${field}.validation.nativeGreedyParity.tokens`,
|
| 436 |
-
),
|
| 437 |
-
temperature: 0,
|
| 438 |
-
},
|
| 439 |
-
acceptanceBenchmark: 'pending',
|
| 440 |
-
browserSpeedGate: 'pending',
|
| 441 |
-
},
|
| 442 |
-
pairings: {
|
| 443 |
-
'1_7b': unsupported1_7b,
|
| 444 |
-
'4b': unsupported4b,
|
| 445 |
-
'8b': unsupported8b,
|
| 446 |
-
'27b': {
|
| 447 |
-
support: 'conditional',
|
| 448 |
-
compatibilityStatus: 'native-greedy-parity-pass',
|
| 449 |
-
runtimeAvailable: false,
|
| 450 |
-
blockSize: requirePositiveInteger(pairing27b.blockSize, `${field}.pairings.27b.blockSize`),
|
| 451 |
-
maskTokenId: requireNonNegativeInteger(pairing27b.maskTokenId, `${field}.pairings.27b.maskTokenId`),
|
| 452 |
-
targetLayerIdsZeroBased,
|
| 453 |
-
targetTapsOneBased,
|
| 454 |
-
reason: requireString(pairing27b.reason, `${field}.pairings.27b.reason`),
|
| 455 |
-
},
|
| 456 |
-
},
|
| 457 |
-
};
|
| 458 |
-
}
|
| 459 |
-
|
| 460 |
function validateFileOrder(files: ManifestModelFile[], field: string): void {
|
| 461 |
const matches = files.map((file) => file.path.match(SPLIT_FILE));
|
| 462 |
const splitCount = matches.filter(Boolean).length;
|
|
@@ -549,8 +259,6 @@ export function parseModelManifestV2(value: unknown): ModelManifestV2 {
|
|
| 549 |
assert(models.length === MODEL_IDS.size, 'models must contain exactly the four Bonsai tiers');
|
| 550 |
assert(new Set(models.map((model) => model.id)).size === models.length, 'models contains duplicate ids');
|
| 551 |
assert(models.every((model) => MODEL_IDS.has(model.id)), 'models contains an unsupported id');
|
| 552 |
-
const dflash = parseDFlash(value.dflash);
|
| 553 |
-
|
| 554 |
return {
|
| 555 |
schemaVersion: 2,
|
| 556 |
repository: {
|
|
@@ -558,7 +266,6 @@ export function parseModelManifestV2(value: unknown): ModelManifestV2 {
|
|
| 558 |
revision: repositoryRevision,
|
| 559 |
},
|
| 560 |
models,
|
| 561 |
-
dflash,
|
| 562 |
};
|
| 563 |
}
|
| 564 |
|
|
|
|
| 47 |
};
|
| 48 |
}
|
| 49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
export interface ModelManifestV2 {
|
| 51 |
schemaVersion: 2;
|
| 52 |
repository: {
|
|
|
|
| 54 |
revision: string | null;
|
| 55 |
};
|
| 56 |
models: ManifestModelV2[];
|
|
|
|
| 57 |
}
|
| 58 |
|
| 59 |
const MODEL_IDS = new Set<ModelTierId>(['1_7b', '4b', '8b', '27b']);
|
| 60 |
const SHA256 = /^[0-9a-f]{64}$/;
|
| 61 |
const REVISION = /^[0-9a-f]{40}$/;
|
| 62 |
const SPLIT_FILE = /-(\d{5})-of-(\d{5})\.gguf$/i;
|
|
|
|
| 63 |
|
| 64 |
function isRecord(value: unknown): value is Record<string, unknown> {
|
| 65 |
return typeof value === 'object' && value !== null && !Array.isArray(value);
|
|
|
|
| 112 |
return path;
|
| 113 |
}
|
| 114 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
function parseSource(value: unknown, field: string): ManifestSourceModel {
|
| 116 |
assert(isRecord(value), `${field} must be an object`);
|
| 117 |
return {
|
|
|
|
| 167 |
};
|
| 168 |
}
|
| 169 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 170 |
function validateFileOrder(files: ManifestModelFile[], field: string): void {
|
| 171 |
const matches = files.map((file) => file.path.match(SPLIT_FILE));
|
| 172 |
const splitCount = matches.filter(Boolean).length;
|
|
|
|
| 259 |
assert(models.length === MODEL_IDS.size, 'models must contain exactly the four Bonsai tiers');
|
| 260 |
assert(new Set(models.map((model) => model.id)).size === models.length, 'models contains duplicate ids');
|
| 261 |
assert(models.every((model) => MODEL_IDS.has(model.id)), 'models contains an unsupported id');
|
|
|
|
|
|
|
| 262 |
return {
|
| 263 |
schemaVersion: 2,
|
| 264 |
repository: {
|
|
|
|
| 266 |
revision: repositoryRevision,
|
| 267 |
},
|
| 268 |
models,
|
|
|
|
| 269 |
};
|
| 270 |
}
|
| 271 |
|
src/engine/native-log.test.ts
CHANGED
|
@@ -3,6 +3,7 @@ import {
|
|
| 3 |
NativeLogCollector,
|
| 4 |
parseNativeDeviceLostSignal,
|
| 5 |
parseNativeLog,
|
|
|
|
| 6 |
} from './native-log';
|
| 7 |
|
| 8 |
describe('parseNativeLog', () => {
|
|
@@ -17,9 +18,14 @@ llama_kv_cache: WebGPU KV buffer size = 56.00 MiB
|
|
| 17 |
llama_kv_cache: size = 56.00 MiB, K (f16): 28.00 MiB, V (f16): 28.00 MiB
|
| 18 |
sched_reserve: WebGPU compute buffer size = 301.00 MiB
|
| 19 |
sched_reserve: CPU compute buffer size = 13.01 MiB
|
|
|
|
| 20 |
sched_reserve: graph splits = 1
|
| 21 |
`);
|
| 22 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
expect(report).toEqual({
|
| 24 |
backends: ['WebGPU', 'CPU'],
|
| 25 |
nGraphSplits: 1,
|
|
@@ -29,7 +35,53 @@ sched_reserve: graph splits = 1
|
|
| 29 |
cacheTypeK: 'f16',
|
| 30 |
cacheTypeV: 'f16',
|
| 31 |
webgpuKvBufferBytes: 56 * 1024 ** 2,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
});
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
});
|
| 34 |
|
| 35 |
it('counts explicit CPU graph nodes and keeps the worst graph split observation', () => {
|
|
@@ -100,6 +152,7 @@ sched_reserve: graph splits = 1
|
|
| 100 |
cacheTypeK: null,
|
| 101 |
cacheTypeV: null,
|
| 102 |
webgpuKvBufferBytes: null,
|
|
|
|
| 103 |
});
|
| 104 |
expect(collector.deviceLostSignal()).toMatchObject({ reason: 1, message: 'unknown' });
|
| 105 |
|
|
@@ -107,6 +160,33 @@ sched_reserve: graph splits = 1
|
|
| 107 |
expect(collector.deviceLostSignal()).toBeNull();
|
| 108 |
});
|
| 109 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 110 |
it('notifies an active native-operation watcher without polling', async () => {
|
| 111 |
const collector = new NativeLogCollector();
|
| 112 |
const watch = collector.watchDeviceLost();
|
|
|
|
| 3 |
NativeLogCollector,
|
| 4 |
parseNativeDeviceLostSignal,
|
| 5 |
parseNativeLog,
|
| 6 |
+
parseNativeModelLoadProgress,
|
| 7 |
} from './native-log';
|
| 8 |
|
| 9 |
describe('parseNativeLog', () => {
|
|
|
|
| 18 |
llama_kv_cache: size = 56.00 MiB, K (f16): 28.00 MiB, V (f16): 28.00 MiB
|
| 19 |
sched_reserve: WebGPU compute buffer size = 301.00 MiB
|
| 20 |
sched_reserve: CPU compute buffer size = 13.01 MiB
|
| 21 |
+
llama_context: CPU output buffer size = 0.95 MiB
|
| 22 |
sched_reserve: graph splits = 1
|
| 23 |
`);
|
| 24 |
|
| 25 |
+
const modelBufferBytes = Math.round(72 * 1024 ** 2) + Math.round(3616.77 * 1024 ** 2);
|
| 26 |
+
const computeBufferBytes = Math.round(301 * 1024 ** 2) + Math.round(13.01 * 1024 ** 2);
|
| 27 |
+
const outputBufferBytes = Math.round(0.95 * 1024 ** 2);
|
| 28 |
+
|
| 29 |
expect(report).toEqual({
|
| 30 |
backends: ['WebGPU', 'CPU'],
|
| 31 |
nGraphSplits: 1,
|
|
|
|
| 35 |
cacheTypeK: 'f16',
|
| 36 |
cacheTypeV: 'f16',
|
| 37 |
webgpuKvBufferBytes: 56 * 1024 ** 2,
|
| 38 |
+
modelBufferBytes,
|
| 39 |
+
computeBufferBytes,
|
| 40 |
+
outputBufferBytes,
|
| 41 |
+
allocatedBufferBytes: modelBufferBytes
|
| 42 |
+
+ computeBufferBytes
|
| 43 |
+
+ outputBufferBytes
|
| 44 |
+
+ (56 * 1024 ** 2),
|
| 45 |
+
webgpuTrace: [],
|
| 46 |
+
});
|
| 47 |
+
});
|
| 48 |
+
|
| 49 |
+
it('parses and clamps structured native model-load progress', () => {
|
| 50 |
+
expect(parseNativeModelLoadProgress(
|
| 51 |
+
'@@MODEL_LOAD_PROGRESS@@{"state":"loading","stages":["metadata","tensors"],"current":"tensors","value":0.375}',
|
| 52 |
+
)).toEqual({
|
| 53 |
+
state: 'loading',
|
| 54 |
+
stages: ['metadata', 'tensors'],
|
| 55 |
+
current: 'tensors',
|
| 56 |
+
value: 0.375,
|
| 57 |
});
|
| 58 |
+
expect(parseNativeModelLoadProgress(
|
| 59 |
+
'@@MODEL_LOAD_PROGRESS@@{"state":"loading","value":7}',
|
| 60 |
+
)).toEqual({ state: 'loading', stages: [], current: null, value: 1 });
|
| 61 |
+
expect(parseNativeModelLoadProgress(
|
| 62 |
+
'@@MODEL_LOAD_PROGRESS@@{"state":"ready","value":0}',
|
| 63 |
+
)).toEqual({ state: 'ready', stages: [], current: null, value: 1 });
|
| 64 |
+
expect(parseNativeModelLoadProgress('@@MODEL_LOAD_PROGRESS@@not-json')).toBeNull();
|
| 65 |
+
});
|
| 66 |
+
|
| 67 |
+
it('delivers native model-load progress to active listeners only', () => {
|
| 68 |
+
const collector = new NativeLogCollector();
|
| 69 |
+
const observed: unknown[] = [];
|
| 70 |
+
const unsubscribe = collector.onModelLoadProgress((progress) => observed.push(progress));
|
| 71 |
+
|
| 72 |
+
collector.append([
|
| 73 |
+
'@@MODEL_LOAD_PROGRESS@@{"state":"loading","stages":["tensors"],"current":"tensors","value":0.5}',
|
| 74 |
+
]);
|
| 75 |
+
collector.append(['@@MODEL_LOAD_PROGRESS@@invalid']);
|
| 76 |
+
unsubscribe();
|
| 77 |
+
collector.append(['@@MODEL_LOAD_PROGRESS@@{"state":"ready"}']);
|
| 78 |
+
|
| 79 |
+
expect(observed).toEqual([{
|
| 80 |
+
state: 'loading',
|
| 81 |
+
stages: ['tensors'],
|
| 82 |
+
current: 'tensors',
|
| 83 |
+
value: 0.5,
|
| 84 |
+
}]);
|
| 85 |
});
|
| 86 |
|
| 87 |
it('counts explicit CPU graph nodes and keeps the worst graph split observation', () => {
|
|
|
|
| 152 |
cacheTypeK: null,
|
| 153 |
cacheTypeV: null,
|
| 154 |
webgpuKvBufferBytes: null,
|
| 155 |
+
webgpuTrace: [],
|
| 156 |
});
|
| 157 |
expect(collector.deviceLostSignal()).toMatchObject({ reason: 1, message: 'unknown' });
|
| 158 |
|
|
|
|
| 160 |
expect(collector.deviceLostSignal()).toBeNull();
|
| 161 |
});
|
| 162 |
|
| 163 |
+
it('retains WebGPU trace markers in a separate report channel', () => {
|
| 164 |
+
const collector = new NativeLogCollector();
|
| 165 |
+
collector.append(['@@WEBGPU_TRACE@@completion_begin id=1']);
|
| 166 |
+
collector.append(['@@WEBGPU_TRACE@@graph_end id=1 nodes=3735 dispatches=1940 submits=31']);
|
| 167 |
+
collector.append(['unrelated native output']);
|
| 168 |
+
|
| 169 |
+
expect(collector.report().webgpuTrace).toEqual([
|
| 170 |
+
'@@WEBGPU_TRACE@@completion_begin id=1',
|
| 171 |
+
'@@WEBGPU_TRACE@@graph_end id=1 nodes=3735 dispatches=1940 submits=31',
|
| 172 |
+
]);
|
| 173 |
+
|
| 174 |
+
collector.clear();
|
| 175 |
+
expect(collector.report().webgpuTrace).toEqual([]);
|
| 176 |
+
});
|
| 177 |
+
|
| 178 |
+
it('bounds the trace independently with room for a complete 64-token run', () => {
|
| 179 |
+
const collector = new NativeLogCollector();
|
| 180 |
+
for (let index = 0; index <= 16_384; index += 1) {
|
| 181 |
+
collector.append([`@@WEBGPU_TRACE@@step_end completion=1 step=${index}`]);
|
| 182 |
+
}
|
| 183 |
+
|
| 184 |
+
const trace = collector.report().webgpuTrace;
|
| 185 |
+
expect(trace).toHaveLength(16_384);
|
| 186 |
+
expect(trace[0]).toContain('step=1');
|
| 187 |
+
expect(trace.at(-1)).toContain('step=16384');
|
| 188 |
+
});
|
| 189 |
+
|
| 190 |
it('notifies an active native-operation watcher without polling', async () => {
|
| 191 |
const collector = new NativeLogCollector();
|
| 192 |
const watch = collector.watchDeviceLost();
|
src/engine/native-log.ts
CHANGED
|
@@ -12,6 +12,11 @@ export interface BackendReport {
|
|
| 12 |
cacheTypeK: string | null;
|
| 13 |
cacheTypeV: string | null;
|
| 14 |
webgpuKvBufferBytes: number | null;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
}
|
| 16 |
|
| 17 |
export interface NativeDeviceLostSignal {
|
|
@@ -25,10 +30,20 @@ export interface NativeDeviceLostWatch {
|
|
| 25 |
dispose(): void;
|
| 26 |
}
|
| 27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
const BACKEND_NAMES = ['WebGPU', 'CPU', 'Metal', 'CUDA', 'Vulkan'] as const;
|
| 29 |
const EVIDENCE = /using device|model buffer size|KV buffer size|compute buffer size|output buffer size|offloaded \d+\/\d+ layers|graph splits\s*=|graph_compute\(|backend\s*=\s*CPU|assigned to (?:device )?CPU|(?:ops?|nodes?).*(?:on|to) CPU|flash_attn\s*=|flash attention.*(?:enabled|disabled)|\bK\s*\([^)]+\).*\bV\s*\([^)]+\)/i;
|
| 30 |
const WEBGPU_DEVICE_LOST = /\bdevice\s+lost!?(?:\s+reason:\s*(-?\d+),\s*message:\s*(.*))?/i;
|
| 31 |
const WEBGPU_KV_BUFFER = /\bWebGPU\s+KV buffer size\s*=\s*(\d+(?:\.\d+)?)\s*(KiB|MiB|GiB)\b/i;
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
const BINARY_UNIT_BYTES = {
|
| 34 |
kib: 1024,
|
|
@@ -81,6 +96,26 @@ export function parseNativeDeviceLostSignal(line: string): NativeDeviceLostSigna
|
|
| 81 |
};
|
| 82 |
}
|
| 83 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
export function parseNativeLog(input: string | readonly string[]): BackendReport {
|
| 85 |
const lines = typeof input === 'string' ? input.split(/\r?\n/) : input;
|
| 86 |
const backends = new Set<string>();
|
|
@@ -91,6 +126,9 @@ export function parseNativeLog(input: string | readonly string[]): BackendReport
|
|
| 91 |
let cacheTypeK: string | null = null;
|
| 92 |
let cacheTypeV: string | null = null;
|
| 93 |
let webgpuKvBufferBytes: number | null = null;
|
|
|
|
|
|
|
|
|
|
| 94 |
|
| 95 |
for (const line of lines) {
|
| 96 |
const deviceMatch = line.match(/using device\s+([^\s(]+)\s+\(([^)]+)\)/i);
|
|
@@ -141,9 +179,24 @@ export function parseNativeLog(input: string | readonly string[]): BackendReport
|
|
| 141 |
const bytes = Math.round(Number(kvBufferSize) * unitBytes);
|
| 142 |
webgpuKvBufferBytes = (webgpuKvBufferBytes ?? 0) + bytes;
|
| 143 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 144 |
opsOnCpu += cpuOpsInLine(line);
|
| 145 |
}
|
| 146 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
return {
|
| 148 |
backends: [...backends],
|
| 149 |
nGraphSplits: graphSplits.length > 0 ? Math.max(...graphSplits) : null,
|
|
@@ -153,16 +206,24 @@ export function parseNativeLog(input: string | readonly string[]): BackendReport
|
|
| 153 |
cacheTypeK,
|
| 154 |
cacheTypeV,
|
| 155 |
webgpuKvBufferBytes,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 156 |
};
|
| 157 |
}
|
| 158 |
|
| 159 |
export class NativeLogCollector {
|
| 160 |
private lines: string[] = [];
|
|
|
|
| 161 |
private deviceLost: NativeDeviceLostSignal | null = null;
|
| 162 |
private readonly deviceLostListeners = new Set<(signal: NativeDeviceLostSignal) => void>();
|
|
|
|
| 163 |
|
| 164 |
clear(): void {
|
| 165 |
this.lines = [];
|
|
|
|
| 166 |
this.deviceLost = null;
|
| 167 |
}
|
| 168 |
|
|
@@ -170,6 +231,10 @@ export class NativeLogCollector {
|
|
| 170 |
const line = values
|
| 171 |
.map((value) => (typeof value === 'string' ? value : JSON.stringify(value)))
|
| 172 |
.join(' ');
|
|
|
|
|
|
|
|
|
|
|
|
|
| 173 |
if (!this.deviceLost) {
|
| 174 |
const deviceLost = parseNativeDeviceLostSignal(line);
|
| 175 |
if (deviceLost) {
|
|
@@ -184,10 +249,19 @@ export class NativeLogCollector {
|
|
| 184 |
this.lines.shift();
|
| 185 |
}
|
| 186 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 187 |
}
|
| 188 |
|
| 189 |
report(): BackendReport {
|
| 190 |
-
return
|
|
|
|
|
|
|
|
|
|
| 191 |
}
|
| 192 |
|
| 193 |
deviceLostSignal(): NativeDeviceLostSignal | null {
|
|
@@ -211,4 +285,9 @@ export class NativeLogCollector {
|
|
| 211 |
},
|
| 212 |
};
|
| 213 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 214 |
}
|
|
|
|
| 12 |
cacheTypeK: string | null;
|
| 13 |
cacheTypeV: string | null;
|
| 14 |
webgpuKvBufferBytes: number | null;
|
| 15 |
+
modelBufferBytes?: number;
|
| 16 |
+
computeBufferBytes?: number;
|
| 17 |
+
outputBufferBytes?: number;
|
| 18 |
+
allocatedBufferBytes?: number;
|
| 19 |
+
webgpuTrace: string[];
|
| 20 |
}
|
| 21 |
|
| 22 |
export interface NativeDeviceLostSignal {
|
|
|
|
| 30 |
dispose(): void;
|
| 31 |
}
|
| 32 |
|
| 33 |
+
export interface NativeModelLoadProgress {
|
| 34 |
+
state: 'loading' | 'ready';
|
| 35 |
+
stages: string[];
|
| 36 |
+
current: string | null;
|
| 37 |
+
value: number;
|
| 38 |
+
}
|
| 39 |
+
|
| 40 |
const BACKEND_NAMES = ['WebGPU', 'CPU', 'Metal', 'CUDA', 'Vulkan'] as const;
|
| 41 |
const EVIDENCE = /using device|model buffer size|KV buffer size|compute buffer size|output buffer size|offloaded \d+\/\d+ layers|graph splits\s*=|graph_compute\(|backend\s*=\s*CPU|assigned to (?:device )?CPU|(?:ops?|nodes?).*(?:on|to) CPU|flash_attn\s*=|flash attention.*(?:enabled|disabled)|\bK\s*\([^)]+\).*\bV\s*\([^)]+\)/i;
|
| 42 |
const WEBGPU_DEVICE_LOST = /\bdevice\s+lost!?(?:\s+reason:\s*(-?\d+),\s*message:\s*(.*))?/i;
|
| 43 |
const WEBGPU_KV_BUFFER = /\bWebGPU\s+KV buffer size\s*=\s*(\d+(?:\.\d+)?)\s*(KiB|MiB|GiB)\b/i;
|
| 44 |
+
const WEBGPU_TRACE_PREFIX = '@@WEBGPU_TRACE@@';
|
| 45 |
+
const MODEL_LOAD_PROGRESS_PREFIX = '@@MODEL_LOAD_PROGRESS@@';
|
| 46 |
+
const WEBGPU_TRACE_LINE_LIMIT = 16_384;
|
| 47 |
|
| 48 |
const BINARY_UNIT_BYTES = {
|
| 49 |
kib: 1024,
|
|
|
|
| 96 |
};
|
| 97 |
}
|
| 98 |
|
| 99 |
+
export function parseNativeModelLoadProgress(line: string): NativeModelLoadProgress | null {
|
| 100 |
+
if (!line.startsWith(MODEL_LOAD_PROGRESS_PREFIX)) return null;
|
| 101 |
+
try {
|
| 102 |
+
const payload = JSON.parse(line.slice(MODEL_LOAD_PROGRESS_PREFIX.length)) as Record<string, unknown>;
|
| 103 |
+
if (payload.state !== 'loading' && payload.state !== 'ready') return null;
|
| 104 |
+
const stages = Array.isArray(payload.stages)
|
| 105 |
+
? payload.stages.filter((stage): stage is string => typeof stage === 'string')
|
| 106 |
+
: [];
|
| 107 |
+
const current = typeof payload.current === 'string' ? payload.current : null;
|
| 108 |
+
const value = payload.state === 'ready'
|
| 109 |
+
? 1
|
| 110 |
+
: typeof payload.value === 'number' && Number.isFinite(payload.value)
|
| 111 |
+
? Math.min(1, Math.max(0, payload.value))
|
| 112 |
+
: 0;
|
| 113 |
+
return { state: payload.state, stages, current, value };
|
| 114 |
+
} catch {
|
| 115 |
+
return null;
|
| 116 |
+
}
|
| 117 |
+
}
|
| 118 |
+
|
| 119 |
export function parseNativeLog(input: string | readonly string[]): BackendReport {
|
| 120 |
const lines = typeof input === 'string' ? input.split(/\r?\n/) : input;
|
| 121 |
const backends = new Set<string>();
|
|
|
|
| 126 |
let cacheTypeK: string | null = null;
|
| 127 |
let cacheTypeV: string | null = null;
|
| 128 |
let webgpuKvBufferBytes: number | null = null;
|
| 129 |
+
let modelBufferBytes = 0;
|
| 130 |
+
let computeBufferBytes = 0;
|
| 131 |
+
let outputBufferBytes = 0;
|
| 132 |
|
| 133 |
for (const line of lines) {
|
| 134 |
const deviceMatch = line.match(/using device\s+([^\s(]+)\s+\(([^)]+)\)/i);
|
|
|
|
| 179 |
const bytes = Math.round(Number(kvBufferSize) * unitBytes);
|
| 180 |
webgpuKvBufferBytes = (webgpuKvBufferBytes ?? 0) + bytes;
|
| 181 |
}
|
| 182 |
+
const bufferMatch = line.match(/\b(model|compute|output) buffer size\s*=\s*(\d+(?:\.\d+)?)\s*(KiB|MiB|GiB)\b/i);
|
| 183 |
+
const bufferKind = bufferMatch?.[1]?.toLowerCase();
|
| 184 |
+
const bufferSize = bufferMatch?.[2];
|
| 185 |
+
const bufferUnit = bufferMatch?.[3];
|
| 186 |
+
if (bufferKind && bufferSize && bufferUnit) {
|
| 187 |
+
const unitBytes = BINARY_UNIT_BYTES[bufferUnit.toLowerCase() as keyof typeof BINARY_UNIT_BYTES];
|
| 188 |
+
const bytes = Math.round(Number(bufferSize) * unitBytes);
|
| 189 |
+
if (bufferKind === 'model') modelBufferBytes += bytes;
|
| 190 |
+
if (bufferKind === 'compute') computeBufferBytes += bytes;
|
| 191 |
+
if (bufferKind === 'output') outputBufferBytes += bytes;
|
| 192 |
+
}
|
| 193 |
opsOnCpu += cpuOpsInLine(line);
|
| 194 |
}
|
| 195 |
|
| 196 |
+
const allocatedBufferBytes = modelBufferBytes
|
| 197 |
+
+ computeBufferBytes
|
| 198 |
+
+ outputBufferBytes
|
| 199 |
+
+ (webgpuKvBufferBytes ?? 0);
|
| 200 |
return {
|
| 201 |
backends: [...backends],
|
| 202 |
nGraphSplits: graphSplits.length > 0 ? Math.max(...graphSplits) : null,
|
|
|
|
| 206 |
cacheTypeK,
|
| 207 |
cacheTypeV,
|
| 208 |
webgpuKvBufferBytes,
|
| 209 |
+
...(modelBufferBytes > 0 ? { modelBufferBytes } : {}),
|
| 210 |
+
...(computeBufferBytes > 0 ? { computeBufferBytes } : {}),
|
| 211 |
+
...(outputBufferBytes > 0 ? { outputBufferBytes } : {}),
|
| 212 |
+
...(allocatedBufferBytes > 0 ? { allocatedBufferBytes } : {}),
|
| 213 |
+
webgpuTrace: [],
|
| 214 |
};
|
| 215 |
}
|
| 216 |
|
| 217 |
export class NativeLogCollector {
|
| 218 |
private lines: string[] = [];
|
| 219 |
+
private webgpuTrace: string[] = [];
|
| 220 |
private deviceLost: NativeDeviceLostSignal | null = null;
|
| 221 |
private readonly deviceLostListeners = new Set<(signal: NativeDeviceLostSignal) => void>();
|
| 222 |
+
private readonly modelLoadListeners = new Set<(progress: NativeModelLoadProgress) => void>();
|
| 223 |
|
| 224 |
clear(): void {
|
| 225 |
this.lines = [];
|
| 226 |
+
this.webgpuTrace = [];
|
| 227 |
this.deviceLost = null;
|
| 228 |
}
|
| 229 |
|
|
|
|
| 231 |
const line = values
|
| 232 |
.map((value) => (typeof value === 'string' ? value : JSON.stringify(value)))
|
| 233 |
.join(' ');
|
| 234 |
+
const modelLoadProgress = parseNativeModelLoadProgress(line);
|
| 235 |
+
if (modelLoadProgress) {
|
| 236 |
+
for (const listener of this.modelLoadListeners) listener(modelLoadProgress);
|
| 237 |
+
}
|
| 238 |
if (!this.deviceLost) {
|
| 239 |
const deviceLost = parseNativeDeviceLostSignal(line);
|
| 240 |
if (deviceLost) {
|
|
|
|
| 249 |
this.lines.shift();
|
| 250 |
}
|
| 251 |
}
|
| 252 |
+
if (line.startsWith(WEBGPU_TRACE_PREFIX)) {
|
| 253 |
+
this.webgpuTrace.push(line.slice(0, 4096));
|
| 254 |
+
if (this.webgpuTrace.length > WEBGPU_TRACE_LINE_LIMIT) {
|
| 255 |
+
this.webgpuTrace.shift();
|
| 256 |
+
}
|
| 257 |
+
}
|
| 258 |
}
|
| 259 |
|
| 260 |
report(): BackendReport {
|
| 261 |
+
return {
|
| 262 |
+
...parseNativeLog(this.lines),
|
| 263 |
+
webgpuTrace: [...this.webgpuTrace],
|
| 264 |
+
};
|
| 265 |
}
|
| 266 |
|
| 267 |
deviceLostSignal(): NativeDeviceLostSignal | null {
|
|
|
|
| 285 |
},
|
| 286 |
};
|
| 287 |
}
|
| 288 |
+
|
| 289 |
+
onModelLoadProgress(listener: (progress: NativeModelLoadProgress) => void): () => void {
|
| 290 |
+
this.modelLoadListeners.add(listener);
|
| 291 |
+
return () => this.modelLoadListeners.delete(listener);
|
| 292 |
+
}
|
| 293 |
}
|
src/engine/protocol.ts
CHANGED
|
@@ -340,17 +340,38 @@ export type EngineResponse = {
|
|
| 340 |
};
|
| 341 |
}[EngineMethod];
|
| 342 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 343 |
export type EngineEvent =
|
| 344 |
| {
|
| 345 |
type: 'event';
|
| 346 |
requestId: string;
|
| 347 |
event: 'progress';
|
| 348 |
-
phase: 'manifest' | 'download' | 'load';
|
| 349 |
loadedBytes: number;
|
| 350 |
totalBytes: number;
|
| 351 |
shardIndex: number | null;
|
| 352 |
shardCount: number;
|
| 353 |
shardPath: string | null;
|
|
|
|
|
|
|
|
|
|
|
|
|
| 354 |
}
|
| 355 |
| {
|
| 356 |
type: 'event';
|
|
@@ -358,6 +379,19 @@ export type EngineEvent =
|
|
| 358 |
event: 'token';
|
| 359 |
text: string;
|
| 360 |
reasoningDelta?: string;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 361 |
};
|
| 362 |
|
| 363 |
export type EngineWorkerMessage = EngineResponse | EngineEvent;
|
|
|
|
| 340 |
};
|
| 341 |
}[EngineMethod];
|
| 342 |
|
| 343 |
+
export type EngineShardProgressState =
|
| 344 |
+
| 'queued'
|
| 345 |
+
| 'cached'
|
| 346 |
+
| 'downloading'
|
| 347 |
+
| 'verifying'
|
| 348 |
+
| 'complete'
|
| 349 |
+
| 'error';
|
| 350 |
+
|
| 351 |
+
export interface EngineShardProgress {
|
| 352 |
+
index: number;
|
| 353 |
+
path: string;
|
| 354 |
+
loadedBytes: number;
|
| 355 |
+
verifiedBytes: number;
|
| 356 |
+
totalBytes: number;
|
| 357 |
+
state: EngineShardProgressState;
|
| 358 |
+
}
|
| 359 |
+
|
| 360 |
export type EngineEvent =
|
| 361 |
| {
|
| 362 |
type: 'event';
|
| 363 |
requestId: string;
|
| 364 |
event: 'progress';
|
| 365 |
+
phase: 'manifest' | 'cache' | 'download' | 'verify' | 'load';
|
| 366 |
loadedBytes: number;
|
| 367 |
totalBytes: number;
|
| 368 |
shardIndex: number | null;
|
| 369 |
shardCount: number;
|
| 370 |
shardPath: string | null;
|
| 371 |
+
stageProgress: number | null;
|
| 372 |
+
nativeStage: string | null;
|
| 373 |
+
residentBytes: number | null;
|
| 374 |
+
shards: EngineShardProgress[];
|
| 375 |
}
|
| 376 |
| {
|
| 377 |
type: 'event';
|
|
|
|
| 379 |
event: 'token';
|
| 380 |
text: string;
|
| 381 |
reasoningDelta?: string;
|
| 382 |
+
}
|
| 383 |
+
| {
|
| 384 |
+
type: 'event';
|
| 385 |
+
requestId: string;
|
| 386 |
+
event: 'generation';
|
| 387 |
+
phase: 'prefill' | 'decode';
|
| 388 |
+
promptProcessed: number;
|
| 389 |
+
promptTotal: number;
|
| 390 |
+
promptCached: number;
|
| 391 |
+
completionTokens: number;
|
| 392 |
+
elapsedMs: number;
|
| 393 |
+
promptTokensPerSecond: number;
|
| 394 |
+
decodeTokensPerSecond: number;
|
| 395 |
};
|
| 396 |
|
| 397 |
export type EngineWorkerMessage = EngineResponse | EngineEvent;
|
src/engine/runtime-device-lost.test.ts
CHANGED
|
@@ -70,6 +70,7 @@ describe('BrowserEngineRuntime WebGPU device loss', () => {
|
|
| 70 |
cacheTypeK: null,
|
| 71 |
cacheTypeV: null,
|
| 72 |
webgpuKvBufferBytes: null,
|
|
|
|
| 73 |
});
|
| 74 |
expect(exit).not.toHaveBeenCalled();
|
| 75 |
expect(internals.loaded).not.toBeNull();
|
|
|
|
| 70 |
cacheTypeK: null,
|
| 71 |
cacheTypeV: null,
|
| 72 |
webgpuKvBufferBytes: null,
|
| 73 |
+
webgpuTrace: [],
|
| 74 |
});
|
| 75 |
expect(exit).not.toHaveBeenCalled();
|
| 76 |
expect(internals.loaded).not.toBeNull();
|
src/engine/runtime-download.test.ts
CHANGED
|
@@ -4,7 +4,12 @@ import { verifyBlobSha256 } from './blob-integrity';
|
|
| 4 |
import { EngineRuntimeError } from './errors';
|
| 5 |
import type { ManifestModelV2 } from './manifest';
|
| 6 |
import { isShardDownloadFailureDetails, type EngineEvent } from './protocol';
|
| 7 |
-
import {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
|
| 9 |
type DownloadOptions = {
|
| 10 |
signal?: AbortSignal;
|
|
@@ -455,6 +460,32 @@ describe('resolveLoadTuning', () => {
|
|
| 455 |
});
|
| 456 |
});
|
| 457 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 458 |
describe('selectWasmFlavor', () => {
|
| 459 |
it('uses the browser-native flavor for auto', () => {
|
| 460 |
expect(selectWasmFlavor('auto', false)).toBe('jspi');
|
|
|
|
| 4 |
import { EngineRuntimeError } from './errors';
|
| 5 |
import type { ManifestModelV2 } from './manifest';
|
| 6 |
import { isShardDownloadFailureDetails, type EngineEvent } from './protocol';
|
| 7 |
+
import {
|
| 8 |
+
BrowserEngineRuntime,
|
| 9 |
+
resolveLoadTuning,
|
| 10 |
+
resolveRuntimeBatchShape,
|
| 11 |
+
selectWasmFlavor,
|
| 12 |
+
} from './runtime';
|
| 13 |
|
| 14 |
type DownloadOptions = {
|
| 15 |
signal?: AbortSignal;
|
|
|
|
| 460 |
});
|
| 461 |
});
|
| 462 |
|
| 463 |
+
describe('resolveRuntimeBatchShape', () => {
|
| 464 |
+
const releaseDefaults = resolveLoadTuning(undefined, 'auto');
|
| 465 |
+
|
| 466 |
+
it('caps only the compat release path to short Safari-safe command buffers', () => {
|
| 467 |
+
expect(resolveRuntimeBatchShape(releaseDefaults, 'compat')).toEqual({
|
| 468 |
+
nBatch: 32,
|
| 469 |
+
nUbatch: 16,
|
| 470 |
+
});
|
| 471 |
+
expect(resolveRuntimeBatchShape(releaseDefaults, 'jspi')).toEqual({
|
| 472 |
+
nBatch: undefined,
|
| 473 |
+
nUbatch: undefined,
|
| 474 |
+
});
|
| 475 |
+
});
|
| 476 |
+
|
| 477 |
+
it('preserves explicit tuning and never defaults microbatch above batch', () => {
|
| 478 |
+
expect(resolveRuntimeBatchShape({ nBatch: 96, nUbatch: 24 }, 'compat')).toEqual({
|
| 479 |
+
nBatch: 96,
|
| 480 |
+
nUbatch: 24,
|
| 481 |
+
});
|
| 482 |
+
expect(resolveRuntimeBatchShape({ nBatch: 8, nUbatch: null }, 'compat')).toEqual({
|
| 483 |
+
nBatch: 8,
|
| 484 |
+
nUbatch: 8,
|
| 485 |
+
});
|
| 486 |
+
});
|
| 487 |
+
});
|
| 488 |
+
|
| 489 |
describe('selectWasmFlavor', () => {
|
| 490 |
it('uses the browser-native flavor for auto', () => {
|
| 491 |
expect(selectWasmFlavor('auto', false)).toBe('jspi');
|
src/engine/runtime-generate.test.ts
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
import { describe, expect, it } from 'vitest';
|
| 2 |
import type { ChatCompletionChunk } from '../../vendor/wllama-bonsai/esm/index.js';
|
|
|
|
| 3 |
import { BrowserEngineRuntime } from './runtime';
|
| 4 |
|
| 5 |
interface CompletionOptions {
|
| 6 |
onData(chunk: ChatCompletionChunk): void;
|
| 7 |
logprobs?: boolean;
|
| 8 |
top_logprobs?: number;
|
|
|
|
|
|
|
| 9 |
}
|
| 10 |
|
|
|
|
|
|
|
| 11 |
interface RuntimeInternals {
|
|
|
|
| 12 |
wllama: {
|
| 13 |
isModelLoaded(): boolean;
|
| 14 |
createChatCompletion(options: CompletionOptions): Promise<void>;
|
| 15 |
} | null;
|
| 16 |
loaded: {
|
| 17 |
manifest: unknown;
|
| 18 |
-
backend: 'wasm';
|
| 19 |
model: {
|
| 20 |
id: '1_7b';
|
| 21 |
displayName: string;
|
| 22 |
cpuFallback: true;
|
| 23 |
runtimePolicy: {
|
| 24 |
flashAttention: false;
|
| 25 |
-
tokenEmbeddingOnWebGPU:
|
| 26 |
-
requireSingleWebGPUGraph:
|
| 27 |
};
|
| 28 |
};
|
| 29 |
} | null;
|
|
@@ -145,6 +151,204 @@ describe('BrowserEngineRuntime generation telemetry', () => {
|
|
| 145 |
expect(completionOptions).not.toHaveProperty('top_logprobs');
|
| 146 |
});
|
| 147 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 148 |
it('reconciles incomplete streamed usage with llama.cpp timing counts', async () => {
|
| 149 |
const runtime = new BrowserEngineRuntime();
|
| 150 |
const internals = runtime as unknown as RuntimeInternals;
|
|
|
|
| 1 |
+
import { describe, expect, it, vi } from 'vitest';
|
| 2 |
import type { ChatCompletionChunk } from '../../vendor/wllama-bonsai/esm/index.js';
|
| 3 |
+
import type { EngineEvent } from './protocol';
|
| 4 |
import { BrowserEngineRuntime } from './runtime';
|
| 5 |
|
| 6 |
interface CompletionOptions {
|
| 7 |
onData(chunk: ChatCompletionChunk): void;
|
| 8 |
logprobs?: boolean;
|
| 9 |
top_logprobs?: number;
|
| 10 |
+
return_progress?: boolean;
|
| 11 |
+
timings_per_token?: boolean;
|
| 12 |
}
|
| 13 |
|
| 14 |
+
type GenerationProgressEvent = Extract<EngineEvent, { event: 'generation' }>;
|
| 15 |
+
|
| 16 |
interface RuntimeInternals {
|
| 17 |
+
wllamaFlavor: 'compat' | 'jspi' | null;
|
| 18 |
wllama: {
|
| 19 |
isModelLoaded(): boolean;
|
| 20 |
createChatCompletion(options: CompletionOptions): Promise<void>;
|
| 21 |
} | null;
|
| 22 |
loaded: {
|
| 23 |
manifest: unknown;
|
| 24 |
+
backend: 'wasm' | 'webgpu';
|
| 25 |
model: {
|
| 26 |
id: '1_7b';
|
| 27 |
displayName: string;
|
| 28 |
cpuFallback: true;
|
| 29 |
runtimePolicy: {
|
| 30 |
flashAttention: false;
|
| 31 |
+
tokenEmbeddingOnWebGPU: boolean;
|
| 32 |
+
requireSingleWebGPUGraph: boolean;
|
| 33 |
};
|
| 34 |
};
|
| 35 |
} | null;
|
|
|
|
| 151 |
expect(completionOptions).not.toHaveProperty('top_logprobs');
|
| 152 |
});
|
| 153 |
|
| 154 |
+
it('streams separate live rates without trusting unstable first-token timings', async () => {
|
| 155 |
+
const runtime = new BrowserEngineRuntime();
|
| 156 |
+
const internals = runtime as unknown as RuntimeInternals;
|
| 157 |
+
let completionOptions: CompletionOptions | null = null;
|
| 158 |
+
internals.loaded = {
|
| 159 |
+
manifest: {},
|
| 160 |
+
backend: 'wasm',
|
| 161 |
+
model: {
|
| 162 |
+
id: '1_7b',
|
| 163 |
+
displayName: 'Fixture Bonsai',
|
| 164 |
+
cpuFallback: true,
|
| 165 |
+
runtimePolicy: {
|
| 166 |
+
flashAttention: false,
|
| 167 |
+
tokenEmbeddingOnWebGPU: true,
|
| 168 |
+
requireSingleWebGPUGraph: false,
|
| 169 |
+
},
|
| 170 |
+
},
|
| 171 |
+
};
|
| 172 |
+
internals.wllama = {
|
| 173 |
+
isModelLoaded: () => true,
|
| 174 |
+
createChatCompletion: async (options) => {
|
| 175 |
+
completionOptions = options;
|
| 176 |
+
const earlyPrefill = chunk('', {
|
| 177 |
+
timings: {
|
| 178 |
+
cache_n: 2,
|
| 179 |
+
prompt_n: 7,
|
| 180 |
+
prompt_ms: 0.001,
|
| 181 |
+
prompt_per_token_ms: 0.001 / 5,
|
| 182 |
+
prompt_per_second: 5_000_000,
|
| 183 |
+
predicted_n: 0,
|
| 184 |
+
predicted_ms: 0,
|
| 185 |
+
predicted_per_token_ms: 0,
|
| 186 |
+
predicted_per_second: 0,
|
| 187 |
+
},
|
| 188 |
+
}) as ChatCompletionChunk & {
|
| 189 |
+
prompt_progress: { total: number; cache: number; processed: number; time_ms: number };
|
| 190 |
+
};
|
| 191 |
+
const earlyPrefillChoice = earlyPrefill.choices[0];
|
| 192 |
+
if (!earlyPrefillChoice) throw new Error('Prefill fixture requires one choice.');
|
| 193 |
+
earlyPrefillChoice.finish_reason = null;
|
| 194 |
+
earlyPrefill.prompt_progress = { total: 12, cache: 2, processed: 7, time_ms: 1 };
|
| 195 |
+
options.onData(earlyPrefill);
|
| 196 |
+
const prefill = chunk('') as ChatCompletionChunk & {
|
| 197 |
+
prompt_progress: { total: number; cache: number; processed: number; time_ms: number };
|
| 198 |
+
};
|
| 199 |
+
const prefillChoice = prefill.choices[0];
|
| 200 |
+
if (!prefillChoice) throw new Error('Prefill fixture requires one choice.');
|
| 201 |
+
prefillChoice.finish_reason = null;
|
| 202 |
+
prefill.prompt_progress = { total: 12, cache: 2, processed: 7, time_ms: 100 };
|
| 203 |
+
options.onData(prefill);
|
| 204 |
+
options.onData(chunk('A', {
|
| 205 |
+
timings: {
|
| 206 |
+
cache_n: 2,
|
| 207 |
+
prompt_n: 12,
|
| 208 |
+
prompt_ms: 100,
|
| 209 |
+
prompt_per_token_ms: 10,
|
| 210 |
+
prompt_per_second: 100,
|
| 211 |
+
predicted_n: 1,
|
| 212 |
+
predicted_ms: 0.001,
|
| 213 |
+
predicted_per_token_ms: 0.001,
|
| 214 |
+
predicted_per_second: 1_000_000,
|
| 215 |
+
},
|
| 216 |
+
}));
|
| 217 |
+
options.onData(chunk('B', {
|
| 218 |
+
usage: { prompt_tokens: 12, completion_tokens: 3, total_tokens: 15 },
|
| 219 |
+
timings: {
|
| 220 |
+
cache_n: 2,
|
| 221 |
+
prompt_n: 12,
|
| 222 |
+
prompt_ms: 100,
|
| 223 |
+
prompt_per_token_ms: 10,
|
| 224 |
+
prompt_per_second: 100,
|
| 225 |
+
predicted_n: 3,
|
| 226 |
+
predicted_ms: 200,
|
| 227 |
+
predicted_per_token_ms: 200 / 3,
|
| 228 |
+
predicted_per_second: 15,
|
| 229 |
+
},
|
| 230 |
+
}));
|
| 231 |
+
options.onData(chunk(''));
|
| 232 |
+
},
|
| 233 |
+
};
|
| 234 |
+
const progress: GenerationProgressEvent[] = [];
|
| 235 |
+
|
| 236 |
+
const result = await runtime.generate(
|
| 237 |
+
'request-live-progress',
|
| 238 |
+
{ messages: [{ role: 'user', content: 'Count.' }] },
|
| 239 |
+
new AbortController().signal,
|
| 240 |
+
(event) => {
|
| 241 |
+
if (event.event === 'generation') progress.push(event);
|
| 242 |
+
},
|
| 243 |
+
);
|
| 244 |
+
|
| 245 |
+
expect(completionOptions).toHaveProperty('return_progress', true);
|
| 246 |
+
expect(completionOptions).toHaveProperty('timings_per_token', true);
|
| 247 |
+
expect(progress.map(({ phase }) => phase)).toEqual([
|
| 248 |
+
'prefill',
|
| 249 |
+
'prefill',
|
| 250 |
+
'prefill',
|
| 251 |
+
'decode',
|
| 252 |
+
'decode',
|
| 253 |
+
]);
|
| 254 |
+
expect(progress[1]).toMatchObject({
|
| 255 |
+
promptProcessed: 7,
|
| 256 |
+
promptTotal: 12,
|
| 257 |
+
promptCached: 2,
|
| 258 |
+
elapsedMs: 1,
|
| 259 |
+
promptTokensPerSecond: 0,
|
| 260 |
+
decodeTokensPerSecond: 0,
|
| 261 |
+
});
|
| 262 |
+
expect(progress[2]).toMatchObject({
|
| 263 |
+
promptProcessed: 7,
|
| 264 |
+
promptTotal: 12,
|
| 265 |
+
promptCached: 2,
|
| 266 |
+
elapsedMs: 100,
|
| 267 |
+
promptTokensPerSecond: 50,
|
| 268 |
+
decodeTokensPerSecond: 0,
|
| 269 |
+
});
|
| 270 |
+
expect(progress[3]).toMatchObject({
|
| 271 |
+
completionTokens: 1,
|
| 272 |
+
elapsedMs: 0.001,
|
| 273 |
+
decodeTokensPerSecond: 0,
|
| 274 |
+
});
|
| 275 |
+
expect(progress[4]).toMatchObject({ completionTokens: 3, elapsedMs: 200 });
|
| 276 |
+
expect(progress[4]?.decodeTokensPerSecond).toBeCloseTo(10, 3);
|
| 277 |
+
expect(result.text).toBe('AB');
|
| 278 |
+
expect(result.usage).toEqual({ promptTokens: 12, completionTokens: 3, totalTokens: 15 });
|
| 279 |
+
});
|
| 280 |
+
|
| 281 |
+
it('uses token arrival time for live decode telemetry in the compat runtime', async () => {
|
| 282 |
+
const runtime = new BrowserEngineRuntime();
|
| 283 |
+
const internals = runtime as unknown as RuntimeInternals;
|
| 284 |
+
let completionOptions: CompletionOptions | null = null;
|
| 285 |
+
internals.wllamaFlavor = 'compat';
|
| 286 |
+
internals.loaded = {
|
| 287 |
+
manifest: {},
|
| 288 |
+
backend: 'wasm',
|
| 289 |
+
model: {
|
| 290 |
+
id: '1_7b',
|
| 291 |
+
displayName: 'Fixture Bonsai',
|
| 292 |
+
cpuFallback: true,
|
| 293 |
+
runtimePolicy: {
|
| 294 |
+
flashAttention: false,
|
| 295 |
+
tokenEmbeddingOnWebGPU: true,
|
| 296 |
+
requireSingleWebGPUGraph: false,
|
| 297 |
+
},
|
| 298 |
+
},
|
| 299 |
+
};
|
| 300 |
+
internals.wllama = {
|
| 301 |
+
isModelLoaded: () => true,
|
| 302 |
+
createChatCompletion: async (options) => {
|
| 303 |
+
completionOptions = options;
|
| 304 |
+
options.onData(chunk('A'));
|
| 305 |
+
options.onData(chunk('B', {
|
| 306 |
+
usage: { prompt_tokens: 8, completion_tokens: 2, total_tokens: 10 },
|
| 307 |
+
timings: {
|
| 308 |
+
cache_n: 0,
|
| 309 |
+
prompt_n: 8,
|
| 310 |
+
prompt_ms: 200,
|
| 311 |
+
prompt_per_token_ms: 25,
|
| 312 |
+
prompt_per_second: 40,
|
| 313 |
+
predicted_n: 2,
|
| 314 |
+
predicted_ms: 500,
|
| 315 |
+
predicted_per_token_ms: 250,
|
| 316 |
+
predicted_per_second: 4,
|
| 317 |
+
},
|
| 318 |
+
}));
|
| 319 |
+
options.onData(chunk(''));
|
| 320 |
+
},
|
| 321 |
+
};
|
| 322 |
+
const now = vi.spyOn(performance, 'now')
|
| 323 |
+
.mockReturnValueOnce(1_000)
|
| 324 |
+
.mockReturnValueOnce(1_250);
|
| 325 |
+
const progress: GenerationProgressEvent[] = [];
|
| 326 |
+
|
| 327 |
+
try {
|
| 328 |
+
const result = await runtime.generate(
|
| 329 |
+
'request-compat-progress',
|
| 330 |
+
{ messages: [{ role: 'user', content: 'Count.' }] },
|
| 331 |
+
new AbortController().signal,
|
| 332 |
+
(event) => {
|
| 333 |
+
if (event.event === 'generation') progress.push(event);
|
| 334 |
+
},
|
| 335 |
+
);
|
| 336 |
+
|
| 337 |
+
expect(completionOptions).not.toHaveProperty('timings_per_token');
|
| 338 |
+
expect(completionOptions).not.toHaveProperty('return_progress');
|
| 339 |
+
expect(progress.map(({ phase }) => phase)).toEqual([
|
| 340 |
+
'prefill',
|
| 341 |
+
'decode',
|
| 342 |
+
'decode',
|
| 343 |
+
]);
|
| 344 |
+
expect(progress[1]).toMatchObject({ completionTokens: 1, decodeTokensPerSecond: 0 });
|
| 345 |
+
expect(progress[2]).toMatchObject({ completionTokens: 2, decodeTokensPerSecond: 4 });
|
| 346 |
+
expect(result.timings).toEqual({ promptTokensPerSecond: 40, predictedTokensPerSecond: 4 });
|
| 347 |
+
} finally {
|
| 348 |
+
now.mockRestore();
|
| 349 |
+
}
|
| 350 |
+
});
|
| 351 |
+
|
| 352 |
it('reconciles incomplete streamed usage with llama.cpp timing counts', async () => {
|
| 353 |
const runtime = new BrowserEngineRuntime();
|
| 354 |
const internals = runtime as unknown as RuntimeInternals;
|
src/engine/runtime-score-sequence.test.ts
CHANGED
|
@@ -52,6 +52,7 @@ const backendReport: BackendReport = {
|
|
| 52 |
cacheTypeK: 'f16',
|
| 53 |
cacheTypeV: 'f16',
|
| 54 |
webgpuKvBufferBytes: 128 * 1024 ** 2,
|
|
|
|
| 55 |
};
|
| 56 |
|
| 57 |
function configuredRuntime(createCompletion: (options: RawCompletionOptions) => Promise<unknown>) {
|
|
|
|
| 52 |
cacheTypeK: 'f16',
|
| 53 |
cacheTypeV: 'f16',
|
| 54 |
webgpuKvBufferBytes: 128 * 1024 ** 2,
|
| 55 |
+
webgpuTrace: [],
|
| 56 |
};
|
| 57 |
|
| 58 |
function configuredRuntime(createCompletion: (options: RawCompletionOptions) => Promise<unknown>) {
|
src/engine/runtime.ts
CHANGED
|
@@ -38,6 +38,7 @@ import type {
|
|
| 38 |
EngineCapabilities,
|
| 39 |
EngineEvent,
|
| 40 |
EngineSampledTokenTraceEntry,
|
|
|
|
| 41 |
GenerateParams,
|
| 42 |
GenerateResult,
|
| 43 |
LoadModelParams,
|
|
@@ -57,6 +58,9 @@ const WLLAMA_COMPAT_WORKER_PATH = '/wasm/wllama-compat.js';
|
|
| 57 |
const MAX_GLUE_INT = 2_147_483_647;
|
| 58 |
const DEVICE_LOST_EXIT_GRACE_MS = 100;
|
| 59 |
const TOKEN_TRACE_TOP_LOGPROBS = 5;
|
|
|
|
|
|
|
|
|
|
| 60 |
const STATE_DRIFT_REFERENCE_TOKENS = 1_024;
|
| 61 |
const STATE_DRIFT_PROMPT_TOKENS = 38;
|
| 62 |
const TEACHER_FORCE_LOGIT_BIAS = 1_000;
|
|
@@ -70,6 +74,23 @@ export interface ResolvedLoadTuning {
|
|
| 70 |
wasmFlavor: BenchmarkWasmFlavor;
|
| 71 |
}
|
| 72 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
function validateBatchValue(value: unknown, label: 'n_batch' | 'n_ubatch'): number | null {
|
| 74 |
if (value === undefined) return null;
|
| 75 |
if (!Number.isSafeInteger(value) || (value as number) <= 0 || (value as number) > MAX_GLUE_INT) {
|
|
@@ -201,6 +222,33 @@ interface LoadedModelState {
|
|
| 201 |
interface CachedShardState {
|
| 202 |
blobs: Array<Blob | null>;
|
| 203 |
cachedBytes: number;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
}
|
| 205 |
|
| 206 |
function errorText(error: unknown): string {
|
|
@@ -277,13 +325,41 @@ function installWorkerDocumentShim(): void {
|
|
| 277 |
function emitProgress(
|
| 278 |
sink: EventSink,
|
| 279 |
requestId: string,
|
| 280 |
-
values: Omit<
|
|
|
|
|
|
|
|
|
|
| 281 |
): void {
|
| 282 |
sink({
|
| 283 |
type: 'event',
|
| 284 |
requestId,
|
| 285 |
event: 'progress',
|
| 286 |
...values,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 287 |
});
|
| 288 |
}
|
| 289 |
|
|
@@ -730,13 +806,32 @@ export class BrowserEngineRuntime {
|
|
| 730 |
}
|
| 731 |
|
| 732 |
private async inspectCachedShards(
|
|
|
|
| 733 |
wllama: Wllama,
|
| 734 |
model: ManifestModelV2,
|
| 735 |
urls: readonly string[],
|
| 736 |
signal: AbortSignal,
|
|
|
|
| 737 |
): Promise<CachedShardState> {
|
| 738 |
const blobs: Array<Blob | null> = [];
|
|
|
|
| 739 |
let cachedBytes = 0;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 740 |
for (let index = 0; index < urls.length; index += 1) {
|
| 741 |
const url = urls[index];
|
| 742 |
const file = model.files[index];
|
|
@@ -749,10 +844,23 @@ export class BrowserEngineRuntime {
|
|
| 749 |
wllama.cacheManager.getMetadata(key),
|
| 750 |
]);
|
| 751 |
if (blob?.size === file.bytes && metadata?.sha256 === file.sha256) {
|
| 752 |
-
const
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 753 |
if (integrity.matches) {
|
| 754 |
blobs.push(blob);
|
| 755 |
cachedBytes += file.bytes;
|
|
|
|
|
|
|
|
|
|
|
|
|
| 756 |
continue;
|
| 757 |
}
|
| 758 |
}
|
|
@@ -760,8 +868,15 @@ export class BrowserEngineRuntime {
|
|
| 760 |
await wllama.cacheManager.delete(url);
|
| 761 |
}
|
| 762 |
blobs.push(null);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 763 |
}
|
| 764 |
-
return { blobs, cachedBytes };
|
| 765 |
}
|
| 766 |
|
| 767 |
private async downloadShards(
|
|
@@ -779,15 +894,21 @@ export class BrowserEngineRuntime {
|
|
| 779 |
else signal.addEventListener('abort', abortDownloads, { once: true });
|
| 780 |
const downloadSignal = downloadController.signal;
|
| 781 |
const loadedByShard = model.files.map((file, index) => cached.blobs[index] ? file.bytes : 0);
|
| 782 |
-
const
|
|
|
|
| 783 |
const file = model.files[shardIndex] ?? null;
|
| 784 |
emitProgress(sink, requestId, {
|
| 785 |
-
phase
|
| 786 |
loadedBytes: loadedByShard.reduce((sum, value) => sum + value, 0),
|
| 787 |
totalBytes: model.downloadBytes,
|
| 788 |
shardIndex,
|
| 789 |
shardCount: model.files.length,
|
| 790 |
shardPath: file?.path ?? null,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 791 |
});
|
| 792 |
};
|
| 793 |
emitDownloadProgress(0);
|
|
@@ -810,7 +931,13 @@ export class BrowserEngineRuntime {
|
|
| 810 |
if (!file || !url) {
|
| 811 |
throw new EngineRuntimeError('INVALID_SHARD_ORDER', 'Manifest shard URL and file counts differ.');
|
| 812 |
}
|
|
|
|
|
|
|
| 813 |
try {
|
|
|
|
|
|
|
|
|
|
|
|
|
| 814 |
await wllama.cacheManager.download(url, {
|
| 815 |
signal: downloadSignal,
|
| 816 |
metadataAdditional: {
|
|
@@ -820,6 +947,7 @@ export class BrowserEngineRuntime {
|
|
| 820 |
},
|
| 821 |
progressCallback: ({ loaded }) => {
|
| 822 |
loadedByShard[index] = Math.min(loaded, file.bytes);
|
|
|
|
| 823 |
emitDownloadProgress(index);
|
| 824 |
},
|
| 825 |
});
|
|
@@ -831,7 +959,15 @@ export class BrowserEngineRuntime {
|
|
| 831 |
`Downloaded shard ${file.path} has ${blob?.size ?? 0} bytes; expected ${file.bytes}.`,
|
| 832 |
);
|
| 833 |
}
|
| 834 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 835 |
if (!integrity.matches) {
|
| 836 |
throw new EngineRuntimeError(
|
| 837 |
'SHARD_HASH_MISMATCH',
|
|
@@ -847,6 +983,9 @@ export class BrowserEngineRuntime {
|
|
| 847 |
});
|
| 848 |
cached.blobs[index] = blob;
|
| 849 |
loadedByShard[index] = file.bytes;
|
|
|
|
|
|
|
|
|
|
| 850 |
emitDownloadProgress(index);
|
| 851 |
} catch (error) {
|
| 852 |
const failure = classifyShardFailure(error, downloadSignal);
|
|
@@ -855,6 +994,9 @@ export class BrowserEngineRuntime {
|
|
| 855 |
if (isPrimaryFailure) firstFailureReserved = true;
|
| 856 |
cached.blobs[index] = null;
|
| 857 |
loadedByShard[index] = 0;
|
|
|
|
|
|
|
|
|
|
| 858 |
let partialDeleted = false;
|
| 859 |
let cleanupError: unknown;
|
| 860 |
try {
|
|
@@ -863,7 +1005,7 @@ export class BrowserEngineRuntime {
|
|
| 863 |
} catch (deleteError) {
|
| 864 |
cleanupError = deleteError;
|
| 865 |
}
|
| 866 |
-
emitDownloadProgress(index);
|
| 867 |
const normalizedFailure = shardFailure(
|
| 868 |
error,
|
| 869 |
failure,
|
|
@@ -922,6 +1064,9 @@ export class BrowserEngineRuntime {
|
|
| 922 |
shardIndex: null,
|
| 923 |
shardCount: 0,
|
| 924 |
shardPath: null,
|
|
|
|
|
|
|
|
|
|
| 925 |
});
|
| 926 |
const manifest = params.manifestUrl !== undefined
|
| 927 |
? await loadModelManifestV2(params.manifestUrl, signal)
|
|
@@ -934,6 +1079,9 @@ export class BrowserEngineRuntime {
|
|
| 934 |
shardIndex: null,
|
| 935 |
shardCount: 0,
|
| 936 |
shardPath: null,
|
|
|
|
|
|
|
|
|
|
| 937 |
});
|
| 938 |
const model = findManifestModel(manifest, params.modelId);
|
| 939 |
const contextSize = params.contextSize ?? model.defaultContext;
|
|
@@ -951,7 +1099,7 @@ export class BrowserEngineRuntime {
|
|
| 951 |
}
|
| 952 |
const urls = orderedShardUrls(manifest, model);
|
| 953 |
const wllama = await this.ensureWllama(wasmFlavor);
|
| 954 |
-
const cached = await this.inspectCachedShards(wllama, model, urls, signal);
|
| 955 |
const [adapter, storage] = await Promise.all([
|
| 956 |
inspectWebGpuAdapter(),
|
| 957 |
estimateStorage(),
|
|
@@ -971,25 +1119,45 @@ export class BrowserEngineRuntime {
|
|
| 971 |
throwIfAborted(signal);
|
| 972 |
emitProgress(sink, requestId, {
|
| 973 |
phase: 'load',
|
| 974 |
-
loadedBytes:
|
| 975 |
totalBytes: model.downloadBytes,
|
| 976 |
shardIndex: null,
|
| 977 |
shardCount: model.files.length,
|
| 978 |
shardPath: null,
|
|
|
|
|
|
|
|
|
|
| 979 |
});
|
| 980 |
const defaultThreads = Math.max(1, Math.floor((navigator.hardwareConcurrency || 1) / 2));
|
| 981 |
const threads = params.threads ?? (gate.selectedBackend === 'wasm' ? defaultThreads : 1);
|
| 982 |
if (!Number.isSafeInteger(threads) || threads <= 0) {
|
| 983 |
throw new EngineRuntimeError('INVALID_THREAD_COUNT', 'Thread count must be a positive integer.');
|
| 984 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 985 |
try {
|
| 986 |
await this.raceWebGpuDeviceLoss('load', gate.selectedBackend, model, wllama.loadModel(blobs, {
|
| 987 |
n_gpu_layers: gate.selectedBackend === 'webgpu' ? 99999 : 0,
|
| 988 |
offload_token_embedding: gate.selectedBackend === 'webgpu',
|
| 989 |
n_ctx: contextSize,
|
| 990 |
n_threads: threads,
|
| 991 |
-
n_batch:
|
| 992 |
-
n_ubatch:
|
| 993 |
seed: 42,
|
| 994 |
flash_attn: tuning.flashMode === 'auto',
|
| 995 |
warmup: false,
|
|
@@ -1075,6 +1243,9 @@ export class BrowserEngineRuntime {
|
|
| 1075 |
shardIndex: null,
|
| 1076 |
shardCount: model.files.length,
|
| 1077 |
shardPath: null,
|
|
|
|
|
|
|
|
|
|
| 1078 |
});
|
| 1079 |
return {
|
| 1080 |
modelId: model.id,
|
|
@@ -1106,6 +1277,8 @@ export class BrowserEngineRuntime {
|
|
| 1106 |
this.wllamaFlavor = null;
|
| 1107 |
this.loaded = null;
|
| 1108 |
throw error;
|
|
|
|
|
|
|
| 1109 |
}
|
| 1110 |
}
|
| 1111 |
|
|
@@ -1130,8 +1303,55 @@ export class BrowserEngineRuntime {
|
|
| 1130 |
let finishReason: GenerateResult['finishReason'] = null;
|
| 1131 |
let usage: GenerateResult['usage'] = null;
|
| 1132 |
let timings: GenerateResult['timings'] = null;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1133 |
const tokenTrace: EngineSampledTokenTraceEntry[] | null = params.returnTokenIds === true ? [] : null;
|
| 1134 |
const streamedToolCalls = new ToolCallAccumulator();
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1135 |
try {
|
| 1136 |
await this.raceWebGpuDeviceLoss('generate', loaded.backend, loaded.model, wllama.createChatCompletion({
|
| 1137 |
messages: params.messages as ChatCompletionMessage[],
|
|
@@ -1149,13 +1369,74 @@ export class BrowserEngineRuntime {
|
|
| 1149 |
? { logprobs: true, top_logprobs: TOKEN_TRACE_TOP_LOGPROBS }
|
| 1150 |
: {}),
|
| 1151 |
abortSignal: signal,
|
| 1152 |
-
timings_per_token: true,
|
| 1153 |
onData: (chunk: ChatCompletionChunk) => {
|
| 1154 |
-
|
| 1155 |
-
streamedToolCalls.append(chunk.choices[0]?.delta.tool_calls);
|
| 1156 |
const delta = chunk.choices[0]?.delta.content;
|
| 1157 |
const textDelta = typeof delta === 'string' ? delta : '';
|
| 1158 |
const reasoningDelta = asReasoningDelta(chunk);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1159 |
if (textDelta || reasoningDelta) {
|
| 1160 |
text += textDelta;
|
| 1161 |
reasoningText += reasoningDelta ?? '';
|
|
|
|
| 38 |
EngineCapabilities,
|
| 39 |
EngineEvent,
|
| 40 |
EngineSampledTokenTraceEntry,
|
| 41 |
+
EngineShardProgress,
|
| 42 |
GenerateParams,
|
| 43 |
GenerateResult,
|
| 44 |
LoadModelParams,
|
|
|
|
| 58 |
const MAX_GLUE_INT = 2_147_483_647;
|
| 59 |
const DEVICE_LOST_EXIT_GRACE_MS = 100;
|
| 60 |
const TOKEN_TRACE_TOP_LOGPROBS = 5;
|
| 61 |
+
const MIN_LIVE_RATE_WINDOW_MS = 50;
|
| 62 |
+
const COMPAT_STABLE_BATCH_SIZE = 32;
|
| 63 |
+
const COMPAT_STABLE_MICRO_BATCH_SIZE = 16;
|
| 64 |
const STATE_DRIFT_REFERENCE_TOKENS = 1_024;
|
| 65 |
const STATE_DRIFT_PROMPT_TOKENS = 38;
|
| 66 |
const TEACHER_FORCE_LOGIT_BIAS = 1_000;
|
|
|
|
| 74 |
wasmFlavor: BenchmarkWasmFlavor;
|
| 75 |
}
|
| 76 |
|
| 77 |
+
export function resolveRuntimeBatchShape(
|
| 78 |
+
tuning: Pick<ResolvedLoadTuning, 'nBatch' | 'nUbatch'>,
|
| 79 |
+
wasmFlavor: 'jspi' | 'compat',
|
| 80 |
+
): { nBatch: number | undefined; nUbatch: number | undefined } {
|
| 81 |
+
if (wasmFlavor !== 'compat') {
|
| 82 |
+
return {
|
| 83 |
+
nBatch: tuning.nBatch ?? undefined,
|
| 84 |
+
nUbatch: tuning.nUbatch ?? undefined,
|
| 85 |
+
};
|
| 86 |
+
}
|
| 87 |
+
const nBatch = tuning.nBatch ?? COMPAT_STABLE_BATCH_SIZE;
|
| 88 |
+
return {
|
| 89 |
+
nBatch,
|
| 90 |
+
nUbatch: tuning.nUbatch ?? Math.min(COMPAT_STABLE_MICRO_BATCH_SIZE, nBatch),
|
| 91 |
+
};
|
| 92 |
+
}
|
| 93 |
+
|
| 94 |
function validateBatchValue(value: unknown, label: 'n_batch' | 'n_ubatch'): number | null {
|
| 95 |
if (value === undefined) return null;
|
| 96 |
if (!Number.isSafeInteger(value) || (value as number) <= 0 || (value as number) > MAX_GLUE_INT) {
|
|
|
|
| 222 |
interface CachedShardState {
|
| 223 |
blobs: Array<Blob | null>;
|
| 224 |
cachedBytes: number;
|
| 225 |
+
shards?: EngineShardProgress[];
|
| 226 |
+
}
|
| 227 |
+
|
| 228 |
+
function initialShardProgress(
|
| 229 |
+
model: ManifestModelV2,
|
| 230 |
+
blobs: readonly (Blob | null)[] = [],
|
| 231 |
+
): EngineShardProgress[] {
|
| 232 |
+
return model.files.map((file, index) => ({
|
| 233 |
+
index,
|
| 234 |
+
path: file.path,
|
| 235 |
+
loadedBytes: blobs[index] ? file.bytes : 0,
|
| 236 |
+
verifiedBytes: blobs[index] ? file.bytes : 0,
|
| 237 |
+
totalBytes: file.bytes,
|
| 238 |
+
state: blobs[index] ? 'cached' : 'queued',
|
| 239 |
+
}));
|
| 240 |
+
}
|
| 241 |
+
|
| 242 |
+
function copyShardProgress(shards: readonly EngineShardProgress[]): EngineShardProgress[] {
|
| 243 |
+
return shards.map((shard) => ({ ...shard }));
|
| 244 |
+
}
|
| 245 |
+
|
| 246 |
+
function totalShardProgress(shards: readonly EngineShardProgress[]): number {
|
| 247 |
+
return shards.reduce((sum, shard) => sum + shard.loadedBytes, 0);
|
| 248 |
+
}
|
| 249 |
+
|
| 250 |
+
function totalVerifiedShardProgress(shards: readonly EngineShardProgress[]): number {
|
| 251 |
+
return shards.reduce((sum, shard) => sum + shard.verifiedBytes, 0);
|
| 252 |
}
|
| 253 |
|
| 254 |
function errorText(error: unknown): string {
|
|
|
|
| 325 |
function emitProgress(
|
| 326 |
sink: EventSink,
|
| 327 |
requestId: string,
|
| 328 |
+
values: Omit<
|
| 329 |
+
Extract<EngineEvent, { event: 'progress' }>,
|
| 330 |
+
'type' | 'requestId' | 'event' | 'nativeStage'
|
| 331 |
+
> & { nativeStage?: string | null },
|
| 332 |
): void {
|
| 333 |
sink({
|
| 334 |
type: 'event',
|
| 335 |
requestId,
|
| 336 |
event: 'progress',
|
| 337 |
...values,
|
| 338 |
+
nativeStage: values.nativeStage ?? null,
|
| 339 |
+
});
|
| 340 |
+
}
|
| 341 |
+
|
| 342 |
+
interface PromptProgressSample {
|
| 343 |
+
total: number;
|
| 344 |
+
cache: number;
|
| 345 |
+
processed: number;
|
| 346 |
+
time_ms: number;
|
| 347 |
+
}
|
| 348 |
+
|
| 349 |
+
type LiveCompletionChunk = ChatCompletionChunk & {
|
| 350 |
+
prompt_progress?: PromptProgressSample;
|
| 351 |
+
};
|
| 352 |
+
|
| 353 |
+
function emitGenerationProgress(
|
| 354 |
+
sink: EventSink,
|
| 355 |
+
requestId: string,
|
| 356 |
+
values: Omit<Extract<EngineEvent, { event: 'generation' }>, 'type' | 'requestId' | 'event'>,
|
| 357 |
+
): void {
|
| 358 |
+
sink({
|
| 359 |
+
type: 'event',
|
| 360 |
+
requestId,
|
| 361 |
+
event: 'generation',
|
| 362 |
+
...values,
|
| 363 |
});
|
| 364 |
}
|
| 365 |
|
|
|
|
| 806 |
}
|
| 807 |
|
| 808 |
private async inspectCachedShards(
|
| 809 |
+
requestId: string,
|
| 810 |
wllama: Wllama,
|
| 811 |
model: ManifestModelV2,
|
| 812 |
urls: readonly string[],
|
| 813 |
signal: AbortSignal,
|
| 814 |
+
sink: EventSink,
|
| 815 |
): Promise<CachedShardState> {
|
| 816 |
const blobs: Array<Blob | null> = [];
|
| 817 |
+
const shards = initialShardProgress(model);
|
| 818 |
let cachedBytes = 0;
|
| 819 |
+
const emitCacheProgress = (shardIndex: number, phase: 'cache' | 'verify'): void => {
|
| 820 |
+
const shard = shards[shardIndex] ?? null;
|
| 821 |
+
emitProgress(sink, requestId, {
|
| 822 |
+
phase,
|
| 823 |
+
loadedBytes: totalShardProgress(shards),
|
| 824 |
+
totalBytes: model.downloadBytes,
|
| 825 |
+
shardIndex,
|
| 826 |
+
shardCount: model.files.length,
|
| 827 |
+
shardPath: shard?.path ?? null,
|
| 828 |
+
stageProgress: phase === 'verify'
|
| 829 |
+
? totalVerifiedShardProgress(shards) / model.downloadBytes
|
| 830 |
+
: totalShardProgress(shards) / model.downloadBytes,
|
| 831 |
+
residentBytes: null,
|
| 832 |
+
shards: copyShardProgress(shards),
|
| 833 |
+
});
|
| 834 |
+
};
|
| 835 |
for (let index = 0; index < urls.length; index += 1) {
|
| 836 |
const url = urls[index];
|
| 837 |
const file = model.files[index];
|
|
|
|
| 844 |
wllama.cacheManager.getMetadata(key),
|
| 845 |
]);
|
| 846 |
if (blob?.size === file.bytes && metadata?.sha256 === file.sha256) {
|
| 847 |
+
const shard = shards[index];
|
| 848 |
+
if (!shard) throw new EngineRuntimeError('INVALID_SHARD_ORDER', 'Manifest shard progress is incomplete.');
|
| 849 |
+
shard.state = 'verifying';
|
| 850 |
+
shard.loadedBytes = file.bytes;
|
| 851 |
+
shard.verifiedBytes = 0;
|
| 852 |
+
emitCacheProgress(index, 'verify');
|
| 853 |
+
const integrity = await verifyBlobSha256(blob, file.sha256, signal, (loadedBytes) => {
|
| 854 |
+
shard.verifiedBytes = Math.min(loadedBytes, shard.totalBytes);
|
| 855 |
+
emitCacheProgress(index, 'verify');
|
| 856 |
+
});
|
| 857 |
if (integrity.matches) {
|
| 858 |
blobs.push(blob);
|
| 859 |
cachedBytes += file.bytes;
|
| 860 |
+
shard.loadedBytes = file.bytes;
|
| 861 |
+
shard.verifiedBytes = file.bytes;
|
| 862 |
+
shard.state = 'cached';
|
| 863 |
+
emitCacheProgress(index, 'cache');
|
| 864 |
continue;
|
| 865 |
}
|
| 866 |
}
|
|
|
|
| 868 |
await wllama.cacheManager.delete(url);
|
| 869 |
}
|
| 870 |
blobs.push(null);
|
| 871 |
+
const shard = shards[index];
|
| 872 |
+
if (shard) {
|
| 873 |
+
shard.loadedBytes = 0;
|
| 874 |
+
shard.verifiedBytes = 0;
|
| 875 |
+
shard.state = 'queued';
|
| 876 |
+
emitCacheProgress(index, 'cache');
|
| 877 |
+
}
|
| 878 |
}
|
| 879 |
+
return { blobs, cachedBytes, shards };
|
| 880 |
}
|
| 881 |
|
| 882 |
private async downloadShards(
|
|
|
|
| 894 |
else signal.addEventListener('abort', abortDownloads, { once: true });
|
| 895 |
const downloadSignal = downloadController.signal;
|
| 896 |
const loadedByShard = model.files.map((file, index) => cached.blobs[index] ? file.bytes : 0);
|
| 897 |
+
const shards = cached.shards ?? initialShardProgress(model, cached.blobs);
|
| 898 |
+
const emitDownloadProgress = (shardIndex: number, phase: 'download' | 'verify' = 'download'): void => {
|
| 899 |
const file = model.files[shardIndex] ?? null;
|
| 900 |
emitProgress(sink, requestId, {
|
| 901 |
+
phase,
|
| 902 |
loadedBytes: loadedByShard.reduce((sum, value) => sum + value, 0),
|
| 903 |
totalBytes: model.downloadBytes,
|
| 904 |
shardIndex,
|
| 905 |
shardCount: model.files.length,
|
| 906 |
shardPath: file?.path ?? null,
|
| 907 |
+
stageProgress: phase === 'verify'
|
| 908 |
+
? totalVerifiedShardProgress(shards) / model.downloadBytes
|
| 909 |
+
: totalShardProgress(shards) / model.downloadBytes,
|
| 910 |
+
residentBytes: null,
|
| 911 |
+
shards: copyShardProgress(shards),
|
| 912 |
});
|
| 913 |
};
|
| 914 |
emitDownloadProgress(0);
|
|
|
|
| 931 |
if (!file || !url) {
|
| 932 |
throw new EngineRuntimeError('INVALID_SHARD_ORDER', 'Manifest shard URL and file counts differ.');
|
| 933 |
}
|
| 934 |
+
const shard = shards[index];
|
| 935 |
+
if (!shard) throw new EngineRuntimeError('INVALID_SHARD_ORDER', 'Manifest shard progress is incomplete.');
|
| 936 |
try {
|
| 937 |
+
shard.state = 'downloading';
|
| 938 |
+
shard.loadedBytes = 0;
|
| 939 |
+
shard.verifiedBytes = 0;
|
| 940 |
+
emitDownloadProgress(index);
|
| 941 |
await wllama.cacheManager.download(url, {
|
| 942 |
signal: downloadSignal,
|
| 943 |
metadataAdditional: {
|
|
|
|
| 947 |
},
|
| 948 |
progressCallback: ({ loaded }) => {
|
| 949 |
loadedByShard[index] = Math.min(loaded, file.bytes);
|
| 950 |
+
shard.loadedBytes = loadedByShard[index] ?? 0;
|
| 951 |
emitDownloadProgress(index);
|
| 952 |
},
|
| 953 |
});
|
|
|
|
| 959 |
`Downloaded shard ${file.path} has ${blob?.size ?? 0} bytes; expected ${file.bytes}.`,
|
| 960 |
);
|
| 961 |
}
|
| 962 |
+
shard.state = 'verifying';
|
| 963 |
+
shard.loadedBytes = file.bytes;
|
| 964 |
+
shard.verifiedBytes = 0;
|
| 965 |
+
loadedByShard[index] = file.bytes;
|
| 966 |
+
emitDownloadProgress(index, 'verify');
|
| 967 |
+
const integrity = await verifyBlobSha256(blob, file.sha256, downloadSignal, (loadedBytes) => {
|
| 968 |
+
shard.verifiedBytes = Math.min(loadedBytes, file.bytes);
|
| 969 |
+
emitDownloadProgress(index, 'verify');
|
| 970 |
+
});
|
| 971 |
if (!integrity.matches) {
|
| 972 |
throw new EngineRuntimeError(
|
| 973 |
'SHARD_HASH_MISMATCH',
|
|
|
|
| 983 |
});
|
| 984 |
cached.blobs[index] = blob;
|
| 985 |
loadedByShard[index] = file.bytes;
|
| 986 |
+
shard.loadedBytes = file.bytes;
|
| 987 |
+
shard.verifiedBytes = file.bytes;
|
| 988 |
+
shard.state = 'complete';
|
| 989 |
emitDownloadProgress(index);
|
| 990 |
} catch (error) {
|
| 991 |
const failure = classifyShardFailure(error, downloadSignal);
|
|
|
|
| 994 |
if (isPrimaryFailure) firstFailureReserved = true;
|
| 995 |
cached.blobs[index] = null;
|
| 996 |
loadedByShard[index] = 0;
|
| 997 |
+
shard.loadedBytes = 0;
|
| 998 |
+
shard.verifiedBytes = 0;
|
| 999 |
+
shard.state = 'error';
|
| 1000 |
let partialDeleted = false;
|
| 1001 |
let cleanupError: unknown;
|
| 1002 |
try {
|
|
|
|
| 1005 |
} catch (deleteError) {
|
| 1006 |
cleanupError = deleteError;
|
| 1007 |
}
|
| 1008 |
+
emitDownloadProgress(index, failure === 'verification' ? 'verify' : 'download');
|
| 1009 |
const normalizedFailure = shardFailure(
|
| 1010 |
error,
|
| 1011 |
failure,
|
|
|
|
| 1064 |
shardIndex: null,
|
| 1065 |
shardCount: 0,
|
| 1066 |
shardPath: null,
|
| 1067 |
+
stageProgress: 0,
|
| 1068 |
+
residentBytes: null,
|
| 1069 |
+
shards: [],
|
| 1070 |
});
|
| 1071 |
const manifest = params.manifestUrl !== undefined
|
| 1072 |
? await loadModelManifestV2(params.manifestUrl, signal)
|
|
|
|
| 1079 |
shardIndex: null,
|
| 1080 |
shardCount: 0,
|
| 1081 |
shardPath: null,
|
| 1082 |
+
stageProgress: 1,
|
| 1083 |
+
residentBytes: null,
|
| 1084 |
+
shards: [],
|
| 1085 |
});
|
| 1086 |
const model = findManifestModel(manifest, params.modelId);
|
| 1087 |
const contextSize = params.contextSize ?? model.defaultContext;
|
|
|
|
| 1099 |
}
|
| 1100 |
const urls = orderedShardUrls(manifest, model);
|
| 1101 |
const wllama = await this.ensureWllama(wasmFlavor);
|
| 1102 |
+
const cached = await this.inspectCachedShards(requestId, wllama, model, urls, signal, sink);
|
| 1103 |
const [adapter, storage] = await Promise.all([
|
| 1104 |
inspectWebGpuAdapter(),
|
| 1105 |
estimateStorage(),
|
|
|
|
| 1119 |
throwIfAborted(signal);
|
| 1120 |
emitProgress(sink, requestId, {
|
| 1121 |
phase: 'load',
|
| 1122 |
+
loadedBytes: model.downloadBytes,
|
| 1123 |
totalBytes: model.downloadBytes,
|
| 1124 |
shardIndex: null,
|
| 1125 |
shardCount: model.files.length,
|
| 1126 |
shardPath: null,
|
| 1127 |
+
stageProgress: 0,
|
| 1128 |
+
residentBytes: null,
|
| 1129 |
+
shards: copyShardProgress(cached.shards ?? initialShardProgress(model, blobs)),
|
| 1130 |
});
|
| 1131 |
const defaultThreads = Math.max(1, Math.floor((navigator.hardwareConcurrency || 1) / 2));
|
| 1132 |
const threads = params.threads ?? (gate.selectedBackend === 'wasm' ? defaultThreads : 1);
|
| 1133 |
if (!Number.isSafeInteger(threads) || threads <= 0) {
|
| 1134 |
throw new EngineRuntimeError('INVALID_THREAD_COUNT', 'Thread count must be a positive integer.');
|
| 1135 |
}
|
| 1136 |
+
const loadShards = copyShardProgress(cached.shards ?? initialShardProgress(model, blobs));
|
| 1137 |
+
const runtimeBatch = resolveRuntimeBatchShape(tuning, wasmFlavor);
|
| 1138 |
+
const stopLoadProgress = this.nativeLog.onModelLoadProgress((progress) => {
|
| 1139 |
+
const liveReport = this.nativeLog.report();
|
| 1140 |
+
emitProgress(sink, requestId, {
|
| 1141 |
+
phase: 'load',
|
| 1142 |
+
loadedBytes: model.downloadBytes,
|
| 1143 |
+
totalBytes: model.downloadBytes,
|
| 1144 |
+
shardIndex: null,
|
| 1145 |
+
shardCount: model.files.length,
|
| 1146 |
+
shardPath: null,
|
| 1147 |
+
stageProgress: progress.value,
|
| 1148 |
+
nativeStage: progress.current,
|
| 1149 |
+
residentBytes: liveReport.allocatedBufferBytes ?? null,
|
| 1150 |
+
shards: copyShardProgress(loadShards),
|
| 1151 |
+
});
|
| 1152 |
+
});
|
| 1153 |
try {
|
| 1154 |
await this.raceWebGpuDeviceLoss('load', gate.selectedBackend, model, wllama.loadModel(blobs, {
|
| 1155 |
n_gpu_layers: gate.selectedBackend === 'webgpu' ? 99999 : 0,
|
| 1156 |
offload_token_embedding: gate.selectedBackend === 'webgpu',
|
| 1157 |
n_ctx: contextSize,
|
| 1158 |
n_threads: threads,
|
| 1159 |
+
n_batch: runtimeBatch.nBatch,
|
| 1160 |
+
n_ubatch: runtimeBatch.nUbatch,
|
| 1161 |
seed: 42,
|
| 1162 |
flash_attn: tuning.flashMode === 'auto',
|
| 1163 |
warmup: false,
|
|
|
|
| 1243 |
shardIndex: null,
|
| 1244 |
shardCount: model.files.length,
|
| 1245 |
shardPath: null,
|
| 1246 |
+
stageProgress: 1,
|
| 1247 |
+
residentBytes: report.allocatedBufferBytes ?? null,
|
| 1248 |
+
shards: copyShardProgress(loadShards),
|
| 1249 |
});
|
| 1250 |
return {
|
| 1251 |
modelId: model.id,
|
|
|
|
| 1277 |
this.wllamaFlavor = null;
|
| 1278 |
this.loaded = null;
|
| 1279 |
throw error;
|
| 1280 |
+
} finally {
|
| 1281 |
+
stopLoadProgress();
|
| 1282 |
}
|
| 1283 |
}
|
| 1284 |
|
|
|
|
| 1303 |
let finishReason: GenerateResult['finishReason'] = null;
|
| 1304 |
let usage: GenerateResult['usage'] = null;
|
| 1305 |
let timings: GenerateResult['timings'] = null;
|
| 1306 |
+
let livePromptTokensPerSecond = 0;
|
| 1307 |
+
let liveDecodeTokensPerSecond = 0;
|
| 1308 |
+
let livePromptProcessed = 0;
|
| 1309 |
+
let livePromptTotal = 0;
|
| 1310 |
+
let livePromptCached = 0;
|
| 1311 |
+
let fallbackCompletionTokens = 0;
|
| 1312 |
+
let fallbackDecodeStartedAt: number | null = null;
|
| 1313 |
+
const decodeRateSamples: Array<{ tokens: number; elapsedMs: number }> = [];
|
| 1314 |
+
const usePerTokenTimings = this.wllamaFlavor !== 'compat';
|
| 1315 |
+
const sampleDecodeRate = (tokens: number, elapsedMs: number): void => {
|
| 1316 |
+
const lastSample = decodeRateSamples.at(-1);
|
| 1317 |
+
if (
|
| 1318 |
+
tokens <= 0
|
| 1319 |
+
|| !Number.isFinite(elapsedMs)
|
| 1320 |
+
|| (lastSample && (tokens <= lastSample.tokens || elapsedMs < lastSample.elapsedMs))
|
| 1321 |
+
) {
|
| 1322 |
+
return;
|
| 1323 |
+
}
|
| 1324 |
+
decodeRateSamples.push({ tokens, elapsedMs });
|
| 1325 |
+
while (decodeRateSamples.length > 2) {
|
| 1326 |
+
const first = decodeRateSamples[0];
|
| 1327 |
+
if (!first) break;
|
| 1328 |
+
const tokenSpan = tokens - first.tokens;
|
| 1329 |
+
const timeSpan = elapsedMs - first.elapsedMs;
|
| 1330 |
+
if (tokenSpan <= 12 && timeSpan <= 2_000) break;
|
| 1331 |
+
decodeRateSamples.shift();
|
| 1332 |
+
}
|
| 1333 |
+
const first = decodeRateSamples[0];
|
| 1334 |
+
if (!first) return;
|
| 1335 |
+
const tokenSpan = tokens - first.tokens;
|
| 1336 |
+
const timeSpan = elapsedMs - first.elapsedMs;
|
| 1337 |
+
if (tokenSpan <= 0 || timeSpan <= 0) return;
|
| 1338 |
+
const rate = tokenSpan * 1_000 / timeSpan;
|
| 1339 |
+
if (Number.isFinite(rate) && rate >= 0) {
|
| 1340 |
+
liveDecodeTokensPerSecond = rate;
|
| 1341 |
+
}
|
| 1342 |
+
};
|
| 1343 |
const tokenTrace: EngineSampledTokenTraceEntry[] | null = params.returnTokenIds === true ? [] : null;
|
| 1344 |
const streamedToolCalls = new ToolCallAccumulator();
|
| 1345 |
+
emitGenerationProgress(sink, requestId, {
|
| 1346 |
+
phase: 'prefill',
|
| 1347 |
+
promptProcessed: 0,
|
| 1348 |
+
promptTotal: 0,
|
| 1349 |
+
promptCached: 0,
|
| 1350 |
+
completionTokens: 0,
|
| 1351 |
+
elapsedMs: 0,
|
| 1352 |
+
promptTokensPerSecond: 0,
|
| 1353 |
+
decodeTokensPerSecond: 0,
|
| 1354 |
+
});
|
| 1355 |
try {
|
| 1356 |
await this.raceWebGpuDeviceLoss('generate', loaded.backend, loaded.model, wllama.createChatCompletion({
|
| 1357 |
messages: params.messages as ChatCompletionMessage[],
|
|
|
|
| 1369 |
? { logprobs: true, top_logprobs: TOKEN_TRACE_TOP_LOGPROBS }
|
| 1370 |
: {}),
|
| 1371 |
abortSignal: signal,
|
| 1372 |
+
...(usePerTokenTimings ? { timings_per_token: true, return_progress: true } : {}),
|
| 1373 |
onData: (chunk: ChatCompletionChunk) => {
|
| 1374 |
+
const liveChunk = chunk as LiveCompletionChunk;
|
|
|
|
| 1375 |
const delta = chunk.choices[0]?.delta.content;
|
| 1376 |
const textDelta = typeof delta === 'string' ? delta : '';
|
| 1377 |
const reasoningDelta = asReasoningDelta(chunk);
|
| 1378 |
+
const progress = liveChunk.prompt_progress;
|
| 1379 |
+
if (progress) {
|
| 1380 |
+
const processed = Math.max(0, progress.processed);
|
| 1381 |
+
const cached = Math.max(0, Math.min(processed, progress.cache));
|
| 1382 |
+
const elapsedMs = Math.max(0, progress.time_ms);
|
| 1383 |
+
const processedNow = Math.max(0, processed - cached);
|
| 1384 |
+
livePromptProcessed = processed;
|
| 1385 |
+
livePromptTotal = Math.max(processed, progress.total);
|
| 1386 |
+
livePromptCached = cached;
|
| 1387 |
+
const promptRate = elapsedMs >= MIN_LIVE_RATE_WINDOW_MS
|
| 1388 |
+
? processedNow * 1000 / elapsedMs
|
| 1389 |
+
: 0;
|
| 1390 |
+
if (Number.isFinite(promptRate) && promptRate >= 0) {
|
| 1391 |
+
livePromptTokensPerSecond = promptRate;
|
| 1392 |
+
}
|
| 1393 |
+
emitGenerationProgress(sink, requestId, {
|
| 1394 |
+
phase: 'prefill',
|
| 1395 |
+
promptProcessed: processed,
|
| 1396 |
+
promptTotal: livePromptTotal,
|
| 1397 |
+
promptCached: cached,
|
| 1398 |
+
completionTokens: 0,
|
| 1399 |
+
elapsedMs,
|
| 1400 |
+
promptTokensPerSecond: livePromptTokensPerSecond,
|
| 1401 |
+
decodeTokensPerSecond: 0,
|
| 1402 |
+
});
|
| 1403 |
+
} else if (usePerTokenTimings && chunk.timings) {
|
| 1404 |
+
const predictedTokens = Math.max(0, chunk.timings.predicted_n);
|
| 1405 |
+
const predictedMs = Math.max(0, chunk.timings.predicted_ms);
|
| 1406 |
+
sampleDecodeRate(predictedTokens, predictedMs);
|
| 1407 |
+
const finalPromptRate = Math.max(0, chunk.timings.prompt_per_second);
|
| 1408 |
+
if (Number.isFinite(finalPromptRate)) {
|
| 1409 |
+
livePromptTokensPerSecond = finalPromptRate;
|
| 1410 |
+
}
|
| 1411 |
+
emitGenerationProgress(sink, requestId, {
|
| 1412 |
+
phase: 'decode',
|
| 1413 |
+
promptProcessed: Math.max(0, chunk.timings.prompt_n),
|
| 1414 |
+
promptTotal: Math.max(0, chunk.timings.prompt_n),
|
| 1415 |
+
promptCached: Math.max(0, chunk.timings.cache_n),
|
| 1416 |
+
completionTokens: predictedTokens,
|
| 1417 |
+
elapsedMs: predictedMs,
|
| 1418 |
+
promptTokensPerSecond: livePromptTokensPerSecond,
|
| 1419 |
+
decodeTokensPerSecond: liveDecodeTokensPerSecond,
|
| 1420 |
+
});
|
| 1421 |
+
} else if (!usePerTokenTimings && (textDelta || reasoningDelta)) {
|
| 1422 |
+
fallbackCompletionTokens += 1;
|
| 1423 |
+
const now = performance.now();
|
| 1424 |
+
fallbackDecodeStartedAt ??= now;
|
| 1425 |
+
const elapsedMs = now - fallbackDecodeStartedAt;
|
| 1426 |
+
sampleDecodeRate(fallbackCompletionTokens, elapsedMs);
|
| 1427 |
+
emitGenerationProgress(sink, requestId, {
|
| 1428 |
+
phase: 'decode',
|
| 1429 |
+
promptProcessed: livePromptProcessed,
|
| 1430 |
+
promptTotal: livePromptTotal,
|
| 1431 |
+
promptCached: livePromptCached,
|
| 1432 |
+
completionTokens: fallbackCompletionTokens,
|
| 1433 |
+
elapsedMs,
|
| 1434 |
+
promptTokensPerSecond: livePromptTokensPerSecond,
|
| 1435 |
+
decodeTokensPerSecond: liveDecodeTokensPerSecond,
|
| 1436 |
+
});
|
| 1437 |
+
}
|
| 1438 |
+
if (tokenTrace !== null) appendSampledTokenTrace(tokenTrace, chunk);
|
| 1439 |
+
streamedToolCalls.append(chunk.choices[0]?.delta.tool_calls);
|
| 1440 |
if (textDelta || reasoningDelta) {
|
| 1441 |
text += textDelta;
|
| 1442 |
reasoningText += reasoningDelta ?? '';
|
src/lib/contracts.ts
CHANGED
|
@@ -19,7 +19,6 @@ export interface ModelOption {
|
|
| 19 |
capability: string;
|
| 20 |
contextLimit: number;
|
| 21 |
defaultContext: number;
|
| 22 |
-
dflash: 'unsupported' | 'conditional';
|
| 23 |
limitReason?: string;
|
| 24 |
}
|
| 25 |
|
|
@@ -34,6 +33,7 @@ export interface ToolRun {
|
|
| 34 |
export interface MessageMetrics {
|
| 35 |
tokens: number;
|
| 36 |
tokensPerSecond: number;
|
|
|
|
| 37 |
timeToFirstTokenMs: number;
|
| 38 |
}
|
| 39 |
|
|
@@ -59,6 +59,12 @@ export interface TelemetrySnapshot {
|
|
| 59 |
contextUsed: number;
|
| 60 |
contextLimit: number;
|
| 61 |
tokensPerSecond: number;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
timeToFirstTokenMs: number;
|
| 63 |
gpuLayers: string;
|
| 64 |
graphSplits: number | null;
|
|
@@ -95,12 +101,25 @@ export interface StorageStatus {
|
|
| 95 |
downloads: DownloadItem[];
|
| 96 |
}
|
| 97 |
|
| 98 |
-
export interface
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
}
|
| 105 |
|
| 106 |
export interface RuntimeSettings {
|
|
@@ -122,10 +141,10 @@ export interface BonsaiShellProps {
|
|
| 122 |
loadedModelId: ModelId | null;
|
| 123 |
messages: ChatMessage[];
|
| 124 |
streamState: StreamState;
|
|
|
|
| 125 |
telemetry: TelemetrySnapshot;
|
| 126 |
toolEvents: ToolEvent[];
|
| 127 |
storage: StorageStatus;
|
| 128 |
-
dflash: DFlashState;
|
| 129 |
artifact: ArtifactDocument | null;
|
| 130 |
toolsEnabled: boolean;
|
| 131 |
settings: RuntimeSettings;
|
|
@@ -139,7 +158,6 @@ export interface BonsaiShellProps {
|
|
| 139 |
onRegenerate: () => void;
|
| 140 |
onCancel: () => void;
|
| 141 |
onToolsEnabledChange: (enabled: boolean) => void;
|
| 142 |
-
onDFlashChange: (enabled: boolean) => void;
|
| 143 |
onOpenSettings: () => void;
|
| 144 |
onCloseSettings: () => void;
|
| 145 |
onSettingsChange: (settings: RuntimeSettings) => void;
|
|
@@ -149,5 +167,4 @@ export interface BonsaiShellProps {
|
|
| 149 |
onDismissError: () => void;
|
| 150 |
onRetryShard: (modelId: ModelId) => void;
|
| 151 |
onCloseArtifact: () => void;
|
| 152 |
-
onRequestDownload: (modelId: ModelId) => void;
|
| 153 |
}
|
|
|
|
| 19 |
capability: string;
|
| 20 |
contextLimit: number;
|
| 21 |
defaultContext: number;
|
|
|
|
| 22 |
limitReason?: string;
|
| 23 |
}
|
| 24 |
|
|
|
|
| 33 |
export interface MessageMetrics {
|
| 34 |
tokens: number;
|
| 35 |
tokensPerSecond: number;
|
| 36 |
+
promptTokensPerSecond?: number;
|
| 37 |
timeToFirstTokenMs: number;
|
| 38 |
}
|
| 39 |
|
|
|
|
| 59 |
contextUsed: number;
|
| 60 |
contextLimit: number;
|
| 61 |
tokensPerSecond: number;
|
| 62 |
+
prefillTokensPerSecond: number;
|
| 63 |
+
decodeTokensPerSecond: number;
|
| 64 |
+
inferencePhase: 'idle' | 'prefill' | 'decode' | 'complete';
|
| 65 |
+
promptProcessed: number;
|
| 66 |
+
promptTotal: number;
|
| 67 |
+
completionTokens: number;
|
| 68 |
timeToFirstTokenMs: number;
|
| 69 |
gpuLayers: string;
|
| 70 |
graphSplits: number | null;
|
|
|
|
| 101 |
downloads: DownloadItem[];
|
| 102 |
}
|
| 103 |
|
| 104 |
+
export interface ModelShardProgress {
|
| 105 |
+
index: number;
|
| 106 |
+
path: string;
|
| 107 |
+
loadedBytes: number;
|
| 108 |
+
verifiedBytes: number;
|
| 109 |
+
totalBytes: number;
|
| 110 |
+
state: 'queued' | 'cached' | 'downloading' | 'verifying' | 'complete' | 'error';
|
| 111 |
+
}
|
| 112 |
+
|
| 113 |
+
export interface ModelLoadProgress {
|
| 114 |
+
modelId: ModelId | null;
|
| 115 |
+
modelLabel: string;
|
| 116 |
+
stage: 'idle' | 'manifest' | 'cache' | 'download' | 'verify' | 'load' | 'ready' | 'error';
|
| 117 |
+
stageProgress: number;
|
| 118 |
+
downloadedBytes: number;
|
| 119 |
+
totalBytes: number;
|
| 120 |
+
residentBytes: number;
|
| 121 |
+
nativeStage: string | null;
|
| 122 |
+
shards: ModelShardProgress[];
|
| 123 |
}
|
| 124 |
|
| 125 |
export interface RuntimeSettings {
|
|
|
|
| 141 |
loadedModelId: ModelId | null;
|
| 142 |
messages: ChatMessage[];
|
| 143 |
streamState: StreamState;
|
| 144 |
+
modelLoadProgress: ModelLoadProgress;
|
| 145 |
telemetry: TelemetrySnapshot;
|
| 146 |
toolEvents: ToolEvent[];
|
| 147 |
storage: StorageStatus;
|
|
|
|
| 148 |
artifact: ArtifactDocument | null;
|
| 149 |
toolsEnabled: boolean;
|
| 150 |
settings: RuntimeSettings;
|
|
|
|
| 158 |
onRegenerate: () => void;
|
| 159 |
onCancel: () => void;
|
| 160 |
onToolsEnabledChange: (enabled: boolean) => void;
|
|
|
|
| 161 |
onOpenSettings: () => void;
|
| 162 |
onCloseSettings: () => void;
|
| 163 |
onSettingsChange: (settings: RuntimeSettings) => void;
|
|
|
|
| 167 |
onDismissError: () => void;
|
| 168 |
onRetryShard: (modelId: ModelId) => void;
|
| 169 |
onCloseArtifact: () => void;
|
|
|
|
| 170 |
}
|
src/vite-env.d.ts
CHANGED
|
@@ -1 +1,7 @@
|
|
| 1 |
/// <reference types="vite/client" />
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
/// <reference types="vite/client" />
|
| 2 |
+
|
| 3 |
+
import type { BackendReport } from './engine/native-log';
|
| 4 |
+
|
| 5 |
+
declare global {
|
| 6 |
+
var __bonsaiBackendReport: BackendReport | undefined;
|
| 7 |
+
}
|
vendor/wllama-bonsai/PATCH.diff
CHANGED
|
@@ -23,7 +23,7 @@ index f1496a4..8883324 100644
|
|
| 23 |
GLUE_FIELD(int, n_threads)
|
| 24 |
GLUE_FIELD_NULLABLE(str, model_alias)
|
| 25 |
diff --git a/cpp/wllama-context.h b/cpp/wllama-context.h
|
| 26 |
-
index 0dafe6d..
|
| 27 |
--- a/cpp/wllama-context.h
|
| 28 |
+++ b/cpp/wllama-context.h
|
| 29 |
@@ -19,6 +19,7 @@
|
|
@@ -34,7 +34,15 @@ index 0dafe6d..d261dfe 100644
|
|
| 34 |
|
| 35 |
#include "ggml-cpu.h"
|
| 36 |
#include "ggml-backend.h"
|
| 37 |
-
@@ -
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
server_task task = server_task(SERVER_TASK_TYPE_COMPLETION);
|
| 39 |
task.id = rd->get_new_id();
|
| 40 |
task.index = 0;
|
|
@@ -67,7 +75,7 @@ index 0dafe6d..d261dfe 100644
|
|
| 67 |
|
| 68 |
rd->post_task({std::move(task)});
|
| 69 |
}
|
| 70 |
-
@@ -415,6 +
|
| 71 |
params.embedding = req.embeddings.value;
|
| 72 |
if (req.n_batch.not_null())
|
| 73 |
params.n_batch = req.n_batch.value;
|
|
@@ -76,10 +84,16 @@ index 0dafe6d..d261dfe 100644
|
|
| 76 |
if (req.n_parallel.not_null())
|
| 77 |
params.n_parallel = req.n_parallel.value;
|
| 78 |
if (req.pooling_type.not_null())
|
| 79 |
-
@@ -568,6 +
|
| 80 |
// load model
|
| 81 |
llama_backend_init();
|
| 82 |
llama_numa_init(params.numa);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
+ if (req.offload_token_embedding.not_null() && req.offload_token_embedding.value)
|
| 84 |
+ {
|
| 85 |
+ auto *webgpu_dev = ggml_backend_dev_by_name("WebGPU");
|
|
@@ -126,10 +140,18 @@ index 08f0d87..c1001da 100644
|
|
| 126 |
n_threads: number;
|
| 127 |
model_alias?: string | undefined;
|
| 128 |
diff --git a/src/types/oai-compat.ts b/src/types/oai-compat.ts
|
| 129 |
-
index dbeeaa2..
|
| 130 |
--- a/src/types/oai-compat.ts
|
| 131 |
+++ b/src/types/oai-compat.ts
|
| 132 |
-
@@ -
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 133 |
// Response types----------
|
| 134 |
|
| 135 |
export interface ChatCompletionLogprob {
|
|
@@ -137,6 +159,28 @@ index dbeeaa2..fd41519 100644
|
|
| 137 |
token: string;
|
| 138 |
logprob: number;
|
| 139 |
bytes: number[] | null;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 140 |
diff --git a/src/types/types.ts b/src/types/types.ts
|
| 141 |
index 2b3cca7..e91b395 100644
|
| 142 |
--- a/src/types/types.ts
|
|
@@ -285,3 +329,26 @@ index be62030..9f5144a 100644
|
|
| 285 |
// length of output buffer is written at the first 4 bytes of input buffer
|
| 286 |
const outputLen = new Uint32Array(
|
| 287 |
getHeapU8().buffer,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
GLUE_FIELD(int, n_threads)
|
| 24 |
GLUE_FIELD_NULLABLE(str, model_alias)
|
| 25 |
diff --git a/cpp/wllama-context.h b/cpp/wllama-context.h
|
| 26 |
+
index 0dafe6d..4782268 100644
|
| 27 |
--- a/cpp/wllama-context.h
|
| 28 |
+++ b/cpp/wllama-context.h
|
| 29 |
@@ -19,6 +19,7 @@
|
|
|
|
| 34 |
|
| 35 |
#include "ggml-cpu.h"
|
| 36 |
#include "ggml-backend.h"
|
| 37 |
+
@@ -263,7 +264,6 @@ struct wllama_context
|
| 38 |
+
// using unique_ptr to allow late initialization
|
| 39 |
+
std::unique_ptr<server_response_reader> rd;
|
| 40 |
+
std::unique_ptr<const server_context_meta> meta;
|
| 41 |
+
-
|
| 42 |
+
struct console
|
| 43 |
+
{
|
| 44 |
+
struct spinner
|
| 45 |
+
@@ -300,16 +300,28 @@ struct wllama_context
|
| 46 |
server_task task = server_task(SERVER_TASK_TYPE_COMPLETION);
|
| 47 |
task.id = rd->get_new_id();
|
| 48 |
task.index = 0;
|
|
|
|
| 75 |
|
| 76 |
rd->post_task({std::move(task)});
|
| 77 |
}
|
| 78 |
+
@@ -415,6 +427,8 @@ struct wllama_context
|
| 79 |
params.embedding = req.embeddings.value;
|
| 80 |
if (req.n_batch.not_null())
|
| 81 |
params.n_batch = req.n_batch.value;
|
|
|
|
| 84 |
if (req.n_parallel.not_null())
|
| 85 |
params.n_parallel = req.n_parallel.value;
|
| 86 |
if (req.pooling_type.not_null())
|
| 87 |
+
@@ -568,6 +582,30 @@ struct wllama_context
|
| 88 |
// load model
|
| 89 |
llama_backend_init();
|
| 90 |
llama_numa_init(params.numa);
|
| 91 |
+
+ ctx_server.set_state_callback([](server_state state, json payload)
|
| 92 |
+
+ {
|
| 93 |
+
+ payload["state"] = server_state_to_str(state);
|
| 94 |
+
+ const std::string serialized = payload.dump();
|
| 95 |
+
+ fprintf(stderr, "@@MODEL_LOAD_PROGRESS@@%s\n", serialized.c_str());
|
| 96 |
+
+ });
|
| 97 |
+ if (req.offload_token_embedding.not_null() && req.offload_token_embedding.value)
|
| 98 |
+ {
|
| 99 |
+ auto *webgpu_dev = ggml_backend_dev_by_name("WebGPU");
|
|
|
|
| 140 |
n_threads: number;
|
| 141 |
model_alias?: string | undefined;
|
| 142 |
diff --git a/src/types/oai-compat.ts b/src/types/oai-compat.ts
|
| 143 |
+
index dbeeaa2..e2f960e 100644
|
| 144 |
--- a/src/types/oai-compat.ts
|
| 145 |
+++ b/src/types/oai-compat.ts
|
| 146 |
+
@@ -128,6 +128,7 @@ export type ChatCompletionParams = {
|
| 147 |
+
// llama-server-specific
|
| 148 |
+
chat_template_kwargs?: Record<string, any>;
|
| 149 |
+
cache_prompt?: boolean;
|
| 150 |
+
+ return_progress?: boolean;
|
| 151 |
+
return_tokens?: boolean;
|
| 152 |
+
timings_per_token?: boolean;
|
| 153 |
+
} & SamplingParams;
|
| 154 |
+
@@ -135,6 +136,7 @@ export type ChatCompletionParams = {
|
| 155 |
// Response types----------
|
| 156 |
|
| 157 |
export interface ChatCompletionLogprob {
|
|
|
|
| 159 |
token: string;
|
| 160 |
logprob: number;
|
| 161 |
bytes: number[] | null;
|
| 162 |
+
@@ -214,6 +216,13 @@ export interface ResultTimings {
|
| 163 |
+
predicted_per_second: number;
|
| 164 |
+
}
|
| 165 |
+
|
| 166 |
+
+export interface PromptProgress {
|
| 167 |
+
+ total: number;
|
| 168 |
+
+ cache: number;
|
| 169 |
+
+ processed: number;
|
| 170 |
+
+ time_ms: number;
|
| 171 |
+
+}
|
| 172 |
+
+
|
| 173 |
+
/** Response when stream=true — one chunk per SSE event */
|
| 174 |
+
export interface ChatCompletionChunk {
|
| 175 |
+
id: string;
|
| 176 |
+
@@ -223,6 +232,7 @@ export interface ChatCompletionChunk {
|
| 177 |
+
choices: ChatCompletionChunkChoice[];
|
| 178 |
+
usage?: ChatCompletionUsage | null;
|
| 179 |
+
timings?: ResultTimings;
|
| 180 |
+
+ prompt_progress?: PromptProgress;
|
| 181 |
+
}
|
| 182 |
+
|
| 183 |
+
// Raw (text) completion
|
| 184 |
diff --git a/src/types/types.ts b/src/types/types.ts
|
| 185 |
index 2b3cca7..e91b395 100644
|
| 186 |
--- a/src/types/types.ts
|
|
|
|
| 329 |
// length of output buffer is written at the first 4 bytes of input buffer
|
| 330 |
const outputLen = new Uint32Array(
|
| 331 |
getHeapU8().buffer,
|
| 332 |
+
diff --git a/llama.cpp/ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp b/llama.cpp/ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
|
| 333 |
+
index d769236..131b82c 100644
|
| 334 |
+
--- a/llama.cpp/ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
|
| 335 |
+
+++ b/llama.cpp/ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
|
| 336 |
+
@@ -50,6 +50,7 @@
|
| 337 |
+
|
| 338 |
+
// Matrix-vector multiplication parameters
|
| 339 |
+
#define WEBGPU_MUL_MAT_VEC_WG_SIZE 256
|
| 340 |
+
+#define WEBGPU_MUL_MAT_VEC_Q1_NO_SG_WG_SIZE 128
|
| 341 |
+
|
| 342 |
+
#define WEBGPU_MUL_MAT_VEC_FLOAT_OUTPUTS_PER_WG 4
|
| 343 |
+
#define WEBGPU_MUL_MAT_VEC_LEGACY_Q_OUTPUTS_PER_WG 4
|
| 344 |
+
@@ -1995,6 +1996,10 @@ class ggml_webgpu_shader_lib {
|
| 345 |
+
|
| 346 |
+
if (key.src0_type == GGML_TYPE_Q1_0) {
|
| 347 |
+
outputs_per_wg = WEBGPU_MUL_MAT_VEC_LEGACY_Q_OUTPUTS_PER_WG;
|
| 348 |
+
+ if (!context.supports_subgroups) {
|
| 349 |
+
+ wg_size = WEBGPU_MUL_MAT_VEC_Q1_NO_SG_WG_SIZE;
|
| 350 |
+
+ variant += "_wg128";
|
| 351 |
+
+ }
|
| 352 |
+
} else if (key.src0_type >= GGML_TYPE_Q2_K) {
|
| 353 |
+
outputs_per_wg = WEBGPU_MUL_MAT_VEC_K_Q_OUTPUTS_PER_WG;
|
| 354 |
+
} else if (key.src0_type >= GGML_TYPE_Q4_0) {
|
vendor/wllama-bonsai/SOURCE.json
CHANGED
|
@@ -4,16 +4,17 @@
|
|
| 4 |
"wllamaRevision": "912c18b75d4358c1405a64646b8dbe43a205943b",
|
| 5 |
"llamaCppRevision": "00fa7cb284cbf133fc426733bd64238a3588a33e",
|
| 6 |
"license": "MIT",
|
| 7 |
-
"patch": "Expose offload_token_embedding and n_ubatch; place token_embd.weight on WebGPU whenever GPU layers are requested; fail loudly when the requested WebGPU buffer is unavailable; add an opt-in forceCompat selector for controlled benchmark A/B runs; type sampled token ids in OAI logprob payloads; accept one raw token-id completion prompt
|
| 8 |
"patchSet": {
|
| 9 |
"format": "git-diff-binary",
|
| 10 |
"path": "vendor/wllama-bonsai/PATCH.diff",
|
| 11 |
-
"bytes":
|
| 12 |
-
"sha256": "
|
| 13 |
"files": [
|
| 14 |
"CMakeLists.txt",
|
| 15 |
"cpp/glue.hpp",
|
| 16 |
"cpp/wllama-context.h",
|
|
|
|
| 17 |
"src/glue/messages.ts",
|
| 18 |
"src/types/oai-compat.ts",
|
| 19 |
"src/types/types.ts",
|
|
@@ -30,18 +31,18 @@
|
|
| 30 |
"files": [
|
| 31 |
{
|
| 32 |
"path": "vendor/wllama-bonsai/esm/index.js",
|
| 33 |
-
"bytes":
|
| 34 |
-
"sha256": "
|
| 35 |
},
|
| 36 |
{
|
| 37 |
"path": "public/wasm/wllama.wasm",
|
| 38 |
-
"bytes":
|
| 39 |
-
"sha256": "
|
| 40 |
},
|
| 41 |
{
|
| 42 |
"path": "public/wasm/wllama-compat.wasm",
|
| 43 |
-
"bytes":
|
| 44 |
-
"sha256": "
|
| 45 |
},
|
| 46 |
{
|
| 47 |
"path": "public/wasm/wllama-compat.js",
|
|
|
|
| 4 |
"wllamaRevision": "912c18b75d4358c1405a64646b8dbe43a205943b",
|
| 5 |
"llamaCppRevision": "00fa7cb284cbf133fc426733bd64238a3588a33e",
|
| 6 |
"license": "MIT",
|
| 7 |
+
"patch": "Expose offload_token_embedding and n_ubatch; place token_embd.weight on WebGPU whenever GPU layers are requested; fail loudly when the requested WebGPU buffer is unavailable; add an opt-in forceCompat selector for controlled benchmark A/B runs; expose native model-load and prompt-prefill progress; type sampled token ids in OAI logprob payloads; accept one raw token-id completion prompt; fail loudly on null action responses; and cap non-subgroup Q1_0 matrix-vector workgroups at the validated 128 threads.",
|
| 8 |
"patchSet": {
|
| 9 |
"format": "git-diff-binary",
|
| 10 |
"path": "vendor/wllama-bonsai/PATCH.diff",
|
| 11 |
+
"bytes": 13467,
|
| 12 |
+
"sha256": "e94aef3f672211e72bab039385b3bcbeda46f0e3218db7a86a9b5468186ab491",
|
| 13 |
"files": [
|
| 14 |
"CMakeLists.txt",
|
| 15 |
"cpp/glue.hpp",
|
| 16 |
"cpp/wllama-context.h",
|
| 17 |
+
"llama.cpp/ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp",
|
| 18 |
"src/glue/messages.ts",
|
| 19 |
"src/types/oai-compat.ts",
|
| 20 |
"src/types/types.ts",
|
|
|
|
| 31 |
"files": [
|
| 32 |
{
|
| 33 |
"path": "vendor/wllama-bonsai/esm/index.js",
|
| 34 |
+
"bytes": 365544,
|
| 35 |
+
"sha256": "b7e883ebeed6aad1b4587d89c20cc4ee8e69e7a0e94e1d307dcad756d657a41a"
|
| 36 |
},
|
| 37 |
{
|
| 38 |
"path": "public/wasm/wllama.wasm",
|
| 39 |
+
"bytes": 8020809,
|
| 40 |
+
"sha256": "5a8686c10f7639653e7e1854d34f39c31ddba404de876a5ae15d36c42263ddc4"
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"path": "public/wasm/wllama-compat.wasm",
|
| 44 |
+
"bytes": 14871898,
|
| 45 |
+
"sha256": "c823684229149dd2386308978288e366f2ff209b231b78485507db8cb735bbdd"
|
| 46 |
},
|
| 47 |
{
|
| 48 |
"path": "public/wasm/wllama-compat.js",
|
vendor/wllama-bonsai/esm/index.cjs
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
vendor/wllama-bonsai/esm/index.js
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
vendor/wllama-bonsai/esm/index.min.js
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
vendor/wllama-bonsai/esm/index.min.js.map
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|