Spaces:
Running
Running
Release v1.1.2 benchmark runtime evidence
Browse filesAdd fail-loud Flash/KV benchmark controls, requested-versus-observed runtime reporting, controlled JSPI/compat selection, per-tier DFlash disposition, tests, provenance, and a prebuilt static bundle.
- README.md +21 -10
- dist/assets/index-AGrRq4Jz.js +0 -0
- dist/assets/{index-Do-82RUc.css → index-C-2tm91j.css} +1 -1
- dist/assets/index-jtL5gUVS.js +0 -0
- dist/assets/{worker-Dh0o2G2j.js → worker-gvajdY5k.js} +0 -0
- dist/index.html +2 -2
- scripts/verify-wllama-assets.mjs +7 -0
- src/app/styles.css +9 -3
- src/bench/BenchApp.tsx +131 -28
- src/bench/report.test.ts +208 -34
- src/bench/report.ts +21 -29
- src/engine/device-gate.test.ts +4 -0
- src/engine/native-log.test.ts +39 -0
- src/engine/native-log.ts +44 -1
- src/engine/protocol.ts +34 -8
- src/engine/runtime-device-lost.test.ts +4 -0
- src/engine/runtime-download.test.ts +160 -20
- src/engine/runtime.ts +180 -33
- vendor/wllama-bonsai/PATCH.diff +61 -5
- vendor/wllama-bonsai/SOURCE.json +5 -5
- vendor/wllama-bonsai/esm/index.cjs +15 -3
- vendor/wllama-bonsai/esm/index.js +15 -3
- vendor/wllama-bonsai/esm/index.min.js +0 -0
- vendor/wllama-bonsai/esm/index.min.js.map +0 -0
- vendor/wllama-bonsai/esm/wllama.d.ts +8 -1
README.md
CHANGED
|
@@ -74,12 +74,20 @@ prefill/decode throughput, graph placement, device limits, engine revisions,
|
|
| 74 |
the exact runtime artifact hash, and a fixed temperature-0 prompt. The report
|
| 75 |
exports as JSON; generated model text is deliberately excluded.
|
| 76 |
|
| 77 |
-
`n_batch`
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
|
| 84 |
The public prewarmed-cache baseline below is one Apple `metal-3` / headless
|
| 85 |
Chrome sample, not a cross-device study and not a first-download claim:
|
|
@@ -117,9 +125,11 @@ stay in the browser; only model/static asset downloads contact Hugging Face.
|
|
| 117 |
- Dawn native WebGPU validation: `18eb229ef5f707c1464cc581252e7603c73a3ef0`
|
| 118 |
- custom wllama source: `912c18b75d4358c1405a64646b8dbe43a205943b`
|
| 119 |
- custom nested llama.cpp: `00fa7cb284cbf133fc426733bd64238a3588a33e`
|
| 120 |
-
- custom patch set: `
|
|
|
|
| 121 |
- JSPI WASM: `dd71f58c75f32c677a64eeb2e56efab7dedbe7395f730bb08bc80021bd182caa`
|
| 122 |
- compat WASM: `eca9754d0d8a490b5c7b58cb384d67e31aa8ab26dd330e4ad1460982df6dd82e`
|
|
|
|
| 123 |
|
| 124 |
The vendored runtime exposes the validated token-embedding WebGPU placement
|
| 125 |
needed for single-graph Bonsai inference and fails closed when placement or the
|
|
@@ -128,9 +138,10 @@ backend tripwire disagrees. Its exact JS/WASM hashes are checked during
|
|
| 128 |
`vendor/wllama-bonsai/SOURCE.json`. Third-party notices are in
|
| 129 |
`THIRD_PARTY_NOTICES.md` and `public/licenses/`.
|
| 130 |
|
| 131 |
-
The pinned WebGPU source implements Flash Attention with Q4_0/Q8_0 K/V
|
| 132 |
-
|
| 133 |
-
|
|
|
|
| 134 |
|
| 135 |
## Licenses and attribution
|
| 136 |
|
|
|
|
| 74 |
the exact runtime artifact hash, and a fixed temperature-0 prompt. The report
|
| 75 |
exports as JSON; generated model text is deliberately excluded.
|
| 76 |
|
| 77 |
+
`n_batch`, `n_ubatch`, Flash Attention, paired K/V cache types, and the pinned
|
| 78 |
+
JSPI/compat WASM flavor are available there as explicitly experimental
|
| 79 |
+
controls. The browser report separates requested and native-observed values,
|
| 80 |
+
records the actually loaded artifact hashes, and fails the run when they
|
| 81 |
+
disagree. A pinned local 1.7B WebGPU sweep proved Flash off/auto with F16, Q8_0,
|
| 82 |
+
and Q4_0 KV: every row retained one graph, 29/29 GPU layers, zero CPU ops, and
|
| 83 |
+
the same short deterministic output hash. Observed KV allocation fell from
|
| 84 |
+
112 MiB for F16 to 59.5 MiB for Q8_0 and 31.5 MiB for Q4_0.
|
| 85 |
+
|
| 86 |
+
A same-Chrome A/B with one warmup and three alternating measured runs per arm
|
| 87 |
+
also proved explicit JSPI/compat switching. Median warm load was 2,257.7 ms for
|
| 88 |
+
JSPI and 2,799.1 ms for compat; all eight outputs and graph tripwires matched.
|
| 89 |
+
These are experimental single-device results, not release recommendations;
|
| 90 |
+
normal chat keeps Flash off, F16/F16, and automatic WASM selection.
|
| 91 |
|
| 92 |
The public prewarmed-cache baseline below is one Apple `metal-3` / headless
|
| 93 |
Chrome sample, not a cross-device study and not a first-download claim:
|
|
|
|
| 125 |
- Dawn native WebGPU validation: `18eb229ef5f707c1464cc581252e7603c73a3ef0`
|
| 126 |
- custom wllama source: `912c18b75d4358c1405a64646b8dbe43a205943b`
|
| 127 |
- custom nested llama.cpp: `00fa7cb284cbf133fc426733bd64238a3588a33e`
|
| 128 |
+
- custom patch set: `ad8786a295eaeb75ee03752d7f84d30f14ea373407844d5c589f7ab46800bb4b`
|
| 129 |
+
- custom ESM: `24bcef8aea8e27fb7b7e2d9e6ea94ba8ced7bfbffd09a0675821b9eb1b4a4c9f`
|
| 130 |
- JSPI WASM: `dd71f58c75f32c677a64eeb2e56efab7dedbe7395f730bb08bc80021bd182caa`
|
| 131 |
- compat WASM: `eca9754d0d8a490b5c7b58cb384d67e31aa8ab26dd330e4ad1460982df6dd82e`
|
| 132 |
+
- compat worker: `0b667db536815fb9e5dc9ffbfbd6ba29affdda38a4ba892d5372ddb41ce9a8c8`
|
| 133 |
|
| 134 |
The vendored runtime exposes the validated token-embedding WebGPU placement
|
| 135 |
needed for single-graph Bonsai inference and fails closed when placement or the
|
|
|
|
| 138 |
`vendor/wllama-bonsai/SOURCE.json`. Third-party notices are in
|
| 139 |
`THIRD_PARTY_NOTICES.md` and `public/licenses/`.
|
| 140 |
|
| 141 |
+
The pinned WebGPU source implements Flash Attention with Q4_0/Q8_0 K/V. The
|
| 142 |
+
benchmark lane has short deterministic-output, memory, and graph-placement
|
| 143 |
+
evidence for those combinations, but long-state, 27B, and cross-device gates
|
| 144 |
+
remain open. Flash Attention therefore remains off for the release path.
|
| 145 |
|
| 146 |
## Licenses and attribution
|
| 147 |
|
dist/assets/index-AGrRq4Jz.js
DELETED
|
The diff for this file is too large to render.
See raw diff
|
|
|
dist/assets/{index-Do-82RUc.css → index-C-2tm91j.css}
RENAMED
|
@@ -1 +1 @@
|
|
| 1 |
-
:root{--lightningcss-light: ;--lightningcss-dark:initial;color-scheme:dark;--ink:#111512;--ink-raised:#191e1a;--ink-soft:#252b25;--paper:#e8dfca;--paper-deep:#d8cdb4;--paper-ghost:#e8dfca9e;--phosphor:#b9ff45;--phosphor-dim:#b9ff4529;--moss:#6f8062;--rust:#c96f43;--line-light:#e8dfca24;--line-dark:#1115122e;--display:"Bodoni 72", Didot, "Iowan Old Style", Baskerville, serif;--body:"Avenir Next Condensed", "Gill Sans", "Trebuchet MS", sans-serif;--mono:"Berkeley Mono", Iosevka, "SFMono-Regular", Consolas, monospace;font-family:var(--body);font-synthesis:none;text-rendering:optimizelegibility}*{box-sizing:border-box}html{background:var(--ink)}body{background:linear-gradient(90deg, #1115120f 1px, transparent 1px) 0 0 / 22px 22px, linear-gradient(#1115120b 1px, transparent 1px) 0 0 / 22px 22px, var(--paper);min-width:320px;min-height:100vh;color:var(--paper);margin:0}button,textarea,select{font:inherit}button{color:inherit}button:focus-visible,textarea:focus-visible,select:focus-visible,summary:focus-visible{outline:2px solid var(--phosphor);outline-offset:3px}button:disabled{cursor:not-allowed}::selection{color:var(--ink);background:var(--phosphor)}.station-frame{isolation:isolate;min-height:100dvh;animation:.58s cubic-bezier(.16,1,.3,1) both station-in}.station-frame:before{z-index:20;pointer-events:none;content:"";opacity:.14;mix-blend-mode:soft-light;background-image:url("data:image/svg+xml,%3Csvg viewBox='0 0 180 180' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='3' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)' opacity='.3'/%3E%3C/svg%3E");position:fixed;inset:0}.station-header{background:var(--ink);border-bottom:1px solid var(--phosphor);z-index:10;grid-template-columns:minmax(330px,1fr) auto minmax(220px,1fr);align-items:stretch;min-height:76px;display:grid;position:relative;box-shadow:0 12px 38px #11151238}.brand-block{align-items:center;gap:14px;padding:12px 24px;display:flex}.brand-block h1{font:400 clamp(25px, 2vw, 34px) / .9 var(--display);letter-spacing:-.035em;margin:0}.brand-block h1 i{color:var(--phosphor);font-weight:400}.brand-mark{background:repeating-radial-gradient(at 52% 62%,#0000 0 4px,#b9ff456b 4.5px 5px);border:1px solid #b9ff457a;border-radius:50%;flex:0 0 42px;place-items:center;width:42px;height:42px;display:grid;transform:rotate(-11deg)}.brand-mark:before,.brand-mark:after,.brand-mark span:before,.brand-mark span:after{content:"";background:var(--phosphor);transform-origin:bottom;width:1px;position:absolute}.brand-mark:before{height:25px;transform:translate(-1px,4px)rotate(0)}.brand-mark:after{height:13px;transform:translate(4px,-2px)rotate(43deg)}.brand-mark span:before{height:11px;transform:translate(-5px,-1px)rotate(-47deg)}.brand-mark span:after{height:8px;transform:translate(3px,-9px)rotate(72deg)}.eyebrow{color:#e8dfca85;font:600 9px/1 var(--mono);letter-spacing:.18em;text-transform:uppercase;margin:0 0 5px}.eyebrow.ink{color:#11151285}.local-promise{border-inline:1px solid var(--line-light);background:linear-gradient(115deg,#b9ff4514,#0000 66%);align-items:center;gap:11px;min-width:360px;padding:0 28px;display:flex}.signal-dot,.live-label i,.sampling-label i{background:var(--phosphor);width:7px;height:7px;box-shadow:0 0 0 5px var(--phosphor-dim), 0 0 17px #b9ff459e;border-radius:50%;display:inline-block}.local-promise span:last-child{gap:3px;display:grid}.local-promise strong{color:var(--phosphor);font:600 10px/1 var(--mono);letter-spacing:.13em;text-transform:uppercase}.local-promise small{color:var(--paper-ghost);font:11px/1.2 var(--body)}.header-actions{justify-self:end;align-items:center;padding-right:18px;display:flex}.station-clock{color:var(--paper-ghost);font-family:var(--mono);justify-items:end;padding:0 18px;display:grid}.station-clock small{letter-spacing:.18em;font-size:8px}.station-clock span{letter-spacing:.09em;margin-top:4px;font-size:10px}.icon-button{cursor:pointer;background:0 0;border:1px solid #e8dfca3d;border-radius:50%;place-items:center;width:38px;height:38px;transition:color .16s,border-color .16s,transform .16s;display:grid}.icon-button:hover{color:var(--phosphor);border-color:var(--phosphor);transform:rotate(-7deg)}.icon-button svg{fill:none;stroke:currentColor;stroke-width:1.4px;width:18px}.workspace-grid{grid-template-columns:clamp(240px,18vw,286px) minmax(510px,1fr) clamp(310px,23vw,370px);grid-template-areas:"models chat inspector";min-height:calc(100dvh - 76px);display:grid}.model-rail,.inspector{background:var(--ink);min-width:0}.model-rail{border-right:1px solid var(--line-light);flex-direction:column;grid-area:models;padding:24px 16px 14px;animation:.52s cubic-bezier(.16,1,.3,1) .1s both rise-in;display:flex;position:relative;overflow:hidden}.model-rail:before{z-index:0;content:"";opacity:.18;background:repeating-radial-gradient(ellipse at 51% 47%, transparent 0 7px, var(--moss) 7.5px 8.5px);border-radius:43% 57% 52% 48%;width:265px;height:265px;position:absolute;bottom:44px;left:-103px;transform:rotate(18deg)}.rail-heading,.section-heading{z-index:1;align-items:flex-start;gap:11px;display:flex;position:relative}.rail-heading>span,.section-heading>span:first-child{color:var(--phosphor);font:500 9px/1 var(--mono);border-top:1px solid var(--phosphor);padding-top:5px}.rail-heading h2,.section-heading h2{font:400 22px/1 var(--display);letter-spacing:-.025em;margin:0}.model-list{z-index:1;gap:5px;margin-top:21px;display:grid;position:relative}.model-card{border:1px solid #0000;transition:border-color .18s,background .18s}.model-card.is-active{border-color:var(--phosphor);background:var(--phosphor-dim);box-shadow:inset 3px 0 0 var(--phosphor)}.model-select{text-align:left;width:100%;color:var(--paper);cursor:pointer;background:0 0;border:0;grid-template-columns:23px 1fr auto;gap:8px;padding:12px 8px 11px;display:grid}.model-card:not(.is-active) .model-select:hover{background:#e8dfca0e}.model-index{color:#e8dfca61;font:8px/1 var(--mono);padding-top:4px}.model-copy{gap:3px;min-width:0;display:grid}.model-title-line{justify-content:space-between;align-items:center;gap:7px;display:flex}.model-title-line strong{font:500 20px/1 var(--display)}.model-copy em{color:#e8dfca75;font:italic 10px/1.2 var(--display);text-overflow:ellipsis;white-space:nowrap;overflow:hidden}.model-copy>span:last-child{color:var(--paper-ghost);margin-top:3px;font-size:10px}.availability{font:600 7px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;border:1px solid var(--line-light);padding:2px 4px}.availability-loaded,.availability-loading,.availability-cached{color:var(--phosphor);border-color:#b9ff4552}.availability-remote,.availability-limited{color:var(--rust);border-color:#c96f4366}.model-footprint{color:#e8dfca63;font:8px/1 var(--mono);white-space:nowrap;align-self:end}.text-action{width:calc(100% - 30px);color:var(--rust);font:8px/1 var(--mono);text-align:left;cursor:pointer;background:0 0;border:0;border-top:1px dashed #c96f4359;margin:-3px 8px 9px 30px;padding:7px 0}.text-action span{float:right}.text-action.is-status{cursor:default;text-overflow:ellipsis;white-space:nowrap;display:block;overflow:hidden}.model-limit-reason{color:#e8dfca94;font:8px/1.35 var(--mono);margin:-4px 8px 9px 30px;display:block}.dflash-card{z-index:1;background:#111512b8;border:1px dashed #e8dfca3b;margin-top:auto;padding:13px 12px 11px;position:relative}.dflash-card.is-available{border-color:#b9ff457a}.dflash-heading{justify-content:space-between;gap:12px;display:flex}.dflash-heading h3{font:500 14px/1 var(--display);margin:7px 0 0}.experimental-stamp{color:var(--rust);font:700 7px/1 var(--mono);letter-spacing:.12em;text-transform:uppercase;border:1px solid;padding:3px 5px 2px;display:inline-block;transform:rotate(-2deg)}.switch{cursor:pointer;background:0 0;border:1px solid #e8dfca4d;border-radius:10px;width:35px;height:18px;padding:2px}.switch span{background:#e8dfca6b;border-radius:50%;width:12px;height:12px;transition:transform .17s,background .17s;display:block}.switch[aria-checked=true]{border-color:var(--phosphor)}.switch[aria-checked=true] span{background:var(--phosphor);transform:translate(17px)}.switch:disabled{opacity:.38}.dflash-card p{color:#e8dfca80;font:9px/1.45 var(--body);margin:9px 0 5px}.dflash-contract{color:#c96f43b8;font:7px/1.3 var(--mono);text-transform:uppercase;margin-bottom:11px;display:block}.dflash-axis{border-top:1px solid #e8dfca1f;justify-content:space-between;align-items:center;display:flex}.dflash-axis span{background:#e8dfca47;width:1px;height:5px;margin-top:-1px}.rail-footnote{z-index:1;color:#e8dfca4d;font:7px/1 var(--mono);letter-spacing:.03em;justify-content:space-between;gap:8px;margin-top:12px;display:flex;position:relative}.chat-surface{min-width:0;height:calc(100dvh - 76px);min-height:calc(100dvh - 76px);color:var(--ink);background:radial-gradient(ellipse at 82% 16%, transparent 0 77px, #1115120e 78px 79px, transparent 80px 89px, #1115120b 90px 91px, transparent 92px), linear-gradient(100deg, transparent 0 67%, #11151206 67.2% 67.35%, transparent 67.5%), var(--paper);grid-area:chat;grid-template-rows:auto 19px minmax(0,1fr) auto;animation:.56s cubic-bezier(.16,1,.3,1) .16s both rise-in;display:grid;position:relative;overflow:hidden}.chat-header{border-bottom:1px solid var(--line-dark);justify-content:space-between;align-items:center;gap:20px;min-height:82px;padding:17px clamp(21px,3vw,44px) 13px;display:flex}.chat-header h2{font:500 clamp(25px, 2.4vw, 37px)/1 var(--display);letter-spacing:-.045em;margin:0;display:inline}.botanical-subtitle{color:#11151275;font:italic 12px/1 var(--display);margin-left:10px}.chat-header-meta{color:#1115128c;font:8px/1 var(--mono);text-transform:uppercase;letter-spacing:.07em;grid-template-columns:repeat(2,auto);gap:5px 13px;display:grid}.header-load-button{color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 7px/1 var(--mono);letter-spacing:.06em;text-transform:uppercase;cursor:pointer;grid-column:span 2;justify-self:end;padding:6px 9px}.header-load-button:disabled{opacity:.42}.live-label{color:#3d551d;grid-column:span 2;justify-self:end;align-items:center;gap:7px;display:flex}.live-label i{background:#5d7e2e;width:5px;height:5px;box-shadow:0 0 0 3px #5a782b21}.live-label.is-dormant{color:#1115127a}.live-label.is-dormant i{background:#1115125c;box-shadow:0 0 0 3px #11151217}.runtime-error{z-index:4;color:var(--paper);border:1px solid var(--rust);background:#111512f5;justify-content:space-between;align-items:center;gap:16px;padding:11px 13px;display:flex;position:absolute;top:105px;left:clamp(21px,3vw,44px);right:clamp(21px,3vw,44px);box-shadow:4px 4px #c96f433d}.runtime-error div{gap:3px;display:grid}.runtime-error strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.runtime-error span{font:10px/1.35 var(--mono);overflow-wrap:anywhere}.runtime-error button{font:7px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:1px solid #e8dfca4d;flex:none;padding:5px 8px}.runtime-error button:disabled{cursor:wait;opacity:.5}.runtime-error.download-retry button{max-width:132px;line-height:1.25}.chat-ruler{color:#11151266;font:7px/1 var(--mono);background:repeating-linear-gradient(90deg,#0000 0 14px,#11151233 14px 15px) 0 100%/100% 4px no-repeat;border-bottom:1px solid #11151217;justify-content:space-between;padding-inline:clamp(21px,3vw,44px);display:flex}.transcript{scrollbar-color:#11151247 transparent;min-height:0;padding:20px clamp(21px,4vw,64px) 32px;overflow-y:auto}.transcript-date{color:#11151273;font:8px/1 var(--mono);letter-spacing:.1em;text-transform:uppercase;align-items:center;gap:10px;margin-bottom:22px;display:flex}.transcript-date:before,.transcript-date:after{content:"";background:#11151224;flex:1;height:1px}.transcript-empty{text-align:center;color:#11151280;max-width:430px;margin:12vh auto 0}.transcript-empty>span{color:#507324;font:42px/1 var(--display);display:block}.transcript-empty h3{color:var(--ink);font:400 25px/1 var(--display);margin:8px 0 6px}.transcript-empty p{margin:0;font-size:11px;line-height:1.5}.message{max-width:690px;margin-bottom:25px;animation:.43s cubic-bezier(.16,1,.3,1) both note-in}.message>header{color:#11151278;font:8px/1 var(--mono);text-transform:uppercase;letter-spacing:.07em;align-items:center;gap:8px;margin-bottom:8px;display:flex}.message>header strong{color:var(--ink)}.message>header time{margin-left:auto}.message-ordinal{color:#4d6728;min-width:22px}.message-content{font:15px/1.55 var(--body)}.message-content>p{margin:0}.markdown-content>:first-child{margin-top:0}.markdown-content>:last-child{margin-bottom:0}.markdown-content p,.markdown-content ul,.markdown-content ol,.markdown-content blockquote{margin:0 0 10px}.markdown-content ul,.markdown-content ol{padding-left:22px}.markdown-content h3,.markdown-content h4,.markdown-content h5{font-family:var(--display);margin:16px 0 7px;font-weight:500;line-height:1.1}.markdown-content blockquote{color:#111512ad;white-space:pre-wrap;border-left:2px solid #5073248c;padding-left:12px}.markdown-content code{font:.78em/1.4 var(--mono);background:#11151214;padding:1px 4px}.markdown-content pre{max-width:100%;color:var(--paper);background:var(--ink);border-left:2px solid var(--phosphor);margin:11px 0;padding:12px;position:relative;overflow:auto}.markdown-content pre code{white-space:pre;background:0 0;padding:0}.markdown-content a{color:#3f6417;text-underline-offset:2px;text-decoration-thickness:1px}.message-user .markdown-content code{background:#e8dfca1f}.reasoning-block{color:#1115129e;background:#11151209;border:1px dashed #1115123d;margin:0 0 11px;padding:8px 10px;font-size:11px}.reasoning-block summary{color:#4d6728;font:700 7px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;cursor:pointer}.reasoning-block[open] summary{margin-bottom:8px}.generation-placeholder{color:#11151273;margin:0;font-style:italic}.message-error{color:#8d321b;font:9px/1.4 var(--mono);margin:9px 0 0}.message.has-error{border-color:var(--rust)}.message-assistant{border-left:2px solid var(--ink);padding-left:22px}.message-assistant .message-content>p:first-letter{float:left;color:#38511d;font:42px/.75 var(--display);margin:1px 6px 0 0}.message-user{max-width:min(78%,610px);color:var(--paper);background:var(--ink);clip-path:polygon(0 0,calc(100% - 10px) 0,100% 10px,100% 100%,0 100%);margin-left:auto;padding:12px 15px 13px;box-shadow:5px 5px #1115121f}.message-user>header,.message-user>header strong{color:#e8dfca8f}.message-user .message-ordinal{color:var(--phosphor)}.message>footer{color:#1115126e;font:8px/1 var(--mono);border-top:1px dashed #11151230;gap:12px;margin-top:9px;padding-top:7px;display:flex}.stream-cursor{vertical-align:-2px;background:#507324;width:7px;height:14px;margin-left:4px;animation:.86s step-end infinite cursor-blink;display:inline-block}.tool-call{background:#1115120b;border:1px solid #1115122e;margin-top:13px}.tool-call summary{cursor:pointer;grid-template-columns:auto 1fr auto;align-items:center;gap:8px;padding:8px 10px;list-style:none;display:grid}.tool-call summary::-webkit-details-marker{display:none}.tool-call code{font:9px/1.4 var(--mono)}.tool-call summary>span:last-child{color:#1115127a;font:7px/1 var(--mono);text-transform:uppercase}.tool-state,.event-pip{background:var(--moss);border-radius:50%;width:6px;height:6px}.tool-state-complete,.event-complete{background:#55772c}.tool-state-running,.event-running{background:var(--phosphor);box-shadow:0 0 0 3px #5a782b26}.tool-state-error,.event-error{background:var(--rust)}.tool-call>div{border-top:1px solid #11151221;padding:9px 10px}.tool-call>div p{color:#11151294;margin:5px 0 0;font-size:10px}.composer{border-top:1px solid var(--line-dark);-webkit-backdrop-filter:blur(9px);backdrop-filter:blur(9px);background:#d8cdb4d1;padding:10px clamp(21px,3vw,44px) 18px;position:relative;box-shadow:0 -15px 38px #1115120f}.composer:before{content:"";background:repeating-linear-gradient(90deg, var(--ink) 0 1px, transparent 1px 12px);opacity:.23;height:5px;position:absolute;top:-5px;left:0;right:0}.composer-meta{color:#11151275;font:8px/1 var(--mono);justify-content:space-between;align-items:center;gap:12px;margin-bottom:7px;display:flex}.tool-toggle{color:#1115128a;font:700 8px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:0;align-items:center;gap:5px;padding:0;display:flex}.tool-toggle.is-on{color:#425f1f}.regenerate-button{color:#1115128a;font:700 8px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:0;margin-right:auto;padding:0}.regenerate-button:disabled{cursor:default;opacity:.45}.tool-toggle svg{fill:none;stroke:currentColor;stroke-width:1.3px;width:12px}.composer-field{border:1px solid var(--ink);background:#e8dfcad4;grid-template-columns:1fr auto;min-height:76px;display:grid;position:relative;box-shadow:4px 4px #1115121a}.composer-field label{color:#11151273;font:7px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase;position:absolute;top:7px;left:11px}.composer-field textarea{resize:none;min-width:0;color:var(--ink);font:14px/1.4 var(--body);background:0 0;border:0;outline:none;padding:23px 13px 9px}.composer-field textarea::placeholder{color:#11151257}.send-button{min-width:82px;color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 9px/1 var(--mono);text-transform:uppercase;cursor:pointer;justify-content:center;align-items:center;gap:7px;margin:7px;transition:transform .15s,box-shadow .15s,background .15s;display:flex}.send-button:hover:not(:disabled){box-shadow:3px 3px 0 var(--ink);transform:translate(-2px,-2px)}.send-button:disabled{opacity:.34}.send-button svg{fill:none;stroke:currentColor;stroke-width:1.4px;width:15px}.send-button.stop{color:var(--paper);background:var(--ink)}.send-button.stop span{background:var(--rust);width:7px;height:7px}.inspector{border-left:1px solid var(--line-light);scrollbar-color:#e8dfca2e transparent;grid-area:inspector;height:calc(100dvh - 76px);min-height:calc(100dvh - 76px);animation:.57s cubic-bezier(.16,1,.3,1) .22s both rise-in;overflow-y:auto}.inspector-section{border-bottom:1px solid var(--line-light);padding:20px 18px}.section-heading{align-items:center;margin-bottom:17px}.section-heading.compact{margin-bottom:14px}.section-heading>div{flex:1}.section-heading h2{font-size:20px}.sampling-label,.persistence{color:#e8dfca6b;font:7px/1 var(--mono);text-transform:uppercase;letter-spacing:.06em;align-items:center;gap:6px;display:flex}.sampling-label i{width:4px;height:4px}.speed-gauge{grid-template-columns:116px 1fr;align-items:center;gap:15px;display:grid}.gauge-ring{background:radial-gradient(circle, var(--ink) 0 51%, transparent 52%), conic-gradient(var(--phosphor) 0 var(--gauge), #e8dfca1c var(--gauge) 100%);border-radius:49% 51% 47% 53%;place-items:center;width:116px;height:116px;display:grid;transform:rotate(-9deg);box-shadow:inset 0 0 0 1px #e8dfca1a,0 0 30px #b9ff450e}.gauge-ring:after{content:"";border:1px dashed #e8dfca21;border-radius:45% 55% 52% 48%;width:128px;height:128px;position:absolute}.gauge-ring>div{z-index:1;justify-items:center;display:grid;position:relative;transform:rotate(9deg)}.gauge-ring strong{color:var(--phosphor);font:400 29px/1 var(--display)}.gauge-ring small{color:var(--paper-ghost);font:7px/1 var(--mono);text-transform:uppercase;margin-top:3px}.speed-gauge dl{margin:0}.speed-gauge dl div{border-bottom:1px solid var(--line-light);font:8px/1 var(--mono);justify-content:space-between;padding:7px 0;display:flex}.speed-gauge dt{color:#e8dfca6e}.speed-gauge dd{color:var(--paper);margin:0}.device-card{background:linear-gradient(90deg,#b9ff4514,#0000);border:1px solid #b9ff453b;grid-template-columns:auto 1fr auto;align-items:center;gap:10px;margin:19px 0 14px;padding:10px;display:grid}.device-icon{border:1px solid #b9ff4540;align-items:flex-end;gap:3px;width:28px;height:28px;padding:5px;display:flex}.device-icon i{background:var(--phosphor);width:4px}.device-icon i:first-child{opacity:.45;height:7px}.device-icon i:nth-child(2){height:15px}.device-icon i:nth-child(3){opacity:.7;height:11px}.device-card>div{gap:3px;display:grid}.device-card small,.memory-grid small,.meter small,.download-item small{color:#e8dfca6b;font:7px/1 var(--mono);text-transform:uppercase}.device-card strong{font:500 11px/1 var(--mono)}.device-card>span:last-child{color:var(--phosphor);font:600 13px/1 var(--mono);justify-items:end;display:grid}.device-card>span:last-child small{color:#e8dfca5e;margin-top:3px;font-size:6px}.meter{margin-top:13px}.meter>div:first-child{font:8px/1 var(--mono);justify-content:space-between;margin-bottom:6px;display:flex}.meter-track{border:1px solid #e8dfca36;height:5px;padding:1px}.meter-track>span{background:repeating-linear-gradient(90deg, var(--phosphor) 0 5px, transparent 5px 7px);height:100%;display:block}.memory-grid{background:var(--line-light);grid-template-columns:1fr 1fr;gap:1px;margin-top:14px;display:grid}.memory-grid>div{background:var(--ink);gap:6px;padding:10px;display:grid}.memory-grid strong{font:500 12px/1 var(--mono)}.event-log{gap:0;margin:0;padding:0;list-style:none;display:grid;position:relative}.event-log:before{content:"";background:var(--line-light);width:1px;position:absolute;top:5px;bottom:5px;left:3px}.event-log li{grid-template-columns:8px 64px 1fr;align-items:start;gap:8px;padding:7px 0;display:grid}.event-log li.event-empty{color:#e8dfca5c;font:8px/1.4 var(--mono);padding:5px 0;display:block}.event-pip{z-index:1;border:1px solid var(--ink);margin-top:2px}.event-log time{color:#e8dfca57;font:7px/1.2 var(--mono)}.event-log div{gap:3px;display:grid}.event-log strong{font:500 9px/1 var(--mono)}.event-log small{color:#e8dfca6b;font-size:8px}.persistence{border:1px solid var(--line-light);padding:3px 5px}.persistence.is-persistent{color:var(--phosphor);border-color:#b9ff4547}.download-item{border:1px solid var(--line-light);grid-template-columns:1fr auto;gap:8px;margin-top:16px;padding:11px 10px 10px;display:grid;position:relative}.download-item>div:first-child{gap:5px;display:grid}.download-item strong{font:500 10px/1 var(--mono)}.download-state{width:fit-content;color:var(--rust);font:700 6px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase}.state-downloading{color:var(--phosphor)}.download-percent{color:var(--phosphor);font:500 15px/1 var(--display)}.download-track{background:#e8dfca1a;grid-column:span 2;height:3px}.download-track span{background:var(--phosphor);height:100%;display:block;box-shadow:0 0 9px #b9ff4559}.artifact-close{border:1px solid var(--line-light);cursor:pointer;background:0 0;border-radius:50%;width:25px;height:25px}.artifact-empty{color:#e8dfca66;font:8px/1.5 var(--mono);margin:0}.artifact-pane{border:1px solid var(--line-light);background:#f8f5eb}.artifact-tabs{color:var(--paper);border-bottom:1px solid var(--line-light);background:var(--ink-raised);grid-template-columns:auto auto 1fr;align-items:center;display:grid}.artifact-tabs button{border:0;border-right:1px solid var(--line-light);font:7px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;padding:7px 9px}.artifact-tabs button[aria-selected=true]{color:var(--ink);background:var(--phosphor)}.artifact-tabs span{color:#e8dfca73;font:7px/1 var(--mono);text-align:right;text-overflow:ellipsis;white-space:nowrap;padding:0 8px;overflow:hidden}.artifact-pane iframe{background:#fff;border:0;width:100%;height:260px;display:block}.artifact-running{position:relative}.artifact-running>button{z-index:2;color:var(--paper);border:1px solid var(--rust);font:7px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:#111512e6;padding:5px 7px;position:absolute;top:7px;right:7px}.artifact-launch{min-height:180px;color:var(--ink);text-align:center;align-content:center;justify-items:center;gap:13px;padding:20px;display:grid}.artifact-launch p{color:#1115129e;max-width:250px;font:9px/1.5 var(--mono);margin:0}.artifact-launch button{color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 7px/1 var(--mono);text-transform:uppercase;cursor:pointer;padding:8px 10px}.artifact-code{height:260px;color:var(--paper);font:8px/1.45 var(--mono);white-space:pre-wrap;background:#0d100e;margin:0;padding:10px;overflow:auto}.settings-layer{z-index:50;-webkit-backdrop-filter:blur(4px);backdrop-filter:blur(4px);background:#080b09a3;justify-content:flex-end;animation:.16s both station-in;display:flex;position:fixed;inset:0}.settings-sheet{width:min(430px,100%);min-height:100%;color:var(--paper);border-left:1px solid var(--phosphor);background:repeating-radial-gradient(ellipse at 104% 101%, transparent 0 13px, #b9ff450e 13.5px 14.5px), var(--ink);padding:30px 27px;animation:.3s cubic-bezier(.16,1,.3,1) both sheet-in;overflow-y:auto;box-shadow:-24px 0 70px #00000052}.settings-sheet>header{border-bottom:1px solid var(--line-light);justify-content:space-between;align-items:flex-start;gap:20px;margin-bottom:37px;padding-bottom:20px;display:flex}.settings-sheet h2{font:400 31px/1 var(--display);margin:0}.sheet-close{border:1px solid var(--line-light);width:35px;height:35px;font:300 22px/1 var(--body);cursor:pointer;background:0 0;border-radius:50%}.sheet-close:hover{color:var(--phosphor);border-color:var(--phosphor)}.setting-group{border-bottom:1px solid var(--line-light);grid-template-columns:24px 1fr auto;align-items:center;gap:12px;padding:15px 0;display:grid}.setting-number{color:var(--phosphor);font:8px/1 var(--mono)}.setting-group>div,.range-setting>span:first-child{gap:5px;display:grid}.setting-group small,.range-setting small{color:#e8dfca6e;font:7px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.setting-group strong,.range-setting strong{font:500 13px/1 var(--mono)}.setting-locked{color:#e8dfca5c;font:7px/1 var(--mono);text-transform:uppercase}.setting-select select{max-width:112px;color:var(--paper);border:1px solid var(--line-light);background:var(--ink-raised);font:8px/1 var(--mono);border-radius:0;padding:6px 7px}.range-setting{gap:13px;margin-top:29px;display:grid}.range-setting>span:first-child{justify-content:space-between;align-items:end;display:flex}.range-setting input{appearance:none;background:linear-gradient(90deg, var(--phosphor) 0 35%, #e8dfca1c 35%);border:1px solid #e8dfca3d;border-radius:0;width:100%;height:4px;margin:0}.range-setting:first-of-type input{background:var(--phosphor)}.range-setting input::-webkit-slider-thumb{appearance:none;border:2px solid var(--ink);background:var(--phosphor);width:12px;height:18px;box-shadow:0 0 0 1px var(--phosphor);cursor:grab}.range-setting input::-moz-range-thumb{border:2px solid var(--ink);background:var(--phosphor);width:10px;height:16px;box-shadow:0 0 0 1px var(--phosphor);cursor:grab;border-radius:0}.range-ticks{pointer-events:none;justify-content:space-between;margin-top:-13px;display:flex}.range-limits{color:#e8dfca59;font:7px/1 var(--mono);justify-content:space-between;display:flex}.range-limits i{font-style:normal}.system-prompt-setting{gap:9px;margin-top:29px;display:grid}.system-prompt-setting>span{color:#e8dfca6e;font:7px/1 var(--mono);letter-spacing:.05em;text-transform:uppercase;justify-content:space-between;display:flex}.system-prompt-setting textarea{resize:vertical;color:var(--paper);border:1px solid var(--line-light);background:var(--ink-raised);font:10px/1.45 var(--mono);padding:10px}.settings-actions{grid-template-columns:1fr 1fr;gap:7px;margin-top:29px;display:grid}.settings-actions button{color:var(--paper);border:1px solid var(--line-light);font:7px/1.2 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;padding:9px}.settings-actions button:hover{color:var(--phosphor);border-color:var(--phosphor)}.settings-actions .danger-action{color:#e78a62;border-color:#c96f4373;grid-column:span 2}.boot-screen{min-height:100dvh;color:var(--paper);background:var(--ink);justify-content:center;align-items:center;gap:18px;padding:24px;display:flex}.boot-screen h1{color:var(--phosphor);font:400 34px/1 var(--display);margin:0}.boot-screen p:last-child{color:var(--paper-ghost);font:10px/1.5 var(--mono)}.range-ticks i{background:#e8dfca40;width:1px;height:5px}.settings-note{background:#b9ff450f;border:1px solid #b9ff4540;align-items:flex-start;gap:13px;margin-top:42px;padding:15px;display:flex}.settings-note .signal-dot{flex:0 0 7px;margin-top:6px}.settings-note p{color:#e8dfca94;margin:0;font-size:10px;line-height:1.5}.settings-note strong{color:var(--phosphor)}@keyframes station-in{0%{opacity:0}to{opacity:1}}@keyframes rise-in{0%{opacity:0;transform:translateY(11px)}to{opacity:1;transform:translateY(0)}}@keyframes note-in{0%{opacity:0;transform:translateY(7px)}to{opacity:1;transform:translateY(0)}}@keyframes cursor-blink{0%,52%{opacity:1}53%,to{opacity:0}}@keyframes sheet-in{0%{opacity:0;transform:translate(34px)}to{opacity:1;transform:translate(0)}}@media (width<=1279px){.station-header{grid-template-columns:minmax(320px,1fr) auto auto}.local-promise{min-width:auto}.local-promise small{display:none}.workspace-grid{grid-template-columns:260px minmax(500px,1fr);grid-template-areas:"models chat""inspector inspector"}.model-rail,.chat-surface{height:calc(100dvh - 76px);min-height:calc(100dvh - 76px)}.inspector{border-top:1px solid var(--phosphor);border-left:0;grid-template-columns:repeat(2,minmax(0,1fr));height:auto;min-height:auto;display:grid;overflow:visible}.inspector-section{border-right:1px solid var(--line-light);border-bottom:0}}@media (width<=820px){.station-header{min-height:70px;display:block}.brand-block{padding:11px 70px 11px 15px}.brand-mark{flex-basis:36px;width:36px;height:36px}.local-promise{border:0;border-top:1px solid var(--line-light);min-height:31px;padding:0 16px}.header-actions{position:absolute;top:10px;right:0}.station-clock{display:none}.workspace-grid{display:block}.model-rail,.chat-surface,.inspector{height:auto;min-height:auto}.model-rail{border-right:0;border-bottom:1px solid var(--phosphor);padding:18px 15px;display:block}.model-list{scroll-snap-type:x mandatory;gap:7px;padding-bottom:7px;display:flex;overflow-x:auto}.model-card{scroll-snap-align:start;min-width:205px}.dflash-card{margin-top:13px}.rail-footnote{display:none}.chat-surface{grid-template-rows:auto 19px minmax(450px,auto) auto;display:grid;overflow:visible}.transcript{padding-inline:22px;overflow:visible}.chat-header{padding:17px 19px}.chat-header-meta span:not(.live-label){display:none}.chat-header-meta .header-load-button{display:block}.live-label{grid-column:1}.composer{z-index:5;padding-inline:14px;position:sticky;bottom:0}.inspector{border-top:1px solid var(--phosphor);display:block}.inspector-section{border-right:0;border-bottom:1px solid var(--line-light)}}@media (width<=520px){.brand-block h1{font-size:24px}.brand-block .eyebrow,.local-promise strong{font-size:7px}.chat-header{align-items:flex-end}.botanical-subtitle{margin:4px 0 0;display:block}.message-user{max-width:91%}.composer-meta>span{display:none}.composer-field{grid-template-columns:1fr;padding-bottom:50px}.send-button{height:35px;position:absolute;bottom:0;right:0}.speed-gauge{grid-template-columns:106px 1fr}.gauge-ring{width:102px;height:102px}.gauge-ring:after{width:113px;height:113px}}@media (prefers-reduced-motion:reduce){*,:before,:after{scroll-behavior:auto!important;transition-duration:.01ms!important;animation-duration:.01ms!important;animation-iteration-count:1!important}}.bench-page{min-height:100dvh;color:var(--ink);background:var(--paper)}.bench-masthead{min-height:86px;color:var(--paper);border-bottom:1px solid var(--phosphor);background:var(--ink);grid-template-columns:minmax(310px,1fr) minmax(360px,1.2fr) auto;align-items:stretch;display:grid}.bench-masthead .brand-block h1{color:var(--paper)}.bench-masthead-note{border-inline:1px solid var(--line-light);background:linear-gradient(110deg,#b9ff4517,#0000 72%);flex-direction:column;justify-content:center;gap:5px;padding:14px 28px;display:flex}.bench-masthead-note strong{color:var(--phosphor);font:700 9px/1 var(--mono);letter-spacing:.11em;text-transform:uppercase}.bench-masthead-note span{color:#e8dfca8c;max-width:520px;font:10px/1.35 var(--body)}.bench-return{color:var(--paper);font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;align-items:center;gap:8px;padding:0 25px;text-decoration:none;display:flex}.bench-return:hover{color:var(--phosphor)}.bench-layout{grid-template-columns:minmax(275px,330px) minmax(0,1fr);min-height:calc(100dvh - 86px);display:grid}.bench-controls{color:var(--paper);border-right:1px solid var(--phosphor);background:repeating-radial-gradient(ellipse at -12% 102%, transparent 0 24px, #b9ff450b 25px 26px), var(--ink);padding:28px 22px}.bench-controls h2{font:400 30px/1 var(--display);letter-spacing:-.03em;margin:0}.bench-control-intro{color:#e8dfca80;margin:9px 0 22px;font-size:11px;line-height:1.45}.bench-controls form{gap:15px;display:grid}.bench-field{gap:6px;min-width:0;display:grid}.bench-field>span{color:#e8dfcac7;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.bench-field select,.bench-field input{width:100%;min-width:0;height:39px;color:var(--paper);background:var(--ink-raised);font:10px/1 var(--mono);border:1px solid #e8dfca38;border-radius:0;outline:0;padding:0 10px}.bench-field input::placeholder{color:#e8dfca40}.bench-field select:disabled,.bench-field input:disabled{opacity:.5}.bench-field small{color:#e8dfca59;font:8px/1.2 var(--mono)}.bench-field-pair{grid-template-columns:repeat(2,minmax(0,1fr));gap:10px;display:grid}.bench-experimental{background:#c96f430d;border:1px dashed #c96f437a;padding:11px}.bench-experimental summary{color:#e19a76;font:700 8px/1 var(--mono);letter-spacing:.07em;text-transform:uppercase;cursor:pointer}.bench-experimental>p{color:#e8dfca75;margin:10px 0;font-size:9px;line-height:1.4}.bench-locks{color:#e8dfca70;font:7px/1.35 var(--mono);gap:5px;margin:10px 0 0;display:grid}.bench-locks div{justify-content:space-between;gap:8px;display:flex}.bench-locks dt,.bench-locks dd{margin:0}.bench-locks dt{color:#e8dfcab8}.bench-gate{border-left:2px solid var(--rust);background:#c96f4314;gap:5px;padding:10px;display:grid}.bench-gate.pass{border-color:var(--phosphor);background:var(--phosphor-dim)}.bench-gate strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.06em;text-transform:uppercase}.bench-gate.pass strong{color:var(--phosphor)}.bench-gate span{color:#e8dfca80;font:8px/1.4 var(--mono);overflow-wrap:anywhere}.bench-run-actions{grid-template-columns:1fr auto;gap:8px;display:grid}.bench-run-button,.bench-stop-button,.bench-section-heading button{border:1px solid var(--ink);background:var(--phosphor);min-height:38px;font:800 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;cursor:pointer;border-radius:0;padding:0 13px}.bench-run-button:disabled{opacity:.35}.bench-stop-button{color:#ffb08a;border-color:var(--rust);background:0 0}.bench-workbench{background:linear-gradient(90deg, #1115120b 1px, transparent 1px) 0 0 / 22px 22px, linear-gradient(#1115120a 1px, transparent 1px) 0 0 / 22px 22px, var(--paper);min-width:0;padding:clamp(26px,4vw,58px)}.bench-hero{border-bottom:1px solid var(--line-dark);grid-template-columns:minmax(320px,1.3fr) minmax(260px,.7fr);align-items:end;gap:34px;padding-bottom:29px;display:grid}.bench-hero h2{max-width:690px;font:400 clamp(42px, 6vw, 82px)/.84 var(--display);letter-spacing:-.065em;margin:0}.bench-hero h2 em{color:#4f7125;font-weight:400}.bench-hero>p{color:#11151294;max-width:430px;margin:0;font-size:11px;line-height:1.55}.bench-progress{gap:8px;padding:16px 0;display:grid}.bench-progress>div{font:8px/1 var(--mono);letter-spacing:.05em;text-transform:uppercase;justify-content:space-between;gap:20px;display:flex}.bench-progress>div span{color:#11151273}.bench-progress-track{background:#11151221;height:3px}.bench-progress-track i{background:#527426;height:100%;transition:width .18s;display:block}.bench-error{color:var(--paper);border-left:3px solid var(--rust);background:var(--ink);gap:5px;margin-bottom:18px;padding:12px 14px;display:grid}.bench-error strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.bench-error span{font:9px/1.4 var(--mono);overflow-wrap:anywhere}.bench-core-section{margin-top:13px}.bench-section-heading{justify-content:space-between;align-items:end;gap:20px;margin-bottom:13px;display:flex}.bench-section-heading h3,.bench-output h3{font:500 24px/1 var(--display);margin:0}.bench-section-heading button{min-height:31px;color:var(--paper);border-color:var(--ink);background:var(--ink)}.bench-core-sample{border:1px solid var(--ink);background:var(--paper-deep);grid-template-columns:repeat(4,minmax(0,1fr));display:grid;position:relative;overflow:hidden}.bench-core-sample:before{content:"";pointer-events:none;opacity:.35;background:repeating-radial-gradient(#0000 0 34px,#1115122b 35px 36px);position:absolute;inset:-130% -5%}.bench-core-sample article{background:#e8dfca91;border-right:1px solid #11151242;flex-direction:column;justify-content:space-between;min-height:128px;padding:17px;display:flex;position:relative}.bench-core-sample article:nth-child(2n){background:#d8cdb499}.bench-core-sample article:last-child{color:var(--paper);background:#111512e8;border-right:0}.bench-core-sample article>span{font:800 8px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase}.bench-core-sample article strong{font:400 clamp(28px, 3vw, 43px)/1 var(--display);letter-spacing:-.04em;margin-block:auto}.bench-core-sample article small{color:#11151280;font:7px/1.2 var(--mono);text-transform:uppercase}.bench-core-sample article:last-child small{color:#e8dfca7a}.bench-detail-grid{grid-template-columns:repeat(2,minmax(0,1fr));gap:12px;margin-top:12px;display:grid}.bench-data-card{border:1px solid var(--line-dark);background:#e8dfcaad;padding:16px}.bench-runtime-card{grid-column:span 2}.bench-data-card dl{grid-template-columns:repeat(2,minmax(0,1fr));gap:12px 26px;margin:0;display:grid}.bench-runtime-card dl{grid-template-columns:minmax(180px,1.5fr) repeat(3,minmax(100px,.7fr))}.bench-data-card dl div{border-top:1px solid #11151221;gap:4px;min-width:0;padding-top:8px;display:grid}.bench-data-card dt{color:#11151273;font:7px/1 var(--mono);letter-spacing:.07em;text-transform:uppercase}.bench-data-card dd{min-width:0;font:500 16px/1.2 var(--body);text-overflow:ellipsis;white-space:nowrap;margin:0;overflow:hidden}.bench-output{border:1px dashed #1115124d;grid-template-columns:repeat(2,minmax(0,1fr));gap:34px;margin-top:12px;padding:21px;display:grid}.bench-output>div:last-child{border-left:1px solid var(--line-dark);padding-left:34px}.bench-output p:not(.eyebrow){color:#11151299;white-space:pre-wrap;margin:8px 0 0;font-size:11px;line-height:1.5}@media (width<=1050px){.bench-masthead{grid-template-columns:minmax(310px,1fr) auto}.bench-masthead-note{display:none}.bench-layout{grid-template-columns:285px minmax(0,1fr)}.bench-core-sample{grid-template-columns:repeat(2,minmax(0,1fr))}.bench-core-sample article:nth-child(2){border-right:0}.bench-core-sample article:nth-child(-n+2){border-bottom:1px solid #11151242}.bench-runtime-card dl{grid-template-columns:repeat(2,minmax(0,1fr))}}@media (width<=760px){.bench-masthead{align-items:center;display:flex}.bench-masthead .brand-block{flex:1;min-width:0}.bench-return{padding:0 15px;font-size:0}.bench-return span{font-size:17px}.bench-layout{display:block}.bench-controls{border-right:0;border-bottom:1px solid var(--phosphor)}.bench-workbench{padding:28px 16px}.bench-hero,.bench-output{grid-template-columns:1fr;gap:18px}.bench-hero h2{font-size:clamp(42px,14vw,68px)}.bench-detail-grid{display:block}.bench-data-card+.bench-data-card{margin-top:10px}.bench-output>div:last-child{border-top:1px solid var(--line-dark);border-left:0;padding-top:18px;padding-left:0}}@media (width<=480px){.bench-masthead .brand-mark{display:none}.bench-core-sample,.bench-data-card dl,.bench-runtime-card dl{grid-template-columns:1fr}.bench-core-sample article{border-bottom:1px solid #11151242;border-right:0;min-height:105px}.bench-core-sample article:last-child{border-bottom:0}}
|
|
|
|
| 1 |
+
:root{--lightningcss-light: ;--lightningcss-dark:initial;color-scheme:dark;--ink:#111512;--ink-raised:#191e1a;--ink-soft:#252b25;--paper:#e8dfca;--paper-deep:#d8cdb4;--paper-ghost:#e8dfca9e;--phosphor:#b9ff45;--phosphor-dim:#b9ff4529;--moss:#6f8062;--rust:#c96f43;--line-light:#e8dfca24;--line-dark:#1115122e;--display:"Bodoni 72", Didot, "Iowan Old Style", Baskerville, serif;--body:"Avenir Next Condensed", "Gill Sans", "Trebuchet MS", sans-serif;--mono:"Berkeley Mono", Iosevka, "SFMono-Regular", Consolas, monospace;font-family:var(--body);font-synthesis:none;text-rendering:optimizelegibility}*{box-sizing:border-box}html{background:var(--ink)}body{background:linear-gradient(90deg, #1115120f 1px, transparent 1px) 0 0 / 22px 22px, linear-gradient(#1115120b 1px, transparent 1px) 0 0 / 22px 22px, var(--paper);min-width:320px;min-height:100vh;color:var(--paper);margin:0}button,textarea,select{font:inherit}button{color:inherit}button:focus-visible,textarea:focus-visible,select:focus-visible,summary:focus-visible{outline:2px solid var(--phosphor);outline-offset:3px}button:disabled{cursor:not-allowed}::selection{color:var(--ink);background:var(--phosphor)}.station-frame{isolation:isolate;min-height:100dvh;animation:.58s cubic-bezier(.16,1,.3,1) both station-in}.station-frame:before{z-index:20;pointer-events:none;content:"";opacity:.14;mix-blend-mode:soft-light;background-image:url("data:image/svg+xml,%3Csvg viewBox='0 0 180 180' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='3' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)' opacity='.3'/%3E%3C/svg%3E");position:fixed;inset:0}.station-header{background:var(--ink);border-bottom:1px solid var(--phosphor);z-index:10;grid-template-columns:minmax(330px,1fr) auto minmax(220px,1fr);align-items:stretch;min-height:76px;display:grid;position:relative;box-shadow:0 12px 38px #11151238}.brand-block{align-items:center;gap:14px;padding:12px 24px;display:flex}.brand-block h1{font:400 clamp(25px, 2vw, 34px) / .9 var(--display);letter-spacing:-.035em;margin:0}.brand-block h1 i{color:var(--phosphor);font-weight:400}.brand-mark{background:repeating-radial-gradient(at 52% 62%,#0000 0 4px,#b9ff456b 4.5px 5px);border:1px solid #b9ff457a;border-radius:50%;flex:0 0 42px;place-items:center;width:42px;height:42px;display:grid;transform:rotate(-11deg)}.brand-mark:before,.brand-mark:after,.brand-mark span:before,.brand-mark span:after{content:"";background:var(--phosphor);transform-origin:bottom;width:1px;position:absolute}.brand-mark:before{height:25px;transform:translate(-1px,4px)rotate(0)}.brand-mark:after{height:13px;transform:translate(4px,-2px)rotate(43deg)}.brand-mark span:before{height:11px;transform:translate(-5px,-1px)rotate(-47deg)}.brand-mark span:after{height:8px;transform:translate(3px,-9px)rotate(72deg)}.eyebrow{color:#e8dfca85;font:600 9px/1 var(--mono);letter-spacing:.18em;text-transform:uppercase;margin:0 0 5px}.eyebrow.ink{color:#11151285}.local-promise{border-inline:1px solid var(--line-light);background:linear-gradient(115deg,#b9ff4514,#0000 66%);align-items:center;gap:11px;min-width:360px;padding:0 28px;display:flex}.signal-dot,.live-label i,.sampling-label i{background:var(--phosphor);width:7px;height:7px;box-shadow:0 0 0 5px var(--phosphor-dim), 0 0 17px #b9ff459e;border-radius:50%;display:inline-block}.local-promise span:last-child{gap:3px;display:grid}.local-promise strong{color:var(--phosphor);font:600 10px/1 var(--mono);letter-spacing:.13em;text-transform:uppercase}.local-promise small{color:var(--paper-ghost);font:11px/1.2 var(--body)}.header-actions{justify-self:end;align-items:center;padding-right:18px;display:flex}.station-clock{color:var(--paper-ghost);font-family:var(--mono);justify-items:end;padding:0 18px;display:grid}.station-clock small{letter-spacing:.18em;font-size:8px}.station-clock span{letter-spacing:.09em;margin-top:4px;font-size:10px}.icon-button{cursor:pointer;background:0 0;border:1px solid #e8dfca3d;border-radius:50%;place-items:center;width:38px;height:38px;transition:color .16s,border-color .16s,transform .16s;display:grid}.icon-button:hover{color:var(--phosphor);border-color:var(--phosphor);transform:rotate(-7deg)}.icon-button svg{fill:none;stroke:currentColor;stroke-width:1.4px;width:18px}.workspace-grid{grid-template-columns:clamp(240px,18vw,286px) minmax(510px,1fr) clamp(310px,23vw,370px);grid-template-areas:"models chat inspector";min-height:calc(100dvh - 76px);display:grid}.model-rail,.inspector{background:var(--ink);min-width:0}.model-rail{border-right:1px solid var(--line-light);flex-direction:column;grid-area:models;padding:24px 16px 14px;animation:.52s cubic-bezier(.16,1,.3,1) .1s both rise-in;display:flex;position:relative;overflow:hidden}.model-rail:before{z-index:0;content:"";opacity:.18;background:repeating-radial-gradient(ellipse at 51% 47%, transparent 0 7px, var(--moss) 7.5px 8.5px);border-radius:43% 57% 52% 48%;width:265px;height:265px;position:absolute;bottom:44px;left:-103px;transform:rotate(18deg)}.rail-heading,.section-heading{z-index:1;align-items:flex-start;gap:11px;display:flex;position:relative}.rail-heading>span,.section-heading>span:first-child{color:var(--phosphor);font:500 9px/1 var(--mono);border-top:1px solid var(--phosphor);padding-top:5px}.rail-heading h2,.section-heading h2{font:400 22px/1 var(--display);letter-spacing:-.025em;margin:0}.model-list{z-index:1;gap:5px;margin-top:21px;display:grid;position:relative}.model-card{border:1px solid #0000;transition:border-color .18s,background .18s}.model-card.is-active{border-color:var(--phosphor);background:var(--phosphor-dim);box-shadow:inset 3px 0 0 var(--phosphor)}.model-select{text-align:left;width:100%;color:var(--paper);cursor:pointer;background:0 0;border:0;grid-template-columns:23px 1fr auto;gap:8px;padding:12px 8px 11px;display:grid}.model-card:not(.is-active) .model-select:hover{background:#e8dfca0e}.model-index{color:#e8dfca61;font:8px/1 var(--mono);padding-top:4px}.model-copy{gap:3px;min-width:0;display:grid}.model-title-line{justify-content:space-between;align-items:center;gap:7px;display:flex}.model-title-line strong{font:500 20px/1 var(--display)}.model-copy em{color:#e8dfca75;font:italic 10px/1.2 var(--display);text-overflow:ellipsis;white-space:nowrap;overflow:hidden}.model-copy>span:last-child{color:var(--paper-ghost);margin-top:3px;font-size:10px}.availability{font:600 7px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;border:1px solid var(--line-light);padding:2px 4px}.availability-loaded,.availability-loading,.availability-cached{color:var(--phosphor);border-color:#b9ff4552}.availability-remote,.availability-limited{color:var(--rust);border-color:#c96f4366}.model-footprint{color:#e8dfca63;font:8px/1 var(--mono);white-space:nowrap;align-self:end}.text-action{width:calc(100% - 30px);color:var(--rust);font:8px/1 var(--mono);text-align:left;cursor:pointer;background:0 0;border:0;border-top:1px dashed #c96f4359;margin:-3px 8px 9px 30px;padding:7px 0}.text-action span{float:right}.text-action.is-status{cursor:default;text-overflow:ellipsis;white-space:nowrap;display:block;overflow:hidden}.model-limit-reason{color:#e8dfca94;font:8px/1.35 var(--mono);margin:-4px 8px 9px 30px;display:block}.dflash-card{z-index:1;background:#111512b8;border:1px dashed #e8dfca3b;margin-top:auto;padding:13px 12px 11px;position:relative}.dflash-card.is-available{border-color:#b9ff457a}.dflash-heading{justify-content:space-between;gap:12px;display:flex}.dflash-heading h3{font:500 14px/1 var(--display);margin:7px 0 0}.experimental-stamp{color:var(--rust);font:700 7px/1 var(--mono);letter-spacing:.12em;text-transform:uppercase;border:1px solid;padding:3px 5px 2px;display:inline-block;transform:rotate(-2deg)}.switch{cursor:pointer;background:0 0;border:1px solid #e8dfca4d;border-radius:10px;width:35px;height:18px;padding:2px}.switch span{background:#e8dfca6b;border-radius:50%;width:12px;height:12px;transition:transform .17s,background .17s;display:block}.switch[aria-checked=true]{border-color:var(--phosphor)}.switch[aria-checked=true] span{background:var(--phosphor);transform:translate(17px)}.switch:disabled{opacity:.38}.dflash-card p{color:#e8dfca80;font:9px/1.45 var(--body);margin:9px 0 5px}.dflash-contract{color:#c96f43b8;font:7px/1.3 var(--mono);text-transform:uppercase;margin-bottom:11px;display:block}.dflash-axis{border-top:1px solid #e8dfca1f;justify-content:space-between;align-items:center;display:flex}.dflash-axis span{background:#e8dfca47;width:1px;height:5px;margin-top:-1px}.rail-footnote{z-index:1;color:#e8dfca4d;font:7px/1 var(--mono);letter-spacing:.03em;justify-content:space-between;gap:8px;margin-top:12px;display:flex;position:relative}.chat-surface{min-width:0;height:calc(100dvh - 76px);min-height:calc(100dvh - 76px);color:var(--ink);background:radial-gradient(ellipse at 82% 16%, transparent 0 77px, #1115120e 78px 79px, transparent 80px 89px, #1115120b 90px 91px, transparent 92px), linear-gradient(100deg, transparent 0 67%, #11151206 67.2% 67.35%, transparent 67.5%), var(--paper);grid-area:chat;grid-template-rows:auto 19px minmax(0,1fr) auto;animation:.56s cubic-bezier(.16,1,.3,1) .16s both rise-in;display:grid;position:relative;overflow:hidden}.chat-header{border-bottom:1px solid var(--line-dark);justify-content:space-between;align-items:center;gap:20px;min-height:82px;padding:17px clamp(21px,3vw,44px) 13px;display:flex}.chat-header h2{font:500 clamp(25px, 2.4vw, 37px)/1 var(--display);letter-spacing:-.045em;margin:0;display:inline}.botanical-subtitle{color:#11151275;font:italic 12px/1 var(--display);margin-left:10px}.chat-header-meta{color:#1115128c;font:8px/1 var(--mono);text-transform:uppercase;letter-spacing:.07em;grid-template-columns:repeat(2,auto);gap:5px 13px;display:grid}.header-load-button{color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 7px/1 var(--mono);letter-spacing:.06em;text-transform:uppercase;cursor:pointer;grid-column:span 2;justify-self:end;padding:6px 9px}.header-load-button:disabled{opacity:.42}.live-label{color:#3d551d;grid-column:span 2;justify-self:end;align-items:center;gap:7px;display:flex}.live-label i{background:#5d7e2e;width:5px;height:5px;box-shadow:0 0 0 3px #5a782b21}.live-label.is-dormant{color:#1115127a}.live-label.is-dormant i{background:#1115125c;box-shadow:0 0 0 3px #11151217}.runtime-error{z-index:4;color:var(--paper);border:1px solid var(--rust);background:#111512f5;justify-content:space-between;align-items:center;gap:16px;padding:11px 13px;display:flex;position:absolute;top:105px;left:clamp(21px,3vw,44px);right:clamp(21px,3vw,44px);box-shadow:4px 4px #c96f433d}.runtime-error div{gap:3px;display:grid}.runtime-error strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.runtime-error span{font:10px/1.35 var(--mono);overflow-wrap:anywhere}.runtime-error button{font:7px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:1px solid #e8dfca4d;flex:none;padding:5px 8px}.runtime-error button:disabled{cursor:wait;opacity:.5}.runtime-error.download-retry button{max-width:132px;line-height:1.25}.chat-ruler{color:#11151266;font:7px/1 var(--mono);background:repeating-linear-gradient(90deg,#0000 0 14px,#11151233 14px 15px) 0 100%/100% 4px no-repeat;border-bottom:1px solid #11151217;justify-content:space-between;padding-inline:clamp(21px,3vw,44px);display:flex}.transcript{scrollbar-color:#11151247 transparent;min-height:0;padding:20px clamp(21px,4vw,64px) 32px;overflow-y:auto}.transcript-date{color:#11151273;font:8px/1 var(--mono);letter-spacing:.1em;text-transform:uppercase;align-items:center;gap:10px;margin-bottom:22px;display:flex}.transcript-date:before,.transcript-date:after{content:"";background:#11151224;flex:1;height:1px}.transcript-empty{text-align:center;color:#11151280;max-width:430px;margin:12vh auto 0}.transcript-empty>span{color:#507324;font:42px/1 var(--display);display:block}.transcript-empty h3{color:var(--ink);font:400 25px/1 var(--display);margin:8px 0 6px}.transcript-empty p{margin:0;font-size:11px;line-height:1.5}.message{max-width:690px;margin-bottom:25px;animation:.43s cubic-bezier(.16,1,.3,1) both note-in}.message>header{color:#11151278;font:8px/1 var(--mono);text-transform:uppercase;letter-spacing:.07em;align-items:center;gap:8px;margin-bottom:8px;display:flex}.message>header strong{color:var(--ink)}.message>header time{margin-left:auto}.message-ordinal{color:#4d6728;min-width:22px}.message-content{font:15px/1.55 var(--body)}.message-content>p{margin:0}.markdown-content>:first-child{margin-top:0}.markdown-content>:last-child{margin-bottom:0}.markdown-content p,.markdown-content ul,.markdown-content ol,.markdown-content blockquote{margin:0 0 10px}.markdown-content ul,.markdown-content ol{padding-left:22px}.markdown-content h3,.markdown-content h4,.markdown-content h5{font-family:var(--display);margin:16px 0 7px;font-weight:500;line-height:1.1}.markdown-content blockquote{color:#111512ad;white-space:pre-wrap;border-left:2px solid #5073248c;padding-left:12px}.markdown-content code{font:.78em/1.4 var(--mono);background:#11151214;padding:1px 4px}.markdown-content pre{max-width:100%;color:var(--paper);background:var(--ink);border-left:2px solid var(--phosphor);margin:11px 0;padding:12px;position:relative;overflow:auto}.markdown-content pre code{white-space:pre;background:0 0;padding:0}.markdown-content a{color:#3f6417;text-underline-offset:2px;text-decoration-thickness:1px}.message-user .markdown-content code{background:#e8dfca1f}.reasoning-block{color:#1115129e;background:#11151209;border:1px dashed #1115123d;margin:0 0 11px;padding:8px 10px;font-size:11px}.reasoning-block summary{color:#4d6728;font:700 7px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;cursor:pointer}.reasoning-block[open] summary{margin-bottom:8px}.generation-placeholder{color:#11151273;margin:0;font-style:italic}.message-error{color:#8d321b;font:9px/1.4 var(--mono);margin:9px 0 0}.message.has-error{border-color:var(--rust)}.message-assistant{border-left:2px solid var(--ink);padding-left:22px}.message-assistant .message-content>p:first-letter{float:left;color:#38511d;font:42px/.75 var(--display);margin:1px 6px 0 0}.message-user{max-width:min(78%,610px);color:var(--paper);background:var(--ink);clip-path:polygon(0 0,calc(100% - 10px) 0,100% 10px,100% 100%,0 100%);margin-left:auto;padding:12px 15px 13px;box-shadow:5px 5px #1115121f}.message-user>header,.message-user>header strong{color:#e8dfca8f}.message-user .message-ordinal{color:var(--phosphor)}.message>footer{color:#1115126e;font:8px/1 var(--mono);border-top:1px dashed #11151230;gap:12px;margin-top:9px;padding-top:7px;display:flex}.stream-cursor{vertical-align:-2px;background:#507324;width:7px;height:14px;margin-left:4px;animation:.86s step-end infinite cursor-blink;display:inline-block}.tool-call{background:#1115120b;border:1px solid #1115122e;margin-top:13px}.tool-call summary{cursor:pointer;grid-template-columns:auto 1fr auto;align-items:center;gap:8px;padding:8px 10px;list-style:none;display:grid}.tool-call summary::-webkit-details-marker{display:none}.tool-call code{font:9px/1.4 var(--mono)}.tool-call summary>span:last-child{color:#1115127a;font:7px/1 var(--mono);text-transform:uppercase}.tool-state,.event-pip{background:var(--moss);border-radius:50%;width:6px;height:6px}.tool-state-complete,.event-complete{background:#55772c}.tool-state-running,.event-running{background:var(--phosphor);box-shadow:0 0 0 3px #5a782b26}.tool-state-error,.event-error{background:var(--rust)}.tool-call>div{border-top:1px solid #11151221;padding:9px 10px}.tool-call>div p{color:#11151294;margin:5px 0 0;font-size:10px}.composer{border-top:1px solid var(--line-dark);-webkit-backdrop-filter:blur(9px);backdrop-filter:blur(9px);background:#d8cdb4d1;padding:10px clamp(21px,3vw,44px) 18px;position:relative;box-shadow:0 -15px 38px #1115120f}.composer:before{content:"";background:repeating-linear-gradient(90deg, var(--ink) 0 1px, transparent 1px 12px);opacity:.23;height:5px;position:absolute;top:-5px;left:0;right:0}.composer-meta{color:#11151275;font:8px/1 var(--mono);justify-content:space-between;align-items:center;gap:12px;margin-bottom:7px;display:flex}.tool-toggle{color:#1115128a;font:700 8px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:0;align-items:center;gap:5px;padding:0;display:flex}.tool-toggle.is-on{color:#425f1f}.regenerate-button{color:#1115128a;font:700 8px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;border:0;margin-right:auto;padding:0}.regenerate-button:disabled{cursor:default;opacity:.45}.tool-toggle svg{fill:none;stroke:currentColor;stroke-width:1.3px;width:12px}.composer-field{border:1px solid var(--ink);background:#e8dfcad4;grid-template-columns:1fr auto;min-height:76px;display:grid;position:relative;box-shadow:4px 4px #1115121a}.composer-field label{color:#11151273;font:7px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase;position:absolute;top:7px;left:11px}.composer-field textarea{resize:none;min-width:0;color:var(--ink);font:14px/1.4 var(--body);background:0 0;border:0;outline:none;padding:23px 13px 9px}.composer-field textarea::placeholder{color:#11151257}.send-button{min-width:82px;color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 9px/1 var(--mono);text-transform:uppercase;cursor:pointer;justify-content:center;align-items:center;gap:7px;margin:7px;transition:transform .15s,box-shadow .15s,background .15s;display:flex}.send-button:hover:not(:disabled){box-shadow:3px 3px 0 var(--ink);transform:translate(-2px,-2px)}.send-button:disabled{opacity:.34}.send-button svg{fill:none;stroke:currentColor;stroke-width:1.4px;width:15px}.send-button.stop{color:var(--paper);background:var(--ink)}.send-button.stop span{background:var(--rust);width:7px;height:7px}.inspector{border-left:1px solid var(--line-light);scrollbar-color:#e8dfca2e transparent;grid-area:inspector;height:calc(100dvh - 76px);min-height:calc(100dvh - 76px);animation:.57s cubic-bezier(.16,1,.3,1) .22s both rise-in;overflow-y:auto}.inspector-section{border-bottom:1px solid var(--line-light);padding:20px 18px}.section-heading{align-items:center;margin-bottom:17px}.section-heading.compact{margin-bottom:14px}.section-heading>div{flex:1}.section-heading h2{font-size:20px}.sampling-label,.persistence{color:#e8dfca6b;font:7px/1 var(--mono);text-transform:uppercase;letter-spacing:.06em;align-items:center;gap:6px;display:flex}.sampling-label i{width:4px;height:4px}.speed-gauge{grid-template-columns:116px 1fr;align-items:center;gap:15px;display:grid}.gauge-ring{background:radial-gradient(circle, var(--ink) 0 51%, transparent 52%), conic-gradient(var(--phosphor) 0 var(--gauge), #e8dfca1c var(--gauge) 100%);border-radius:49% 51% 47% 53%;place-items:center;width:116px;height:116px;display:grid;transform:rotate(-9deg);box-shadow:inset 0 0 0 1px #e8dfca1a,0 0 30px #b9ff450e}.gauge-ring:after{content:"";border:1px dashed #e8dfca21;border-radius:45% 55% 52% 48%;width:128px;height:128px;position:absolute}.gauge-ring>div{z-index:1;justify-items:center;display:grid;position:relative;transform:rotate(9deg)}.gauge-ring strong{color:var(--phosphor);font:400 29px/1 var(--display)}.gauge-ring small{color:var(--paper-ghost);font:7px/1 var(--mono);text-transform:uppercase;margin-top:3px}.speed-gauge dl{margin:0}.speed-gauge dl div{border-bottom:1px solid var(--line-light);font:8px/1 var(--mono);justify-content:space-between;padding:7px 0;display:flex}.speed-gauge dt{color:#e8dfca6e}.speed-gauge dd{color:var(--paper);margin:0}.device-card{background:linear-gradient(90deg,#b9ff4514,#0000);border:1px solid #b9ff453b;grid-template-columns:auto 1fr auto;align-items:center;gap:10px;margin:19px 0 14px;padding:10px;display:grid}.device-icon{border:1px solid #b9ff4540;align-items:flex-end;gap:3px;width:28px;height:28px;padding:5px;display:flex}.device-icon i{background:var(--phosphor);width:4px}.device-icon i:first-child{opacity:.45;height:7px}.device-icon i:nth-child(2){height:15px}.device-icon i:nth-child(3){opacity:.7;height:11px}.device-card>div{gap:3px;display:grid}.device-card small,.memory-grid small,.meter small,.download-item small{color:#e8dfca6b;font:7px/1 var(--mono);text-transform:uppercase}.device-card strong{font:500 11px/1 var(--mono)}.device-card>span:last-child{color:var(--phosphor);font:600 13px/1 var(--mono);justify-items:end;display:grid}.device-card>span:last-child small{color:#e8dfca5e;margin-top:3px;font-size:6px}.meter{margin-top:13px}.meter>div:first-child{font:8px/1 var(--mono);justify-content:space-between;margin-bottom:6px;display:flex}.meter-track{border:1px solid #e8dfca36;height:5px;padding:1px}.meter-track>span{background:repeating-linear-gradient(90deg, var(--phosphor) 0 5px, transparent 5px 7px);height:100%;display:block}.memory-grid{background:var(--line-light);grid-template-columns:1fr 1fr;gap:1px;margin-top:14px;display:grid}.memory-grid>div{background:var(--ink);gap:6px;padding:10px;display:grid}.memory-grid strong{font:500 12px/1 var(--mono)}.event-log{gap:0;margin:0;padding:0;list-style:none;display:grid;position:relative}.event-log:before{content:"";background:var(--line-light);width:1px;position:absolute;top:5px;bottom:5px;left:3px}.event-log li{grid-template-columns:8px 64px 1fr;align-items:start;gap:8px;padding:7px 0;display:grid}.event-log li.event-empty{color:#e8dfca5c;font:8px/1.4 var(--mono);padding:5px 0;display:block}.event-pip{z-index:1;border:1px solid var(--ink);margin-top:2px}.event-log time{color:#e8dfca57;font:7px/1.2 var(--mono)}.event-log div{gap:3px;display:grid}.event-log strong{font:500 9px/1 var(--mono)}.event-log small{color:#e8dfca6b;font-size:8px}.persistence{border:1px solid var(--line-light);padding:3px 5px}.persistence.is-persistent{color:var(--phosphor);border-color:#b9ff4547}.download-item{border:1px solid var(--line-light);grid-template-columns:1fr auto;gap:8px;margin-top:16px;padding:11px 10px 10px;display:grid;position:relative}.download-item>div:first-child{gap:5px;display:grid}.download-item strong{font:500 10px/1 var(--mono)}.download-state{width:fit-content;color:var(--rust);font:700 6px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase}.state-downloading{color:var(--phosphor)}.download-percent{color:var(--phosphor);font:500 15px/1 var(--display)}.download-track{background:#e8dfca1a;grid-column:span 2;height:3px}.download-track span{background:var(--phosphor);height:100%;display:block;box-shadow:0 0 9px #b9ff4559}.artifact-close{border:1px solid var(--line-light);cursor:pointer;background:0 0;border-radius:50%;width:25px;height:25px}.artifact-empty{color:#e8dfca66;font:8px/1.5 var(--mono);margin:0}.artifact-pane{border:1px solid var(--line-light);background:#f8f5eb}.artifact-tabs{color:var(--paper);border-bottom:1px solid var(--line-light);background:var(--ink-raised);grid-template-columns:auto auto 1fr;align-items:center;display:grid}.artifact-tabs button{border:0;border-right:1px solid var(--line-light);font:7px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;padding:7px 9px}.artifact-tabs button[aria-selected=true]{color:var(--ink);background:var(--phosphor)}.artifact-tabs span{color:#e8dfca73;font:7px/1 var(--mono);text-align:right;text-overflow:ellipsis;white-space:nowrap;padding:0 8px;overflow:hidden}.artifact-pane iframe{background:#fff;border:0;width:100%;height:260px;display:block}.artifact-running{position:relative}.artifact-running>button{z-index:2;color:var(--paper);border:1px solid var(--rust);font:7px/1 var(--mono);text-transform:uppercase;cursor:pointer;background:#111512e6;padding:5px 7px;position:absolute;top:7px;right:7px}.artifact-launch{min-height:180px;color:var(--ink);text-align:center;align-content:center;justify-items:center;gap:13px;padding:20px;display:grid}.artifact-launch p{color:#1115129e;max-width:250px;font:9px/1.5 var(--mono);margin:0}.artifact-launch button{color:var(--ink);border:1px solid var(--ink);background:var(--phosphor);font:700 7px/1 var(--mono);text-transform:uppercase;cursor:pointer;padding:8px 10px}.artifact-code{height:260px;color:var(--paper);font:8px/1.45 var(--mono);white-space:pre-wrap;background:#0d100e;margin:0;padding:10px;overflow:auto}.settings-layer{z-index:50;-webkit-backdrop-filter:blur(4px);backdrop-filter:blur(4px);background:#080b09a3;justify-content:flex-end;animation:.16s both station-in;display:flex;position:fixed;inset:0}.settings-sheet{width:min(430px,100%);min-height:100%;color:var(--paper);border-left:1px solid var(--phosphor);background:repeating-radial-gradient(ellipse at 104% 101%, transparent 0 13px, #b9ff450e 13.5px 14.5px), var(--ink);padding:30px 27px;animation:.3s cubic-bezier(.16,1,.3,1) both sheet-in;overflow-y:auto;box-shadow:-24px 0 70px #00000052}.settings-sheet>header{border-bottom:1px solid var(--line-light);justify-content:space-between;align-items:flex-start;gap:20px;margin-bottom:37px;padding-bottom:20px;display:flex}.settings-sheet h2{font:400 31px/1 var(--display);margin:0}.sheet-close{border:1px solid var(--line-light);width:35px;height:35px;font:300 22px/1 var(--body);cursor:pointer;background:0 0;border-radius:50%}.sheet-close:hover{color:var(--phosphor);border-color:var(--phosphor)}.setting-group{border-bottom:1px solid var(--line-light);grid-template-columns:24px 1fr auto;align-items:center;gap:12px;padding:15px 0;display:grid}.setting-number{color:var(--phosphor);font:8px/1 var(--mono)}.setting-group>div,.range-setting>span:first-child{gap:5px;display:grid}.setting-group small,.range-setting small{color:#e8dfca6e;font:7px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.setting-group strong,.range-setting strong{font:500 13px/1 var(--mono)}.setting-locked{color:#e8dfca5c;font:7px/1 var(--mono);text-transform:uppercase}.setting-select select{max-width:112px;color:var(--paper);border:1px solid var(--line-light);background:var(--ink-raised);font:8px/1 var(--mono);border-radius:0;padding:6px 7px}.range-setting{gap:13px;margin-top:29px;display:grid}.range-setting>span:first-child{justify-content:space-between;align-items:end;display:flex}.range-setting input{appearance:none;background:linear-gradient(90deg, var(--phosphor) 0 35%, #e8dfca1c 35%);border:1px solid #e8dfca3d;border-radius:0;width:100%;height:4px;margin:0}.range-setting:first-of-type input{background:var(--phosphor)}.range-setting input::-webkit-slider-thumb{appearance:none;border:2px solid var(--ink);background:var(--phosphor);width:12px;height:18px;box-shadow:0 0 0 1px var(--phosphor);cursor:grab}.range-setting input::-moz-range-thumb{border:2px solid var(--ink);background:var(--phosphor);width:10px;height:16px;box-shadow:0 0 0 1px var(--phosphor);cursor:grab;border-radius:0}.range-ticks{pointer-events:none;justify-content:space-between;margin-top:-13px;display:flex}.range-limits{color:#e8dfca59;font:7px/1 var(--mono);justify-content:space-between;display:flex}.range-limits i{font-style:normal}.system-prompt-setting{gap:9px;margin-top:29px;display:grid}.system-prompt-setting>span{color:#e8dfca6e;font:7px/1 var(--mono);letter-spacing:.05em;text-transform:uppercase;justify-content:space-between;display:flex}.system-prompt-setting textarea{resize:vertical;color:var(--paper);border:1px solid var(--line-light);background:var(--ink-raised);font:10px/1.45 var(--mono);padding:10px}.settings-actions{grid-template-columns:1fr 1fr;gap:7px;margin-top:29px;display:grid}.settings-actions button{color:var(--paper);border:1px solid var(--line-light);font:7px/1.2 var(--mono);text-transform:uppercase;cursor:pointer;background:0 0;padding:9px}.settings-actions button:hover{color:var(--phosphor);border-color:var(--phosphor)}.settings-actions .danger-action{color:#e78a62;border-color:#c96f4373;grid-column:span 2}.boot-screen{min-height:100dvh;color:var(--paper);background:var(--ink);justify-content:center;align-items:center;gap:18px;padding:24px;display:flex}.boot-screen h1{color:var(--phosphor);font:400 34px/1 var(--display);margin:0}.boot-screen p:last-child{color:var(--paper-ghost);font:10px/1.5 var(--mono)}.range-ticks i{background:#e8dfca40;width:1px;height:5px}.settings-note{background:#b9ff450f;border:1px solid #b9ff4540;align-items:flex-start;gap:13px;margin-top:42px;padding:15px;display:flex}.settings-note .signal-dot{flex:0 0 7px;margin-top:6px}.settings-note p{color:#e8dfca94;margin:0;font-size:10px;line-height:1.5}.settings-note strong{color:var(--phosphor)}@keyframes station-in{0%{opacity:0}to{opacity:1}}@keyframes rise-in{0%{opacity:0;transform:translateY(11px)}to{opacity:1;transform:translateY(0)}}@keyframes note-in{0%{opacity:0;transform:translateY(7px)}to{opacity:1;transform:translateY(0)}}@keyframes cursor-blink{0%,52%{opacity:1}53%,to{opacity:0}}@keyframes sheet-in{0%{opacity:0;transform:translate(34px)}to{opacity:1;transform:translate(0)}}@media (width<=1279px){.station-header{grid-template-columns:minmax(320px,1fr) auto auto}.local-promise{min-width:auto}.local-promise small{display:none}.workspace-grid{grid-template-columns:260px minmax(500px,1fr);grid-template-areas:"models chat""inspector inspector"}.model-rail,.chat-surface{height:calc(100dvh - 76px);min-height:calc(100dvh - 76px)}.inspector{border-top:1px solid var(--phosphor);border-left:0;grid-template-columns:repeat(2,minmax(0,1fr));height:auto;min-height:auto;display:grid;overflow:visible}.inspector-section{border-right:1px solid var(--line-light);border-bottom:0}}@media (width<=820px){.station-header{min-height:70px;display:block}.brand-block{padding:11px 70px 11px 15px}.brand-mark{flex-basis:36px;width:36px;height:36px}.local-promise{border:0;border-top:1px solid var(--line-light);min-height:31px;padding:0 16px}.header-actions{position:absolute;top:10px;right:0}.station-clock{display:none}.workspace-grid{display:block}.model-rail,.chat-surface,.inspector{height:auto;min-height:auto}.model-rail{border-right:0;border-bottom:1px solid var(--phosphor);padding:18px 15px;display:block}.model-list{scroll-snap-type:x mandatory;gap:7px;padding-bottom:7px;display:flex;overflow-x:auto}.model-card{scroll-snap-align:start;min-width:205px}.dflash-card{margin-top:13px}.rail-footnote{display:none}.chat-surface{grid-template-rows:auto 19px minmax(450px,auto) auto;display:grid;overflow:visible}.transcript{padding-inline:22px;overflow:visible}.chat-header{padding:17px 19px}.chat-header-meta span:not(.live-label){display:none}.chat-header-meta .header-load-button{display:block}.live-label{grid-column:1}.composer{z-index:5;padding-inline:14px;position:sticky;bottom:0}.inspector{border-top:1px solid var(--phosphor);display:block}.inspector-section{border-right:0;border-bottom:1px solid var(--line-light)}}@media (width<=520px){.brand-block h1{font-size:24px}.brand-block .eyebrow,.local-promise strong{font-size:7px}.chat-header{align-items:flex-end}.botanical-subtitle{margin:4px 0 0;display:block}.message-user{max-width:91%}.composer-meta>span{display:none}.composer-field{grid-template-columns:1fr;padding-bottom:50px}.send-button{height:35px;position:absolute;bottom:0;right:0}.speed-gauge{grid-template-columns:106px 1fr}.gauge-ring{width:102px;height:102px}.gauge-ring:after{width:113px;height:113px}}@media (prefers-reduced-motion:reduce){*,:before,:after{scroll-behavior:auto!important;transition-duration:.01ms!important;animation-duration:.01ms!important;animation-iteration-count:1!important}}.bench-page{min-height:100dvh;color:var(--ink);background:var(--paper)}.bench-masthead{min-height:86px;color:var(--paper);border-bottom:1px solid var(--phosphor);background:var(--ink);grid-template-columns:minmax(310px,1fr) minmax(360px,1.2fr) auto;align-items:stretch;display:grid}.bench-masthead .brand-block h1{color:var(--paper)}.bench-masthead-note{border-inline:1px solid var(--line-light);background:linear-gradient(110deg,#b9ff4517,#0000 72%);flex-direction:column;justify-content:center;gap:5px;padding:14px 28px;display:flex}.bench-masthead-note strong{color:var(--phosphor);font:700 9px/1 var(--mono);letter-spacing:.11em;text-transform:uppercase}.bench-masthead-note span{color:#e8dfca8c;max-width:520px;font:10px/1.35 var(--body)}.bench-return{color:var(--paper);font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;align-items:center;gap:8px;padding:0 25px;text-decoration:none;display:flex}.bench-return:hover{color:var(--phosphor)}.bench-layout{grid-template-columns:minmax(275px,330px) minmax(0,1fr);min-height:calc(100dvh - 86px);display:grid}.bench-controls{color:var(--paper);border-right:1px solid var(--phosphor);background:repeating-radial-gradient(ellipse at -12% 102%, transparent 0 24px, #b9ff450b 25px 26px), var(--ink);padding:28px 22px}.bench-controls h2{font:400 30px/1 var(--display);letter-spacing:-.03em;margin:0}.bench-control-intro{color:#e8dfca80;margin:9px 0 22px;font-size:11px;line-height:1.45}.bench-controls form{gap:15px;display:grid}.bench-field{gap:6px;min-width:0;display:grid}.bench-field>span{color:#e8dfcac7;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.bench-field select,.bench-field input{width:100%;min-width:0;height:39px;color:var(--paper);background:var(--ink-raised);font:10px/1 var(--mono);border:1px solid #e8dfca38;border-radius:0;outline:0;padding:0 10px}.bench-field input::placeholder{color:#e8dfca40}.bench-field select:disabled,.bench-field input:disabled{opacity:.5}.bench-field small{color:#e8dfca59;font:8px/1.2 var(--mono)}.bench-field-pair{grid-template-columns:repeat(2,minmax(0,1fr));gap:10px;display:grid}.bench-experimental{background:#c96f430d;border:1px dashed #c96f437a;padding:11px}.bench-experimental summary{color:#e19a76;font:700 8px/1 var(--mono);letter-spacing:.07em;text-transform:uppercase;cursor:pointer}.bench-experimental>p{color:#e8dfca75;margin:10px 0;font-size:9px;line-height:1.4}.bench-locks{color:#e8dfca70;font:7px/1.35 var(--mono);gap:5px;margin:10px 0 0;display:grid}.bench-locks div{justify-content:space-between;gap:8px;display:flex}.bench-locks dt,.bench-locks dd{margin:0}.bench-locks dt{color:#e8dfcab8}.bench-gate{border-left:2px solid var(--rust);background:#c96f4314;gap:5px;padding:10px;display:grid}.bench-gate.pass{border-color:var(--phosphor);background:var(--phosphor-dim)}.bench-gate strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.06em;text-transform:uppercase}.bench-gate.pass strong{color:var(--phosphor)}.bench-gate span{color:#e8dfca80;font:8px/1.4 var(--mono);overflow-wrap:anywhere}.bench-run-actions{grid-template-columns:1fr auto;gap:8px;display:grid}.bench-run-button,.bench-stop-button,.bench-section-heading button{border:1px solid var(--ink);background:var(--phosphor);min-height:38px;font:800 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase;cursor:pointer;border-radius:0;padding:0 13px}.bench-run-button:disabled{opacity:.35}.bench-stop-button{color:#ffb08a;border-color:var(--rust);background:0 0}.bench-workbench{background:linear-gradient(90deg, #1115120b 1px, transparent 1px) 0 0 / 22px 22px, linear-gradient(#1115120a 1px, transparent 1px) 0 0 / 22px 22px, var(--paper);min-width:0;padding:clamp(26px,4vw,58px)}.bench-hero{border-bottom:1px solid var(--line-dark);grid-template-columns:minmax(320px,1.3fr) minmax(260px,.7fr);align-items:end;gap:34px;padding-bottom:29px;display:grid}.bench-hero h2{max-width:690px;font:400 clamp(42px, 6vw, 82px)/.84 var(--display);letter-spacing:-.065em;margin:0}.bench-hero h2 em{color:#4f7125;font-weight:400}.bench-hero>p{color:#11151294;max-width:430px;margin:0;font-size:11px;line-height:1.55}.bench-progress{gap:8px;padding:16px 0;display:grid}.bench-progress>div{font:8px/1 var(--mono);letter-spacing:.05em;text-transform:uppercase;justify-content:space-between;gap:20px;display:flex}.bench-progress>div span{color:#11151273}.bench-progress-track{background:#11151221;height:3px}.bench-progress-track i{background:#527426;height:100%;transition:width .18s;display:block}.bench-error{color:var(--paper);border-left:3px solid var(--rust);background:var(--ink);gap:5px;margin-bottom:18px;padding:12px 14px;display:grid}.bench-error strong{color:#ff9a68;font:700 8px/1 var(--mono);letter-spacing:.08em;text-transform:uppercase}.bench-error span{font:9px/1.4 var(--mono);overflow-wrap:anywhere}.bench-core-section{margin-top:13px}.bench-section-heading{justify-content:space-between;align-items:end;gap:20px;margin-bottom:13px;display:flex}.bench-section-heading h3,.bench-output h3{font:500 24px/1 var(--display);margin:0}.bench-section-heading button{min-height:31px;color:var(--paper);border-color:var(--ink);background:var(--ink)}.bench-core-sample{border:1px solid var(--ink);background:var(--paper-deep);grid-template-columns:repeat(4,minmax(0,1fr));display:grid;position:relative;overflow:hidden}.bench-core-sample:before{content:"";pointer-events:none;opacity:.35;background:repeating-radial-gradient(#0000 0 34px,#1115122b 35px 36px);position:absolute;inset:-130% -5%}.bench-core-sample article{background:#e8dfca91;border-right:1px solid #11151242;flex-direction:column;justify-content:space-between;min-height:128px;padding:17px;display:flex;position:relative}.bench-core-sample article:nth-child(2n){background:#d8cdb499}.bench-core-sample article:last-child{color:var(--paper);background:#111512e8;border-right:0}.bench-core-sample article>span{font:800 8px/1 var(--mono);letter-spacing:.09em;text-transform:uppercase}.bench-core-sample article strong{font:400 clamp(28px, 3vw, 43px)/1 var(--display);letter-spacing:-.04em;margin-block:auto}.bench-core-sample article small{color:#11151280;font:7px/1.2 var(--mono);text-transform:uppercase}.bench-core-sample article:last-child small{color:#e8dfca7a}.bench-detail-grid{grid-template-columns:repeat(2,minmax(0,1fr));gap:12px;margin-top:12px;display:grid}.bench-data-card{border:1px solid var(--line-dark);background:#e8dfcaad;padding:16px}.bench-runtime-card,.bench-tuning-card{grid-column:span 2}.bench-data-card dl{grid-template-columns:repeat(2,minmax(0,1fr));gap:12px 26px;margin:0;display:grid}.bench-runtime-card dl{grid-template-columns:minmax(180px,1.5fr) repeat(4,minmax(100px,.7fr))}.bench-tuning-card dl{grid-template-columns:repeat(4,minmax(0,1fr))}.bench-data-card dl div{border-top:1px solid #11151221;gap:4px;min-width:0;padding-top:8px;display:grid}.bench-data-card dt{color:#11151273;font:7px/1 var(--mono);letter-spacing:.07em;text-transform:uppercase}.bench-data-card dd{min-width:0;font:500 16px/1.2 var(--body);text-overflow:ellipsis;white-space:nowrap;margin:0;overflow:hidden}.bench-output{border:1px dashed #1115124d;grid-template-columns:repeat(2,minmax(0,1fr));gap:34px;margin-top:12px;padding:21px;display:grid}.bench-output>div:last-child{border-left:1px solid var(--line-dark);padding-left:34px}.bench-output p:not(.eyebrow){color:#11151299;white-space:pre-wrap;margin:8px 0 0;font-size:11px;line-height:1.5}@media (width<=1050px){.bench-masthead{grid-template-columns:minmax(310px,1fr) auto}.bench-masthead-note{display:none}.bench-layout{grid-template-columns:285px minmax(0,1fr)}.bench-core-sample{grid-template-columns:repeat(2,minmax(0,1fr))}.bench-core-sample article:nth-child(2){border-right:0}.bench-core-sample article:nth-child(-n+2){border-bottom:1px solid #11151242}.bench-runtime-card dl{grid-template-columns:repeat(2,minmax(0,1fr))}}@media (width<=760px){.bench-masthead{align-items:center;display:flex}.bench-masthead .brand-block{flex:1;min-width:0}.bench-return{padding:0 15px;font-size:0}.bench-return span{font-size:17px}.bench-layout{display:block}.bench-controls{border-right:0;border-bottom:1px solid var(--phosphor)}.bench-workbench{padding:28px 16px}.bench-hero,.bench-output{grid-template-columns:1fr;gap:18px}.bench-hero h2{font-size:clamp(42px,14vw,68px)}.bench-detail-grid{display:block}.bench-data-card+.bench-data-card{margin-top:10px}.bench-output>div:last-child{border-top:1px solid var(--line-dark);border-left:0;padding-top:18px;padding-left:0}}@media (width<=480px){.bench-masthead .brand-mark{display:none}.bench-core-sample,.bench-data-card dl,.bench-runtime-card dl,.bench-tuning-card dl{grid-template-columns:1fr}.bench-core-sample article{border-bottom:1px solid #11151242;border-right:0;min-height:105px}.bench-core-sample article:last-child{border-bottom:0}}
|
dist/assets/index-jtL5gUVS.js
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
dist/assets/{worker-Dh0o2G2j.js → worker-gvajdY5k.js}
RENAMED
|
The diff for this file is too large to render.
See raw diff
|
|
|
dist/index.html
CHANGED
|
@@ -10,8 +10,8 @@
|
|
| 10 |
content="Bonsai local inference field station — private WebGPU language models in your browser."
|
| 11 |
/>
|
| 12 |
<title>Bonsai Field Station</title>
|
| 13 |
-
<script type="module" crossorigin src="/assets/index-
|
| 14 |
-
<link rel="stylesheet" crossorigin href="/assets/index-
|
| 15 |
</head>
|
| 16 |
<body>
|
| 17 |
<div id="root"></div>
|
|
|
|
| 10 |
content="Bonsai local inference field station — private WebGPU language models in your browser."
|
| 11 |
/>
|
| 12 |
<title>Bonsai Field Station</title>
|
| 13 |
+
<script type="module" crossorigin src="/assets/index-jtL5gUVS.js"></script>
|
| 14 |
+
<link rel="stylesheet" crossorigin href="/assets/index-C-2tm91j.css">
|
| 15 |
</head>
|
| 16 |
<body>
|
| 17 |
<div id="root"></div>
|
scripts/verify-wllama-assets.mjs
CHANGED
|
@@ -40,7 +40,14 @@ if (!declarations.includes('offload_token_embedding?: boolean')) {
|
|
| 40 |
if (!declarations.includes('n_ubatch?: number')) {
|
| 41 |
throw new Error('Vendored Bonsai wllama declarations lack n_ubatch');
|
| 42 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
const moduleSource = await readFile(resolve(root, 'vendor/wllama-bonsai/esm/index.js'), 'utf8');
|
| 44 |
if (!moduleSource.includes('n_ubatch: params.n_ubatch')) {
|
| 45 |
throw new Error('Vendored Bonsai wllama module does not forward n_ubatch');
|
| 46 |
}
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
if (!declarations.includes('n_ubatch?: number')) {
|
| 41 |
throw new Error('Vendored Bonsai wllama declarations lack n_ubatch');
|
| 42 |
}
|
| 43 |
+
const wllamaDeclarations = await readFile(resolve(root, 'vendor/wllama-bonsai/esm/wllama.d.ts'), 'utf8');
|
| 44 |
+
if (!wllamaDeclarations.includes('forceCompat?: boolean')) {
|
| 45 |
+
throw new Error('Vendored Bonsai wllama declarations lack forceCompat');
|
| 46 |
+
}
|
| 47 |
const moduleSource = await readFile(resolve(root, 'vendor/wllama-bonsai/esm/index.js'), 'utf8');
|
| 48 |
if (!moduleSource.includes('n_ubatch: params.n_ubatch')) {
|
| 49 |
throw new Error('Vendored Bonsai wllama module does not forward n_ubatch');
|
| 50 |
}
|
| 51 |
+
if (!moduleSource.includes('const forceCompat = this.config.forceCompat === true')) {
|
| 52 |
+
throw new Error('Vendored Bonsai wllama module does not implement forceCompat');
|
| 53 |
+
}
|
src/app/styles.css
CHANGED
|
@@ -2590,7 +2590,8 @@ button:disabled {
|
|
| 2590 |
background: rgba(232, 223, 202, 0.68);
|
| 2591 |
}
|
| 2592 |
|
| 2593 |
-
.bench-runtime-card
|
|
|
|
| 2594 |
grid-column: span 2;
|
| 2595 |
}
|
| 2596 |
|
|
@@ -2602,7 +2603,11 @@ button:disabled {
|
|
| 2602 |
}
|
| 2603 |
|
| 2604 |
.bench-runtime-card dl {
|
| 2605 |
-
grid-template-columns: minmax(180px, 1.5fr) repeat(
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2606 |
}
|
| 2607 |
|
| 2608 |
.bench-data-card dl div {
|
|
@@ -2747,7 +2752,8 @@ button:disabled {
|
|
| 2747 |
|
| 2748 |
.bench-core-sample,
|
| 2749 |
.bench-data-card dl,
|
| 2750 |
-
.bench-runtime-card dl
|
|
|
|
| 2751 |
grid-template-columns: 1fr;
|
| 2752 |
}
|
| 2753 |
|
|
|
|
| 2590 |
background: rgba(232, 223, 202, 0.68);
|
| 2591 |
}
|
| 2592 |
|
| 2593 |
+
.bench-runtime-card,
|
| 2594 |
+
.bench-tuning-card {
|
| 2595 |
grid-column: span 2;
|
| 2596 |
}
|
| 2597 |
|
|
|
|
| 2603 |
}
|
| 2604 |
|
| 2605 |
.bench-runtime-card dl {
|
| 2606 |
+
grid-template-columns: minmax(180px, 1.5fr) repeat(4, minmax(100px, 0.7fr));
|
| 2607 |
+
}
|
| 2608 |
+
|
| 2609 |
+
.bench-tuning-card dl {
|
| 2610 |
+
grid-template-columns: repeat(4, minmax(0, 1fr));
|
| 2611 |
}
|
| 2612 |
|
| 2613 |
.bench-data-card dl div {
|
|
|
|
| 2752 |
|
| 2753 |
.bench-core-sample,
|
| 2754 |
.bench-data-card dl,
|
| 2755 |
+
.bench-runtime-card dl,
|
| 2756 |
+
.bench-tuning-card dl {
|
| 2757 |
grid-template-columns: 1fr;
|
| 2758 |
}
|
| 2759 |
|
src/bench/BenchApp.tsx
CHANGED
|
@@ -4,6 +4,9 @@ import {
|
|
| 4 |
EngineClientError,
|
| 5 |
evaluateModelGate,
|
| 6 |
loadModelManifestV2,
|
|
|
|
|
|
|
|
|
|
| 7 |
type EngineCapabilities,
|
| 8 |
type ModelManifestV2,
|
| 9 |
type ModelTierId,
|
|
@@ -21,6 +24,7 @@ import {
|
|
| 21 |
|
| 22 |
const MANIFEST_PATH = 'manifest/models.json';
|
| 23 |
const DEFAULT_MODEL: ModelTierId = '1_7b';
|
|
|
|
| 24 |
|
| 25 |
interface RunProgress {
|
| 26 |
label: string;
|
|
@@ -57,7 +61,7 @@ function throwIfAborted(signal: AbortSignal): void {
|
|
| 57 |
function parseOptionalInteger(value: string): number | null {
|
| 58 |
if (value.trim() === '') return null;
|
| 59 |
const parsed = Number(value);
|
| 60 |
-
return Number.isSafeInteger(parsed) && parsed > 0 ? parsed : null;
|
| 61 |
}
|
| 62 |
|
| 63 |
function formatDuration(value: number | null | undefined): string {
|
|
@@ -91,6 +95,9 @@ export function BenchApp() {
|
|
| 91 |
const [maxTokens, setMaxTokens] = useState(64);
|
| 92 |
const [nBatchInput, setNBatchInput] = useState('');
|
| 93 |
const [nUbatchInput, setNUbatchInput] = useState('');
|
|
|
|
|
|
|
|
|
|
| 94 |
const [running, setRunning] = useState(false);
|
| 95 |
const [progress, setProgress] = useState<RunProgress>({
|
| 96 |
label: 'Waiting for an operator-initiated run',
|
|
@@ -99,7 +106,8 @@ export function BenchApp() {
|
|
| 99 |
});
|
| 100 |
const [report, setReport] = useState<BenchmarkReport | null>(null);
|
| 101 |
const [output, setOutput] = useState('');
|
| 102 |
-
const [
|
|
|
|
| 103 |
const operationRef = useRef<AbortController | null>(null);
|
| 104 |
|
| 105 |
useEffect(() => {
|
|
@@ -125,7 +133,7 @@ export function BenchApp() {
|
|
| 125 |
});
|
| 126 |
}).catch((bootstrapError) => {
|
| 127 |
if (cancelled || controller.signal.aborted) return;
|
| 128 |
-
|
| 129 |
setProgress((current) => ({ ...current, label: 'Runtime inspection failed' }));
|
| 130 |
});
|
| 131 |
return () => {
|
|
@@ -156,17 +164,44 @@ export function BenchApp() {
|
|
| 156 |
: model?.contextLength ?? 4_096;
|
| 157 |
const nBatch = parseOptionalInteger(nBatchInput);
|
| 158 |
const nUbatch = parseOptionalInteger(nUbatchInput);
|
| 159 |
-
const
|
| 160 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 161 |
const configurationValid = Number.isSafeInteger(contextSize)
|
| 162 |
&& contextSize > 0
|
| 163 |
&& contextSize <= contextLimit
|
| 164 |
&& Number.isSafeInteger(maxTokens)
|
| 165 |
&& maxTokens >= 8
|
| 166 |
&& maxTokens <= 512
|
| 167 |
-
&&
|
| 168 |
const canRun = Boolean(manifest && capabilities && model && deviceGate?.allowed && configurationValid);
|
| 169 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 170 |
const handleModelChange = (nextId: ModelTierId) => {
|
| 171 |
if (running || !manifest) return;
|
| 172 |
const next = manifest.models.find((candidate) => candidate.id === nextId);
|
|
@@ -174,9 +209,7 @@ export function BenchApp() {
|
|
| 174 |
setModelId(nextId);
|
| 175 |
setContextSize(next.defaultContext);
|
| 176 |
if (!next.cpuFallback && backend === 'wasm') setBackend('auto');
|
| 177 |
-
|
| 178 |
-
setOutput('');
|
| 179 |
-
setError(null);
|
| 180 |
};
|
| 181 |
|
| 182 |
const runBenchmark = () => {
|
|
@@ -186,7 +219,7 @@ export function BenchApp() {
|
|
| 186 |
setRunning(true);
|
| 187 |
setReport(null);
|
| 188 |
setOutput('');
|
| 189 |
-
|
| 190 |
const startedAt = new Date().toISOString();
|
| 191 |
|
| 192 |
const run = async () => {
|
|
@@ -209,9 +242,13 @@ export function BenchApp() {
|
|
| 209 |
modelId: model.id,
|
| 210 |
backend,
|
| 211 |
contextSize,
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 215 |
};
|
| 216 |
|
| 217 |
const coldStarted = performance.now();
|
|
@@ -252,6 +289,11 @@ export function BenchApp() {
|
|
| 252 |
if (coldLoad.backend !== warmLoad.backend) {
|
| 253 |
throw new Error(`Backend changed between cold and warm loads (${coldLoad.backend} → ${warmLoad.backend}).`);
|
| 254 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 255 |
throwIfAborted(controller.signal);
|
| 256 |
|
| 257 |
let firstTokenAt: number | null = null;
|
|
@@ -294,8 +336,6 @@ export function BenchApp() {
|
|
| 294 |
requestedBackend: backend,
|
| 295 |
contextSize: warmLoad.context.size,
|
| 296 |
maxTokens,
|
| 297 |
-
nBatch,
|
| 298 |
-
nUbatch,
|
| 299 |
coldLoadMs,
|
| 300 |
warmLoadMs,
|
| 301 |
coldCachedBytes,
|
|
@@ -321,7 +361,7 @@ export function BenchApp() {
|
|
| 321 |
setProgress({ label: 'Benchmark stopped by operator', loadedBytes: 0, totalBytes: 0 });
|
| 322 |
return;
|
| 323 |
}
|
| 324 |
-
|
| 325 |
setProgress({ label: 'Benchmark failed safely · no report exported', loadedBytes: 0, totalBytes: 0 });
|
| 326 |
}).finally(() => {
|
| 327 |
if (operationRef.current === controller) operationRef.current = null;
|
|
@@ -379,8 +419,7 @@ export function BenchApp() {
|
|
| 379 |
disabled={running}
|
| 380 |
onChange={(event) => {
|
| 381 |
setBackend(event.target.value as RequestedBackend);
|
| 382 |
-
|
| 383 |
-
setOutput('');
|
| 384 |
}}
|
| 385 |
>
|
| 386 |
<option value="auto">Auto · gated</option>
|
|
@@ -402,7 +441,7 @@ export function BenchApp() {
|
|
| 402 |
disabled={running}
|
| 403 |
onChange={(event) => {
|
| 404 |
setContextSize(Number(event.target.value));
|
| 405 |
-
|
| 406 |
}}
|
| 407 |
/>
|
| 408 |
<small>Max {contextLimit.toLocaleString()}</small>
|
|
@@ -418,7 +457,7 @@ export function BenchApp() {
|
|
| 418 |
disabled={running}
|
| 419 |
onChange={(event) => {
|
| 420 |
setMaxTokens(Number(event.target.value));
|
| 421 |
-
|
| 422 |
}}
|
| 423 |
/>
|
| 424 |
<small>8–512</small>
|
|
@@ -426,21 +465,22 @@ export function BenchApp() {
|
|
| 426 |
</div>
|
| 427 |
|
| 428 |
<details className="bench-experimental">
|
| 429 |
-
<summary>Experimental
|
| 430 |
-
<p>Blank
|
| 431 |
<div className="bench-field-pair">
|
| 432 |
<label className="bench-field">
|
| 433 |
<span>nBatch</span>
|
| 434 |
<input
|
| 435 |
type="number"
|
| 436 |
min="1"
|
|
|
|
| 437 |
step="1"
|
| 438 |
placeholder="runtime default"
|
| 439 |
value={nBatchInput}
|
| 440 |
disabled={running}
|
| 441 |
onChange={(event) => {
|
| 442 |
setNBatchInput(event.target.value);
|
| 443 |
-
|
| 444 |
}}
|
| 445 |
/>
|
| 446 |
</label>
|
|
@@ -449,23 +489,75 @@ export function BenchApp() {
|
|
| 449 |
<input
|
| 450 |
type="number"
|
| 451 |
min="1"
|
|
|
|
| 452 |
step="1"
|
| 453 |
placeholder="runtime default"
|
| 454 |
value={nUbatchInput}
|
| 455 |
disabled={running}
|
| 456 |
onChange={(event) => {
|
| 457 |
setNUbatchInput(event.target.value);
|
| 458 |
-
|
| 459 |
}}
|
| 460 |
/>
|
| 461 |
</label>
|
| 462 |
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 463 |
<dl className="bench-locks">
|
| 464 |
-
<div><dt>
|
| 465 |
-
<div><dt>
|
| 466 |
</dl>
|
| 467 |
</details>
|
| 468 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 469 |
<div className={`bench-gate ${deviceGate?.allowed ? 'pass' : 'fail'}`}>
|
| 470 |
<strong>{deviceGate?.allowed ? 'Device preflight passed' : 'Device preflight blocked'}</strong>
|
| 471 |
<span>
|
|
@@ -510,7 +602,8 @@ export function BenchApp() {
|
|
| 510 |
<span className="bench-progress-track"><i style={{ width: `${progressPercent}%` }} /></span>
|
| 511 |
</section>
|
| 512 |
|
| 513 |
-
{
|
|
|
|
| 514 |
|
| 515 |
<section className="bench-core-section" aria-label="Benchmark measurements">
|
| 516 |
<div className="bench-section-heading">
|
|
@@ -567,6 +660,16 @@ export function BenchApp() {
|
|
| 567 |
<div><dt>wllama</dt><dd title={report?.runtime.wllamaRevision}>{report?.runtime.wllamaRevision.slice(0, 12) ?? '—'}</dd></div>
|
| 568 |
<div><dt>llama.cpp</dt><dd title={report?.runtime.llamaCppRevision}>{report?.runtime.llamaCppRevision.slice(0, 12) ?? '—'}</dd></div>
|
| 569 |
<div><dt>Weights</dt><dd title={report?.runtime.manifestRevision ?? undefined}>{report?.runtime.manifestRevision?.slice(0, 12) ?? '—'}</dd></div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 570 |
</dl>
|
| 571 |
</article>
|
| 572 |
</section>
|
|
@@ -579,7 +682,7 @@ export function BenchApp() {
|
|
| 579 |
</div>
|
| 580 |
<div>
|
| 581 |
<p className="eyebrow ink">Generated text · excluded from export</p>
|
| 582 |
-
<p>{output || 'Run the assay to inspect the generated specimen.'}</p>
|
| 583 |
</div>
|
| 584 |
</section>
|
| 585 |
</main>
|
|
|
|
| 4 |
EngineClientError,
|
| 5 |
evaluateModelGate,
|
| 6 |
loadModelManifestV2,
|
| 7 |
+
type BenchmarkFlashMode,
|
| 8 |
+
type BenchmarkKvCacheType,
|
| 9 |
+
type BenchmarkWasmFlavor,
|
| 10 |
type EngineCapabilities,
|
| 11 |
type ModelManifestV2,
|
| 12 |
type ModelTierId,
|
|
|
|
| 24 |
|
| 25 |
const MANIFEST_PATH = 'manifest/models.json';
|
| 26 |
const DEFAULT_MODEL: ModelTierId = '1_7b';
|
| 27 |
+
const MAX_GLUE_INT = 2_147_483_647;
|
| 28 |
|
| 29 |
interface RunProgress {
|
| 30 |
label: string;
|
|
|
|
| 61 |
function parseOptionalInteger(value: string): number | null {
|
| 62 |
if (value.trim() === '') return null;
|
| 63 |
const parsed = Number(value);
|
| 64 |
+
return Number.isSafeInteger(parsed) && parsed > 0 && parsed <= MAX_GLUE_INT ? parsed : null;
|
| 65 |
}
|
| 66 |
|
| 67 |
function formatDuration(value: number | null | undefined): string {
|
|
|
|
| 95 |
const [maxTokens, setMaxTokens] = useState(64);
|
| 96 |
const [nBatchInput, setNBatchInput] = useState('');
|
| 97 |
const [nUbatchInput, setNUbatchInput] = useState('');
|
| 98 |
+
const [flashMode, setFlashMode] = useState<BenchmarkFlashMode>('off');
|
| 99 |
+
const [kvCacheType, setKvCacheType] = useState<BenchmarkKvCacheType>('f16');
|
| 100 |
+
const [wasmFlavor, setWasmFlavor] = useState<BenchmarkWasmFlavor>('auto');
|
| 101 |
const [running, setRunning] = useState(false);
|
| 102 |
const [progress, setProgress] = useState<RunProgress>({
|
| 103 |
label: 'Waiting for an operator-initiated run',
|
|
|
|
| 106 |
});
|
| 107 |
const [report, setReport] = useState<BenchmarkReport | null>(null);
|
| 108 |
const [output, setOutput] = useState('');
|
| 109 |
+
const [bootstrapError, setBootstrapError] = useState<string | null>(null);
|
| 110 |
+
const [runError, setRunError] = useState<string | null>(null);
|
| 111 |
const operationRef = useRef<AbortController | null>(null);
|
| 112 |
|
| 113 |
useEffect(() => {
|
|
|
|
| 133 |
});
|
| 134 |
}).catch((bootstrapError) => {
|
| 135 |
if (cancelled || controller.signal.aborted) return;
|
| 136 |
+
setBootstrapError(errorMessage(bootstrapError));
|
| 137 |
setProgress((current) => ({ ...current, label: 'Runtime inspection failed' }));
|
| 138 |
});
|
| 139 |
return () => {
|
|
|
|
| 164 |
: model?.contextLength ?? 4_096;
|
| 165 |
const nBatch = parseOptionalInteger(nBatchInput);
|
| 166 |
const nUbatch = parseOptionalInteger(nUbatchInput);
|
| 167 |
+
const batchTuningError = nBatchInput.trim() !== '' && nBatch === null
|
| 168 |
+
? `nBatch must be an integer from 1 to ${MAX_GLUE_INT.toLocaleString()}.`
|
| 169 |
+
: nUbatchInput.trim() !== '' && nUbatch === null
|
| 170 |
+
? `nUbatch must be an integer from 1 to ${MAX_GLUE_INT.toLocaleString()}.`
|
| 171 |
+
: nBatch !== null && nUbatch !== null && nUbatch > nBatch
|
| 172 |
+
? 'nUbatch must not exceed nBatch.'
|
| 173 |
+
: null;
|
| 174 |
+
const runtimeTuningError = flashMode === 'off' && kvCacheType !== 'f16'
|
| 175 |
+
? 'Quantized KV requires Flash Attention auto.'
|
| 176 |
+
: flashMode === 'auto' && backend !== 'webgpu'
|
| 177 |
+
? 'Flash Attention experiments require the explicit WebGPU backend.'
|
| 178 |
+
: null;
|
| 179 |
+
const wasmTuningError = wasmFlavor === 'jspi' && capabilities?.runtime.wasmFlavor === 'compat'
|
| 180 |
+
? 'JSPI is unavailable in this browser; use auto or compat.'
|
| 181 |
+
: null;
|
| 182 |
+
const configurationTuningError = batchTuningError ?? runtimeTuningError ?? wasmTuningError;
|
| 183 |
const configurationValid = Number.isSafeInteger(contextSize)
|
| 184 |
&& contextSize > 0
|
| 185 |
&& contextSize <= contextLimit
|
| 186 |
&& Number.isSafeInteger(maxTokens)
|
| 187 |
&& maxTokens >= 8
|
| 188 |
&& maxTokens <= 512
|
| 189 |
+
&& configurationTuningError === null;
|
| 190 |
const canRun = Boolean(manifest && capabilities && model && deviceGate?.allowed && configurationValid);
|
| 191 |
|
| 192 |
+
const clearPreviousRun = () => {
|
| 193 |
+
setReport(null);
|
| 194 |
+
setOutput('');
|
| 195 |
+
setRunError(null);
|
| 196 |
+
if (manifest && capabilities) {
|
| 197 |
+
setProgress({
|
| 198 |
+
label: 'Ready · settings changed; run again to produce matching evidence',
|
| 199 |
+
loadedBytes: 0,
|
| 200 |
+
totalBytes: 0,
|
| 201 |
+
});
|
| 202 |
+
}
|
| 203 |
+
};
|
| 204 |
+
|
| 205 |
const handleModelChange = (nextId: ModelTierId) => {
|
| 206 |
if (running || !manifest) return;
|
| 207 |
const next = manifest.models.find((candidate) => candidate.id === nextId);
|
|
|
|
| 209 |
setModelId(nextId);
|
| 210 |
setContextSize(next.defaultContext);
|
| 211 |
if (!next.cpuFallback && backend === 'wasm') setBackend('auto');
|
| 212 |
+
clearPreviousRun();
|
|
|
|
|
|
|
| 213 |
};
|
| 214 |
|
| 215 |
const runBenchmark = () => {
|
|
|
|
| 219 |
setRunning(true);
|
| 220 |
setReport(null);
|
| 221 |
setOutput('');
|
| 222 |
+
setRunError(null);
|
| 223 |
const startedAt = new Date().toISOString();
|
| 224 |
|
| 225 |
const run = async () => {
|
|
|
|
| 242 |
modelId: model.id,
|
| 243 |
backend,
|
| 244 |
contextSize,
|
| 245 |
+
benchmarkTuning: {
|
| 246 |
+
...(nBatch === null ? {} : { nBatch }),
|
| 247 |
+
...(nUbatch === null ? {} : { nUbatch }),
|
| 248 |
+
flashMode,
|
| 249 |
+
kvCacheType,
|
| 250 |
+
wasmFlavor,
|
| 251 |
+
},
|
| 252 |
};
|
| 253 |
|
| 254 |
const coldStarted = performance.now();
|
|
|
|
| 289 |
if (coldLoad.backend !== warmLoad.backend) {
|
| 290 |
throw new Error(`Backend changed between cold and warm loads (${coldLoad.backend} → ${warmLoad.backend}).`);
|
| 291 |
}
|
| 292 |
+
if (coldLoad.tuning.observed.wasmFlavor !== warmLoad.tuning.observed.wasmFlavor) {
|
| 293 |
+
throw new Error(
|
| 294 |
+
`WASM flavor changed between cold and warm loads (${coldLoad.tuning.observed.wasmFlavor} → ${warmLoad.tuning.observed.wasmFlavor}).`,
|
| 295 |
+
);
|
| 296 |
+
}
|
| 297 |
throwIfAborted(controller.signal);
|
| 298 |
|
| 299 |
let firstTokenAt: number | null = null;
|
|
|
|
| 336 |
requestedBackend: backend,
|
| 337 |
contextSize: warmLoad.context.size,
|
| 338 |
maxTokens,
|
|
|
|
|
|
|
| 339 |
coldLoadMs,
|
| 340 |
warmLoadMs,
|
| 341 |
coldCachedBytes,
|
|
|
|
| 361 |
setProgress({ label: 'Benchmark stopped by operator', loadedBytes: 0, totalBytes: 0 });
|
| 362 |
return;
|
| 363 |
}
|
| 364 |
+
setRunError(errorMessage(runError));
|
| 365 |
setProgress({ label: 'Benchmark failed safely · no report exported', loadedBytes: 0, totalBytes: 0 });
|
| 366 |
}).finally(() => {
|
| 367 |
if (operationRef.current === controller) operationRef.current = null;
|
|
|
|
| 419 |
disabled={running}
|
| 420 |
onChange={(event) => {
|
| 421 |
setBackend(event.target.value as RequestedBackend);
|
| 422 |
+
clearPreviousRun();
|
|
|
|
| 423 |
}}
|
| 424 |
>
|
| 425 |
<option value="auto">Auto · gated</option>
|
|
|
|
| 441 |
disabled={running}
|
| 442 |
onChange={(event) => {
|
| 443 |
setContextSize(Number(event.target.value));
|
| 444 |
+
clearPreviousRun();
|
| 445 |
}}
|
| 446 |
/>
|
| 447 |
<small>Max {contextLimit.toLocaleString()}</small>
|
|
|
|
| 457 |
disabled={running}
|
| 458 |
onChange={(event) => {
|
| 459 |
setMaxTokens(Number(event.target.value));
|
| 460 |
+
clearPreviousRun();
|
| 461 |
}}
|
| 462 |
/>
|
| 463 |
<small>8–512</small>
|
|
|
|
| 465 |
</div>
|
| 466 |
|
| 467 |
<details className="bench-experimental">
|
| 468 |
+
<summary>Experimental runtime tuning</summary>
|
| 469 |
+
<p>Blank batch fields use the pinned runtime defaults. Every option is exported as an experiment, not a release recommendation.</p>
|
| 470 |
<div className="bench-field-pair">
|
| 471 |
<label className="bench-field">
|
| 472 |
<span>nBatch</span>
|
| 473 |
<input
|
| 474 |
type="number"
|
| 475 |
min="1"
|
| 476 |
+
max={MAX_GLUE_INT}
|
| 477 |
step="1"
|
| 478 |
placeholder="runtime default"
|
| 479 |
value={nBatchInput}
|
| 480 |
disabled={running}
|
| 481 |
onChange={(event) => {
|
| 482 |
setNBatchInput(event.target.value);
|
| 483 |
+
clearPreviousRun();
|
| 484 |
}}
|
| 485 |
/>
|
| 486 |
</label>
|
|
|
|
| 489 |
<input
|
| 490 |
type="number"
|
| 491 |
min="1"
|
| 492 |
+
max={MAX_GLUE_INT}
|
| 493 |
step="1"
|
| 494 |
placeholder="runtime default"
|
| 495 |
value={nUbatchInput}
|
| 496 |
disabled={running}
|
| 497 |
onChange={(event) => {
|
| 498 |
setNUbatchInput(event.target.value);
|
| 499 |
+
clearPreviousRun();
|
| 500 |
}}
|
| 501 |
/>
|
| 502 |
</label>
|
| 503 |
</div>
|
| 504 |
+
<div className="bench-field-pair">
|
| 505 |
+
<label className="bench-field">
|
| 506 |
+
<span>Flash Attention request</span>
|
| 507 |
+
<select
|
| 508 |
+
value={flashMode}
|
| 509 |
+
disabled={running}
|
| 510 |
+
onChange={(event) => {
|
| 511 |
+
setFlashMode(event.target.value as BenchmarkFlashMode);
|
| 512 |
+
clearPreviousRun();
|
| 513 |
+
}}
|
| 514 |
+
>
|
| 515 |
+
<option value="off">Off · default</option>
|
| 516 |
+
<option value="auto">Auto · observed on</option>
|
| 517 |
+
</select>
|
| 518 |
+
</label>
|
| 519 |
+
<label className="bench-field">
|
| 520 |
+
<span>KV cache K / V</span>
|
| 521 |
+
<select
|
| 522 |
+
value={kvCacheType}
|
| 523 |
+
disabled={running}
|
| 524 |
+
onChange={(event) => {
|
| 525 |
+
setKvCacheType(event.target.value as BenchmarkKvCacheType);
|
| 526 |
+
clearPreviousRun();
|
| 527 |
+
}}
|
| 528 |
+
>
|
| 529 |
+
<option value="f16">f16 / f16 · default</option>
|
| 530 |
+
<option value="q8_0">q8_0 / q8_0 · exp.</option>
|
| 531 |
+
<option value="q4_0">q4_0 / q4_0 · exp.</option>
|
| 532 |
+
</select>
|
| 533 |
+
</label>
|
| 534 |
+
</div>
|
| 535 |
+
<label className="bench-field">
|
| 536 |
+
<span>WASM runtime</span>
|
| 537 |
+
<select
|
| 538 |
+
value={wasmFlavor}
|
| 539 |
+
disabled={running}
|
| 540 |
+
onChange={(event) => {
|
| 541 |
+
setWasmFlavor(event.target.value as BenchmarkWasmFlavor);
|
| 542 |
+
clearPreviousRun();
|
| 543 |
+
}}
|
| 544 |
+
>
|
| 545 |
+
<option value="auto">Auto · browser-selected</option>
|
| 546 |
+
<option value="jspi">JSPI · explicit</option>
|
| 547 |
+
<option value="compat">Compat / Asyncify · explicit</option>
|
| 548 |
+
</select>
|
| 549 |
+
<small>Current auto path: {capabilities?.runtime.wasmFlavor ?? 'inspecting'}</small>
|
| 550 |
+
</label>
|
| 551 |
<dl className="bench-locks">
|
| 552 |
+
<div><dt>Chat release</dt><dd>Flash off · KV f16/f16</dd></div>
|
| 553 |
+
<div><dt>Evidence rule</dt><dd>native log must match or the run fails</dd></div>
|
| 554 |
</dl>
|
| 555 |
</details>
|
| 556 |
|
| 557 |
+
{configurationTuningError && (
|
| 558 |
+
<p className="bench-inline-warning" role="alert">{configurationTuningError}</p>
|
| 559 |
+
)}
|
| 560 |
+
|
| 561 |
<div className={`bench-gate ${deviceGate?.allowed ? 'pass' : 'fail'}`}>
|
| 562 |
<strong>{deviceGate?.allowed ? 'Device preflight passed' : 'Device preflight blocked'}</strong>
|
| 563 |
<span>
|
|
|
|
| 602 |
<span className="bench-progress-track"><i style={{ width: `${progressPercent}%` }} /></span>
|
| 603 |
</section>
|
| 604 |
|
| 605 |
+
{bootstrapError && <div className="bench-error" role="alert"><strong>Runtime inspection failed</strong><span>{bootstrapError}</span></div>}
|
| 606 |
+
{runError && <div className="bench-error" role="alert"><strong>Run rejected</strong><span>{runError}</span></div>}
|
| 607 |
|
| 608 |
<section className="bench-core-section" aria-label="Benchmark measurements">
|
| 609 |
<div className="bench-section-heading">
|
|
|
|
| 660 |
<div><dt>wllama</dt><dd title={report?.runtime.wllamaRevision}>{report?.runtime.wllamaRevision.slice(0, 12) ?? '—'}</dd></div>
|
| 661 |
<div><dt>llama.cpp</dt><dd title={report?.runtime.llamaCppRevision}>{report?.runtime.llamaCppRevision.slice(0, 12) ?? '—'}</dd></div>
|
| 662 |
<div><dt>Weights</dt><dd title={report?.runtime.manifestRevision ?? undefined}>{report?.runtime.manifestRevision?.slice(0, 12) ?? '—'}</dd></div>
|
| 663 |
+
<div><dt>WASM</dt><dd>{report?.runtime.wasmFlavor.toUpperCase() ?? '—'}</dd></div>
|
| 664 |
+
</dl>
|
| 665 |
+
</article>
|
| 666 |
+
<article className="bench-data-card bench-tuning-card">
|
| 667 |
+
<p className="eyebrow ink">Native tuning</p>
|
| 668 |
+
<dl>
|
| 669 |
+
<div><dt>Applied</dt><dd>{report ? (report.load.tuning.applied ? 'yes' : 'no') : '—'}</dd></div>
|
| 670 |
+
<div><dt>Flash</dt><dd>{report ? `${report.load.tuning.requested.flashMode} → ${String(report.load.tuning.observed.flashAttention)}` : '—'}</dd></div>
|
| 671 |
+
<div><dt>K/V</dt><dd>{report ? `${report.load.tuning.observed.cacheTypeK}/${report.load.tuning.observed.cacheTypeV}` : '—'}</dd></div>
|
| 672 |
+
<div><dt>KV bytes</dt><dd>{report?.load.tuning.observed.kvBufferBytes === null || report?.load.tuning.observed.kvBufferBytes === undefined ? '—' : formatBytes(report.load.tuning.observed.kvBufferBytes)}</dd></div>
|
| 673 |
</dl>
|
| 674 |
</article>
|
| 675 |
</section>
|
|
|
|
| 682 |
</div>
|
| 683 |
<div>
|
| 684 |
<p className="eyebrow ink">Generated text · excluded from export</p>
|
| 685 |
+
<p data-testid="bench-generated-output">{output || 'Run the assay to inspect the generated specimen.'}</p>
|
| 686 |
</div>
|
| 687 |
</section>
|
| 688 |
</main>
|
src/bench/report.test.ts
CHANGED
|
@@ -1,5 +1,6 @@
|
|
| 1 |
import { describe, expect, it } from 'vitest';
|
| 2 |
import type {
|
|
|
|
| 3 |
EngineCapabilities,
|
| 4 |
LoadModelResult,
|
| 5 |
ManifestModelV2,
|
|
@@ -13,6 +14,21 @@ import {
|
|
| 13 |
type BenchmarkObservation,
|
| 14 |
} from './report';
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
const model = {
|
| 17 |
id: '8b',
|
| 18 |
displayName: 'Bonsai 8B',
|
|
@@ -25,8 +41,16 @@ const manifest = {
|
|
| 25 |
id: 'WaveCut/Bonsai-web-GGUF',
|
| 26 |
revision: '112ea7a1a6229bde132b176b9a72477a7ecfde64',
|
| 27 |
},
|
| 28 |
-
dflash: {
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
const capabilities = {
|
| 32 |
crossOriginIsolated: true,
|
|
@@ -55,18 +79,76 @@ const capabilities = {
|
|
| 55 |
patchSetSha256: 'f'.repeat(64),
|
| 56 |
moduleSha256: 'e'.repeat(64),
|
| 57 |
wasmFlavor: 'jspi',
|
| 58 |
-
wasmSha256:
|
| 59 |
compatWorkerSha256: null,
|
| 60 |
tokenEmbeddingOnWebGPU: true,
|
| 61 |
tensorPlacementOverrides: false,
|
| 62 |
},
|
| 63 |
} satisfies EngineCapabilities;
|
| 64 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
const loadResult = {
|
|
|
|
| 66 |
backend: 'webgpu',
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
function observation(overrides: Partial<BenchmarkObservation> = {}): BenchmarkObservation {
|
| 72 |
return {
|
|
@@ -78,8 +160,6 @@ function observation(overrides: Partial<BenchmarkObservation> = {}): BenchmarkOb
|
|
| 78 |
requestedBackend: 'auto',
|
| 79 |
contextSize: 4_096,
|
| 80 |
maxTokens: 64,
|
| 81 |
-
nBatch: null,
|
| 82 |
-
nUbatch: null,
|
| 83 |
coldLoadMs: 1_234.56,
|
| 84 |
warmLoadMs: 234.54,
|
| 85 |
coldCachedBytes: 250,
|
|
@@ -95,12 +175,7 @@ function observation(overrides: Partial<BenchmarkObservation> = {}): BenchmarkOb
|
|
| 95 |
generationElapsedMs: 900.04,
|
| 96 |
timeToFirstTokenMs: 123.45,
|
| 97 |
streamedTokenEvents: 5,
|
| 98 |
-
backendReport
|
| 99 |
-
backends: ['WebGPU'],
|
| 100 |
-
nGraphSplits: 1,
|
| 101 |
-
opsOnCpu: 0,
|
| 102 |
-
layersGpu: { offloaded: 49, total: 49 },
|
| 103 |
-
},
|
| 104 |
...overrides,
|
| 105 |
};
|
| 106 |
}
|
|
@@ -118,14 +193,27 @@ describe('benchmark report', () => {
|
|
| 118 |
|
| 119 |
expect(report.load).toMatchObject({
|
| 120 |
tuning: {
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
},
|
| 130 |
cold: { durationMs: 1_234.6, cacheState: 'partial' },
|
| 131 |
warm: { durationMs: 234.5 },
|
|
@@ -149,7 +237,7 @@ describe('benchmark report', () => {
|
|
| 149 |
patchSetSha256: 'f'.repeat(64),
|
| 150 |
moduleSha256: 'e'.repeat(64),
|
| 151 |
wasmFlavor: 'jspi',
|
| 152 |
-
wasmSha256:
|
| 153 |
compatWorkerSha256: null,
|
| 154 |
});
|
| 155 |
expect(report.environment).toEqual({
|
|
@@ -184,24 +272,110 @@ describe('benchmark report', () => {
|
|
| 184 |
dflash: {
|
| 185 |
enabled: false,
|
| 186 |
runtimeAvailable: false,
|
| 187 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 188 |
},
|
| 189 |
});
|
| 190 |
});
|
| 191 |
|
| 192 |
-
it('records
|
| 193 |
-
const
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 194 |
|
| 195 |
expect(report.load.tuning).toEqual({
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
});
|
|
|
|
|
|
|
|
|
|
|
|
|
| 205 |
});
|
| 206 |
|
| 207 |
it('uses streamed events when final engine usage is present but incomplete', () => {
|
|
|
|
| 1 |
import { describe, expect, it } from 'vitest';
|
| 2 |
import type {
|
| 3 |
+
BackendReport,
|
| 4 |
EngineCapabilities,
|
| 5 |
LoadModelResult,
|
| 6 |
ManifestModelV2,
|
|
|
|
| 14 |
type BenchmarkObservation,
|
| 15 |
} from './report';
|
| 16 |
|
| 17 |
+
const JSPI_WASM_SHA256 = 'd'.repeat(64);
|
| 18 |
+
const COMPAT_WASM_SHA256 = 'c'.repeat(64);
|
| 19 |
+
const COMPAT_WORKER_SHA256 = 'b'.repeat(64);
|
| 20 |
+
|
| 21 |
+
const dflash27bPairing = {
|
| 22 |
+
support: 'conditional',
|
| 23 |
+
compatibilityStatus: 'native-greedy-parity-pass',
|
| 24 |
+
runtimeAvailable: false,
|
| 25 |
+
blockSize: 16,
|
| 26 |
+
maskTokenId: 248_070,
|
| 27 |
+
targetLayerIdsZeroBased: [1, 16, 31, 46, 61],
|
| 28 |
+
targetTapsOneBased: [2, 17, 32, 47, 62],
|
| 29 |
+
reason: 'Native parity passes; browser activation remains gated.',
|
| 30 |
+
} as const;
|
| 31 |
+
|
| 32 |
const model = {
|
| 33 |
id: '8b',
|
| 34 |
displayName: 'Bonsai 8B',
|
|
|
|
| 41 |
id: 'WaveCut/Bonsai-web-GGUF',
|
| 42 |
revision: '112ea7a1a6229bde132b176b9a72477a7ecfde64',
|
| 43 |
},
|
| 44 |
+
dflash: {
|
| 45 |
+
status: 'conditional',
|
| 46 |
+
pairings: {
|
| 47 |
+
'1_7b': { support: 'unsupported', reason: 'No compatible 1.7B drafter is locked.' },
|
| 48 |
+
'4b': { support: 'unsupported', reason: 'The 27B drafter is incompatible with Bonsai 4B.' },
|
| 49 |
+
'8b': { support: 'unsupported', reason: 'The 27B drafter is incompatible with Bonsai 8B.' },
|
| 50 |
+
'27b': dflash27bPairing,
|
| 51 |
+
},
|
| 52 |
+
},
|
| 53 |
+
} as unknown as ModelManifestV2;
|
| 54 |
|
| 55 |
const capabilities = {
|
| 56 |
crossOriginIsolated: true,
|
|
|
|
| 79 |
patchSetSha256: 'f'.repeat(64),
|
| 80 |
moduleSha256: 'e'.repeat(64),
|
| 81 |
wasmFlavor: 'jspi',
|
| 82 |
+
wasmSha256: JSPI_WASM_SHA256,
|
| 83 |
compatWorkerSha256: null,
|
| 84 |
tokenEmbeddingOnWebGPU: true,
|
| 85 |
tensorPlacementOverrides: false,
|
| 86 |
},
|
| 87 |
} satisfies EngineCapabilities;
|
| 88 |
|
| 89 |
+
const backendReport = {
|
| 90 |
+
backends: ['WebGPU'],
|
| 91 |
+
nGraphSplits: 1,
|
| 92 |
+
opsOnCpu: 0,
|
| 93 |
+
layersGpu: { offloaded: 49, total: 49 },
|
| 94 |
+
flashAttention: false,
|
| 95 |
+
cacheTypeK: 'f16',
|
| 96 |
+
cacheTypeV: 'f16',
|
| 97 |
+
webgpuKvBufferBytes: 56 * 1024 ** 2,
|
| 98 |
+
} satisfies BackendReport;
|
| 99 |
+
|
| 100 |
const loadResult = {
|
| 101 |
+
modelId: '8b',
|
| 102 |
backend: 'webgpu',
|
| 103 |
+
gate: {
|
| 104 |
+
allowed: true,
|
| 105 |
+
requestedBackend: 'auto',
|
| 106 |
+
selectedBackend: 'webgpu',
|
| 107 |
+
reasons: [],
|
| 108 |
+
warnings: [],
|
| 109 |
+
requiredBytes: 1_000,
|
| 110 |
+
availableStorageBytes: 9_000,
|
| 111 |
+
},
|
| 112 |
+
shardUrls: ['https://models.test/bonsai-8b.gguf'],
|
| 113 |
+
context: {
|
| 114 |
+
size: 4_096,
|
| 115 |
+
trainingSize: 32_768,
|
| 116 |
+
layerCount: 49,
|
| 117 |
+
vocabularySize: 151_936,
|
| 118 |
+
batchSize: 2_048,
|
| 119 |
+
microBatchSize: 512,
|
| 120 |
+
},
|
| 121 |
+
tuning: {
|
| 122 |
+
scope: 'release-defaults',
|
| 123 |
+
requested: {
|
| 124 |
+
nBatch: null,
|
| 125 |
+
nUbatch: null,
|
| 126 |
+
flashMode: 'off',
|
| 127 |
+
cacheTypeK: 'f16',
|
| 128 |
+
cacheTypeV: 'f16',
|
| 129 |
+
wasmFlavor: 'auto',
|
| 130 |
+
},
|
| 131 |
+
observed: {
|
| 132 |
+
nBatch: 2_048,
|
| 133 |
+
nUbatch: 512,
|
| 134 |
+
flashAttention: false,
|
| 135 |
+
cacheTypeK: 'f16',
|
| 136 |
+
cacheTypeV: 'f16',
|
| 137 |
+
kvBufferBytes: 56 * 1024 ** 2,
|
| 138 |
+
wasmFlavor: 'jspi',
|
| 139 |
+
wasmSha256: JSPI_WASM_SHA256,
|
| 140 |
+
compatWorkerSha256: null,
|
| 141 |
+
},
|
| 142 |
+
applied: true,
|
| 143 |
+
},
|
| 144 |
+
chatTemplate: {
|
| 145 |
+
bytes: 1_024,
|
| 146 |
+
hasThinkMarker: true,
|
| 147 |
+
hasToolCallMarker: true,
|
| 148 |
+
hasToolResponseMarker: true,
|
| 149 |
+
},
|
| 150 |
+
backendReport,
|
| 151 |
+
} satisfies LoadModelResult;
|
| 152 |
|
| 153 |
function observation(overrides: Partial<BenchmarkObservation> = {}): BenchmarkObservation {
|
| 154 |
return {
|
|
|
|
| 160 |
requestedBackend: 'auto',
|
| 161 |
contextSize: 4_096,
|
| 162 |
maxTokens: 64,
|
|
|
|
|
|
|
| 163 |
coldLoadMs: 1_234.56,
|
| 164 |
warmLoadMs: 234.54,
|
| 165 |
coldCachedBytes: 250,
|
|
|
|
| 175 |
generationElapsedMs: 900.04,
|
| 176 |
timeToFirstTokenMs: 123.45,
|
| 177 |
streamedTokenEvents: 5,
|
| 178 |
+
backendReport,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 179 |
...overrides,
|
| 180 |
};
|
| 181 |
}
|
|
|
|
| 193 |
|
| 194 |
expect(report.load).toMatchObject({
|
| 195 |
tuning: {
|
| 196 |
+
scope: 'release-defaults',
|
| 197 |
+
requested: {
|
| 198 |
+
nBatch: null,
|
| 199 |
+
nUbatch: null,
|
| 200 |
+
flashMode: 'off',
|
| 201 |
+
cacheTypeK: 'f16',
|
| 202 |
+
cacheTypeV: 'f16',
|
| 203 |
+
wasmFlavor: 'auto',
|
| 204 |
+
},
|
| 205 |
+
observed: {
|
| 206 |
+
nBatch: 2_048,
|
| 207 |
+
nUbatch: 512,
|
| 208 |
+
flashAttention: false,
|
| 209 |
+
cacheTypeK: 'f16',
|
| 210 |
+
cacheTypeV: 'f16',
|
| 211 |
+
kvBufferBytes: 56 * 1024 ** 2,
|
| 212 |
+
wasmFlavor: 'jspi',
|
| 213 |
+
wasmSha256: JSPI_WASM_SHA256,
|
| 214 |
+
compatWorkerSha256: null,
|
| 215 |
+
},
|
| 216 |
+
applied: true,
|
| 217 |
},
|
| 218 |
cold: { durationMs: 1_234.6, cacheState: 'partial' },
|
| 219 |
warm: { durationMs: 234.5 },
|
|
|
|
| 237 |
patchSetSha256: 'f'.repeat(64),
|
| 238 |
moduleSha256: 'e'.repeat(64),
|
| 239 |
wasmFlavor: 'jspi',
|
| 240 |
+
wasmSha256: JSPI_WASM_SHA256,
|
| 241 |
compatWorkerSha256: null,
|
| 242 |
});
|
| 243 |
expect(report.environment).toEqual({
|
|
|
|
| 272 |
dflash: {
|
| 273 |
enabled: false,
|
| 274 |
runtimeAvailable: false,
|
| 275 |
+
globalStatus: 'conditional',
|
| 276 |
+
pairing: {
|
| 277 |
+
support: 'unsupported',
|
| 278 |
+
reason: 'The 27B drafter is incompatible with Bonsai 8B.',
|
| 279 |
+
},
|
| 280 |
},
|
| 281 |
});
|
| 282 |
});
|
| 283 |
|
| 284 |
+
it('records requested and observed experimental tuning without relabelling it as release policy', () => {
|
| 285 |
+
const experimentalLoadResult: LoadModelResult = {
|
| 286 |
+
...loadResult,
|
| 287 |
+
context: {
|
| 288 |
+
...loadResult.context,
|
| 289 |
+
batchSize: 512,
|
| 290 |
+
microBatchSize: 128,
|
| 291 |
+
},
|
| 292 |
+
tuning: {
|
| 293 |
+
scope: 'benchmark',
|
| 294 |
+
requested: {
|
| 295 |
+
nBatch: 512,
|
| 296 |
+
nUbatch: 128,
|
| 297 |
+
flashMode: 'auto',
|
| 298 |
+
cacheTypeK: 'q8_0',
|
| 299 |
+
cacheTypeV: 'q8_0',
|
| 300 |
+
wasmFlavor: 'compat',
|
| 301 |
+
},
|
| 302 |
+
observed: {
|
| 303 |
+
nBatch: 512,
|
| 304 |
+
nUbatch: 128,
|
| 305 |
+
flashAttention: true,
|
| 306 |
+
cacheTypeK: 'q8_0',
|
| 307 |
+
cacheTypeV: 'q8_0',
|
| 308 |
+
kvBufferBytes: 28 * 1024 ** 2,
|
| 309 |
+
wasmFlavor: 'compat',
|
| 310 |
+
wasmSha256: COMPAT_WASM_SHA256,
|
| 311 |
+
compatWorkerSha256: COMPAT_WORKER_SHA256,
|
| 312 |
+
},
|
| 313 |
+
applied: true,
|
| 314 |
+
},
|
| 315 |
+
};
|
| 316 |
+
const report = buildBenchmarkReport(observation({ loadResult: experimentalLoadResult }));
|
| 317 |
|
| 318 |
expect(report.load.tuning).toEqual({
|
| 319 |
+
scope: 'benchmark',
|
| 320 |
+
requested: {
|
| 321 |
+
nBatch: 512,
|
| 322 |
+
nUbatch: 128,
|
| 323 |
+
flashMode: 'auto',
|
| 324 |
+
cacheTypeK: 'q8_0',
|
| 325 |
+
cacheTypeV: 'q8_0',
|
| 326 |
+
wasmFlavor: 'compat',
|
| 327 |
+
},
|
| 328 |
+
observed: {
|
| 329 |
+
nBatch: 512,
|
| 330 |
+
nUbatch: 128,
|
| 331 |
+
flashAttention: true,
|
| 332 |
+
cacheTypeK: 'q8_0',
|
| 333 |
+
cacheTypeV: 'q8_0',
|
| 334 |
+
kvBufferBytes: 28 * 1024 ** 2,
|
| 335 |
+
wasmFlavor: 'compat',
|
| 336 |
+
wasmSha256: COMPAT_WASM_SHA256,
|
| 337 |
+
compatWorkerSha256: COMPAT_WORKER_SHA256,
|
| 338 |
+
},
|
| 339 |
+
applied: true,
|
| 340 |
+
});
|
| 341 |
+
expect(report.runtime).toMatchObject({
|
| 342 |
+
wasmFlavor: 'compat',
|
| 343 |
+
wasmSha256: COMPAT_WASM_SHA256,
|
| 344 |
+
compatWorkerSha256: COMPAT_WORKER_SHA256,
|
| 345 |
+
});
|
| 346 |
+
});
|
| 347 |
+
|
| 348 |
+
it('exports the selected model DFlash pairing for unsupported 1.7B and conditional 27B', () => {
|
| 349 |
+
const reportFor = (modelId: ManifestModelV2['id']) => buildBenchmarkReport(observation({
|
| 350 |
+
model: {
|
| 351 |
+
...model,
|
| 352 |
+
id: modelId,
|
| 353 |
+
displayName: `Bonsai ${modelId}`,
|
| 354 |
+
} as ManifestModelV2,
|
| 355 |
+
loadResult: { ...loadResult, modelId },
|
| 356 |
+
}));
|
| 357 |
+
|
| 358 |
+
expect(reportFor('1_7b').releasePolicy.dflash).toEqual({
|
| 359 |
+
enabled: false,
|
| 360 |
+
runtimeAvailable: false,
|
| 361 |
+
globalStatus: 'conditional',
|
| 362 |
+
pairing: {
|
| 363 |
+
support: 'unsupported',
|
| 364 |
+
reason: 'No compatible 1.7B drafter is locked.',
|
| 365 |
+
},
|
| 366 |
+
});
|
| 367 |
+
|
| 368 |
+
const report27b = reportFor('27b');
|
| 369 |
+
expect(report27b.releasePolicy.dflash).toEqual({
|
| 370 |
+
enabled: false,
|
| 371 |
+
runtimeAvailable: false,
|
| 372 |
+
globalStatus: 'conditional',
|
| 373 |
+
pairing: dflash27bPairing,
|
| 374 |
});
|
| 375 |
+
const exportedPairing = report27b.releasePolicy.dflash.pairing;
|
| 376 |
+
expect(exportedPairing.support).toBe('conditional');
|
| 377 |
+
if (exportedPairing.support !== 'conditional') throw new Error('Expected the conditional 27B pairing.');
|
| 378 |
+
expect(exportedPairing.targetLayerIdsZeroBased).not.toBe(dflash27bPairing.targetLayerIdsZeroBased);
|
| 379 |
});
|
| 380 |
|
| 381 |
it('uses streamed events when final engine usage is present but incomplete', () => {
|
src/bench/report.ts
CHANGED
|
@@ -22,8 +22,6 @@ export interface BenchmarkObservation {
|
|
| 22 |
requestedBackend: RequestedBackend;
|
| 23 |
contextSize: number;
|
| 24 |
maxTokens: number;
|
| 25 |
-
nBatch: number | null;
|
| 26 |
-
nUbatch: number | null;
|
| 27 |
coldLoadMs: number;
|
| 28 |
warmLoadMs: number;
|
| 29 |
coldCachedBytes: number | null;
|
|
@@ -37,7 +35,7 @@ export interface BenchmarkObservation {
|
|
| 37 |
}
|
| 38 |
|
| 39 |
export interface BenchmarkReport {
|
| 40 |
-
schemaVersion:
|
| 41 |
kind: 'bonsai-browser-benchmark';
|
| 42 |
startedAt: string;
|
| 43 |
completedAt: string;
|
|
@@ -49,16 +47,7 @@ export interface BenchmarkReport {
|
|
| 49 |
contextSize: number;
|
| 50 |
};
|
| 51 |
load: {
|
| 52 |
-
tuning:
|
| 53 |
-
requestedNBatch: number | null;
|
| 54 |
-
requestedNUbatch: number | null;
|
| 55 |
-
nBatch: number;
|
| 56 |
-
nUbatch: number;
|
| 57 |
-
flashAttention: boolean;
|
| 58 |
-
cacheTypeK: LoadModelResult['tuning']['cacheTypeK'];
|
| 59 |
-
cacheTypeV: LoadModelResult['tuning']['cacheTypeV'];
|
| 60 |
-
status: 'release-defaults' | 'experimental-batch-shape';
|
| 61 |
-
};
|
| 62 |
cold: {
|
| 63 |
durationMs: number | null;
|
| 64 |
cachedBytesAtStart: number | null;
|
|
@@ -133,7 +122,8 @@ export interface BenchmarkReport {
|
|
| 133 |
dflash: {
|
| 134 |
enabled: false;
|
| 135 |
runtimeAvailable: false;
|
| 136 |
-
|
|
|
|
| 137 |
};
|
| 138 |
};
|
| 139 |
}
|
|
@@ -179,9 +169,10 @@ export function buildBenchmarkReport(observation: BenchmarkObservation): Benchma
|
|
| 179 |
const device = loadResult.backend === 'webgpu'
|
| 180 |
? capabilities.webgpu.name || 'WebGPU adapter'
|
| 181 |
: `CPU · ${capabilities.hardwareConcurrency} logical cores`;
|
|
|
|
| 182 |
|
| 183 |
return {
|
| 184 |
-
schemaVersion:
|
| 185 |
kind: 'bonsai-browser-benchmark',
|
| 186 |
startedAt: observation.startedAt,
|
| 187 |
completedAt: observation.completedAt,
|
|
@@ -194,16 +185,10 @@ export function buildBenchmarkReport(observation: BenchmarkObservation): Benchma
|
|
| 194 |
},
|
| 195 |
load: {
|
| 196 |
tuning: {
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
flashAttention: loadResult.tuning.flashAttention,
|
| 202 |
-
cacheTypeK: loadResult.tuning.cacheTypeK,
|
| 203 |
-
cacheTypeV: loadResult.tuning.cacheTypeV,
|
| 204 |
-
status: observation.nBatch === null && observation.nUbatch === null
|
| 205 |
-
? 'release-defaults'
|
| 206 |
-
: 'experimental-batch-shape',
|
| 207 |
},
|
| 208 |
cold: {
|
| 209 |
durationMs: roundedNonNegative(observation.coldLoadMs, 1),
|
|
@@ -255,9 +240,9 @@ export function buildBenchmarkReport(observation: BenchmarkObservation): Benchma
|
|
| 255 |
llamaCppRevision: capabilities.runtime.llamaCppRevision,
|
| 256 |
patchSetSha256: capabilities.runtime.patchSetSha256,
|
| 257 |
moduleSha256: capabilities.runtime.moduleSha256,
|
| 258 |
-
wasmFlavor:
|
| 259 |
-
wasmSha256:
|
| 260 |
-
compatWorkerSha256:
|
| 261 |
manifestRepository: observation.manifest.repository.id,
|
| 262 |
manifestRevision: observation.manifest.repository.revision,
|
| 263 |
crossOriginIsolated: capabilities.crossOriginIsolated,
|
|
@@ -288,7 +273,14 @@ export function buildBenchmarkReport(observation: BenchmarkObservation): Benchma
|
|
| 288 |
dflash: {
|
| 289 |
enabled: false,
|
| 290 |
runtimeAvailable: false,
|
| 291 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 292 |
},
|
| 293 |
},
|
| 294 |
};
|
|
|
|
| 22 |
requestedBackend: RequestedBackend;
|
| 23 |
contextSize: number;
|
| 24 |
maxTokens: number;
|
|
|
|
|
|
|
| 25 |
coldLoadMs: number;
|
| 26 |
warmLoadMs: number;
|
| 27 |
coldCachedBytes: number | null;
|
|
|
|
| 35 |
}
|
| 36 |
|
| 37 |
export interface BenchmarkReport {
|
| 38 |
+
schemaVersion: 2;
|
| 39 |
kind: 'bonsai-browser-benchmark';
|
| 40 |
startedAt: string;
|
| 41 |
completedAt: string;
|
|
|
|
| 47 |
contextSize: number;
|
| 48 |
};
|
| 49 |
load: {
|
| 50 |
+
tuning: LoadModelResult['tuning'];
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
cold: {
|
| 52 |
durationMs: number | null;
|
| 53 |
cachedBytesAtStart: number | null;
|
|
|
|
| 122 |
dflash: {
|
| 123 |
enabled: false;
|
| 124 |
runtimeAvailable: false;
|
| 125 |
+
globalStatus: ModelManifestV2['dflash']['status'];
|
| 126 |
+
pairing: ModelManifestV2['dflash']['pairings'][ManifestModelV2['id']];
|
| 127 |
};
|
| 128 |
};
|
| 129 |
}
|
|
|
|
| 169 |
const device = loadResult.backend === 'webgpu'
|
| 170 |
? capabilities.webgpu.name || 'WebGPU adapter'
|
| 171 |
: `CPU · ${capabilities.hardwareConcurrency} logical cores`;
|
| 172 |
+
const dflashPairing = observation.manifest.dflash.pairings[observation.model.id];
|
| 173 |
|
| 174 |
return {
|
| 175 |
+
schemaVersion: 2,
|
| 176 |
kind: 'bonsai-browser-benchmark',
|
| 177 |
startedAt: observation.startedAt,
|
| 178 |
completedAt: observation.completedAt,
|
|
|
|
| 185 |
},
|
| 186 |
load: {
|
| 187 |
tuning: {
|
| 188 |
+
scope: loadResult.tuning.scope,
|
| 189 |
+
requested: { ...loadResult.tuning.requested },
|
| 190 |
+
observed: { ...loadResult.tuning.observed },
|
| 191 |
+
applied: loadResult.tuning.applied,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 192 |
},
|
| 193 |
cold: {
|
| 194 |
durationMs: roundedNonNegative(observation.coldLoadMs, 1),
|
|
|
|
| 240 |
llamaCppRevision: capabilities.runtime.llamaCppRevision,
|
| 241 |
patchSetSha256: capabilities.runtime.patchSetSha256,
|
| 242 |
moduleSha256: capabilities.runtime.moduleSha256,
|
| 243 |
+
wasmFlavor: loadResult.tuning.observed.wasmFlavor,
|
| 244 |
+
wasmSha256: loadResult.tuning.observed.wasmSha256,
|
| 245 |
+
compatWorkerSha256: loadResult.tuning.observed.compatWorkerSha256,
|
| 246 |
manifestRepository: observation.manifest.repository.id,
|
| 247 |
manifestRevision: observation.manifest.repository.revision,
|
| 248 |
crossOriginIsolated: capabilities.crossOriginIsolated,
|
|
|
|
| 273 |
dflash: {
|
| 274 |
enabled: false,
|
| 275 |
runtimeAvailable: false,
|
| 276 |
+
globalStatus: observation.manifest.dflash.status,
|
| 277 |
+
pairing: dflashPairing.support === 'conditional'
|
| 278 |
+
? {
|
| 279 |
+
...dflashPairing,
|
| 280 |
+
targetLayerIdsZeroBased: [...dflashPairing.targetLayerIdsZeroBased],
|
| 281 |
+
targetTapsOneBased: [...dflashPairing.targetTapsOneBased],
|
| 282 |
+
}
|
| 283 |
+
: { ...dflashPairing },
|
| 284 |
},
|
| 285 |
},
|
| 286 |
};
|
src/engine/device-gate.test.ts
CHANGED
|
@@ -171,6 +171,10 @@ describe('assertBackendPolicy', () => {
|
|
| 171 |
nGraphSplits: 1,
|
| 172 |
opsOnCpu: 0,
|
| 173 |
layersGpu: { offloaded: 65, total: 65 },
|
|
|
|
|
|
|
|
|
|
|
|
|
| 174 |
};
|
| 175 |
|
| 176 |
it('accepts a proven single WebGPU graph', () => {
|
|
|
|
| 171 |
nGraphSplits: 1,
|
| 172 |
opsOnCpu: 0,
|
| 173 |
layersGpu: { offloaded: 65, total: 65 },
|
| 174 |
+
flashAttention: false,
|
| 175 |
+
cacheTypeK: 'f16',
|
| 176 |
+
cacheTypeV: 'f16',
|
| 177 |
+
webgpuKvBufferBytes: 56 * 1024 ** 2,
|
| 178 |
};
|
| 179 |
|
| 180 |
it('accepts a proven single WebGPU graph', () => {
|
src/engine/native-log.test.ts
CHANGED
|
@@ -12,6 +12,9 @@ llama_prepare_model_devices: using device WebGPU (WebGPU) (unknown id)
|
|
| 12 |
load_tensors: offloaded 65/65 layers to GPU
|
| 13 |
load_tensors: CPU model buffer size = 72.00 MiB
|
| 14 |
load_tensors: WebGPU model buffer size = 3616.77 MiB
|
|
|
|
|
|
|
|
|
|
| 15 |
sched_reserve: WebGPU compute buffer size = 301.00 MiB
|
| 16 |
sched_reserve: CPU compute buffer size = 13.01 MiB
|
| 17 |
sched_reserve: graph splits = 1
|
|
@@ -22,6 +25,10 @@ sched_reserve: graph splits = 1
|
|
| 22 |
nGraphSplits: 1,
|
| 23 |
opsOnCpu: 0,
|
| 24 |
layersGpu: { offloaded: 65, total: 65 },
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
});
|
| 26 |
});
|
| 27 |
|
|
@@ -41,6 +48,34 @@ sched_reserve: graph splits = 1
|
|
| 41 |
expect(parseNativeLog('load_tensors: offloaded 29/29 layers to GPU').nGraphSplits).toBeNull();
|
| 42 |
});
|
| 43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
it('parses the native llama.cpp WebGPU device-lost callback', () => {
|
| 45 |
expect(parseNativeDeviceLostSignal(
|
| 46 |
'ggml_webgpu: Device lost! Reason: 2, Message: GPU process crashed',
|
|
@@ -61,6 +96,10 @@ sched_reserve: graph splits = 1
|
|
| 61 |
nGraphSplits: null,
|
| 62 |
opsOnCpu: 0,
|
| 63 |
layersGpu: null,
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
});
|
| 65 |
expect(collector.deviceLostSignal()).toMatchObject({ reason: 1, message: 'unknown' });
|
| 66 |
|
|
|
|
| 12 |
load_tensors: offloaded 65/65 layers to GPU
|
| 13 |
load_tensors: CPU model buffer size = 72.00 MiB
|
| 14 |
load_tensors: WebGPU model buffer size = 3616.77 MiB
|
| 15 |
+
llama_context: flash_attn = disabled
|
| 16 |
+
llama_kv_cache: WebGPU KV buffer size = 56.00 MiB
|
| 17 |
+
llama_kv_cache: size = 56.00 MiB, K (f16): 28.00 MiB, V (f16): 28.00 MiB
|
| 18 |
sched_reserve: WebGPU compute buffer size = 301.00 MiB
|
| 19 |
sched_reserve: CPU compute buffer size = 13.01 MiB
|
| 20 |
sched_reserve: graph splits = 1
|
|
|
|
| 25 |
nGraphSplits: 1,
|
| 26 |
opsOnCpu: 0,
|
| 27 |
layersGpu: { offloaded: 65, total: 65 },
|
| 28 |
+
flashAttention: false,
|
| 29 |
+
cacheTypeK: 'f16',
|
| 30 |
+
cacheTypeV: 'f16',
|
| 31 |
+
webgpuKvBufferBytes: 56 * 1024 ** 2,
|
| 32 |
});
|
| 33 |
});
|
| 34 |
|
|
|
|
| 48 |
expect(parseNativeLog('load_tensors: offloaded 29/29 layers to GPU').nGraphSplits).toBeNull();
|
| 49 |
});
|
| 50 |
|
| 51 |
+
it('reports only resolved Flash Attention state and lets later evidence win', () => {
|
| 52 |
+
expect(parseNativeLog('llama_context: flash_attn = auto').flashAttention).toBeNull();
|
| 53 |
+
expect(parseNativeLog([
|
| 54 |
+
'llama_context: flash_attn = auto',
|
| 55 |
+
'resolve_fused_ops: Flash Attention enabled',
|
| 56 |
+
]).flashAttention).toBe(true);
|
| 57 |
+
expect(parseNativeLog([
|
| 58 |
+
'llama_context: flash_attn = enabled',
|
| 59 |
+
'resolve_fused_ops: Flash Attention not supported, set to disabled',
|
| 60 |
+
]).flashAttention).toBe(false);
|
| 61 |
+
});
|
| 62 |
+
|
| 63 |
+
it('retains cache tuning evidence and sums binary WebGPU KV buffer units', () => {
|
| 64 |
+
const collector = new NativeLogCollector();
|
| 65 |
+
collector.append(['llama_context: flash_attn = disabled']);
|
| 66 |
+
collector.append(['llama_kv_cache: K (Q8_0): 1.00 MiB, V (Q4_0): 1.00 MiB']);
|
| 67 |
+
collector.append(['llama_kv_cache: WebGPU KV buffer size = 512 KiB']);
|
| 68 |
+
collector.append(['llama_kv_cache: WebGPU KV buffer size = 2.50 MiB']);
|
| 69 |
+
collector.append(['llama_kv_cache: WebGPU KV buffer size = 1.00 GiB']);
|
| 70 |
+
|
| 71 |
+
expect(collector.report()).toMatchObject({
|
| 72 |
+
flashAttention: false,
|
| 73 |
+
cacheTypeK: 'q8_0',
|
| 74 |
+
cacheTypeV: 'q4_0',
|
| 75 |
+
webgpuKvBufferBytes: (512 * 1024) + (2.5 * 1024 ** 2) + (1024 ** 3),
|
| 76 |
+
});
|
| 77 |
+
});
|
| 78 |
+
|
| 79 |
it('parses the native llama.cpp WebGPU device-lost callback', () => {
|
| 80 |
expect(parseNativeDeviceLostSignal(
|
| 81 |
'ggml_webgpu: Device lost! Reason: 2, Message: GPU process crashed',
|
|
|
|
| 96 |
nGraphSplits: null,
|
| 97 |
opsOnCpu: 0,
|
| 98 |
layersGpu: null,
|
| 99 |
+
flashAttention: null,
|
| 100 |
+
cacheTypeK: null,
|
| 101 |
+
cacheTypeV: null,
|
| 102 |
+
webgpuKvBufferBytes: null,
|
| 103 |
});
|
| 104 |
expect(collector.deviceLostSignal()).toMatchObject({ reason: 1, message: 'unknown' });
|
| 105 |
|
src/engine/native-log.ts
CHANGED
|
@@ -8,6 +8,10 @@ export interface BackendReport {
|
|
| 8 |
nGraphSplits: number | null;
|
| 9 |
opsOnCpu: number;
|
| 10 |
layersGpu: GpuLayerReport | null;
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
}
|
| 12 |
|
| 13 |
export interface NativeDeviceLostSignal {
|
|
@@ -22,8 +26,15 @@ export interface NativeDeviceLostWatch {
|
|
| 22 |
}
|
| 23 |
|
| 24 |
const BACKEND_NAMES = ['WebGPU', 'CPU', 'Metal', 'CUDA', 'Vulkan'] as const;
|
| 25 |
-
const EVIDENCE = /using device|model buffer size|KV buffer size|compute buffer size|output buffer size|offloaded \d+\/\d+ layers|graph splits\s*=|graph_compute\(|backend\s*=\s*CPU|assigned to (?:device )?CPU|(?:ops?|nodes?).*(?:on|to) CPU/i;
|
| 26 |
const WEBGPU_DEVICE_LOST = /\bdevice\s+lost!?(?:\s+reason:\s*(-?\d+),\s*message:\s*(.*))?/i;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
function canonicalBackend(value: string): string {
|
| 29 |
const match = BACKEND_NAMES.find((backend) => value.toLowerCase().startsWith(backend.toLowerCase()));
|
|
@@ -76,6 +87,10 @@ export function parseNativeLog(input: string | readonly string[]): BackendReport
|
|
| 76 |
const graphSplits: number[] = [];
|
| 77 |
let opsOnCpu = 0;
|
| 78 |
let layersGpu: GpuLayerReport | null = null;
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
|
| 80 |
for (const line of lines) {
|
| 81 |
const deviceMatch = line.match(/using device\s+([^\s(]+)\s+\(([^)]+)\)/i);
|
|
@@ -102,6 +117,30 @@ export function parseNativeLog(input: string | readonly string[]): BackendReport
|
|
| 102 |
if (graphMatch) {
|
| 103 |
graphSplits.push(Number(graphMatch[1]));
|
| 104 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
opsOnCpu += cpuOpsInLine(line);
|
| 106 |
}
|
| 107 |
|
|
@@ -110,6 +149,10 @@ export function parseNativeLog(input: string | readonly string[]): BackendReport
|
|
| 110 |
nGraphSplits: graphSplits.length > 0 ? Math.max(...graphSplits) : null,
|
| 111 |
opsOnCpu,
|
| 112 |
layersGpu,
|
|
|
|
|
|
|
|
|
|
|
|
|
| 113 |
};
|
| 114 |
}
|
| 115 |
|
|
|
|
| 8 |
nGraphSplits: number | null;
|
| 9 |
opsOnCpu: number;
|
| 10 |
layersGpu: GpuLayerReport | null;
|
| 11 |
+
flashAttention: boolean | null;
|
| 12 |
+
cacheTypeK: string | null;
|
| 13 |
+
cacheTypeV: string | null;
|
| 14 |
+
webgpuKvBufferBytes: number | null;
|
| 15 |
}
|
| 16 |
|
| 17 |
export interface NativeDeviceLostSignal {
|
|
|
|
| 26 |
}
|
| 27 |
|
| 28 |
const BACKEND_NAMES = ['WebGPU', 'CPU', 'Metal', 'CUDA', 'Vulkan'] as const;
|
| 29 |
+
const EVIDENCE = /using device|model buffer size|KV buffer size|compute buffer size|output buffer size|offloaded \d+\/\d+ layers|graph splits\s*=|graph_compute\(|backend\s*=\s*CPU|assigned to (?:device )?CPU|(?:ops?|nodes?).*(?:on|to) CPU|flash_attn\s*=|flash attention.*(?:enabled|disabled)|\bK\s*\([^)]+\).*\bV\s*\([^)]+\)/i;
|
| 30 |
const WEBGPU_DEVICE_LOST = /\bdevice\s+lost!?(?:\s+reason:\s*(-?\d+),\s*message:\s*(.*))?/i;
|
| 31 |
+
const WEBGPU_KV_BUFFER = /\bWebGPU\s+KV buffer size\s*=\s*(\d+(?:\.\d+)?)\s*(KiB|MiB|GiB)\b/i;
|
| 32 |
+
|
| 33 |
+
const BINARY_UNIT_BYTES = {
|
| 34 |
+
kib: 1024,
|
| 35 |
+
mib: 1024 ** 2,
|
| 36 |
+
gib: 1024 ** 3,
|
| 37 |
+
} as const;
|
| 38 |
|
| 39 |
function canonicalBackend(value: string): string {
|
| 40 |
const match = BACKEND_NAMES.find((backend) => value.toLowerCase().startsWith(backend.toLowerCase()));
|
|
|
|
| 87 |
const graphSplits: number[] = [];
|
| 88 |
let opsOnCpu = 0;
|
| 89 |
let layersGpu: GpuLayerReport | null = null;
|
| 90 |
+
let flashAttention: boolean | null = null;
|
| 91 |
+
let cacheTypeK: string | null = null;
|
| 92 |
+
let cacheTypeV: string | null = null;
|
| 93 |
+
let webgpuKvBufferBytes: number | null = null;
|
| 94 |
|
| 95 |
for (const line of lines) {
|
| 96 |
const deviceMatch = line.match(/using device\s+([^\s(]+)\s+\(([^)]+)\)/i);
|
|
|
|
| 117 |
if (graphMatch) {
|
| 118 |
graphSplits.push(Number(graphMatch[1]));
|
| 119 |
}
|
| 120 |
+
|
| 121 |
+
const contextFlashMatch = line.match(/\bflash_attn\s*=\s*(enabled|disabled)\b/i);
|
| 122 |
+
const resolvedFlashMatch = line.match(/\bFlash Attention\b.*\b(enabled|disabled)\b/i);
|
| 123 |
+
const flashState = (contextFlashMatch ?? resolvedFlashMatch)?.[1];
|
| 124 |
+
if (flashState) {
|
| 125 |
+
flashAttention = flashState.toLowerCase() === 'enabled';
|
| 126 |
+
}
|
| 127 |
+
|
| 128 |
+
const cacheTypesMatch = line.match(/\bK\s*\(\s*([^)]+?)\s*\).*\bV\s*\(\s*([^)]+?)\s*\)/i);
|
| 129 |
+
const cacheK = cacheTypesMatch?.[1];
|
| 130 |
+
const cacheV = cacheTypesMatch?.[2];
|
| 131 |
+
if (cacheK && cacheV) {
|
| 132 |
+
cacheTypeK = cacheK.trim().toLowerCase();
|
| 133 |
+
cacheTypeV = cacheV.trim().toLowerCase();
|
| 134 |
+
}
|
| 135 |
+
|
| 136 |
+
const kvBufferMatch = line.match(WEBGPU_KV_BUFFER);
|
| 137 |
+
const kvBufferSize = kvBufferMatch?.[1];
|
| 138 |
+
const kvBufferUnit = kvBufferMatch?.[2];
|
| 139 |
+
if (kvBufferSize && kvBufferUnit) {
|
| 140 |
+
const unitBytes = BINARY_UNIT_BYTES[kvBufferUnit.toLowerCase() as keyof typeof BINARY_UNIT_BYTES];
|
| 141 |
+
const bytes = Math.round(Number(kvBufferSize) * unitBytes);
|
| 142 |
+
webgpuKvBufferBytes = (webgpuKvBufferBytes ?? 0) + bytes;
|
| 143 |
+
}
|
| 144 |
opsOnCpu += cpuOpsInLine(line);
|
| 145 |
}
|
| 146 |
|
|
|
|
| 149 |
nGraphSplits: graphSplits.length > 0 ? Math.max(...graphSplits) : null,
|
| 150 |
opsOnCpu,
|
| 151 |
layersGpu,
|
| 152 |
+
flashAttention,
|
| 153 |
+
cacheTypeK,
|
| 154 |
+
cacheTypeV,
|
| 155 |
+
webgpuKvBufferBytes,
|
| 156 |
};
|
| 157 |
}
|
| 158 |
|
src/engine/protocol.ts
CHANGED
|
@@ -64,6 +64,18 @@ export interface FutureTensorPlacement {
|
|
| 64 |
}>;
|
| 65 |
}
|
| 66 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
export interface EngineChatToolCall {
|
| 68 |
id: string;
|
| 69 |
type: 'function';
|
|
@@ -106,11 +118,7 @@ export type LoadModelParams = ManifestSource & {
|
|
| 106 |
backend: RequestedBackend;
|
| 107 |
contextSize?: number;
|
| 108 |
threads?: number;
|
| 109 |
-
|
| 110 |
-
nUbatch?: number;
|
| 111 |
-
flashAttention?: boolean;
|
| 112 |
-
cacheTypeK?: 'f32' | 'f16' | 'q8_0' | 'q5_1' | 'q5_0' | 'q4_1' | 'q4_0';
|
| 113 |
-
cacheTypeV?: 'f32' | 'f16' | 'q8_0' | 'q5_1' | 'q5_0' | 'q4_1' | 'q4_0';
|
| 114 |
tensorPlacement?: FutureTensorPlacement;
|
| 115 |
};
|
| 116 |
|
|
@@ -141,9 +149,27 @@ export interface LoadModelResult {
|
|
| 141 |
microBatchSize: number;
|
| 142 |
};
|
| 143 |
tuning: {
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
};
|
| 148 |
chatTemplate: {
|
| 149 |
bytes: number;
|
|
|
|
| 64 |
}>;
|
| 65 |
}
|
| 66 |
|
| 67 |
+
export type BenchmarkFlashMode = 'off' | 'auto';
|
| 68 |
+
export type BenchmarkKvCacheType = 'f16' | 'q8_0' | 'q4_0';
|
| 69 |
+
export type BenchmarkWasmFlavor = 'auto' | 'jspi' | 'compat';
|
| 70 |
+
|
| 71 |
+
export interface BenchmarkLoadTuning {
|
| 72 |
+
nBatch?: number;
|
| 73 |
+
nUbatch?: number;
|
| 74 |
+
flashMode: BenchmarkFlashMode;
|
| 75 |
+
kvCacheType: BenchmarkKvCacheType;
|
| 76 |
+
wasmFlavor: BenchmarkWasmFlavor;
|
| 77 |
+
}
|
| 78 |
+
|
| 79 |
export interface EngineChatToolCall {
|
| 80 |
id: string;
|
| 81 |
type: 'function';
|
|
|
|
| 118 |
backend: RequestedBackend;
|
| 119 |
contextSize?: number;
|
| 120 |
threads?: number;
|
| 121 |
+
benchmarkTuning?: BenchmarkLoadTuning;
|
|
|
|
|
|
|
|
|
|
|
|
|
| 122 |
tensorPlacement?: FutureTensorPlacement;
|
| 123 |
};
|
| 124 |
|
|
|
|
| 149 |
microBatchSize: number;
|
| 150 |
};
|
| 151 |
tuning: {
|
| 152 |
+
scope: 'release-defaults' | 'benchmark';
|
| 153 |
+
requested: {
|
| 154 |
+
nBatch: number | null;
|
| 155 |
+
nUbatch: number | null;
|
| 156 |
+
flashMode: BenchmarkFlashMode;
|
| 157 |
+
cacheTypeK: BenchmarkKvCacheType;
|
| 158 |
+
cacheTypeV: BenchmarkKvCacheType;
|
| 159 |
+
wasmFlavor: BenchmarkWasmFlavor;
|
| 160 |
+
};
|
| 161 |
+
observed: {
|
| 162 |
+
nBatch: number;
|
| 163 |
+
nUbatch: number;
|
| 164 |
+
flashAttention: boolean | null;
|
| 165 |
+
cacheTypeK: string | null;
|
| 166 |
+
cacheTypeV: string | null;
|
| 167 |
+
kvBufferBytes: number | null;
|
| 168 |
+
wasmFlavor: 'jspi' | 'compat';
|
| 169 |
+
wasmSha256: string;
|
| 170 |
+
compatWorkerSha256: string | null;
|
| 171 |
+
};
|
| 172 |
+
applied: boolean;
|
| 173 |
};
|
| 174 |
chatTemplate: {
|
| 175 |
bytes: number;
|
src/engine/runtime-device-lost.test.ts
CHANGED
|
@@ -66,6 +66,10 @@ describe('BrowserEngineRuntime WebGPU device loss', () => {
|
|
| 66 |
nGraphSplits: null,
|
| 67 |
opsOnCpu: 0,
|
| 68 |
layersGpu: null,
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
});
|
| 70 |
expect(exit).not.toHaveBeenCalled();
|
| 71 |
expect(internals.loaded).not.toBeNull();
|
|
|
|
| 66 |
nGraphSplits: null,
|
| 67 |
opsOnCpu: 0,
|
| 68 |
layersGpu: null,
|
| 69 |
+
flashAttention: null,
|
| 70 |
+
cacheTypeK: null,
|
| 71 |
+
cacheTypeV: null,
|
| 72 |
+
webgpuKvBufferBytes: null,
|
| 73 |
});
|
| 74 |
expect(exit).not.toHaveBeenCalled();
|
| 75 |
expect(internals.loaded).not.toBeNull();
|
src/engine/runtime-download.test.ts
CHANGED
|
@@ -4,7 +4,7 @@ import { verifyBlobSha256 } from './blob-integrity';
|
|
| 4 |
import { EngineRuntimeError } from './errors';
|
| 5 |
import type { ManifestModelV2 } from './manifest';
|
| 6 |
import { isShardDownloadFailureDetails, type EngineEvent } from './protocol';
|
| 7 |
-
import { BrowserEngineRuntime } from './runtime';
|
| 8 |
|
| 9 |
type DownloadOptions = {
|
| 10 |
signal?: AbortSignal;
|
|
@@ -313,29 +313,169 @@ describe('BrowserEngineRuntime per-shard retry', () => {
|
|
| 313 |
});
|
| 314 |
});
|
| 315 |
|
| 316 |
-
describe('
|
| 317 |
-
const
|
| 318 |
-
'
|
| 319 |
-
|
| 320 |
-
|
| 321 |
-
|
| 322 |
-
|
| 323 |
-
|
| 324 |
-
|
| 325 |
-
|
| 326 |
-
|
| 327 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 328 |
|
| 329 |
-
it('rejects
|
| 330 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 331 |
});
|
| 332 |
|
| 333 |
-
it('
|
| 334 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 335 |
});
|
| 336 |
|
| 337 |
-
it('rejects
|
| 338 |
-
|
| 339 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 340 |
});
|
| 341 |
});
|
|
|
|
| 4 |
import { EngineRuntimeError } from './errors';
|
| 5 |
import type { ManifestModelV2 } from './manifest';
|
| 6 |
import { isShardDownloadFailureDetails, type EngineEvent } from './protocol';
|
| 7 |
+
import { BrowserEngineRuntime, resolveLoadTuning, selectWasmFlavor } from './runtime';
|
| 8 |
|
| 9 |
type DownloadOptions = {
|
| 10 |
signal?: AbortSignal;
|
|
|
|
| 313 |
});
|
| 314 |
});
|
| 315 |
|
| 316 |
+
describe('resolveLoadTuning', () => {
|
| 317 |
+
const benchmarkDefaults = {
|
| 318 |
+
flashMode: 'off',
|
| 319 |
+
kvCacheType: 'f16',
|
| 320 |
+
wasmFlavor: 'auto',
|
| 321 |
+
} as const;
|
| 322 |
+
|
| 323 |
+
function tuningError(
|
| 324 |
+
input: unknown,
|
| 325 |
+
backend: 'auto' | 'webgpu' | 'wasm' = 'webgpu',
|
| 326 |
+
): EngineRuntimeError {
|
| 327 |
+
try {
|
| 328 |
+
resolveLoadTuning(input, backend);
|
| 329 |
+
} catch (error) {
|
| 330 |
+
if (error instanceof EngineRuntimeError) return error;
|
| 331 |
+
throw error;
|
| 332 |
+
}
|
| 333 |
+
throw new Error('Expected benchmark tuning to be rejected.');
|
| 334 |
+
}
|
| 335 |
+
|
| 336 |
+
it('uses immutable release defaults when benchmark tuning is omitted', () => {
|
| 337 |
+
expect(resolveLoadTuning(undefined, 'auto')).toEqual({
|
| 338 |
+
scope: 'release-defaults',
|
| 339 |
+
nBatch: null,
|
| 340 |
+
nUbatch: null,
|
| 341 |
+
flashMode: 'off',
|
| 342 |
+
kvCacheType: 'f16',
|
| 343 |
+
wasmFlavor: 'auto',
|
| 344 |
+
});
|
| 345 |
+
expect(resolveLoadTuning(undefined, 'wasm')).toEqual({
|
| 346 |
+
scope: 'release-defaults',
|
| 347 |
+
nBatch: null,
|
| 348 |
+
nUbatch: null,
|
| 349 |
+
flashMode: 'off',
|
| 350 |
+
kvCacheType: 'f16',
|
| 351 |
+
wasmFlavor: 'auto',
|
| 352 |
+
});
|
| 353 |
+
});
|
| 354 |
+
|
| 355 |
+
it('accepts an explicit benchmark batch shape without changing release defaults', () => {
|
| 356 |
+
expect(resolveLoadTuning({
|
| 357 |
+
...benchmarkDefaults,
|
| 358 |
+
nBatch: 512,
|
| 359 |
+
nUbatch: 128,
|
| 360 |
+
}, 'auto')).toEqual({
|
| 361 |
+
scope: 'benchmark',
|
| 362 |
+
nBatch: 512,
|
| 363 |
+
nUbatch: 128,
|
| 364 |
+
flashMode: 'off',
|
| 365 |
+
kvCacheType: 'f16',
|
| 366 |
+
wasmFlavor: 'auto',
|
| 367 |
+
});
|
| 368 |
+
});
|
| 369 |
|
| 370 |
+
it('rejects a microbatch larger than the requested batch', () => {
|
| 371 |
+
expect(tuningError({
|
| 372 |
+
...benchmarkDefaults,
|
| 373 |
+
nBatch: 128,
|
| 374 |
+
nUbatch: 256,
|
| 375 |
+
})).toMatchObject({
|
| 376 |
+
code: 'INVALID_BENCHMARK_TUNING',
|
| 377 |
+
message: 'Benchmark n_ubatch must not exceed n_batch.',
|
| 378 |
+
});
|
| 379 |
});
|
| 380 |
|
| 381 |
+
it('accepts Flash Attention and quantized KV only on the explicit WebGPU backend', () => {
|
| 382 |
+
expect(resolveLoadTuning({
|
| 383 |
+
...benchmarkDefaults,
|
| 384 |
+
flashMode: 'auto',
|
| 385 |
+
}, 'webgpu')).toMatchObject({
|
| 386 |
+
scope: 'benchmark',
|
| 387 |
+
flashMode: 'auto',
|
| 388 |
+
kvCacheType: 'f16',
|
| 389 |
+
wasmFlavor: 'auto',
|
| 390 |
+
});
|
| 391 |
+
for (const kvCacheType of ['q8_0', 'q4_0'] as const) {
|
| 392 |
+
expect(resolveLoadTuning({
|
| 393 |
+
...benchmarkDefaults,
|
| 394 |
+
flashMode: 'auto',
|
| 395 |
+
kvCacheType,
|
| 396 |
+
}, 'webgpu')).toMatchObject({
|
| 397 |
+
scope: 'benchmark',
|
| 398 |
+
flashMode: 'auto',
|
| 399 |
+
kvCacheType,
|
| 400 |
+
wasmFlavor: 'auto',
|
| 401 |
+
});
|
| 402 |
+
}
|
| 403 |
+
|
| 404 |
+
expect(tuningError({
|
| 405 |
+
...benchmarkDefaults,
|
| 406 |
+
flashMode: 'auto',
|
| 407 |
+
}, 'auto')).toMatchObject({
|
| 408 |
+
code: 'INVALID_BENCHMARK_TUNING',
|
| 409 |
+
});
|
| 410 |
+
expect(tuningError({
|
| 411 |
+
...benchmarkDefaults,
|
| 412 |
+
flashMode: 'auto',
|
| 413 |
+
kvCacheType: 'q8_0',
|
| 414 |
+
}, 'wasm')).toMatchObject({
|
| 415 |
+
code: 'INVALID_BENCHMARK_TUNING',
|
| 416 |
+
});
|
| 417 |
});
|
| 418 |
|
| 419 |
+
it('rejects malformed and incomplete benchmark tuning objects', () => {
|
| 420 |
+
for (const input of [null, [], 'off', 42]) {
|
| 421 |
+
expect(tuningError(input)).toMatchObject({ code: 'INVALID_BENCHMARK_TUNING' });
|
| 422 |
+
}
|
| 423 |
+
for (const input of [
|
| 424 |
+
{ kvCacheType: 'f16' },
|
| 425 |
+
{ flashMode: 'off' },
|
| 426 |
+
{ flashMode: 'off', kvCacheType: 'f16' },
|
| 427 |
+
{ ...benchmarkDefaults, flashMode: 'enabled' },
|
| 428 |
+
{ ...benchmarkDefaults, kvCacheType: 'q5_0' },
|
| 429 |
+
{ ...benchmarkDefaults, wasmFlavor: 'future' },
|
| 430 |
+
]) {
|
| 431 |
+
expect(tuningError(input)).toMatchObject({ code: 'INVALID_BENCHMARK_TUNING' });
|
| 432 |
+
}
|
| 433 |
+
});
|
| 434 |
+
|
| 435 |
+
it('rejects quantized V cache unless Flash Attention is in auto mode', () => {
|
| 436 |
+
for (const kvCacheType of ['q8_0', 'q4_0'] as const) {
|
| 437 |
+
expect(tuningError({ ...benchmarkDefaults, kvCacheType })).toMatchObject({
|
| 438 |
+
code: 'INVALID_BENCHMARK_TUNING',
|
| 439 |
+
message: expect.stringMatching(/requires Flash Attention/i),
|
| 440 |
+
});
|
| 441 |
+
}
|
| 442 |
+
});
|
| 443 |
+
|
| 444 |
+
it('rejects invalid and overflowing signed glue batch values', () => {
|
| 445 |
+
for (const nBatch of [0, -1, 1.5, Number.NaN, Number.POSITIVE_INFINITY, 2_147_483_648]) {
|
| 446 |
+
expect(tuningError({ ...benchmarkDefaults, nBatch })).toMatchObject({
|
| 447 |
+
code: 'INVALID_BATCH_SIZE',
|
| 448 |
+
});
|
| 449 |
+
}
|
| 450 |
+
for (const nUbatch of [0, -1, 1.5, Number.NaN, Number.POSITIVE_INFINITY, Number.MAX_SAFE_INTEGER]) {
|
| 451 |
+
expect(tuningError({ ...benchmarkDefaults, nUbatch })).toMatchObject({
|
| 452 |
+
code: 'INVALID_UBATCH_SIZE',
|
| 453 |
+
});
|
| 454 |
+
}
|
| 455 |
+
});
|
| 456 |
+
});
|
| 457 |
+
|
| 458 |
+
describe('selectWasmFlavor', () => {
|
| 459 |
+
it('uses the browser-native flavor for auto', () => {
|
| 460 |
+
expect(selectWasmFlavor('auto', false)).toBe('jspi');
|
| 461 |
+
expect(selectWasmFlavor('auto', true)).toBe('compat');
|
| 462 |
+
});
|
| 463 |
+
|
| 464 |
+
it('forces compat even when JSPI is available', () => {
|
| 465 |
+
expect(selectWasmFlavor('compat', false)).toBe('compat');
|
| 466 |
+
expect(selectWasmFlavor('compat', true)).toBe('compat');
|
| 467 |
+
});
|
| 468 |
+
|
| 469 |
+
it('selects JSPI when the browser supports it', () => {
|
| 470 |
+
expect(selectWasmFlavor('jspi', false)).toBe('jspi');
|
| 471 |
+
});
|
| 472 |
+
|
| 473 |
+
it('fails loud when JSPI is requested but unavailable', () => {
|
| 474 |
+
expect(() => selectWasmFlavor('jspi', true)).toThrowError(
|
| 475 |
+
expect.objectContaining({
|
| 476 |
+
code: 'BENCHMARK_WASM_FLAVOR_UNAVAILABLE',
|
| 477 |
+
message: expect.stringMatching(/requires the compatibility runtime/i),
|
| 478 |
+
}),
|
| 479 |
+
);
|
| 480 |
});
|
| 481 |
});
|
src/engine/runtime.ts
CHANGED
|
@@ -31,6 +31,9 @@ import {
|
|
| 31 |
import { NativeLogCollector, type BackendReport } from './native-log';
|
| 32 |
import { ToolCallAccumulator } from './tool-call-accumulator';
|
| 33 |
import type {
|
|
|
|
|
|
|
|
|
|
| 34 |
EngineCapabilities,
|
| 35 |
EngineEvent,
|
| 36 |
GenerateParams,
|
|
@@ -50,6 +53,95 @@ const WLLAMA_COMPAT_WORKER_PATH = '/wasm/wllama-compat.js';
|
|
| 50 |
const MAX_GLUE_INT = 2_147_483_647;
|
| 51 |
const DEVICE_LOST_EXIT_GRACE_MS = 100;
|
| 52 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
function runtimeArtifact(path: string): { bytes: number; sha256: string } {
|
| 54 |
const artifact = runtimeSource.files.find((file) => file.path === path);
|
| 55 |
if (!artifact) throw new Error(`Runtime provenance is missing ${path}`);
|
|
@@ -68,6 +160,23 @@ function needsCompatWasm(): boolean {
|
|
| 68 |
}
|
| 69 |
}
|
| 70 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
type EventSink = (event: EngineEvent) => void;
|
| 72 |
|
| 73 |
interface LoadedModelState {
|
|
@@ -215,6 +324,7 @@ function asReasoningDelta(chunk: ChatCompletionChunk): string | undefined {
|
|
| 215 |
|
| 216 |
export class BrowserEngineRuntime {
|
| 217 |
private wllama: Wllama | null = null;
|
|
|
|
| 218 |
private compatWorkerCode: string | null = null;
|
| 219 |
private loaded: LoadedModelState | null = null;
|
| 220 |
private readonly nativeLog = new NativeLogCollector();
|
|
@@ -253,6 +363,7 @@ export class BrowserEngineRuntime {
|
|
| 253 |
const invalidatedWllama = this.wllama;
|
| 254 |
this.loaded = null;
|
| 255 |
this.wllama = null;
|
|
|
|
| 256 |
if (invalidatedWllama) {
|
| 257 |
let timeout: ReturnType<typeof setTimeout> | undefined;
|
| 258 |
try {
|
|
@@ -287,10 +398,16 @@ export class BrowserEngineRuntime {
|
|
| 287 |
);
|
| 288 |
}
|
| 289 |
|
| 290 |
-
private async ensureWllama(): Promise<Wllama> {
|
| 291 |
-
if (this.wllama) {
|
| 292 |
return this.wllama;
|
| 293 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 294 |
installWorkerDocumentShim();
|
| 295 |
if (this.compatWorkerCode === null) {
|
| 296 |
const response = await fetch(absoluteAssetUrl(WLLAMA_COMPAT_WORKER_PATH));
|
|
@@ -306,6 +423,7 @@ export class BrowserEngineRuntime {
|
|
| 306 |
{ default: absoluteAssetUrl(WLLAMA_WASM_PATH) },
|
| 307 |
{
|
| 308 |
parallelDownloads: 3,
|
|
|
|
| 309 |
logger: {
|
| 310 |
debug: (...values: unknown[]) => this.nativeLog.append(values),
|
| 311 |
log: (...values: unknown[]) => this.nativeLog.append(values),
|
|
@@ -319,6 +437,7 @@ export class BrowserEngineRuntime {
|
|
| 319 |
worker: { code: this.compatWorkerCode },
|
| 320 |
}, 'firefox_safari');
|
| 321 |
this.wllama = wllama;
|
|
|
|
| 322 |
return wllama;
|
| 323 |
}
|
| 324 |
|
|
@@ -541,26 +660,8 @@ export class BrowserEngineRuntime {
|
|
| 541 |
params.tensorPlacement,
|
| 542 |
);
|
| 543 |
}
|
| 544 |
-
|
| 545 |
-
|
| 546 |
-
|| params.nBatch <= 0
|
| 547 |
-
|| params.nBatch > MAX_GLUE_INT
|
| 548 |
-
)) {
|
| 549 |
-
throw new EngineRuntimeError(
|
| 550 |
-
'INVALID_BATCH_SIZE',
|
| 551 |
-
`n_batch must be a positive signed 32-bit integer (maximum ${MAX_GLUE_INT}).`,
|
| 552 |
-
);
|
| 553 |
-
}
|
| 554 |
-
if (params.nUbatch !== undefined && (
|
| 555 |
-
!Number.isSafeInteger(params.nUbatch)
|
| 556 |
-
|| params.nUbatch <= 0
|
| 557 |
-
|| params.nUbatch > MAX_GLUE_INT
|
| 558 |
-
)) {
|
| 559 |
-
throw new EngineRuntimeError(
|
| 560 |
-
'INVALID_UBATCH_SIZE',
|
| 561 |
-
`n_ubatch must be a positive signed 32-bit integer (maximum ${MAX_GLUE_INT}).`,
|
| 562 |
-
);
|
| 563 |
-
}
|
| 564 |
await this.unload();
|
| 565 |
this.nativeLog.clear();
|
| 566 |
emitProgress(sink, requestId, {
|
|
@@ -585,7 +686,7 @@ export class BrowserEngineRuntime {
|
|
| 585 |
});
|
| 586 |
const model = findManifestModel(manifest, params.modelId);
|
| 587 |
const urls = orderedShardUrls(manifest, model);
|
| 588 |
-
const wllama = await this.ensureWllama();
|
| 589 |
const cached = await this.inspectCachedShards(wllama, model, urls, signal);
|
| 590 |
const [adapter, storage] = await Promise.all([
|
| 591 |
inspectWebGpuAdapter(),
|
|
@@ -630,13 +731,13 @@ export class BrowserEngineRuntime {
|
|
| 630 |
offload_token_embedding: gate.selectedBackend === 'webgpu',
|
| 631 |
n_ctx: contextSize,
|
| 632 |
n_threads: threads,
|
| 633 |
-
n_batch:
|
| 634 |
-
n_ubatch:
|
| 635 |
seed: 42,
|
| 636 |
-
flash_attn:
|
| 637 |
warmup: false,
|
| 638 |
-
cache_type_k:
|
| 639 |
-
cache_type_v:
|
| 640 |
}));
|
| 641 |
await this.assertWebGpuAlive('load', gate.selectedBackend, model);
|
| 642 |
throwIfAborted(signal);
|
|
@@ -652,6 +753,54 @@ export class BrowserEngineRuntime {
|
|
| 652 |
}
|
| 653 |
const context = wllama.getLoadedContextInfo();
|
| 654 |
const template = wllama.getChatTemplate() ?? '';
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 655 |
this.loaded = { manifest, model, backend: gate.selectedBackend };
|
| 656 |
emitProgress(sink, requestId, {
|
| 657 |
phase: 'load',
|
|
@@ -674,11 +823,7 @@ export class BrowserEngineRuntime {
|
|
| 674 |
batchSize: context.n_batch,
|
| 675 |
microBatchSize: context.n_ubatch,
|
| 676 |
},
|
| 677 |
-
tuning:
|
| 678 |
-
flashAttention: params.flashAttention ?? false,
|
| 679 |
-
cacheTypeK: params.cacheTypeK ?? 'f16',
|
| 680 |
-
cacheTypeV: params.cacheTypeV ?? 'f16',
|
| 681 |
-
},
|
| 682 |
chatTemplate: {
|
| 683 |
bytes: new TextEncoder().encode(template).byteLength,
|
| 684 |
hasThinkMarker: template.includes('<think>'),
|
|
@@ -691,6 +836,8 @@ export class BrowserEngineRuntime {
|
|
| 691 |
if (error instanceof EngineRuntimeError && error.code === 'WEBGPU_DEVICE_LOST') throw error;
|
| 692 |
await this.assertWebGpuAlive('load', gate.selectedBackend, model);
|
| 693 |
await wllama.exit().catch(() => undefined);
|
|
|
|
|
|
|
| 694 |
this.loaded = null;
|
| 695 |
throw error;
|
| 696 |
}
|
|
|
|
| 31 |
import { NativeLogCollector, type BackendReport } from './native-log';
|
| 32 |
import { ToolCallAccumulator } from './tool-call-accumulator';
|
| 33 |
import type {
|
| 34 |
+
BenchmarkFlashMode,
|
| 35 |
+
BenchmarkKvCacheType,
|
| 36 |
+
BenchmarkWasmFlavor,
|
| 37 |
EngineCapabilities,
|
| 38 |
EngineEvent,
|
| 39 |
GenerateParams,
|
|
|
|
| 53 |
const MAX_GLUE_INT = 2_147_483_647;
|
| 54 |
const DEVICE_LOST_EXIT_GRACE_MS = 100;
|
| 55 |
|
| 56 |
+
export interface ResolvedLoadTuning {
|
| 57 |
+
scope: 'release-defaults' | 'benchmark';
|
| 58 |
+
nBatch: number | null;
|
| 59 |
+
nUbatch: number | null;
|
| 60 |
+
flashMode: BenchmarkFlashMode;
|
| 61 |
+
kvCacheType: BenchmarkKvCacheType;
|
| 62 |
+
wasmFlavor: BenchmarkWasmFlavor;
|
| 63 |
+
}
|
| 64 |
+
|
| 65 |
+
function validateBatchValue(value: unknown, label: 'n_batch' | 'n_ubatch'): number | null {
|
| 66 |
+
if (value === undefined) return null;
|
| 67 |
+
if (!Number.isSafeInteger(value) || (value as number) <= 0 || (value as number) > MAX_GLUE_INT) {
|
| 68 |
+
throw new EngineRuntimeError(
|
| 69 |
+
label === 'n_batch' ? 'INVALID_BATCH_SIZE' : 'INVALID_UBATCH_SIZE',
|
| 70 |
+
`${label} must be a positive signed 32-bit integer (maximum ${MAX_GLUE_INT}).`,
|
| 71 |
+
);
|
| 72 |
+
}
|
| 73 |
+
return value as number;
|
| 74 |
+
}
|
| 75 |
+
|
| 76 |
+
export function resolveLoadTuning(
|
| 77 |
+
input: unknown,
|
| 78 |
+
backend: LoadModelParams['backend'],
|
| 79 |
+
): ResolvedLoadTuning {
|
| 80 |
+
if (input === undefined) {
|
| 81 |
+
return {
|
| 82 |
+
scope: 'release-defaults',
|
| 83 |
+
nBatch: null,
|
| 84 |
+
nUbatch: null,
|
| 85 |
+
flashMode: 'off',
|
| 86 |
+
kvCacheType: 'f16',
|
| 87 |
+
wasmFlavor: 'auto',
|
| 88 |
+
};
|
| 89 |
+
}
|
| 90 |
+
if (typeof input !== 'object' || input === null || Array.isArray(input)) {
|
| 91 |
+
throw new EngineRuntimeError('INVALID_BENCHMARK_TUNING', 'Benchmark tuning must be an object.');
|
| 92 |
+
}
|
| 93 |
+
const tuning = input as Record<string, unknown>;
|
| 94 |
+
const flashMode = tuning.flashMode;
|
| 95 |
+
const kvCacheType = tuning.kvCacheType;
|
| 96 |
+
const wasmFlavor = tuning.wasmFlavor;
|
| 97 |
+
if (flashMode !== 'off' && flashMode !== 'auto') {
|
| 98 |
+
throw new EngineRuntimeError(
|
| 99 |
+
'INVALID_BENCHMARK_TUNING',
|
| 100 |
+
'Benchmark flash mode must be off or auto.',
|
| 101 |
+
);
|
| 102 |
+
}
|
| 103 |
+
if (kvCacheType !== 'f16' && kvCacheType !== 'q8_0' && kvCacheType !== 'q4_0') {
|
| 104 |
+
throw new EngineRuntimeError(
|
| 105 |
+
'INVALID_BENCHMARK_TUNING',
|
| 106 |
+
'Benchmark KV cache type must be f16, q8_0, or q4_0.',
|
| 107 |
+
);
|
| 108 |
+
}
|
| 109 |
+
if (wasmFlavor !== 'auto' && wasmFlavor !== 'jspi' && wasmFlavor !== 'compat') {
|
| 110 |
+
throw new EngineRuntimeError(
|
| 111 |
+
'INVALID_BENCHMARK_TUNING',
|
| 112 |
+
'Benchmark WASM flavor must be auto, jspi, or compat.',
|
| 113 |
+
);
|
| 114 |
+
}
|
| 115 |
+
if (kvCacheType !== 'f16' && flashMode !== 'auto') {
|
| 116 |
+
throw new EngineRuntimeError(
|
| 117 |
+
'INVALID_BENCHMARK_TUNING',
|
| 118 |
+
'Quantized V cache requires Flash Attention in auto mode.',
|
| 119 |
+
);
|
| 120 |
+
}
|
| 121 |
+
if ((flashMode === 'auto' || kvCacheType !== 'f16') && backend !== 'webgpu') {
|
| 122 |
+
throw new EngineRuntimeError(
|
| 123 |
+
'INVALID_BENCHMARK_TUNING',
|
| 124 |
+
'Flash Attention and quantized KV experiments require the explicit WebGPU backend.',
|
| 125 |
+
);
|
| 126 |
+
}
|
| 127 |
+
const nBatch = validateBatchValue(tuning.nBatch, 'n_batch');
|
| 128 |
+
const nUbatch = validateBatchValue(tuning.nUbatch, 'n_ubatch');
|
| 129 |
+
if (nBatch !== null && nUbatch !== null && nUbatch > nBatch) {
|
| 130 |
+
throw new EngineRuntimeError(
|
| 131 |
+
'INVALID_BENCHMARK_TUNING',
|
| 132 |
+
'Benchmark n_ubatch must not exceed n_batch.',
|
| 133 |
+
);
|
| 134 |
+
}
|
| 135 |
+
return {
|
| 136 |
+
scope: 'benchmark',
|
| 137 |
+
nBatch,
|
| 138 |
+
nUbatch,
|
| 139 |
+
flashMode,
|
| 140 |
+
kvCacheType,
|
| 141 |
+
wasmFlavor,
|
| 142 |
+
};
|
| 143 |
+
}
|
| 144 |
+
|
| 145 |
function runtimeArtifact(path: string): { bytes: number; sha256: string } {
|
| 146 |
const artifact = runtimeSource.files.find((file) => file.path === path);
|
| 147 |
if (!artifact) throw new Error(`Runtime provenance is missing ${path}`);
|
|
|
|
| 160 |
}
|
| 161 |
}
|
| 162 |
|
| 163 |
+
export function selectWasmFlavor(
|
| 164 |
+
requested: BenchmarkWasmFlavor,
|
| 165 |
+
compatRequired = needsCompatWasm(),
|
| 166 |
+
): 'jspi' | 'compat' {
|
| 167 |
+
if (requested === 'compat') return 'compat';
|
| 168 |
+
if (requested === 'jspi') {
|
| 169 |
+
if (compatRequired) {
|
| 170 |
+
throw new EngineRuntimeError(
|
| 171 |
+
'BENCHMARK_WASM_FLAVOR_UNAVAILABLE',
|
| 172 |
+
'JSPI was requested, but this browser requires the compatibility runtime.',
|
| 173 |
+
);
|
| 174 |
+
}
|
| 175 |
+
return 'jspi';
|
| 176 |
+
}
|
| 177 |
+
return compatRequired ? 'compat' : 'jspi';
|
| 178 |
+
}
|
| 179 |
+
|
| 180 |
type EventSink = (event: EngineEvent) => void;
|
| 181 |
|
| 182 |
interface LoadedModelState {
|
|
|
|
| 324 |
|
| 325 |
export class BrowserEngineRuntime {
|
| 326 |
private wllama: Wllama | null = null;
|
| 327 |
+
private wllamaFlavor: 'jspi' | 'compat' | null = null;
|
| 328 |
private compatWorkerCode: string | null = null;
|
| 329 |
private loaded: LoadedModelState | null = null;
|
| 330 |
private readonly nativeLog = new NativeLogCollector();
|
|
|
|
| 363 |
const invalidatedWllama = this.wllama;
|
| 364 |
this.loaded = null;
|
| 365 |
this.wllama = null;
|
| 366 |
+
this.wllamaFlavor = null;
|
| 367 |
if (invalidatedWllama) {
|
| 368 |
let timeout: ReturnType<typeof setTimeout> | undefined;
|
| 369 |
try {
|
|
|
|
| 398 |
);
|
| 399 |
}
|
| 400 |
|
| 401 |
+
private async ensureWllama(flavor = selectWasmFlavor('auto')): Promise<Wllama> {
|
| 402 |
+
if (this.wllama && this.wllamaFlavor === flavor) {
|
| 403 |
return this.wllama;
|
| 404 |
}
|
| 405 |
+
if (this.wllama) {
|
| 406 |
+
await this.wllama.exit().catch(() => undefined);
|
| 407 |
+
this.wllama = null;
|
| 408 |
+
this.loaded = null;
|
| 409 |
+
this.wllamaFlavor = null;
|
| 410 |
+
}
|
| 411 |
installWorkerDocumentShim();
|
| 412 |
if (this.compatWorkerCode === null) {
|
| 413 |
const response = await fetch(absoluteAssetUrl(WLLAMA_COMPAT_WORKER_PATH));
|
|
|
|
| 423 |
{ default: absoluteAssetUrl(WLLAMA_WASM_PATH) },
|
| 424 |
{
|
| 425 |
parallelDownloads: 3,
|
| 426 |
+
forceCompat: flavor === 'compat',
|
| 427 |
logger: {
|
| 428 |
debug: (...values: unknown[]) => this.nativeLog.append(values),
|
| 429 |
log: (...values: unknown[]) => this.nativeLog.append(values),
|
|
|
|
| 437 |
worker: { code: this.compatWorkerCode },
|
| 438 |
}, 'firefox_safari');
|
| 439 |
this.wllama = wllama;
|
| 440 |
+
this.wllamaFlavor = flavor;
|
| 441 |
return wllama;
|
| 442 |
}
|
| 443 |
|
|
|
|
| 660 |
params.tensorPlacement,
|
| 661 |
);
|
| 662 |
}
|
| 663 |
+
const tuning = resolveLoadTuning(params.benchmarkTuning, params.backend);
|
| 664 |
+
const wasmFlavor = selectWasmFlavor(tuning.wasmFlavor);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 665 |
await this.unload();
|
| 666 |
this.nativeLog.clear();
|
| 667 |
emitProgress(sink, requestId, {
|
|
|
|
| 686 |
});
|
| 687 |
const model = findManifestModel(manifest, params.modelId);
|
| 688 |
const urls = orderedShardUrls(manifest, model);
|
| 689 |
+
const wllama = await this.ensureWllama(wasmFlavor);
|
| 690 |
const cached = await this.inspectCachedShards(wllama, model, urls, signal);
|
| 691 |
const [adapter, storage] = await Promise.all([
|
| 692 |
inspectWebGpuAdapter(),
|
|
|
|
| 731 |
offload_token_embedding: gate.selectedBackend === 'webgpu',
|
| 732 |
n_ctx: contextSize,
|
| 733 |
n_threads: threads,
|
| 734 |
+
n_batch: tuning.nBatch ?? undefined,
|
| 735 |
+
n_ubatch: tuning.nUbatch ?? undefined,
|
| 736 |
seed: 42,
|
| 737 |
+
flash_attn: tuning.flashMode === 'auto',
|
| 738 |
warmup: false,
|
| 739 |
+
cache_type_k: tuning.kvCacheType,
|
| 740 |
+
cache_type_v: tuning.kvCacheType,
|
| 741 |
}));
|
| 742 |
await this.assertWebGpuAlive('load', gate.selectedBackend, model);
|
| 743 |
throwIfAborted(signal);
|
|
|
|
| 753 |
}
|
| 754 |
const context = wllama.getLoadedContextInfo();
|
| 755 |
const template = wllama.getChatTemplate() ?? '';
|
| 756 |
+
const wasmArtifact = runtimeArtifact(
|
| 757 |
+
wasmFlavor === 'compat' ? 'public/wasm/wllama-compat.wasm' : 'public/wasm/wllama.wasm',
|
| 758 |
+
);
|
| 759 |
+
const compatWorkerSha256 = wasmFlavor === 'compat'
|
| 760 |
+
? runtimeArtifact('public/wasm/wllama-compat.js').sha256
|
| 761 |
+
: null;
|
| 762 |
+
const tuningApplied = (tuning.nBatch === null || context.n_batch === tuning.nBatch)
|
| 763 |
+
&& (tuning.nUbatch === null || context.n_ubatch === tuning.nUbatch)
|
| 764 |
+
&& report.flashAttention === (tuning.flashMode === 'auto')
|
| 765 |
+
&& report.cacheTypeK === tuning.kvCacheType
|
| 766 |
+
&& report.cacheTypeV === tuning.kvCacheType
|
| 767 |
+
&& (gate.selectedBackend !== 'webgpu'
|
| 768 |
+
|| (report.webgpuKvBufferBytes !== null && report.webgpuKvBufferBytes > 0))
|
| 769 |
+
&& (tuning.wasmFlavor === 'auto' || tuning.wasmFlavor === wasmFlavor);
|
| 770 |
+
const tuningResult: LoadModelResult['tuning'] = {
|
| 771 |
+
scope: tuning.scope,
|
| 772 |
+
requested: {
|
| 773 |
+
nBatch: tuning.nBatch,
|
| 774 |
+
nUbatch: tuning.nUbatch,
|
| 775 |
+
flashMode: tuning.flashMode,
|
| 776 |
+
cacheTypeK: tuning.kvCacheType,
|
| 777 |
+
cacheTypeV: tuning.kvCacheType,
|
| 778 |
+
wasmFlavor: tuning.wasmFlavor,
|
| 779 |
+
},
|
| 780 |
+
observed: {
|
| 781 |
+
nBatch: context.n_batch,
|
| 782 |
+
nUbatch: context.n_ubatch,
|
| 783 |
+
flashAttention: report.flashAttention,
|
| 784 |
+
cacheTypeK: report.cacheTypeK,
|
| 785 |
+
cacheTypeV: report.cacheTypeV,
|
| 786 |
+
kvBufferBytes: report.webgpuKvBufferBytes,
|
| 787 |
+
wasmFlavor,
|
| 788 |
+
wasmSha256: wasmArtifact.sha256,
|
| 789 |
+
compatWorkerSha256,
|
| 790 |
+
},
|
| 791 |
+
applied: tuningApplied,
|
| 792 |
+
};
|
| 793 |
+
if (!tuningApplied) {
|
| 794 |
+
throw new EngineRuntimeError(
|
| 795 |
+
tuning.scope === 'benchmark'
|
| 796 |
+
? 'BENCHMARK_TUNING_NOT_APPLIED'
|
| 797 |
+
: 'RUNTIME_POLICY_NOT_APPLIED',
|
| 798 |
+
tuning.scope === 'benchmark'
|
| 799 |
+
? 'The native runtime did not apply the requested benchmark tuning exactly.'
|
| 800 |
+
: 'The native runtime did not preserve the fixed release tuning policy.',
|
| 801 |
+
tuningResult,
|
| 802 |
+
);
|
| 803 |
+
}
|
| 804 |
this.loaded = { manifest, model, backend: gate.selectedBackend };
|
| 805 |
emitProgress(sink, requestId, {
|
| 806 |
phase: 'load',
|
|
|
|
| 823 |
batchSize: context.n_batch,
|
| 824 |
microBatchSize: context.n_ubatch,
|
| 825 |
},
|
| 826 |
+
tuning: tuningResult,
|
|
|
|
|
|
|
|
|
|
|
|
|
| 827 |
chatTemplate: {
|
| 828 |
bytes: new TextEncoder().encode(template).byteLength,
|
| 829 |
hasThinkMarker: template.includes('<think>'),
|
|
|
|
| 836 |
if (error instanceof EngineRuntimeError && error.code === 'WEBGPU_DEVICE_LOST') throw error;
|
| 837 |
await this.assertWebGpuAlive('load', gate.selectedBackend, model);
|
| 838 |
await wllama.exit().catch(() => undefined);
|
| 839 |
+
this.wllama = null;
|
| 840 |
+
this.wllamaFlavor = null;
|
| 841 |
this.loaded = null;
|
| 842 |
throw error;
|
| 843 |
}
|
vendor/wllama-bonsai/PATCH.diff
CHANGED
|
@@ -119,10 +119,40 @@ index 2b3cca7..e91b395 100644
|
|
| 119 |
n_threads?: number;
|
| 120 |
embeddings?: boolean;
|
| 121 |
diff --git a/src/wllama.ts b/src/wllama.ts
|
| 122 |
-
index ce74b2b..
|
| 123 |
--- a/src/wllama.ts
|
| 124 |
+++ b/src/wllama.ts
|
| 125 |
-
@@ -
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
if (this.proxy) {
|
| 127 |
throw new WllamaError('Module is already initialized', 'load_error');
|
| 128 |
}
|
|
@@ -150,7 +180,7 @@ index ce74b2b..8057028 100644
|
|
| 150 |
// detect if we can use multi-thread and webgpu
|
| 151 |
const supportMultiThread = await isSupportMultiThread();
|
| 152 |
const hwConccurency = Math.floor((navigator.hardwareConcurrency || 1) / 2);
|
| 153 |
-
@@ -518,7 +
|
| 154 |
// if async read is not supported, use mmap; refer to README-dev.md for more details
|
| 155 |
use_mmap: !canUseAsyncFileRead(workerResources.compat),
|
| 156 |
use_mlock: false,
|
|
@@ -160,7 +190,7 @@ index ce74b2b..8057028 100644
|
|
| 160 |
n_ctx: params.n_ctx ?? 1024,
|
| 161 |
n_threads: this.useMultiThread ? nbThreads : 1,
|
| 162 |
n_ctx_auto: false, // not supported for now
|
| 163 |
-
@@ -529,6 +
|
| 164 |
embeddings: params.embeddings,
|
| 165 |
offload_kqv: params.offload_kqv,
|
| 166 |
n_batch: params.n_batch,
|
|
@@ -168,7 +198,7 @@ index ce74b2b..8057028 100644
|
|
| 168 |
pooling_type: params.pooling_type as string,
|
| 169 |
rope_scaling_type: params.rope_scaling_type as string,
|
| 170 |
rope_freq_base: params.rope_freq_base,
|
| 171 |
-
@@ -579,6 +
|
| 172 |
skip_chat_parsing: params.skip_chat_parsing,
|
| 173 |
prefill_assistant: params.prefill_assistant,
|
| 174 |
});
|
|
@@ -179,3 +209,29 @@ index ce74b2b..8057028 100644
|
|
| 179 |
const loadedCtxInfo: LoadedContextInfo & GlueMsgLoadRes = {
|
| 180 |
...loadResult,
|
| 181 |
metadata: {},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 119 |
n_threads?: number;
|
| 120 |
embeddings?: boolean;
|
| 121 |
diff --git a/src/wllama.ts b/src/wllama.ts
|
| 122 |
+
index ce74b2b..1796ca1 100644
|
| 123 |
--- a/src/wllama.ts
|
| 124 |
+++ b/src/wllama.ts
|
| 125 |
+
@@ -55,6 +55,13 @@ export interface WllamaLogger {
|
| 126 |
+
|
| 127 |
+
// TODO: bring back useCache
|
| 128 |
+
export interface WllamaConfig {
|
| 129 |
+
+ /**
|
| 130 |
+
+ * Always use the configured compat worker JS and WASM pair, even when the
|
| 131 |
+
+ * browser supports the default JSPI runtime.
|
| 132 |
+
+ *
|
| 133 |
+
+ * Default: forceCompat = false
|
| 134 |
+
+ */
|
| 135 |
+
+ forceCompat?: boolean;
|
| 136 |
+
/**
|
| 137 |
+
* If true, suppress all log messages from native CPP code
|
| 138 |
+
*/
|
| 139 |
+
@@ -238,13 +245,13 @@ export class Wllama {
|
| 140 |
+
/**
|
| 141 |
+
* Set compatibility options for Wllama.
|
| 142 |
+
* @param compat Set to null to disable compatibility, or 'default' to use the default compat resources from CDN.
|
| 143 |
+
- * @param mode 'safari' by default; If set to 'firefox_safari', the compat mode will **also** be enabled on Firefox, which will significantly degrade the performance but allow using WebGPU on Firefox.
|
| 144 |
+
+ * @param mode 'safari' by default; If set to 'firefox_safari', the compat mode will **also** be enabled on Firefox, which will significantly degrade the performance but allow using WebGPU on Firefox. Ignored when forceCompat is true.
|
| 145 |
+
*/
|
| 146 |
+
setCompat(
|
| 147 |
+
compat: WllamaCompat | null | 'default',
|
| 148 |
+
mode: 'safari' | 'firefox_safari' = 'safari'
|
| 149 |
+
) {
|
| 150 |
+
- if (mode === 'safari') {
|
| 151 |
+
+ if (mode === 'safari' && !this.config.forceCompat) {
|
| 152 |
+
if (isFirefox()) {
|
| 153 |
+
this.compat = null;
|
| 154 |
+
return;
|
| 155 |
+
@@ -478,6 +485,27 @@ export class Wllama {
|
| 156 |
if (this.proxy) {
|
| 157 |
throw new WllamaError('Module is already initialized', 'load_error');
|
| 158 |
}
|
|
|
|
| 180 |
// detect if we can use multi-thread and webgpu
|
| 181 |
const supportMultiThread = await isSupportMultiThread();
|
| 182 |
const hwConccurency = Math.floor((navigator.hardwareConcurrency || 1) / 2);
|
| 183 |
+
@@ -518,7 +546,8 @@ export class Wllama {
|
| 184 |
// if async read is not supported, use mmap; refer to README-dev.md for more details
|
| 185 |
use_mmap: !canUseAsyncFileRead(workerResources.compat),
|
| 186 |
use_mlock: false,
|
|
|
|
| 190 |
n_ctx: params.n_ctx ?? 1024,
|
| 191 |
n_threads: this.useMultiThread ? nbThreads : 1,
|
| 192 |
n_ctx_auto: false, // not supported for now
|
| 193 |
+
@@ -529,6 +558,7 @@ export class Wllama {
|
| 194 |
embeddings: params.embeddings,
|
| 195 |
offload_kqv: params.offload_kqv,
|
| 196 |
n_batch: params.n_batch,
|
|
|
|
| 198 |
pooling_type: params.pooling_type as string,
|
| 199 |
rope_scaling_type: params.rope_scaling_type as string,
|
| 200 |
rope_freq_base: params.rope_freq_base,
|
| 201 |
+
@@ -579,6 +609,10 @@ export class Wllama {
|
| 202 |
skip_chat_parsing: params.skip_chat_parsing,
|
| 203 |
prefill_assistant: params.prefill_assistant,
|
| 204 |
});
|
|
|
|
| 209 |
const loadedCtxInfo: LoadedContextInfo & GlueMsgLoadRes = {
|
| 210 |
...loadResult,
|
| 211 |
metadata: {},
|
| 212 |
+
@@ -1094,7 +1128,24 @@ export class Wllama {
|
| 213 |
+
wasmPath: absoluteUrl(this.pathConfig['default']),
|
| 214 |
+
compat: false,
|
| 215 |
+
};
|
| 216 |
+
- if (needCompat()) {
|
| 217 |
+
+ const forceCompat = this.config.forceCompat === true;
|
| 218 |
+
+ if (forceCompat || needCompat()) {
|
| 219 |
+
+ const compatWorker = this.compat?.worker;
|
| 220 |
+
+ const hasCompatWorker =
|
| 221 |
+
+ (typeof compatWorker === 'string' && compatWorker.trim().length > 0) ||
|
| 222 |
+
+ (typeof compatWorker === 'object' &&
|
| 223 |
+
+ compatWorker !== null &&
|
| 224 |
+
+ typeof compatWorker.code === 'string' &&
|
| 225 |
+
+ compatWorker.code.trim().length > 0);
|
| 226 |
+
+ const compatWasm = this.compat?.wasm;
|
| 227 |
+
+ const hasCompatWasm =
|
| 228 |
+
+ typeof compatWasm === 'string' && compatWasm.trim().length > 0;
|
| 229 |
+
+ if (forceCompat && (!hasCompatWorker || !hasCompatWasm)) {
|
| 230 |
+
+ throw new WllamaError(
|
| 231 |
+
+ 'forceCompat requires both compat worker JS and compat WASM resources. Configure them with setCompat() before loading.',
|
| 232 |
+
+ 'load_error'
|
| 233 |
+
+ );
|
| 234 |
+
+ }
|
| 235 |
+
if (!this.compat) {
|
| 236 |
+
this.logger().warn(
|
| 237 |
+
'Not using compat mode' +
|
vendor/wllama-bonsai/SOURCE.json
CHANGED
|
@@ -4,12 +4,12 @@
|
|
| 4 |
"wllamaRevision": "912c18b75d4358c1405a64646b8dbe43a205943b",
|
| 5 |
"llamaCppRevision": "00fa7cb284cbf133fc426733bd64238a3588a33e",
|
| 6 |
"license": "MIT",
|
| 7 |
-
"patch": "Expose offload_token_embedding and n_ubatch; place token_embd.weight on WebGPU whenever GPU layers are requested; fail loudly when the requested WebGPU buffer is unavailable.",
|
| 8 |
"patchSet": {
|
| 9 |
"format": "git-diff-binary",
|
| 10 |
"path": "vendor/wllama-bonsai/PATCH.diff",
|
| 11 |
-
"bytes":
|
| 12 |
-
"sha256": "
|
| 13 |
"files": [
|
| 14 |
"CMakeLists.txt",
|
| 15 |
"cpp/glue.hpp",
|
|
@@ -28,8 +28,8 @@
|
|
| 28 |
"files": [
|
| 29 |
{
|
| 30 |
"path": "vendor/wllama-bonsai/esm/index.js",
|
| 31 |
-
"bytes":
|
| 32 |
-
"sha256": "
|
| 33 |
},
|
| 34 |
{
|
| 35 |
"path": "public/wasm/wllama.wasm",
|
|
|
|
| 4 |
"wllamaRevision": "912c18b75d4358c1405a64646b8dbe43a205943b",
|
| 5 |
"llamaCppRevision": "00fa7cb284cbf133fc426733bd64238a3588a33e",
|
| 6 |
"license": "MIT",
|
| 7 |
+
"patch": "Expose offload_token_embedding and n_ubatch; place token_embd.weight on WebGPU whenever GPU layers are requested; fail loudly when the requested WebGPU buffer is unavailable; add an opt-in forceCompat selector for controlled benchmark A/B runs.",
|
| 8 |
"patchSet": {
|
| 9 |
"format": "git-diff-binary",
|
| 10 |
"path": "vendor/wllama-bonsai/PATCH.diff",
|
| 11 |
+
"bytes": 9350,
|
| 12 |
+
"sha256": "ad8786a295eaeb75ee03752d7f84d30f14ea373407844d5c589f7ab46800bb4b",
|
| 13 |
"files": [
|
| 14 |
"CMakeLists.txt",
|
| 15 |
"cpp/glue.hpp",
|
|
|
|
| 28 |
"files": [
|
| 29 |
{
|
| 30 |
"path": "vendor/wllama-bonsai/esm/index.js",
|
| 31 |
+
"bytes": 365397,
|
| 32 |
+
"sha256": "24bcef8aea8e27fb7b7e2d9e6ea94ba8ced7bfbffd09a0675821b9eb1b4a4c9f"
|
| 33 |
},
|
| 34 |
{
|
| 35 |
"path": "public/wasm/wllama.wasm",
|
vendor/wllama-bonsai/esm/index.cjs
CHANGED
|
@@ -2650,10 +2650,10 @@ var Wllama = class {
|
|
| 2650 |
/**
|
| 2651 |
* Set compatibility options for Wllama.
|
| 2652 |
* @param compat Set to null to disable compatibility, or 'default' to use the default compat resources from CDN.
|
| 2653 |
-
* @param mode 'safari' by default; If set to 'firefox_safari', the compat mode will **also** be enabled on Firefox, which will significantly degrade the performance but allow using WebGPU on Firefox.
|
| 2654 |
*/
|
| 2655 |
setCompat(compat, mode = "safari") {
|
| 2656 |
-
if (mode === "safari") {
|
| 2657 |
if (isFirefox()) {
|
| 2658 |
this.compat = null;
|
| 2659 |
return;
|
|
@@ -3395,11 +3395,23 @@ var Wllama = class {
|
|
| 3395 |
});
|
| 3396 |
}
|
| 3397 |
getWorkerResources() {
|
|
|
|
| 3398 |
const workerResources = {
|
| 3399 |
wasmPath: absoluteUrl(this.pathConfig["default"]),
|
| 3400 |
compat: false
|
| 3401 |
};
|
| 3402 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3403 |
if (!this.compat) {
|
| 3404 |
this.logger().warn(
|
| 3405 |
"Not using compat mode" + (isFirefox() ? " (expected on Firefox - WebGPU will be disabled)" : "")
|
|
|
|
| 2650 |
/**
|
| 2651 |
* Set compatibility options for Wllama.
|
| 2652 |
* @param compat Set to null to disable compatibility, or 'default' to use the default compat resources from CDN.
|
| 2653 |
+
* @param mode 'safari' by default; If set to 'firefox_safari', the compat mode will **also** be enabled on Firefox, which will significantly degrade the performance but allow using WebGPU on Firefox. Ignored when forceCompat is true.
|
| 2654 |
*/
|
| 2655 |
setCompat(compat, mode = "safari") {
|
| 2656 |
+
if (mode === "safari" && !this.config.forceCompat) {
|
| 2657 |
if (isFirefox()) {
|
| 2658 |
this.compat = null;
|
| 2659 |
return;
|
|
|
|
| 3395 |
});
|
| 3396 |
}
|
| 3397 |
getWorkerResources() {
|
| 3398 |
+
var _a, _b;
|
| 3399 |
const workerResources = {
|
| 3400 |
wasmPath: absoluteUrl(this.pathConfig["default"]),
|
| 3401 |
compat: false
|
| 3402 |
};
|
| 3403 |
+
const forceCompat = this.config.forceCompat === true;
|
| 3404 |
+
if (forceCompat || needCompat()) {
|
| 3405 |
+
const compatWorker = (_a = this.compat) == null ? void 0 : _a.worker;
|
| 3406 |
+
const hasCompatWorker = typeof compatWorker === "string" && compatWorker.trim().length > 0 || typeof compatWorker === "object" && compatWorker !== null && typeof compatWorker.code === "string" && compatWorker.code.trim().length > 0;
|
| 3407 |
+
const compatWasm = (_b = this.compat) == null ? void 0 : _b.wasm;
|
| 3408 |
+
const hasCompatWasm = typeof compatWasm === "string" && compatWasm.trim().length > 0;
|
| 3409 |
+
if (forceCompat && (!hasCompatWorker || !hasCompatWasm)) {
|
| 3410 |
+
throw new WllamaError(
|
| 3411 |
+
"forceCompat requires both compat worker JS and compat WASM resources. Configure them with setCompat() before loading.",
|
| 3412 |
+
"load_error"
|
| 3413 |
+
);
|
| 3414 |
+
}
|
| 3415 |
if (!this.compat) {
|
| 3416 |
this.logger().warn(
|
| 3417 |
"Not using compat mode" + (isFirefox() ? " (expected on Firefox - WebGPU will be disabled)" : "")
|
vendor/wllama-bonsai/esm/index.js
CHANGED
|
@@ -2615,10 +2615,10 @@ var Wllama = class {
|
|
| 2615 |
/**
|
| 2616 |
* Set compatibility options for Wllama.
|
| 2617 |
* @param compat Set to null to disable compatibility, or 'default' to use the default compat resources from CDN.
|
| 2618 |
-
* @param mode 'safari' by default; If set to 'firefox_safari', the compat mode will **also** be enabled on Firefox, which will significantly degrade the performance but allow using WebGPU on Firefox.
|
| 2619 |
*/
|
| 2620 |
setCompat(compat, mode = "safari") {
|
| 2621 |
-
if (mode === "safari") {
|
| 2622 |
if (isFirefox()) {
|
| 2623 |
this.compat = null;
|
| 2624 |
return;
|
|
@@ -3360,11 +3360,23 @@ var Wllama = class {
|
|
| 3360 |
});
|
| 3361 |
}
|
| 3362 |
getWorkerResources() {
|
|
|
|
| 3363 |
const workerResources = {
|
| 3364 |
wasmPath: absoluteUrl(this.pathConfig["default"]),
|
| 3365 |
compat: false
|
| 3366 |
};
|
| 3367 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3368 |
if (!this.compat) {
|
| 3369 |
this.logger().warn(
|
| 3370 |
"Not using compat mode" + (isFirefox() ? " (expected on Firefox - WebGPU will be disabled)" : "")
|
|
|
|
| 2615 |
/**
|
| 2616 |
* Set compatibility options for Wllama.
|
| 2617 |
* @param compat Set to null to disable compatibility, or 'default' to use the default compat resources from CDN.
|
| 2618 |
+
* @param mode 'safari' by default; If set to 'firefox_safari', the compat mode will **also** be enabled on Firefox, which will significantly degrade the performance but allow using WebGPU on Firefox. Ignored when forceCompat is true.
|
| 2619 |
*/
|
| 2620 |
setCompat(compat, mode = "safari") {
|
| 2621 |
+
if (mode === "safari" && !this.config.forceCompat) {
|
| 2622 |
if (isFirefox()) {
|
| 2623 |
this.compat = null;
|
| 2624 |
return;
|
|
|
|
| 3360 |
});
|
| 3361 |
}
|
| 3362 |
getWorkerResources() {
|
| 3363 |
+
var _a, _b;
|
| 3364 |
const workerResources = {
|
| 3365 |
wasmPath: absoluteUrl(this.pathConfig["default"]),
|
| 3366 |
compat: false
|
| 3367 |
};
|
| 3368 |
+
const forceCompat = this.config.forceCompat === true;
|
| 3369 |
+
if (forceCompat || needCompat()) {
|
| 3370 |
+
const compatWorker = (_a = this.compat) == null ? void 0 : _a.worker;
|
| 3371 |
+
const hasCompatWorker = typeof compatWorker === "string" && compatWorker.trim().length > 0 || typeof compatWorker === "object" && compatWorker !== null && typeof compatWorker.code === "string" && compatWorker.code.trim().length > 0;
|
| 3372 |
+
const compatWasm = (_b = this.compat) == null ? void 0 : _b.wasm;
|
| 3373 |
+
const hasCompatWasm = typeof compatWasm === "string" && compatWasm.trim().length > 0;
|
| 3374 |
+
if (forceCompat && (!hasCompatWorker || !hasCompatWasm)) {
|
| 3375 |
+
throw new WllamaError(
|
| 3376 |
+
"forceCompat requires both compat worker JS and compat WASM resources. Configure them with setCompat() before loading.",
|
| 3377 |
+
"load_error"
|
| 3378 |
+
);
|
| 3379 |
+
}
|
| 3380 |
if (!this.compat) {
|
| 3381 |
this.logger().warn(
|
| 3382 |
"Not using compat mode" + (isFirefox() ? " (expected on Firefox - WebGPU will be disabled)" : "")
|
vendor/wllama-bonsai/esm/index.min.js
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
vendor/wllama-bonsai/esm/index.min.js.map
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
vendor/wllama-bonsai/esm/wllama.d.ts
CHANGED
|
@@ -13,6 +13,13 @@ export interface WllamaLogger {
|
|
| 13 |
error: typeof console.error;
|
| 14 |
}
|
| 15 |
export interface WllamaConfig {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
/**
|
| 17 |
* If true, suppress all log messages from native CPP code
|
| 18 |
*/
|
|
@@ -176,7 +183,7 @@ export declare class Wllama {
|
|
| 176 |
/**
|
| 177 |
* Set compatibility options for Wllama.
|
| 178 |
* @param compat Set to null to disable compatibility, or 'default' to use the default compat resources from CDN.
|
| 179 |
-
* @param mode 'safari' by default; If set to 'firefox_safari', the compat mode will **also** be enabled on Firefox, which will significantly degrade the performance but allow using WebGPU on Firefox.
|
| 180 |
*/
|
| 181 |
setCompat(compat: WllamaCompat | null | 'default', mode?: 'safari' | 'firefox_safari'): void;
|
| 182 |
/**
|
|
|
|
| 13 |
error: typeof console.error;
|
| 14 |
}
|
| 15 |
export interface WllamaConfig {
|
| 16 |
+
/**
|
| 17 |
+
* Always use the configured compat worker JS and WASM pair, even when the
|
| 18 |
+
* browser supports the default JSPI runtime.
|
| 19 |
+
*
|
| 20 |
+
* Default: forceCompat = false
|
| 21 |
+
*/
|
| 22 |
+
forceCompat?: boolean;
|
| 23 |
/**
|
| 24 |
* If true, suppress all log messages from native CPP code
|
| 25 |
*/
|
|
|
|
| 183 |
/**
|
| 184 |
* Set compatibility options for Wllama.
|
| 185 |
* @param compat Set to null to disable compatibility, or 'default' to use the default compat resources from CDN.
|
| 186 |
+
* @param mode 'safari' by default; If set to 'firefox_safari', the compat mode will **also** be enabled on Firefox, which will significantly degrade the performance but allow using WebGPU on Firefox. Ignored when forceCompat is true.
|
| 187 |
*/
|
| 188 |
setCompat(compat: WllamaCompat | null | 'default', mode?: 'safari' | 'firefox_safari'): void;
|
| 189 |
/**
|