Spaces:
Running
Running
Upload folder using huggingface_hub
Browse files- README.md +69 -9
- app.js +243 -0
- heads.json +0 -0
- index.html +65 -0
- meta.json +1 -0
- model.fp16.onnx +3 -0
- style.css +88 -0
README.md
CHANGED
|
@@ -1,13 +1,73 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
-
sdk:
|
| 7 |
-
sdk_version: 6.17.3
|
| 8 |
-
python_version: '3.13'
|
| 9 |
-
app_file: app.py
|
| 10 |
pinned: false
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: LocalAgent Tool Calling (WebGPU)
|
| 3 |
+
emoji: 🛠️
|
| 4 |
+
colorFrom: indigo
|
| 5 |
+
colorTo: purple
|
| 6 |
+
sdk: static
|
|
|
|
|
|
|
|
|
|
| 7 |
pinned: false
|
| 8 |
+
license: mit
|
| 9 |
+
short_description: Sub-100M from-scratch tool-calling agent in the browser
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# LocalAgent — tool calling in the browser (WebGPU)
|
| 13 |
+
|
| 14 |
+
A **28M-parameter, pretrained-from-scratch** byte-level agent that does **grounded tool
|
| 15 |
+
calling** and **multi-step planning** — running **entirely in your browser** on
|
| 16 |
+
[onnxruntime-web](https://onnxruntime.ai/docs/tutorials/web/) with the **WebGPU** backend
|
| 17 |
+
(WASM fallback when WebGPU is unavailable). No server, no API key; the model is downloaded once
|
| 18 |
+
and cached.
|
| 19 |
+
|
| 20 |
+
Model: [`SangbumChoi/localagent-tiny-30m-byte`](https://huggingface.co/SangbumChoi/localagent-tiny-30m-byte).
|
| 21 |
+
Source: [LocalAgent](https://github.com/sangbumchoi/localagent).
|
| 22 |
+
|
| 23 |
+
## What it shows
|
| 24 |
+
|
| 25 |
+
- **Tool selection** — the model's *real* `tool_head` decision (a linear head on the ONNX
|
| 26 |
+
`hidden` output) over the 21-tool surface, with a confidence score, plus **abstention** when no
|
| 27 |
+
tool fits.
|
| 28 |
+
- **Grounded arguments** — arguments copied from spans of your prompt, so the emitted call is
|
| 29 |
+
schema-valid by construction.
|
| 30 |
+
- **Multi-step plans** — the learned `plan_rollout`: pick a tool → ground it → feed back a
|
| 31 |
+
simulated response → pick the next, until the model emits the *stop* (`text`) class.
|
| 32 |
+
|
| 33 |
+
## How it runs (honest version)
|
| 34 |
+
|
| 35 |
+
The transformer forward pass runs on **WebGPU** via an exported ONNX graph that emits both `logits`
|
| 36 |
+
and the last `hidden` state. The **tool head** (one matmul + argmax over `hidden`), the
|
| 37 |
+
**argument grounding**, and the **planner loop** are light JavaScript on top — a faithful port of
|
| 38 |
+
the Python `tool_head` / grounding / `plan_rollout`. Arg grounding in-browser covers the common
|
| 39 |
+
formats (paths, URLs, quoted strings, names, numbers); the full Python grounder is the source of
|
| 40 |
+
truth. First load fetches `model.fp16.onnx` (~tens of MB) and caches it.
|
| 41 |
+
|
| 42 |
+
## Files
|
| 43 |
+
- `index.html` / `style.css` — the UI shell.
|
| 44 |
+
- `app.js` — byte tokenizer, onnxruntime-web session (WebGPU + WASM fallback), tool selection,
|
| 45 |
+
grounding, and the planner rollout.
|
| 46 |
+
- `model.fp16.onnx`, `heads.json`, `meta.json` — the exported inference bundle (**not in the
|
| 47 |
+
source repo**; they are deploy artifacts).
|
| 48 |
+
|
| 49 |
+
## Deploy
|
| 50 |
+
The model bundle is produced from a trained checkpoint, separately from the source tree:
|
| 51 |
+
|
| 52 |
+
```bash
|
| 53 |
+
python -c "from localagent.inference.export.to_onnx import export_web; \
|
| 54 |
+
export_web('runs/tiny-30m-byte-best.pt', 'runs/web_export')"
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
Then upload **the four static files + the three bundle files** into a Hugging Face Space repo
|
| 58 |
+
(`sdk: static`), all at the repo root, using git-lfs for the large ones:
|
| 59 |
+
|
| 60 |
+
```bash
|
| 61 |
+
huggingface-cli upload <user>/localagent-webgpu spaces/localagent-webgpu/ . --repo-type space
|
| 62 |
+
huggingface-cli upload <user>/localagent-webgpu runs/web_export/model.fp16.onnx model.fp16.onnx --repo-type space
|
| 63 |
+
huggingface-cli upload <user>/localagent-webgpu runs/web_export/heads.json heads.json --repo-type space
|
| 64 |
+
huggingface-cli upload <user>/localagent-webgpu runs/web_export/meta.json meta.json --repo-type space
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
`app.js` fetches `model.fp16.onnx` / `heads.json` / `meta.json` relative to the page, so they must
|
| 68 |
+
sit next to `index.html`. The export was verified for onnxruntime-CPU↔PyTorch parity
|
| 69 |
+
(max |Δlogits| 7.6e-6; fp16 drift 1.4e-3, same tool argmax) and the in-browser tool-selection +
|
| 70 |
+
pointer-grounding math was checked against the bundle (`get_weather{city:"Paris"}`,
|
| 71 |
+
`read_file{path:"tests/test_api.py"}`, …). The graph is all standard opset-17 ops; onnxruntime-web
|
| 72 |
+
falls back per-op to WASM for any op without a WebGPU kernel (`Trilu`/`Tile`/`Expand`), with
|
| 73 |
+
identical results.
|
app.js
ADDED
|
@@ -0,0 +1,243 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/* LocalAgent — in-browser tool calling on onnxruntime-web (WebGPU + WASM fallback).
|
| 2 |
+
*
|
| 3 |
+
* The transformer forward pass runs as an ONNX graph emitting `logits` and `hidden`.
|
| 4 |
+
* The tool head, argument grounding, and the planner rollout are ported here from the Python
|
| 5 |
+
* `tool_head` / grounding / `plan_rollout`. Bundle contract (see localagent.inference.export):
|
| 6 |
+
* model.fp16.onnx inputs: input_ids[int64, 1xT] outputs: logits[1,T,256], hidden[1,T,d]
|
| 7 |
+
* heads.json { tool_head:{weight:[C][d], bias:[C], classes:[C], stop_index:int}, ... }
|
| 8 |
+
* meta.json { vocab_size, d_model, pad_id, markers:{...}, tools:[{name,args,schema}], tool_classes }
|
| 9 |
+
*/
|
| 10 |
+
|
| 11 |
+
const MODEL_URL = "model.fp16.onnx";
|
| 12 |
+
let SESSION = null;
|
| 13 |
+
let HEADS = null;
|
| 14 |
+
let META = null;
|
| 15 |
+
let BACKEND = "wasm";
|
| 16 |
+
|
| 17 |
+
// ---- bundle loading -------------------------------------------------------
|
| 18 |
+
async function loadBundle() {
|
| 19 |
+
ort.env.wasm.wasmPaths = "https://cdn.jsdelivr.net/npm/onnxruntime-web@1.20.1/dist/";
|
| 20 |
+
[HEADS, META] = await Promise.all([
|
| 21 |
+
fetch("heads.json").then((r) => r.json()),
|
| 22 |
+
fetch("meta.json").then((r) => r.json()),
|
| 23 |
+
]);
|
| 24 |
+
try {
|
| 25 |
+
SESSION = await ort.InferenceSession.create(MODEL_URL, {
|
| 26 |
+
executionProviders: ["webgpu", "wasm"],
|
| 27 |
+
});
|
| 28 |
+
BACKEND = "webgpu";
|
| 29 |
+
} catch (e) {
|
| 30 |
+
console.warn("WebGPU unavailable, falling back to WASM:", e);
|
| 31 |
+
SESSION = await ort.InferenceSession.create(MODEL_URL, { executionProviders: ["wasm"] });
|
| 32 |
+
BACKEND = "wasm";
|
| 33 |
+
}
|
| 34 |
+
}
|
| 35 |
+
|
| 36 |
+
// ---- byte tokenizer (vocab 256) ------------------------------------------
|
| 37 |
+
// Markers are literal strings encoded as UTF-8 bytes — identical to the Python byte tokenizer.
|
| 38 |
+
const enc = new TextEncoder();
|
| 39 |
+
function bytesOf(s) { return Array.from(enc.encode(s)); }
|
| 40 |
+
function mark(name) { return META.markers[name].text; } // markers carry { text, ids }
|
| 41 |
+
|
| 42 |
+
// Render a user turn the way the model was trained / `plan_rollout` renders it.
|
| 43 |
+
function renderContext(query, steps) {
|
| 44 |
+
let s = mark("user") + query + mark("assistant");
|
| 45 |
+
for (const st of steps || []) {
|
| 46 |
+
s += mark("tool_call_open") + st.tool + "(" + JSON.stringify(st.args) + ")" + mark("tool_call_close");
|
| 47 |
+
s += mark("tool") + mark("tool_response_open") + (st.response || "ok") + mark("tool_response_close");
|
| 48 |
+
s += mark("assistant");
|
| 49 |
+
}
|
| 50 |
+
return bytesOf(s);
|
| 51 |
+
}
|
| 52 |
+
|
| 53 |
+
// ---- model forward --------------------------------------------------------
|
| 54 |
+
async function forward(ids) {
|
| 55 |
+
const arr = BigInt64Array.from(ids.map((x) => BigInt(x)));
|
| 56 |
+
const input = new ort.Tensor("int64", arr, [1, ids.length]);
|
| 57 |
+
const out = await SESSION.run({ input_ids: input });
|
| 58 |
+
return out; // { logits, hidden }
|
| 59 |
+
}
|
| 60 |
+
|
| 61 |
+
// ---- tool head (linear on the last hidden vector) -------------------------
|
| 62 |
+
function softmaxArgmax(logits) {
|
| 63 |
+
let m = -Infinity;
|
| 64 |
+
for (const v of logits) m = Math.max(m, v);
|
| 65 |
+
let z = 0;
|
| 66 |
+
const p = logits.map((v) => { const e = Math.exp(v - m); z += e; return e; });
|
| 67 |
+
let bi = 0;
|
| 68 |
+
for (let i = 1; i < p.length; i++) if (p[i] > p[bi]) bi = i;
|
| 69 |
+
return { index: bi, conf: p[bi] / z };
|
| 70 |
+
}
|
| 71 |
+
|
| 72 |
+
function selectTool(hiddenTensor, T) {
|
| 73 |
+
const d = META.d_model;
|
| 74 |
+
const H = hiddenTensor.data; // Float32Array length T*d
|
| 75 |
+
const off = (T - 1) * d; // last position
|
| 76 |
+
const last = H.subarray ? H.subarray(off, off + d) : Array.from(H).slice(off, off + d);
|
| 77 |
+
const { weight, bias, classes, stop_index } = HEADS.tool_head;
|
| 78 |
+
const logits = new Array(classes.length);
|
| 79 |
+
for (let c = 0; c < classes.length; c++) {
|
| 80 |
+
let acc = bias[c];
|
| 81 |
+
const Wc = weight[c];
|
| 82 |
+
for (let k = 0; k < d; k++) acc += Wc[k] * last[k];
|
| 83 |
+
logits[c] = acc;
|
| 84 |
+
}
|
| 85 |
+
const { index, conf } = softmaxArgmax(logits);
|
| 86 |
+
return { name: classes[index], index, conf, isStop: index === stop_index };
|
| 87 |
+
}
|
| 88 |
+
|
| 89 |
+
// ---- argument grounding via the learned pointer head (port of pointer_head) ----
|
| 90 |
+
// For each copy-arg of the chosen tool, compute start/end span logits over the input positions
|
| 91 |
+
// and slice the value out of the prompt bytes — identical math to the PyTorch pointer head:
|
| 92 |
+
// q = arg_emb[arg_idx[arg]]; qs = start_W·q; qe = end_W·q
|
| 93 |
+
// start = argmax_t hidden[t]·qs; end = argmax_{t>=start} hidden[t]·qe; value = bytes[start..end]
|
| 94 |
+
function matvec(M, v) { // M [d][d] · v [d] -> [d]
|
| 95 |
+
const d = v.length, out = new Float32Array(d);
|
| 96 |
+
for (let i = 0; i < d; i++) { const Mi = M[i]; let a = 0; for (let j = 0; j < d; j++) a += Mi[j] * v[j]; out[i] = a; }
|
| 97 |
+
return out;
|
| 98 |
+
}
|
| 99 |
+
function dotAt(H, t, d, q) { // hidden[t] · q
|
| 100 |
+
const off = t * d; let a = 0;
|
| 101 |
+
for (let k = 0; k < d; k++) a += H[off + k] * q[k];
|
| 102 |
+
return a;
|
| 103 |
+
}
|
| 104 |
+
function pointerSpan(arg, ids, H, T) {
|
| 105 |
+
const ph = HEADS.pointer_head, d = META.d_model;
|
| 106 |
+
const ai = ph.arg_idx[arg];
|
| 107 |
+
if (ai == null) return "";
|
| 108 |
+
const qs = matvec(ph.start_W, ph.arg_emb[ai]);
|
| 109 |
+
const qe = matvec(ph.end_W, ph.arg_emb[ai]);
|
| 110 |
+
let s = 0, sb = -Infinity;
|
| 111 |
+
for (let t = 0; t < T; t++) { const v = dotAt(H, t, d, qs); if (v > sb) { sb = v; s = t; } }
|
| 112 |
+
let e = s, eb = -Infinity;
|
| 113 |
+
for (let t = s; t < T; t++) { const v = dotAt(H, t, d, qe); if (v > eb) { eb = v; e = t; } }
|
| 114 |
+
try { return new TextDecoder().decode(new Uint8Array(ids.slice(s, e + 1))); } catch { return ""; }
|
| 115 |
+
}
|
| 116 |
+
function groundArgs(tool, ids, hiddenTensor, T) {
|
| 117 |
+
const spec = (META.tools || []).find((t) => t.name === tool);
|
| 118 |
+
const args = {};
|
| 119 |
+
if (!spec) return args;
|
| 120 |
+
const H = hiddenTensor.data;
|
| 121 |
+
for (const arg of spec.args || []) {
|
| 122 |
+
args[arg] = HEADS.pointer_head.arg_idx[arg] != null ? pointerSpan(arg, ids, H, T) : "";
|
| 123 |
+
}
|
| 124 |
+
return args;
|
| 125 |
+
}
|
| 126 |
+
|
| 127 |
+
// ---- single grounded call -------------------------------------------------
|
| 128 |
+
async function callOnce(query) {
|
| 129 |
+
const ids = renderContext(query, []);
|
| 130 |
+
const t0 = performance.now();
|
| 131 |
+
const out = await forward(ids);
|
| 132 |
+
const sel = selectTool(out.hidden, ids.length);
|
| 133 |
+
const ms = performance.now() - t0;
|
| 134 |
+
if (sel.isStop) return { abstain: true, conf: sel.conf, ms };
|
| 135 |
+
return { tool: sel.name, args: groundArgs(sel.name, ids, out.hidden, ids.length), conf: sel.conf, ms };
|
| 136 |
+
}
|
| 137 |
+
|
| 138 |
+
// ---- planner rollout (port of plan_rollout) -------------------------------
|
| 139 |
+
async function planRollout(query, maxSteps = 4) {
|
| 140 |
+
const steps = [];
|
| 141 |
+
const t0 = performance.now();
|
| 142 |
+
for (let i = 0; i < maxSteps; i++) {
|
| 143 |
+
const ids = renderContext(query, steps);
|
| 144 |
+
const out = await forward(ids);
|
| 145 |
+
const sel = selectTool(out.hidden, ids.length);
|
| 146 |
+
if (sel.isStop) break;
|
| 147 |
+
const args = groundArgs(sel.name, ids, out.hidden, ids.length);
|
| 148 |
+
steps.push({ tool: sel.name, args, conf: sel.conf, response: simResponse(sel.name, args) });
|
| 149 |
+
}
|
| 150 |
+
return { steps, ms: performance.now() - t0 };
|
| 151 |
+
}
|
| 152 |
+
|
| 153 |
+
// A compact simulated tool response so downstream steps have context (mirrors _sim_response).
|
| 154 |
+
function simResponse(tool, args) {
|
| 155 |
+
if (/read_file|grep/.test(tool)) return Object.values(args)[0] || "ok";
|
| 156 |
+
if (/search|news/.test(tool)) return "result: " + (args.query || "");
|
| 157 |
+
return "ok";
|
| 158 |
+
}
|
| 159 |
+
|
| 160 |
+
// ---- UI -------------------------------------------------------------------
|
| 161 |
+
const $ = (id) => document.getElementById(id);
|
| 162 |
+
|
| 163 |
+
function setStatus(cls, text, backend) {
|
| 164 |
+
const s = $("status");
|
| 165 |
+
s.className = "status " + cls;
|
| 166 |
+
$("status-text").textContent = text;
|
| 167 |
+
const b = $("backend-badge");
|
| 168 |
+
if (backend) { b.hidden = false; b.textContent = backend.toUpperCase(); }
|
| 169 |
+
}
|
| 170 |
+
|
| 171 |
+
function renderCall(step, idx) {
|
| 172 |
+
const div = document.createElement("div");
|
| 173 |
+
div.className = "call" + (step.abstain ? " abstain" : "");
|
| 174 |
+
const conf = step.conf != null ? `<span class="conf">${(step.conf * 100).toFixed(0)}%</span>` : "";
|
| 175 |
+
if (step.abstain) {
|
| 176 |
+
div.innerHTML = `${conf}<span class="tool">— abstains (no tool needed)</span>`;
|
| 177 |
+
} else {
|
| 178 |
+
const ix = idx != null ? `<span class="step-index">${idx + 1}.</span>` : "";
|
| 179 |
+
div.innerHTML = `${conf}${ix}<span class="tool">${step.tool}</span>` +
|
| 180 |
+
`<pre>${JSON.stringify(step.args, null, 2)}</pre>`;
|
| 181 |
+
}
|
| 182 |
+
return div;
|
| 183 |
+
}
|
| 184 |
+
|
| 185 |
+
async function run() {
|
| 186 |
+
const query = $("prompt").value.trim();
|
| 187 |
+
if (!query || !SESSION) return;
|
| 188 |
+
$("run").disabled = true;
|
| 189 |
+
const res = $("result");
|
| 190 |
+
res.hidden = false;
|
| 191 |
+
res.innerHTML = '<div class="call"><span class="tool">…thinking</span></div>';
|
| 192 |
+
try {
|
| 193 |
+
if ($("plan-mode").checked) {
|
| 194 |
+
const { steps, ms } = await planRollout(query);
|
| 195 |
+
res.innerHTML = "";
|
| 196 |
+
if (!steps.length) res.appendChild(renderCall({ abstain: true }));
|
| 197 |
+
steps.forEach((s, i) => res.appendChild(renderCall(s, i)));
|
| 198 |
+
const t = document.createElement("div");
|
| 199 |
+
t.className = "timing";
|
| 200 |
+
t.textContent = `${steps.length} step(s) · ${ms.toFixed(0)} ms · ${BACKEND}`;
|
| 201 |
+
res.appendChild(t);
|
| 202 |
+
} else {
|
| 203 |
+
const out = await callOnce(query);
|
| 204 |
+
res.innerHTML = "";
|
| 205 |
+
res.appendChild(renderCall(out));
|
| 206 |
+
const t = document.createElement("div");
|
| 207 |
+
t.className = "timing";
|
| 208 |
+
t.textContent = `${out.ms.toFixed(0)} ms · ${BACKEND}`;
|
| 209 |
+
res.appendChild(t);
|
| 210 |
+
}
|
| 211 |
+
} catch (e) {
|
| 212 |
+
res.innerHTML = `<div class="call abstain"><span class="tool">error</span><pre>${e}</pre></div>`;
|
| 213 |
+
} finally {
|
| 214 |
+
$("run").disabled = false;
|
| 215 |
+
}
|
| 216 |
+
}
|
| 217 |
+
|
| 218 |
+
function wireUI() {
|
| 219 |
+
$("run").addEventListener("click", run);
|
| 220 |
+
$("prompt").addEventListener("keydown", (e) => {
|
| 221 |
+
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") run();
|
| 222 |
+
});
|
| 223 |
+
document.querySelectorAll(".chip").forEach((c) => {
|
| 224 |
+
c.addEventListener("click", () => {
|
| 225 |
+
$("prompt").value = c.textContent;
|
| 226 |
+
$("plan-mode").checked = c.dataset.plan === "1";
|
| 227 |
+
run();
|
| 228 |
+
});
|
| 229 |
+
});
|
| 230 |
+
}
|
| 231 |
+
|
| 232 |
+
(async function main() {
|
| 233 |
+
wireUI();
|
| 234 |
+
try {
|
| 235 |
+
setStatus("loading", "Loading model… (first load downloads & caches the weights)");
|
| 236 |
+
await loadBundle();
|
| 237 |
+
setStatus("ready", "Model ready — runs locally in your browser.", BACKEND);
|
| 238 |
+
$("run").disabled = false;
|
| 239 |
+
} catch (e) {
|
| 240 |
+
console.error(e);
|
| 241 |
+
setStatus("error", "Failed to load the model bundle: " + e.message);
|
| 242 |
+
}
|
| 243 |
+
})();
|
heads.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
index.html
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!doctype html>
|
| 2 |
+
<html lang="en">
|
| 3 |
+
<head>
|
| 4 |
+
<meta charset="utf-8" />
|
| 5 |
+
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
| 6 |
+
<title>LocalAgent — Tool Calling (WebGPU)</title>
|
| 7 |
+
<link rel="stylesheet" href="style.css" />
|
| 8 |
+
</head>
|
| 9 |
+
<body>
|
| 10 |
+
<main>
|
| 11 |
+
<header>
|
| 12 |
+
<h1>🛠️ LocalAgent <span class="sub">tool calling in your browser</span></h1>
|
| 13 |
+
<p class="tagline">
|
| 14 |
+
A <strong>28M-param, from-scratch</strong> byte-level agent. The transformer runs on
|
| 15 |
+
<strong>WebGPU</strong> (<code>onnxruntime-web</code>); the tool head, grounding, and the
|
| 16 |
+
planner loop are light JS. <a href="https://huggingface.co/SangbumChoi/localagent-tiny-30m-byte" target="_blank" rel="noopener">model</a> ·
|
| 17 |
+
<a href="https://github.com/sangbumchoi/localagent" target="_blank" rel="noopener">code</a>
|
| 18 |
+
</p>
|
| 19 |
+
</header>
|
| 20 |
+
|
| 21 |
+
<section id="status" class="status loading">
|
| 22 |
+
<span class="dot"></span>
|
| 23 |
+
<span id="status-text">Loading model…</span>
|
| 24 |
+
<span id="backend-badge" class="badge" hidden></span>
|
| 25 |
+
</section>
|
| 26 |
+
|
| 27 |
+
<section class="io">
|
| 28 |
+
<label for="prompt">Your request</label>
|
| 29 |
+
<textarea id="prompt" rows="2" placeholder="e.g. What's the weather in Paris?"></textarea>
|
| 30 |
+
|
| 31 |
+
<div class="row">
|
| 32 |
+
<label class="switch">
|
| 33 |
+
<input type="checkbox" id="plan-mode" />
|
| 34 |
+
<span>Multi-step plan (planner rollout)</span>
|
| 35 |
+
</label>
|
| 36 |
+
<button id="run" disabled>Run</button>
|
| 37 |
+
</div>
|
| 38 |
+
|
| 39 |
+
<div class="examples">
|
| 40 |
+
<span>Try:</span>
|
| 41 |
+
<button class="chip" data-plan="0">What's the weather in Paris?</button>
|
| 42 |
+
<button class="chip" data-plan="0">Move src/app.py to backup/app.py.</button>
|
| 43 |
+
<button class="chip" data-plan="0">Send an email to Greta.</button>
|
| 44 |
+
<button class="chip" data-plan="0">How tall is Mount Everest?</button>
|
| 45 |
+
<button class="chip" data-plan="1">Read tests/test_api.py, run the tests, then commit.</button>
|
| 46 |
+
<button class="chip" data-plan="1">Search the web for the news, then save it to Notion.</button>
|
| 47 |
+
</div>
|
| 48 |
+
</section>
|
| 49 |
+
|
| 50 |
+
<section id="result" class="result" hidden></section>
|
| 51 |
+
|
| 52 |
+
<footer>
|
| 53 |
+
<p>
|
| 54 |
+
Runs fully client-side — the model is fetched once and cached. Arg grounding in-browser
|
| 55 |
+
covers common formats; the Python grounder is the source of truth. No data leaves your
|
| 56 |
+
device.
|
| 57 |
+
</p>
|
| 58 |
+
</footer>
|
| 59 |
+
</main>
|
| 60 |
+
|
| 61 |
+
<!-- onnxruntime-web (WASM + WebGPU execution providers) -->
|
| 62 |
+
<script src="https://cdn.jsdelivr.net/npm/onnxruntime-web@1.20.1/dist/ort.webgpu.min.js"></script>
|
| 63 |
+
<script src="app.js"></script>
|
| 64 |
+
</body>
|
| 65 |
+
</html>
|
meta.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"vocab_size": 256, "d_model": 512, "pad_id": 0, "eos_id": 0, "encoding": "utf-8-bytes", "markers": {"user": {"text": "<|user|>", "ids": [60, 124, 117, 115, 101, 114, 124, 62]}, "assistant": {"text": "<|assistant|>", "ids": [60, 124, 97, 115, 115, 105, 115, 116, 97, 110, 116, 124, 62]}, "tool": {"text": "<|tool|>", "ids": [60, 124, 116, 111, 111, 108, 124, 62]}, "tool_call_open": {"text": "<tool_call>", "ids": [60, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]}, "tool_call_close": {"text": "</tool_call>", "ids": [60, 47, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]}, "tool_response_open": {"text": "<tool_response>", "ids": [60, 116, 111, 111, 108, 95, 114, 101, 115, 112, 111, 110, 115, 101, 62]}, "tool_response_close": {"text": "</tool_response>", "ids": [60, 47, 116, 111, 111, 108, 95, 114, 101, 115, 112, 111, 110, 115, 101, 62]}}, "tools": [{"name": "get_weather", "description": "Get the current weather for a city.", "args": ["city", "unit"], "schema": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["c", "f"]}}, "required": ["city"]}}, {"name": "calculator", "description": "Evaluate an arithmetic expression.", "args": ["expression"], "schema": {"type": "object", "properties": {"expression": {"type": "string", "format": "arithmetic"}}, "required": ["expression"]}}, {"name": "web_search", "description": "Search the web.", "args": ["query", "k"], "schema": {"type": "object", "properties": {"query": {"type": "string"}, "k": {"type": "integer"}}, "required": ["query"]}}, {"name": "planner", "description": "Make a plan to achieve a goal.", "args": ["goal"], "schema": {"type": "object", "properties": {"goal": {"type": "string"}}, "required": ["goal"]}}, {"name": "define", "description": "Define a term.", "args": ["term"], "schema": {"type": "object", "properties": {"term": {"type": "string"}}, "required": ["term"]}}, {"name": "play_music", "description": "Play a song.", "args": ["song"], "schema": {"type": "object", "properties": {"song": {"type": "string"}}, "required": ["song"]}}, {"name": "get_news", "description": "Get news on a topic.", "args": ["topic"], "schema": {"type": "object", "properties": {"topic": {"type": "string"}}, "required": ["topic"]}}, {"name": "read_file", "description": "Read a file.", "args": ["path"], "schema": {"type": "object", "properties": {"path": {"type": "string", "format": "path"}}, "required": ["path"]}}, {"name": "write_file", "description": "Create or write a file.", "args": ["path"], "schema": {"type": "object", "properties": {"path": {"type": "string", "format": "path"}}, "required": ["path"]}}, {"name": "grep_search", "description": "Search the codebase for a pattern.", "args": ["pattern"], "schema": {"type": "object", "properties": {"pattern": {"type": "string", "format": "quoted"}}, "required": ["pattern"]}}, {"name": "run_command", "description": "Run a shell command.", "args": ["command"], "schema": {"type": "object", "properties": {"command": {"type": "string", "format": "quoted"}}, "required": ["command"]}}, {"name": "git_commit", "description": "Make a git commit.", "args": ["message"], "schema": {"type": "object", "properties": {"message": {"type": "string", "format": "quoted"}}, "required": ["message"]}}, {"name": "run_tests", "description": "Run the test suite.", "args": [], "schema": {"type": "object", "properties": {}, "required": []}}, {"name": "set_reminder", "description": "Set a reminder for a task.", "args": ["task"], "schema": {"type": "object", "properties": {"task": {"type": "string"}}, "required": ["task"]}}, {"name": "set_timer", "description": "Set a timer for a duration.", "args": ["duration"], "schema": {"type": "object", "properties": {"duration": {"type": "string"}}, "required": ["duration"]}}, {"name": "calendar_event", "description": "Create a Google Calendar event.", "args": ["title"], "schema": {"type": "object", "properties": {"title": {"type": "string", "format": "quoted"}}, "required": ["title"]}}, {"name": "send_email", "description": "Send an email to someone.", "args": ["recipient"], "schema": {"type": "object", "properties": {"recipient": {"type": "string"}}, "required": ["recipient"]}}, {"name": "open_url", "description": "Open a URL in the web browser.", "args": ["url"], "schema": {"type": "object", "properties": {"url": {"type": "string", "format": "url"}}, "required": ["url"]}}, {"name": "notion_write", "description": "Write a note in Notion.", "args": ["content"], "schema": {"type": "object", "properties": {"content": {"type": "string", "format": "quoted"}}, "required": ["content"]}}, {"name": "slack_send", "description": "Send a Slack message.", "args": ["message"], "schema": {"type": "object", "properties": {"message": {"type": "string", "format": "quoted"}}, "required": ["message"]}}, {"name": "jira_issue", "description": "Create a Jira issue.", "args": ["summary"], "schema": {"type": "object", "properties": {"summary": {"type": "string", "format": "quoted"}}, "required": ["summary"]}}], "tool_classes": ["get_weather", "calculator", "web_search", "planner", "define", "play_music", "get_news", "read_file", "write_file", "grep_search", "run_command", "git_commit", "run_tests", "set_reminder", "set_timer", "calendar_event", "send_email", "open_url", "notion_write", "slack_send", "jira_issue", "text"]}
|
model.fp16.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3d21b07579fd496b86d5b92157567eae59e98249afcdea728d349e13e89dfa76
|
| 3 |
+
size 57468760
|
style.css
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
:root {
|
| 2 |
+
--bg: #0f1020;
|
| 3 |
+
--panel: #1a1b35;
|
| 4 |
+
--panel-2: #232450;
|
| 5 |
+
--fg: #e8e8f2;
|
| 6 |
+
--muted: #9a9ac0;
|
| 7 |
+
--accent: #8b7cff;
|
| 8 |
+
--accent-2: #5ad1c0;
|
| 9 |
+
--ok: #5ad17a;
|
| 10 |
+
--warn: #ffcc66;
|
| 11 |
+
--err: #ff7a90;
|
| 12 |
+
--mono: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, monospace;
|
| 13 |
+
}
|
| 14 |
+
|
| 15 |
+
* { box-sizing: border-box; }
|
| 16 |
+
|
| 17 |
+
body {
|
| 18 |
+
margin: 0;
|
| 19 |
+
background: radial-gradient(1200px 600px at 70% -10%, #2a2350 0%, var(--bg) 60%);
|
| 20 |
+
color: var(--fg);
|
| 21 |
+
font-family: system-ui, -apple-system, Segoe UI, Roboto, sans-serif;
|
| 22 |
+
line-height: 1.5;
|
| 23 |
+
}
|
| 24 |
+
|
| 25 |
+
main { max-width: 760px; margin: 0 auto; padding: 32px 20px 64px; }
|
| 26 |
+
|
| 27 |
+
header h1 { font-size: 1.7rem; margin: 0 0 4px; }
|
| 28 |
+
header h1 .sub { font-size: 0.95rem; color: var(--muted); font-weight: 400; }
|
| 29 |
+
.tagline { color: var(--muted); margin: 0 0 20px; font-size: 0.92rem; }
|
| 30 |
+
.tagline a { color: var(--accent-2); text-decoration: none; }
|
| 31 |
+
.tagline a:hover { text-decoration: underline; }
|
| 32 |
+
|
| 33 |
+
.status {
|
| 34 |
+
display: flex; align-items: center; gap: 10px;
|
| 35 |
+
background: var(--panel); border: 1px solid var(--panel-2);
|
| 36 |
+
border-radius: 10px; padding: 10px 14px; font-size: 0.9rem; margin-bottom: 18px;
|
| 37 |
+
}
|
| 38 |
+
.status .dot { width: 10px; height: 10px; border-radius: 50%; background: var(--warn); flex: none; }
|
| 39 |
+
.status.loading .dot { background: var(--warn); animation: pulse 1.2s infinite; }
|
| 40 |
+
.status.ready .dot { background: var(--ok); }
|
| 41 |
+
.status.error .dot { background: var(--err); }
|
| 42 |
+
@keyframes pulse { 0%,100% { opacity: 1; } 50% { opacity: 0.35; } }
|
| 43 |
+
.badge {
|
| 44 |
+
margin-left: auto; font-family: var(--mono); font-size: 0.75rem;
|
| 45 |
+
background: var(--panel-2); padding: 2px 8px; border-radius: 6px; color: var(--accent-2);
|
| 46 |
+
}
|
| 47 |
+
|
| 48 |
+
.io label { display: block; font-size: 0.85rem; color: var(--muted); margin-bottom: 6px; }
|
| 49 |
+
textarea {
|
| 50 |
+
width: 100%; background: var(--panel); color: var(--fg);
|
| 51 |
+
border: 1px solid var(--panel-2); border-radius: 10px; padding: 12px 14px;
|
| 52 |
+
font-size: 1rem; resize: vertical; font-family: inherit;
|
| 53 |
+
}
|
| 54 |
+
textarea:focus { outline: none; border-color: var(--accent); }
|
| 55 |
+
|
| 56 |
+
.row { display: flex; align-items: center; justify-content: space-between; margin: 12px 0; gap: 12px; }
|
| 57 |
+
.switch { display: flex; align-items: center; gap: 8px; font-size: 0.88rem; color: var(--fg); cursor: pointer; }
|
| 58 |
+
.switch input { accent-color: var(--accent); width: 16px; height: 16px; }
|
| 59 |
+
|
| 60 |
+
button#run {
|
| 61 |
+
background: var(--accent); color: #11102a; border: none; font-weight: 600;
|
| 62 |
+
padding: 10px 22px; border-radius: 10px; font-size: 0.95rem; cursor: pointer;
|
| 63 |
+
}
|
| 64 |
+
button#run:disabled { opacity: 0.45; cursor: not-allowed; }
|
| 65 |
+
button#run:not(:disabled):hover { filter: brightness(1.08); }
|
| 66 |
+
|
| 67 |
+
.examples { display: flex; flex-wrap: wrap; gap: 8px; align-items: center; margin-top: 8px; }
|
| 68 |
+
.examples span { color: var(--muted); font-size: 0.82rem; }
|
| 69 |
+
.chip {
|
| 70 |
+
background: var(--panel); color: var(--fg); border: 1px solid var(--panel-2);
|
| 71 |
+
border-radius: 999px; padding: 5px 12px; font-size: 0.82rem; cursor: pointer;
|
| 72 |
+
}
|
| 73 |
+
.chip:hover { border-color: var(--accent); color: #fff; }
|
| 74 |
+
|
| 75 |
+
.result { margin-top: 22px; }
|
| 76 |
+
.call {
|
| 77 |
+
background: var(--panel); border: 1px solid var(--panel-2); border-left: 3px solid var(--accent-2);
|
| 78 |
+
border-radius: 10px; padding: 14px 16px; margin-bottom: 10px;
|
| 79 |
+
}
|
| 80 |
+
.call .tool { font-family: var(--mono); font-size: 1rem; color: var(--accent-2); }
|
| 81 |
+
.call .conf { float: right; color: var(--muted); font-size: 0.8rem; font-family: var(--mono); }
|
| 82 |
+
.call pre { margin: 8px 0 0; font-family: var(--mono); font-size: 0.85rem; color: var(--fg); white-space: pre-wrap; }
|
| 83 |
+
.call.abstain { border-left-color: var(--warn); }
|
| 84 |
+
.call.abstain .tool { color: var(--warn); }
|
| 85 |
+
.step-index { color: var(--muted); font-family: var(--mono); font-size: 0.78rem; margin-right: 6px; }
|
| 86 |
+
.timing { color: var(--muted); font-size: 0.78rem; margin-top: 6px; font-family: var(--mono); }
|
| 87 |
+
|
| 88 |
+
footer { margin-top: 36px; color: var(--muted); font-size: 0.78rem; }
|