README.md

---
license: apache-2.0
base_model: tiiuae/Falcon-Perception
tags: [webgpu, wgsl, int8, browser, object-detection, open-vocabulary, quantized]
---

# falcon-perception-webgpu — FPQ4 (q8 + q4-lm) weights

Custom mixed-precision quants of
[tiiuae/Falcon-Perception](https://huggingface.co/tiiuae/Falcon-Perception)
(0.6B open-vocabulary detection VLM) in the FPQ4 wire format of the
[falcon-perception-webgpu-demo](https://huggingface.co/spaces/borkiss/falcon-perception-webgpu-demo)
browser runtime — hand-written WGSL kernels, no ONNX Runtime for the language
stack, GPU-resident decode loop with speculative decoding for video.

**`falcon-q8-lm4.bin`** (724 MB, single file, everything the engine needs):

- all 113 layer matrices **int8** (group 128) — box quality demands it: any
  single q4 group's noise flips coordinate bins (q4 box-set IoU vs fp32 is
  only 0.43; q8 reaches hidden cosine 0.9993);
- `lm_head` **int4** — bit-exact argmax vs q8, −34 MB and −half the lm-read
  traffic per decode token;
- coord/size decode heads **f16**, Fourier coord/size encoders f32 —
  0/14 bin mismatches vs the ONNX reference;
- embeddings f16.

Numbers on an Apple M4 (10-core GPU, ~120 GB/s) in Chrome:

| | this runtime | original Space (fp32 ONNX/WASM) |
|---|---|---|
| full detection query | **1.2 s** | 7.8–8.7 s |
| decode | 91–95 tok/s | ~8.7 tok/s |
| download | 724 MB | 2.4 GB |
| video keyframes | 2.2–2.5 Hz (≤3 objects) + 30 FPS correlation tracker | — |

Quality vs the fp32 original on identical pixels: 8/8 boxes matched,
mean IoU 0.957. The runtime itself is verified bit-exact per layer (75/75)
against a numpy dequantized-weights reference, decode token-for-token.

## FPQ4 container

`'FPQ4'` magic + u32 manifest length + JSON manifest + 256-byte-aligned blobs,
absolute offsets — one `fetch`, sliced straight into GPU buffers:

```js
const head = new DataView(await file.slice(0, 8).arrayBuffer());
const mlen = head.getUint32(4, true);
const manifest = JSON.parse(new TextDecoder().decode(
    await file.slice(8, 8 + mlen).arrayBuffer()));
// per tensor t: file.slice(t.packed.offset, t.packed.offset + t.packed.length) → writeBuffer
```

q8 matrix ≈ 1.02 bytes/param (packed + f16 scales + i8 zeros, group 128);
q4 ≈ 0.52 bytes/param. Only the mask heads (`anyup`/`segm_head`) stay in ONNX
on the demo's segmentation path; detection runs 100% on WGSL.

Try it: **https://huggingface.co/spaces/borkiss/falcon-perception-webgpu-demo**
(needs WebGPU with `shader-f16` + `subgroups` — Chrome/Edge 125+).