Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,104 @@
+---
+library_name: wllama
+tags:
+  - gemma
+  - gemma-4
+  - webgpu
+  - browser-inference
+  - mixture-of-experts
+  - moe
+  - strix-halo
+  - unified-memory
+  - wllama
+  - first-of-its-kind
+language:
+  - en
+license: apache-2.0
+pipeline_tag: text-generation
+base_model: google/gemma-4-26b-a4b-it
+---
+
+# Gemma-4-26B-A4B in the Browser via WebGPU
+
+**Gemma-4-26B-A4B-it (MoE, 3.8B active params per token) running in a browser tab via WebGPU at 23 tokens/second.** 20GB GGUF loaded into WebGPU memory on AMD Strix Halo iGPU (64GB unified memory).
+
+## What This Is
+
+A working setup for running Gemma-4-26B-A4B-it in the browser using [wllama](https://github.com/ngxson/wllama) (WASM binding for llama.cpp) with a **patched WebGPU backend** that fixes a buffer aliasing bug in the GLU/GeGLU shader.
+
+This is, to our knowledge as of May 2026, the **largest model successfully run in a browser via WebGPU** — 20GB of Q5_K_XL weights loaded into 31.5GB of available WebGPU memory on a consumer iGPU.
+
+## Key Findings
+
+### WebGPU Memory on Strix Halo
+- **31.5 GB** available to a single Chrome tab (tested empirically)
+- 64GB unified memory, no discrete GPU needed
+- `maxBufferSize` reports 2GB per buffer, but total allocation far exceeds this
+
+### Performance
+- **23 tokens/second** decode speed
+- **~2 minutes** model loading (20GB via byte-range fetch)
+- **Quiet operation** — same model through llama-server Vulkan thrashes the machine; browser WebGPU (D3D12 path) runs silently
+
+### The Bug We Fixed
+llama.cpp's WebGPU backend (`ggml-webgpu.cpp`) has a buffer aliasing bug in the GLU shader that crashes all Gemma-4 MoE models. The GeGLU operation binds overlapping regions of the same GPU buffer as separate writable storage bindings — Vulkan allows this, WebGPU forbids it.
+
+**The fix:** When `src0` and `src1` tensor views overlap (share the same backing buffer), force the NO_SPLIT shader variant which reads both halves from a single binding with offset computation. This follows the same pattern as PRs [#22266](https://github.com/ggml-org/llama.cpp/pull/22266) (RMS_NORM_MUL) and [#22456](https://github.com/ggml-org/llama.cpp/pull/22456) (SSM_SCAN).
+
+**Files changed:**
+- `ggml-webgpu-shader-lib.hpp` — Added overlap detection to GLU pipeline key
+- `ggml-webgpu.cpp` — Skip separate src1 binding when overlapping
+- `glu.wgsl` — Added INPLACE mode for src0/dst overlap case
+
+## Quick Start
+
+```bash
+# 1. Clone this repo
+git clone https://huggingface.co/LJTSG/gemma-webgpu
+
+# 2. Split your Gemma GGUF into <2GB chunks
+llama-gguf-split --split-max-size 512M /path/to/gemma-4-26B-A4B.gguf ./model_splits/gemma-26b
+
+# 3. Start the server
+node serve_gemma.js
+
+# 4. Open http://localhost:8150
+# Click "Load Model" → wait for 20GB download → "Generate"
+```
+
+**Requirements:**
+- Gemma-4-26B-A4B-it GGUF (Q5_K_XL or Q4_K_M)
+- Node.js
+- Chrome/Edge with WebGPU support
+- GPU with **20+ GB** accessible via WebGPU (tested: AMD Strix Halo iGPU, 64GB unified)
+
+## Why Browser WebGPU?
+
+On AMD Strix Halo (and likely other unified memory iGPU systems):
+- **Vulkan path** (llama-server): fights the driver for GPU memory, thrashes, machine runs hot and loud
+- **WebGPU path** (browser): goes through D3D12, the path AMD optimizes for. Silent, smooth, same speed
+
+The browser's managed WebGPU context is genuinely better for sustained inference on iGPU hardware than native Vulkan.
+
+## Files
+
+- `index.html` — Test page with Load/Generate buttons
+- `serve_gemma.js` — Node.js server with Range requests + CORS/COEP/COOP headers
+- `memory_test.html` — WebGPU memory ceiling allocation test
+- `wllama-patch/` — The GLU aliasing fix (diff against wllama v3.4.1)
+
+## Related Work
+
+- [LJTSG/mamba-webgpu](https://huggingface.co/LJTSG/mamba-webgpu) — First browser-native Mamba/SSM inference (hand-written WGSL shaders)
+- [wllama](https://github.com/ngxson/wllama) — WASM binding for llama.cpp with WebGPU support
+- [Llamas on the Web](https://reeselevine.github.io/llamas-on-the-web/) — WebGPU backend for llama.cpp
+
+## License
+
+Apache 2.0
+
+## Credits
+
+Built by Joshua ([@LJTSG](https://huggingface.co/LJTSG)) and Claude (Anthropic Opus 4.6).
+Model: [google/gemma-4-26b-a4b-it](https://huggingface.co/google/gemma-4-26b-a4b-it).
+Runtime: [wllama](https://github.com/ngxson/wllama) by ngxson, with patched WebGPU backend.