---
library_name: wllama
tags:
  - gemma
  - gemma-4
  - webgpu
  - browser-inference
  - mixture-of-experts
  - moe
  - strix-halo
  - unified-memory
  - wllama
  - first-of-its-kind
language:
  - en
license: apache-2.0
pipeline_tag: text-generation
base_model: google/gemma-4-26b-a4b-it
---

# Gemma-4-26B-A4B in the Browser via WebGPU

**Gemma-4-26B-A4B-it (MoE, 3.8B active params per token) running in a browser tab via WebGPU at 23 tokens/second.** 20GB GGUF loaded into WebGPU memory on AMD Strix Halo iGPU (64GB unified memory).

## What This Is

A working setup for running Gemma-4-26B-A4B-it in the browser using [wllama](https://github.com/ngxson/wllama) (WASM binding for llama.cpp) with a **patched WebGPU backend** that fixes a buffer aliasing bug in the GLU/GeGLU shader.

This is, to our knowledge as of May 2026, the **largest model successfully run in a browser via WebGPU** — 20GB of Q5_K_XL weights loaded into 31.5GB of available WebGPU memory on a consumer iGPU.

## Key Findings

### WebGPU Memory on Strix Halo
- **31.5 GB** available to a single Chrome tab (tested empirically)
- 64GB unified memory, no discrete GPU needed
- `maxBufferSize` reports 2GB per buffer, but total allocation far exceeds this

### Performance
- **23 tokens/second** decode speed
- **~2 minutes** model loading (20GB via byte-range fetch)
- **Quiet operation** — same model through llama-server Vulkan thrashes the machine; browser WebGPU (D3D12 path) runs silently

### The Bug We Fixed
llama.cpp's WebGPU backend (`ggml-webgpu.cpp`) has a buffer aliasing bug in the GLU shader that crashes all Gemma-4 MoE models. The GeGLU operation binds overlapping regions of the same GPU buffer as separate writable storage bindings — Vulkan allows this, WebGPU forbids it.

**The fix:** When `src0` and `src1` tensor views overlap (share the same backing buffer), force the NO_SPLIT shader variant which reads both halves from a single binding with offset computation. This follows the same pattern as PRs [#22266](https://github.com/ggml-org/llama.cpp/pull/22266) (RMS_NORM_MUL) and [#22456](https://github.com/ggml-org/llama.cpp/pull/22456) (SSM_SCAN).

**Files changed:**
- `ggml-webgpu-shader-lib.hpp` — Added overlap detection to GLU pipeline key
- `ggml-webgpu.cpp` — Skip separate src1 binding when overlapping
- `glu.wgsl` — Added INPLACE mode for src0/dst overlap case

## Quick Start

```bash
# 1. Clone this repo
git clone https://huggingface.co/LJTSG/gemma-webgpu

# 2. Split your Gemma GGUF into <2GB chunks
llama-gguf-split --split-max-size 512M /path/to/gemma-4-26B-A4B.gguf ./model_splits/gemma-26b

# 3. Start the server
node serve_gemma.js

# 4. Open http://localhost:8150
# Click "Load Model" → wait for 20GB download → "Generate"
```

**Requirements:**
- Gemma-4-26B-A4B-it GGUF (Q5_K_XL or Q4_K_M)
- Node.js
- Chrome/Edge with WebGPU support
- GPU with **20+ GB** accessible via WebGPU (tested: AMD Strix Halo iGPU, 64GB unified)

## Why Browser WebGPU?

On AMD Strix Halo (and likely other unified memory iGPU systems):
- **Vulkan path** (llama-server): fights the driver for GPU memory, thrashes, machine runs hot and loud
- **WebGPU path** (browser): goes through D3D12, the path AMD optimizes for. Silent, smooth, same speed

The browser's managed WebGPU context is genuinely better for sustained inference on iGPU hardware than native Vulkan.

## Files

- `index.html` — Test page with Load/Generate buttons
- `serve_gemma.js` — Node.js server with Range requests + CORS/COEP/COOP headers
- `memory_test.html` — WebGPU memory ceiling allocation test
- `wllama-patch/` — The GLU aliasing fix (diff against wllama v3.4.1)

## Related Work

- [LJTSG/mamba-webgpu](https://huggingface.co/LJTSG/mamba-webgpu) — First browser-native Mamba/SSM inference (hand-written WGSL shaders)
- [wllama](https://github.com/ngxson/wllama) — WASM binding for llama.cpp with WebGPU support
- [Llamas on the Web](https://reeselevine.github.io/llamas-on-the-web/) — WebGPU backend for llama.cpp

## License

Apache 2.0

## Credits

Built by Joshua ([@LJTSG](https://huggingface.co/LJTSG)) and Claude (Anthropic Opus 4.6).
Model: [google/gemma-4-26b-a4b-it](https://huggingface.co/google/gemma-4-26b-a4b-it).
Runtime: [wllama](https://github.com/ngxson/wllama) by ngxson, with patched WebGPU backend.