README.md

metadata

title: Gemma 4 WebGPU
sdk: gradio
app_file: app.py
python_version: '3.11'
models:
  - ryanhlewis/gemma-4-E2B-it-qat-q4_0-gguf-webgpu
  - gguf-org/gemma-4-e2b-it-gguf
tags:
  - webgpu
  - gguf
  - llama.cpp
  - wllama
  - gemma

Gemma 4 WebGPU

Browser chat app for ryanhlewis/gemma-4-E2B-it-qat-q4_0-gguf-webgpu.

The Space serves only HTML, CSS, JavaScript, and Wllama WASM assets from the checked-in dist/ build. The GGUF files are downloaded by the browser and inference runs locally through Wllama / llama.cpp WebGPU. The Python entrypoint is only a static file server with cross-origin isolation headers.

Model Assets

The official Google repository contains:

File	Size	Use
gemma-4-E2B_q4_0-it.gguf	3.35 GB	language model
mmproj-gemma-4-e2b-it-q4_0.gguf	336 MB	q4 vision projector

Wllama supports split GGUF loading, so the app defaults to a public mirror containing split shards of the 3.35 GB model file plus a q4 vision projector. This keeps each browser-fetched LLM shard under the per-file browser/WASM limit while preserving the same language model weights.

Local Development

npm install
npm run dev

Open:

http://127.0.0.1:8030/

Cross-origin isolation headers are enabled in Vite and in app.py because Wllama uses browser features such as WebGPU, WebAssembly, OPFS, and SharedArrayBuffer when available.

URL Overrides

The app accepts:

?modelUrl=https://huggingface.co/<repo>/resolve/main/model-00001-of-00005.gguf
?mmprojUrl=https://huggingface.co/<repo>/resolve/main/mmproj-gemma-4-e2b-it-q4_0.gguf
?ctx=4096&threads=4&maxTokens=384

For split models, pass the first shard URL. Wllama discovers the remaining shards from the -00001-of-00005.gguf naming pattern.

Producing Split Shards

Use the helper script after installing Docker and Hugging Face CLI auth:

.\scripts\split-gemma-gguf.ps1

It downloads the official Google GGUF, uses the llama.cpp Docker image to run llama-gguf-split, and uploads the split shards plus the mmproj file to a public Hugging Face model repo.

References

• Wllama supports direct browser inference, WebGPU, multimodal inputs, and split GGUF loading.
• The Hugging Face Space uses the free Gradio runtime as a static file server so it does not require Static Space build credits.
• google/gemma-4-E2B-it-qat-q4_0-gguf is Apache-2.0 licensed.