sjenit/embeddinggemma-2-ONNX-wasm

EmbeddingGemma 2 (ONNX, q4) for the onnxruntime-web WebAssembly backend

This repository holds a modified copy of the q4 text model and vision encoder from onnx-community/embeddinggemma-2-ONNX at commit daa72c51243991dfcaf9f9137d2c573d8f7790c0. The base model is google/embeddinggemma-2, by Google DeepMind.

Why it exists

The q4 and quantized graphs embed token and position tables with the com.microsoft.GatherBlockQuantized operator. onnxruntime-web's WebAssembly build has no kernel for it, so session creation fails without WebGPU:

Could not find an implementation for GatherBlockQuantized(1) node with name 'node_embedding_Quant'

What changed

Only these files differ from the source:

• onnx/model_q4.onnx, onnx/model_q4.onnx_data
• onnx/vision_encoder_q4.onnx, onnx/vision_encoder_q4.onnx_data

Each GatherBlockQuantized node (one in the text model, two in the vision encoder) is replaced with standard ONNX ops that do the same thing. They gather the packed 4-bit rows, scales and zero points, unpack the nibbles with BitwiseAnd and BitShift, and compute (q - zero_point) * scale. Every weight is byte-identical. rewrite.py is the script that made the change.

Checked against the original graphs on native onnxruntime CPU: cosine similarity 1.000000 on sentence_embedding (max abs diff ~1e-6) and on image_features (max abs diff ~3e-5).

The config, tokenizer and processor files are unmodified copies. The audio encoder and other precisions are not included.

Usage (Transformers.js)

Same as the source repo, using dtype: 'q4' and device: 'wasm'. On WebGPU, use the source repo.

License

Apache 2.0, the same as the source model. See LICENSE. Modified by ReelCubby, 2026-10-30.