| --- |
| license: apache-2.0 |
| language: |
| - en |
| - multilingual |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - audio |
| - speech-recognition |
| - gguf |
| - gemma |
| - conformer |
| library_name: ggml |
| base_model: google/gemma-4-E2B-it |
| --- |
| |
| # Gemma-4-E2B-it — GGUF |
|
|
| GGUF conversion of [`google/gemma-4-E2B-it`](https://huggingface.co/google/gemma-4-E2B-it) for use with **[CrispStrobe/CrispASR](https://github.com/CrispStrobe/CrispASR)**. |
|
|
| ## Available variants |
|
|
| | File | Quant | Size | Notes | |
| |---|---|---|---| |
| | `gemma4-e2b-it.gguf` | F16 | ~9.5 GB | Full precision | |
| | `gemma4-e2b-it-q8_0.gguf` | Q8_0 | ~5.0 GB | Near-lossless quant | |
| | `gemma4-e2b-it-q4_k.gguf` | Q4_K | ~2.8 GB | Standard quant | |
| | `gemma4-e2b-it-q2_k.gguf` | Q2_K | ~2.2 GB | Smallest, quality drop | |
| |
| ## Model details |
| |
| - **Architecture:** USM Conformer audio encoder (12L, 1024d, chunked-local attention with relative position bias, LightConv1d, ClippableLinear with QAT scalars) + Gemma4 LLM decoder (35L, 1536d, GQA 8Q/1KV, per-layer embeddings, hybrid sliding/full attention, GeGLU) |
| - **Parameters:** 2.3B effective (5.1B with embeddings) |
| - **Audio:** Gemma4AudioFeatureExtractor — 128-bin mel, 16 kHz, frame_length=320, hop=160, fft_length=512, semicausal padding, log(mel + mel_floor=0.001), no normalisation |
| - **Languages:** 140+ (ASR + speech translation) |
| - **License:** Apache 2.0 |
| - **Source:** [`google/gemma-4-E2B-it`](https://huggingface.co/google/gemma-4-E2B-it) |
|
|
| ## What's included vs an upstream Gemma-4 GGUF |
|
|
| This GGUF is built specifically for ASR with CrispASR and includes the audio path that |
| standard text/vision Gemma-4 GGUFs (unsloth, ggml-org) omit: |
|
|
| - 12-layer audio conformer encoder (~872 tensors total). |
| - Gemma4MultimodalEmbedder audio→LLM adapter (`embed_audio.embedding_projection`, |
| pre-projection RMSNorm). |
| - All ClippableLinear QAT clipping scalars (`.input_min/max`, `.output_min/max`) — these |
| are NOT QAT-only artefacts. HF applies them at inference via |
| `Gemma4ClippableLinear.forward`. Skipping them collapses the encoder past layer 5. |
| - `num_kv_shared_layers`, `layer_full_mask`, `partial_rotary_factor`, |
| `global_head_dim`, `use_double_wide_mlp`, `attention_k_eq_v` — all the per-layer |
| flags the LLM forward needs to honour. |
| - Mel filterbank + Hann window resources (HTK no-norm filters, |
| `frame_length=320` window; the runtime regenerates these too). |
|
|
| Vision tower tensors are excluded. |
|
|
| ## Usage with CrispASR |
|
|
| ```bash |
| # Auto-download (recommended) |
| ./build/bin/crispasr --backend gemma4-e2b -m auto --auto-download -f audio.wav |
| |
| # Or explicit path |
| ./build/bin/crispasr --backend gemma4-e2b -m gemma4-e2b-it-q4_k.gguf -f audio.wav |
| ``` |
|
|
| ## Differential testing |
|
|
| CrispASR ships a stage-by-stage differential test against the HF PyTorch |
| reference. Per-stage cosine similarity vs HF `Gemma4AudioModel`: |
|
|
| ``` |
| mel_spectrogram 1.0000 bit-exact (HF FE faithfully reproduced) |
| audio_subsample_output 0.9994 conv2d + LayerNorm + ReLU |
| audio_layer_0..11 0.97 — 0.99 (with QAT clip scalars) |
| audio_tower_output 0.99+ |
| ``` |
|
|
| Run it yourself: |
|
|
| ```bash |
| # 1. Dump HF reference |
| HF_HOME=/path/to/hf-cache python tools/dump_reference.py \ |
| --backend gemma4 --model-dir google/gemma-4-E2B-it \ |
| --audio samples/jfk.wav --output /tmp/gemma4-ref.gguf |
| |
| # 2. Compare |
| build/bin/crispasr-diff gemma4 \ |
| gemma4-e2b-it-q4_k.gguf /tmp/gemma4-ref.gguf samples/jfk.wav |
| ``` |
|
|
| ## Conversion provenance |
|
|
| This GGUF was produced by `models/convert-gemma4-e2b-to-gguf.py` (CrispASR repo) |
| running on Kaggle T4 nodes (16 GB RAM). Conversion config: |
|
|
| - `--outtype f16` then `crispasr-quantize` for Q-variants. |
| - ClippableLinear QAT scalars persisted as 1-element F32 tensors named |
| `audio.layers.{i}.{linear}.input_min/max, output_min/max`. |
| - Vision tower (`model.vision_tower.*`, `model.embed_vision.*`) skipped. |
|
|
