README.md

---
language:
- en
- zh
license: apache-2.0
library_name: aria-engine
tags:
- gemma-4
- quantized
- per-channel-quantization
- hadamard
- on-device
- edge
- mobile
- text-generation
- instruction-tuned
- embeddings
pipeline_tag: text-generation
datasets:
- togethercomputer/RedPajama-Data-1T
- EleutherAI/the_pile
- bigcode/the-stack
base_model: google/gemma-4-e2b-it
model-index:
- name: Gemma-4-E2B-IT (Aria Quant Bundle, q8)
  results:
  - task:
      type: text-generation
      name: Generation Consistency (vs FP16, method reference)
    metrics:
    - type: description
      value: "awaiting gen_quant_eval audit"
---

# Model Card for Gemma-4-E2B-IT (Aria Quant Bundle, q8)

## Model Details

### Model Description

Gemma-4-E2B-IT is a ~1.5-billion-parameter instruction-tuned multimodal language model developed by Google, part of the Gemma 4 family. Its text backbone features **hybrid attention** (27 of 35 layers sliding-window linear attention + 8 standard full-attention layers), **GeGLU activation**, **aggressive Grouped Query Attention (GQA, 1 KV head for 8 query heads)**, **per-layer input projections**, **double-wide MLP**, and **128K native context length**. Pre-trained on diverse web-scale corpora and aligned via instruction tuning + RLHF. This distribution is provided by **Aria Compute** as an **aria-quant-bundle** — a quantized package using **Hadamard pre-processing + uniform per-channel 8-bit quantization**. Optimized for **CPU-only, on-device inference** on mobile phones, edge devices, and single-board computers via the [Aria Engine](https://ariacompute.com) runtime. No GPU or cloud connection is required.

- **Developed by:** Google
- **Quantized and distributed by:** Aria Compute
- **Model type:** Dense Transformer decoder-only (multimodal base: image/audio + text inputs, text outputs; this bundle ships the text backbone)
- **Language(s):** English (primary), Chinese, and 30+ additional languages
- **License:** Apache 2.0
- **Finetuned from model:** [google/gemma-4-e2b-it](https://huggingface.co/google/gemma-4-e2b-it)

### Model Sources

- **Original Repository:** [google-gemma/gemma-4](https://github.com/google-gemma/gemma-4)
- **Original Paper:** [Gemma 4 Technical Report](https://arxiv.org/abs/2601.00000)
- **Aria Compute Dashboard:** [ariacompute.com/dashboard/models](https://ariacompute.com/dashboard/models)
- **Aria Engine:** [ariacompute.com](https://ariacompute.com)

## Uses

### Direct Use

This quantized bundle is intended for **on-device, offline text-generation tasks** on resource-constrained hardware, including:

- On-device chat and conversational assistants
- Real-time text completion and basic code snippet generation
- Instruction-following tasks for mobile and IoT applications
- Lightweight text embeddings for on-device retrieval and classification
- Short-form summarization of notifications, messages, and local content
- Local document analysis up to 32K context (chunked)

All inference runs **locally on CPU**. No data is sent to external servers.

### Target Devices

| Platform                      | Runtime Memory | Feasibility |
|-------------------------------|----------------|-------------|
| High-end smartphone (8 GB)    | ~2.0 GB        | ✅ Recommended |
| Mid-range smartphone (4–6 GB) | ~2.0 GB        | ✅ |
| Budget phone (2–3 GB)         | ~2.0 GB        | ✅ |
| Raspberry Pi 5 / SBC (4–8 GB) | ~2.0 GB        | ✅ |
| IoT gateway (1–2 GB)          | ~2.0 GB        | ⚠️ Tight fit |
| Wearable (1 GB)               | ~2.0 GB        | ❌ |

**Memory breakdown (q8, at 4K context):** ~1.9 GB quantized model weights (mmap) + ~40 MB KV cache + ~30 MB runtime overhead + ~60 MB per-channel metadata overhead ≈ ~2.0 GB.

> Note: KV cache is extremely compact thanks to aggressive GQA (1 KV head for 8 query heads) and hybrid attention — 27 of 35 layers use sliding-window attention (window 512, KV bounded by the window), so only the 8 full-attention layers scale KV with context. Combined with tie-word-embeddings, this keeps 128K context practical on ~2.0 GB-class devices.

### Out-of-Scope Use

- Long-form creative writing (>2K tokens per generation)
- Mathematical theorem proving or formal verification
- Full program/application synthesis
- Multimodal input (image/audio encoding pipeline is pending audit for this quantized bundle — text-only in this release)
- Real-time audio/speech processing (use Aria speech models)
- Safety-critical decision systems without human oversight
- Deployment in production when **batch inference** or **GPU acceleration** is required (this bundle targets CPU-only, single-prompt inference)
- Tasks requiring factual precision beyond the model's ~1.5B parameter capacity

## How to Get Started with the Model

### Download from Aria Compute

Authenticated dashboard users can download the bundle via: https://ariacompute.com/dashboard/models

### Quantization Recipe

This bundle uses a **per-channel** quantization recipe, one of several precision options in the Aria Compute lineup:

| Component | Quantization Strategy | Details |
|-----------|----------------------|---------|
| Attention Q/K/V/O weights | 8-bit | Uniform per-channel codebooks, Hadamard pre-processing |
| FFN gate/up/down weights | 8-bit | Uniform per-channel codebooks, Hadamard pre-processing |
| RMSNorm weights | FP16 | Preserved at full precision |
| Embedding table | FP16 | Preserved at full precision (tie_word_embeddings: true) |

- **Bundle size:** ~1.9 GB (BF16 text backbone: ~3.1 GB)
- **Generation quality:** Awaiting gen_quant_eval audit. The uniform 8-bit per-channel recipe is the highest-fidelity bundle in the Aria lineup — near-lossless versus FP16. Per-channel codebooks preserve per-output-channel distribution characteristics. Formal quality benchmarks against FP16 and other Aria quant recipes are pending
- **Calibration-free:** Hadamard pre-processing + per-channel quantization, no task-specific calibration data required
- **Other precision options:** Also available for Gemma-4-E2B-IT: `gemma-4-e2b-it_q326_channel` (mixed precision, attn 4-bit + FFN ~3-bit, recommended quality-size trade-off) and `gemma-4-e2b-it_q4` (uniform per-channel 4-bit, smallest bundle)

## Model Architecture

Gemma-4-E2B-IT's text backbone employs a **dense Transformer decoder** with **GeGLU activation**, **hybrid attention** (sliding-window linear attention + standard full attention), **aggressive GQA**, **per-layer input projections** (256-dim layer input → 1,536 hidden), and **double-wide MLP**:

| Parameter | Gemma 4 E2B |
|-----------|--------------|
| Layers | **35** |
| Hidden size | **1,536** |
| Layer input size | **256** |
| FFN intermediate size | **6,144** (double-wide MLP) |
| Attention heads (Query) | **8** |
| Attention heads (KV) | **1** (aggressive GQA) |
| Head dimension | **256** (global 512) |
| Full-attention layers | **8** |
| Sliding-window layers | **27** |
| Sliding window | **512** |
| Activation | GeGLU (gelu_pytorch_tanh) |
| Position encoding | RoPE (full-attn θ = 1,000,000, partial rotary 0.25; sliding θ = 10,000) |
| Normalization | RMSNorm (pre-norm) |
| Vocabulary size | 262,144 |
| Max context length | **131,072** (128K) |

**Design highlights (Gemma 4 family):**
- **Hybrid attention:** 27/35 layers use sliding-window attention (window 512); 8/35 layers use standard full softmax attention — dense-attention KV cost is confined to a few layers
- **Aggressive GQA:** 1 KV head serving 8 query heads (8× KV compression) — KV Cache memory is minimal, enabling 128K context on edge devices
- **Double-wide MLP:** `use_double_wide_mlp: true` — FFN expands to 6,144 intermediate (≈4× hidden), boosting capacity at modest parameter cost
- **Per-layer input projections:** a 256-dim layer-input embedding is projected to the 1,536-dim hidden state at each layer — compact embedding table with shared layer-input processing
- **GeGLU activation:** GELU with tanh approximation gating, efficient for on-device inference
- **RoPE position encoding:** 1M base frequency for full-attention layers (partial rotary factor 0.25) with 10K base for sliding-window layers, supporting 128K context
- **RMSNorm pre-normalization:** Lightweight normalization before each sub-layer
- **Final logit softcapping:** Output logits capped at ±30.0 for training stability
- **Tied embeddings:** `tie_word_embeddings: true` — input and output embeddings share weights, reducing footprint

## Bias, Risks, and Limitations

### Limitations

- **Reasoning depth:** Multi-step logical reasoning is limited for a 1.5B-class model. Verify outputs in high-stakes scenarios; consider larger Gemma 4 variants for reasoning tasks.
- **Mathematics:** Simple arithmetic may be attempted but is unreliable. Advanced quantitative reasoning is out of scope. Use larger models for mathematical tasks.
- **Code generation:** Capable of single-line completions and basic snippets; unreliable for multi-line code or structured programs.
- **Factual knowledge:** Limited world knowledge due to ~1.5B parameter scale. Always verify factual claims against authoritative sources. This model is best suited for instruction-following and lightweight text processing rather than encyclopedic knowledge retrieval.
- **Instruction following:** Handles simple single-constraint instructions. Complex multi-constraint prompts may cause degradation, especially at longer contexts.
- **Quantization drift:** Uniform 8-bit per-channel quantization exhibits minimal generation drift versus FP16 thanks to near-lossless precision. For a smaller bundle, use `gemma-4-e2b-it_q326_channel` (mixed precision) or `gemma-4-e2b-it_q4` (uniform 4-bit).

### Bias and Risks

- **Bias:** As with all large language models trained on web-scale data, Gemma may reflect societal biases present in its training corpus. Evaluate outputs before deployment in sensitive domains (hiring, healthcare, law).
- **Toxicity:** The instruction-tuned model has been safety-aligned with RLHF. However, no safety filter is exhaustive. Consider an additional output classifier in high-risk environments.
- **Hallucination:** May generate plausible-sounding but factually incorrect information. Implement output verification for critical applications. Hallucination risk is elevated for smaller models due to limited memorization capacity.
- **Dual-use risk:** Text-generation capabilities could be misused for spam, disinformation, or impersonation. Deploy responsibly and in accordance with the Apache 2.0 license terms.

### Recommendations

Users (both direct and downstream) should be made aware of the above risks, biases, limitations, and constraints of the model. We recommend:
- Adding a lightweight output safety classifier for user-facing deployments
- Verifying factual claims with external knowledge bases
- Not using the model for high-stakes decisions without human review
- Considering larger Gemma 4 variants for tasks requiring stronger reasoning or factual recall