README.md

---
language:
- en
- zh
license: apache-2.0
library_name: aria-engine
tags:
- gemma-3n
- quantized
- per-channel-quantization
- hadamard
- on-device
- edge
- mobile
- text-generation
- instruction-tuned
- embeddings
pipeline_tag: text-generation
datasets:
- togethercomputer/RedPajama-Data-1T
- EleutherAI/the_pile
- bigcode/the-stack
base_model: google/gemma-3n-e2b-it
model-index:
- name: Gemma-3n-E2B-IT (Aria Quant Bundle, q4)
  results:
  - task:
      type: text-generation
      name: Generation Consistency (vs FP16, method reference)
    metrics:
    - type: description
      value: "awaiting gen_quant_eval audit"
---

# Model Card for Gemma-3n-E2B-IT (Aria Quant Bundle, q4)

## Model Details

### Model Description

Gemma-3n-E2B-IT is a ~2-billion-parameter instruction-tuned multimodal language model developed by Google, part of the Gemma 3n family. Its text backbone features **hybrid attention** (24 of 30 layers sliding-window linear attention with activation sparsity + 6 Laurel low-rank full-attention layers), **GeGLU activation**, **Grouped Query Attention (GQA)**, **per-layer input projections**, and **32K native context length**. Pre-trained on diverse web-scale corpora and aligned via instruction tuning + RLHF. This distribution is provided by **Aria Compute** as an **aria-quant-bundle** — a quantized package using **Hadamard pre-processing + per-channel quantization**. Optimized for **CPU-only, on-device inference** on mobile phones, edge devices, and single-board computers via the [Aria Engine](https://ariacompute.com) runtime. No GPU or cloud connection is required.

- **Developed by:** Google
- **Quantized and distributed by:** Aria Compute
- **Model type:** Dense Transformer decoder-only (multimodal base: image/audio + text inputs, text outputs; this bundle ships the text backbone)
- **Language(s):** English (primary), Chinese, and 30+ additional languages
- **License:** Apache 2.0
- **Finetuned from model:** [google/gemma-3n-e2b-it](https://huggingface.co/google/gemma-3n-e2b-it)

### Model Sources

- **Original Repository:** [google-gemma/gemma-3n](https://github.com/google-gemma/gemma-3n)
- **Original Paper:** [Gemma 3n Technical Report](https://arxiv.org/abs/2506.16392)
- **Aria Compute Dashboard:** [ariacompute.com/dashboard/models](https://ariacompute.com/dashboard/models)
- **Aria Engine:** [ariacompute.com](https://ariacompute.com)

## Uses

### Direct Use

This quantized bundle is intended for **on-device, offline text-generation tasks** on resource-constrained hardware, including:

- On-device chat and conversational assistants
- Real-time text completion and basic code snippet generation
- Instruction-following tasks for mobile and IoT applications
- Lightweight text embeddings for on-device retrieval and classification
- Short-form summarization of notifications, messages, and local content
- Local document analysis up to 32K context (chunked)

All inference runs **locally on CPU**. No data is sent to external servers.

### Target Devices

| Platform                      | Runtime Memory | Feasibility |
|-------------------------------|----------------|-------------|
| High-end smartphone (8 GB)    | ~1.2 GB        | ✅ Recommended |
| Mid-range smartphone (4–6 GB) | ~1.2 GB        | ✅ |
| Budget phone (2–3 GB)         | ~1.2 GB        | ⚠️ Tight fit |
| Raspberry Pi 5 / SBC (4–8 GB) | ~1.2 GB        | ✅ |
| IoT gateway (1–2 GB)          | ~1.2 GB        | ❌ |
| Wearable (1 GB)               | ~1.2 GB        | ❌ |

**Memory breakdown (q4, at 4K context):** ~1.1 GB quantized model weights (mmap) + ~70 MB KV cache + ~30 MB runtime overhead + ~50 MB per-channel metadata overhead ≈ ~1.2 GB.

> Note: KV cache is compact thanks to hybrid attention — 24 of 30 layers use sliding-window attention (window 512, KV bounded by the window), so only the 6 Laurel full-attention layers scale KV with context. Combined with GQA (2 KV heads), this keeps 32K context practical on ~2 GB-class devices.

### Out-of-Scope Use

- Long-form creative writing (>2K tokens per generation)
- Mathematical theorem proving or formal verification
- Full program/application synthesis
- Multimodal input (image/audio encoding pipeline is pending audit for this quantized bundle — text-only in this release)
- Real-time audio/speech processing (use Aria speech models)
- Safety-critical decision systems without human oversight
- Deployment in production when **batch inference** or **GPU acceleration** is required (this bundle targets CPU-only, single-prompt inference)
- Tasks requiring factual precision beyond the model's ~2B parameter capacity

## How to Get Started with the Model

### Download from Aria Compute

Authenticated dashboard users can download the bundle via: https://ariacompute.com/dashboard/models

### Quantization Recipe

This bundle uses a **per-channel** quantization recipe, one of several precision options in the Aria Compute lineup:

| Component | Quantization Strategy | Details |
|-----------|----------------------|---------|
| Attention Q/K/V/O weights | 4-bit | Per-channel codebooks, Hadamard pre-processing |
| FFN gate/up/down weights | 4-bit | Per-channel codebooks, Hadamard pre-processing |
| RMSNorm weights | FP16 | Preserved at full precision |
| Embedding table | FP16 | Preserved at full precision |

- **Bundle size:** ~1.1 GB (BF16 text backbone: ~3.9 GB)
- **Generation quality:** Awaiting gen_quant_eval audit. Per-channel quantization preserves per-output-channel distribution characteristics. Formal quality benchmarks against FP16 and other Aria quant recipes are pending
- **Calibration-free:** Hadamard pre-processing + per-channel quantization, no task-specific calibration data required
- **Other precision options:** Also available for Gemma-3n-E2B-IT: `gemma-3n-e2b-it_q8_channel` (per-channel 8-bit) and `gemma-3n-e2b-it_q326_channel` (mixed precision, attn 4-bit + FFN ~3-bit)

## Model Architecture

Gemma-3n-E2B-IT's text backbone employs a **dense Transformer decoder** with **GeGLU activation**, **hybrid attention** (sliding-window linear attention + Laurel low-rank full attention), and **per-layer input projections** (256-dim layer input → 2,048 hidden):

| Parameter | Gemma 3n E4B | Gemma 3n E2B |
|-----------|--------------|--------------|
| Layers | 35 | **30** |
| Hidden size | 2,048 | **2,048** |
| Layer input size | 256 | **256** |
| FFN intermediate size | 16,384 | **8,192** |
| Attention heads (Query) | 8 | **8** |
| Attention heads (KV) | 2 (GQA) | **2** (GQA) |
| Head dimension | 256 | **256** |
| Full-attention (Laurel) layers | 7 | **6** |
| Sliding-window layers | 28 | **24** |
| Sliding window | 512 | **512** |
| Laurel rank | 64 | **64** |
| Activation | GeGLU | GeGLU |
| Position encoding | RoPE (θ = 1,000,000) | RoPE (θ = 1,000,000) |
| Normalization | RMSNorm (pre-norm) | RMSNorm (pre-norm) |
| Vocabulary size | 262,400 | 262,400 |
| Max context length | 32,768 | **32,768** |

**Design highlights (Gemma 3n family):**
- **Hybrid attention:** 24/30 layers use sliding-window attention (window 512) with **activation sparsity** (first 10 layers at 95% sparsity); only 6/30 layers use full softmax attention with **Laurel** (low-rank attention, rank 64) — dense-attention KV cost is confined to a few layers
- **GQA (Grouped Query Attention):** 2 KV heads serving 8 query heads — KV Cache memory is 4× smaller than full attention
- **Per-layer input projections:** a 256-dim layer-input embedding is projected to the 2,048-dim hidden state at each layer — compact embedding table with shared layer-input processing
- **GeGLU activation:** GELU with tanh approximation gating, efficient for on-device inference
- **RoPE position encoding:** 1M base frequency (global) with 10K local base frequency for sliding-window layers, supporting 32K context
- **RMSNorm pre-normalization:** Lightweight normalization before each sub-layer
- **Final logit softcapping:** Output logits capped at ±30.0 for training stability

**Key architecture differences from Gemma 3n E4B:**
- **~half the parameter count** (~2B vs ~4B) — same hidden size and head configuration, but fewer layers (30 vs 35) and narrower FFN (8,192 vs 16,384)
- **Fewer full-attention layers** (6 vs 7) — a slightly larger share of computation remains in the sliding-window regime
- **Less aggressive KV sharing** (num_kv_shared_layers 10 vs 15)
- **Same head_dim, sliding window, context length, and vocabulary** — architecture-family consistency
- **Instruction-tuned variant** — based on the IT (instruction-tuned) checkpoint, providing better instruction-following capability out of the box

## Bias, Risks, and Limitations

### Limitations

- **Reasoning depth:** Multi-step logical reasoning is limited for a 2B-class model. Verify outputs in high-stakes scenarios; consider larger Gemma 3n variants for reasoning tasks.
- **Mathematics:** Simple arithmetic may be attempted but is unreliable. Advanced quantitative reasoning is out of scope. Use larger models for mathematical tasks.
- **Code generation:** Capable of single-line completions and basic snippets; unreliable for multi-line code or structured programs.
- **Factual knowledge:** Limited world knowledge due to ~2B parameter scale. Always verify factual claims against authoritative sources. This model is best suited for instruction-following and lightweight text processing rather than encyclopedic knowledge retrieval.
- **Instruction following:** Handles simple single-constraint instructions. Complex multi-constraint prompts may cause degradation, especially at longer contexts.
- **Quantization drift:** 4-bit per-channel quantization may exhibit generation drift versus FP16, especially on ambiguous or open-ended prompts. For higher fidelity, use `gemma-3n-e2b-it_q8_channel` (per-channel 8-bit). For even smaller bundles, see `gemma-3n-e2b-it_q326_channel` (mixed precision, attn 4-bit + FFN ~3-bit).

### Bias and Risks

- **Bias:** As with all large language models trained on web-scale data, Gemma 3n may reflect societal biases present in its training corpus. Evaluate outputs before deployment in sensitive domains (hiring, healthcare, law).
- **Toxicity:** The instruction-tuned model has been safety-aligned with RLHF. However, no safety filter is exhaustive. Consider an additional output classifier in high-risk environments.
- **Hallucination:** May generate plausible-sounding but factually incorrect information. Implement output verification for critical applications. Hallucination risk is elevated for smaller models due to limited memorization capacity.
- **Dual-use risk:** Text-generation capabilities could be misused for spam, disinformation, or impersonation. Deploy responsibly and in accordance with the Apache 2.0 license terms.

### Recommendations

Users (both direct and downstream) should be made aware of the above risks, biases, limitations, and constraints of the model. We recommend:
- Adding a lightweight output safety classifier for user-facing deployments
- Verifying factual claims with external knowledge bases
- Not using the model for high-stakes decisions without human review
- Considering larger Gemma 3n variants (E4B) for tasks requiring stronger reasoning or factual recall