ariacompute/gemma-3-1b-it_q4

Model Card for Gemma-3-1B-IT (Aria Quant Bundle, q4)

Model Details

Model Description

Gemma-3-1B-IT is a ~1.1-billion-parameter instruction-tuned, dense Transformer decoder-only language model developed by Google, part of the Gemma 3 family. It features GeGLU activation, Grouped Query Attention (GQA), and 32K native context length. Pre-trained on diverse web-scale corpora and aligned via instruction tuning + RLHF. This distribution is provided by Aria Compute as an aria-quant-bundle — a quantized package using Hadamard pre-processing + per-channel quantization. Optimized for CPU-only, on-device inference on mobile phones, edge devices, and single-board computers via the Aria Engine runtime. No GPU or cloud connection is required.

• Developed by: Google
• Quantized and distributed by: Aria Compute
• Model type: Dense Transformer decoder-only (language, instruction-tuned)
• Language(s): English (primary), Chinese, and 30+ additional languages
• License: Apache 2.0
• Finetuned from model: google/gemma-3-1b-it

Model Sources

• Original Repository: google/gemma-3
• Original Paper: Gemma 3 Technical Report
• Aria Compute Dashboard: ariacompute.com/dashboard/models
• Aria Engine: ariacompute.com

Uses

Direct Use

This quantized bundle is intended for on-device, offline text-generation tasks on resource-constrained hardware, including:

• On-device chat and conversational assistants
• Real-time text completion and multi-line code generation
• Instruction-following tasks for mobile and IoT applications
• Lightweight text embeddings for on-device retrieval and classification
• Short-to-medium form summarization of notifications, messages, and local content
• Local document analysis up to 32K context (chunked)

All inference runs locally on CPU. No data is sent to external servers.

Target Devices

Platform	Runtime Memory	Feasibility
High-end smartphone (8 GB)	~0.60 GB	✅ Recommended
Mid-range smartphone (4–6 GB)	~0.60 GB	✅
Budget phone (2–3 GB)	~0.60 GB	✅
Raspberry Pi 5 / SBC (4–8 GB)	~0.60 GB	✅
IoT gateway (1–2 GB)	~0.60 GB	✅
Wearable (1 GB)	~0.60 GB	⚠️ Tight, functional for short contexts

Memory breakdown (q4, at 4K context): ~0.45 GB quantized model weights (mmap) + ~60 MB KV cache + ~45 MB runtime overhead + ~45 MB per-channel metadata overhead ≈ ~0.60 GB.

Note: KV cache is compact due to aggressive GQA (26 layers × 3 KV heads × head_dim=128), ~40% of an equivalent full-attention model's cache.

Out-of-Scope Use

• Long-form creative writing (>4K tokens per generation)
• Mathematical theorem proving or formal verification
• Full program/application synthesis
• Multimodal input (this model is text-only)
• Real-time audio/speech processing (use Aria speech models)
• Safety-critical decision systems without human oversight
• Deployment in production when batch inference or GPU acceleration is required (this bundle targets CPU-only, single-prompt inference)

How to Get Started with the Model

Download from Aria Compute

Authenticated dashboard users can download the bundle via: https://ariacompute.com/dashboard/models

Quantization Recipe

This bundle uses a per-channel quantization recipe, one of several precision options in the Aria Compute lineup:

Component	Quantization Strategy	Details
Attention Q/K/V/O weights	4-bit	Per-channel codebooks, Hadamard pre-processing
FFN gate/up/down weights	4-bit	Per-channel codebooks, Hadamard pre-processing
RMSNorm weights	FP16	Preserved at full precision
Embedding table	FP16	Preserved at full precision

• Bundle size: ~0.45 GB (BF16 original: ~2.2 GB)
• Generation quality: Awaiting gen_quant_eval audit. Per-channel quantization preserves per-output-channel distribution characteristics. Formal quality benchmarks against FP16 and other Aria quant recipes are pending
• Calibration-free: Hadamard pre-processing + per-channel quantization, no task-specific calibration data required
• Other precision options: Also available for Gemma-3-1B-IT: gemma-3-1b-it_q8_channel (per-channel 8-bit) and gemma-3-1b-it_q326_channel

Model Architecture

Gemma-3-1B-IT employs a dense Transformer decoder architecture with GeGLU activation and Grouped Query Attention:

Parameter	Gemma 2 2B	Gemma 3 1B
Layers	26	26
Hidden size	2,304	1,152
FFN intermediate size	9,216	4,608
Attention heads (Query)	8	9
Attention heads (KV)	4 (GQA, group 2)	3 (GQA, group 3)
Head dimension	288	128
Activation	GeGLU	GeGLU
Position encoding	RoPE (θ = 10,000)	RoPE (θ = 10,000)
Normalization	RMSNorm (pre-norm)	RMSNorm (pre-norm)
Vocabulary size	256,128	256,128
Max context length	8,192	32,768

Design highlights (shared with Gemma family):

• GQA (Grouped Query Attention): 3 KV heads serving 9 query heads (group 3) — KV Cache memory is 3× smaller than full attention
• GeGLU activation: GELU with tanh approximation gating, efficient for on-device inference
• RoPE position encoding: Standard 10K base frequency supporting 32K context length
• RMSNorm pre-normalization: Lightweight normalization before each sub-layer

Key architecture differences from Gemma 2 2B:

• Half hidden width (1,152 vs 2,304) and half FFN width (4,608 vs 9,216) — the primary source of the ~1B parameter reduction from the 2B class
• 4× context length (32K vs 8K) — improved long-context capability despite smaller model size
• Smaller head_dim (128 vs 288) — lighter per-head computation with more query heads (9 vs 8)
• More aggressive GQA (group 3 vs group 2) — further KV cache compression to compensate for longer context
• Instruction-tuned variant — this bundle is based on the IT (instruction-tuned) checkpoint, not the base pre-trained model, providing better instruction-following capability out of the box

Bias, Risks, and Limitations

Limitations

• Reasoning depth: Multi-step logical reasoning (≥3 steps) is limited for a 1B-class model. Verify outputs in high-stakes scenarios.
• Mathematics: Simple arithmetic and GSM8K-level problems are partially supported; advanced quantitative reasoning is unreliable. Use larger models for mathematical precision.
• Code generation: Capable of single-function completions and basic snippets; unreliable for multi-file program synthesis or complex algorithms.
• Factual knowledge: Limited world knowledge due to ~1B parameter scale. Always verify factual claims against authoritative sources.
• Instruction following: Handles simple single-constraint instructions reliably. Complex multi-constraint prompts may cause degradation at longer contexts.
• Quantization drift: Per-channel quantization may exhibit generation drift versus FP16, especially on ambiguous or open-ended prompts. For minimal drift, use gemma-3-1b-it_q8_channel (per-channel 8-bit).

Bias and Risks

• Bias: As with all large language models trained on web-scale data, Gemma 3 may reflect societal biases present in its training corpus. Evaluate outputs before deployment in sensitive domains (hiring, healthcare, law).
• Toxicity: The instruction-tuned model has been safety-aligned with RLHF. However, no safety filter is exhaustive. Consider an additional output classifier in high-risk environments.
• Hallucination: May generate plausible-sounding but factually incorrect information. Implement output verification for critical applications.
• Dual-use risk: Text-generation capabilities could be misused for spam, disinformation, or impersonation. Deploy responsibly and in accordance with the Apache 2.0 license terms.

Recommendations

Users (both direct and downstream) should be made aware of the above risks, biases, limitations, and constraints of the model. We recommend:

• Adding a lightweight output safety classifier for user-facing deployments
• Verifying factual claims with external knowledge bases
• Not using the model for high-stakes decisions without human review