README.md

---
license: gemma
license_link: https://ai.google.dev/gemma/terms
base_model: google/gemma-3n-E2B-it
tags:
  - gguf
  - llama.cpp
  - on-device
  - android
library_name: llama.cpp
---

# Gemma LITE (E2B) — a smaller dictionary, the same mind

A **Model Derivative** of Google's Gemma 3n E2B, made for phones that cannot hold
the original in memory.

**Nothing about the model's reasoning was changed.** All 35 transformer blocks are
byte-for-byte the quantisation they started as. The only modification is to one
tensor.

## What was modified

`per_layer_token_embd.weight` — requantised from **q6_K to q4_0**, matching the
precision of every layer that does the actual reasoning.

That single tensor is **57% of the original file**: 1,837 MiB of a 3,179 MiB
model, shaped `[8960, 262144]`. It is a *lookup table* — one row of about 7 KB is
read per token — and it was stored at **higher precision than the layers doing the
thinking**, every one of which is q4_0.

```
llama-quantize --allow-requantize \
  --tensor-type per_layer_token_embd=q4_0 \
  gemma-3n-E2B-it-q4_0.gguf gemma-lite-e2b-q4.gguf Q4_0
```

| | file | `per_layer_token_embd` |
|---|---|---|
| source (E2B q4_0) | 3.12 GiB | 1,837 MiB (q6_K) |
| **LITE** | **2.56 GiB** | 1,270 MiB (q4_0) |

## ⚠️ A q2_K version of this model was published here and has been withdrawn

An earlier build took the same tensor down to **q2_K**, reaching 2.04 GiB. It
passed a single-turn audition and **failed badly over a real conversation.** It
was replaced on 2026-08-12 and should not be used.

Over 30 turns, q2_K against this q4_0 build:

| | q2_K (withdrawn) | **q4_0 (this file)** |
|---|---|---|
| repetition, second half (4-gram overlap) | 0.175, peaking **0.43** | **0.000** |
| replies containing malformed control tokens | **17 of 30** | 3 of 30 |
| corrupted words (`It't what it is`) | throughout | none |
| latency, second half | 5,600 ms | 3,963 ms |

q2_K collapsed into a verbal tic — *"That's what it is"* in nearly every reply
after turn 12 — emitted malformed control tokens such as `<end_of_of_turn>` into
visible output, and confidently misremembered the conversation it was in.

☠️ **The lesson, if you are quantising embeddings yourself: a single-turn test
cannot see this.** The damage only appears once enough context has to be held at
once. Test over a long conversation, not one reply.

## Measured

Galaxy S22 Ultra (8 GB): the **unmodified** model was killed by Android's
low-memory killer eleven times and never finished loading.

ROG Phone 8 Pro, llama.cpp with OpenCL, 8192 context:

| | |
|---|---|
| unevictable (anonymous) memory | **~178 MiB** |
| GPU / shmem | ~80 MiB |
| prompt eval | **211 tok/s** |
| generation | **10.7 tok/s** |

A ~3,800-token system prompt prefills in about **18 seconds**.

☠️ The GPU path is not an optimisation — it is why this fits. On CPU the same
model holds **1,642 MiB** of unevictable memory, roughly 9× more, which is the
shape of failure that killed the unmodified model on 8 GB.

## Quality

☠️ **No perplexity or standardised benchmark has been run.** What exists is a
30-turn conversational soak and a 17-prompt content probe, both single-sample. If
you need measured quality, measure it.

Note the numbers above are for this file at 4,300 tokens of context; behaviour at
the full 8,192 has not been characterised.

## What it cannot do

The source GGUF carries **no vision or audio encoder tensors**, so this is a
**text-only** model. Gemma 3n's image and audio input paths are not present.

## Licence

Gemma 3n is provided under and subject to the **Gemma Terms of Use**:
https://ai.google.dev/gemma/terms

This is a **modified** version. The modification is described above and is limited
to the requantisation of a single tensor. The Gemma Terms, including the
Prohibited Use Policy, apply to this derivative and to anything derived from it.

Base model: `google/gemma-3n-E2B-it`.