elcooooo's picture
Model card, filtered prompts and integration guide
273a61e verified
|
Raw
History Blame Contribute Delete
5.64 kB
---
license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
base_model: google/gemma-4-E2B-it
library_name: gguf
pipeline_tag: text-generation
tags:
- gguf
- llama.cpp
- on-device
- mobile
- vocabulary-pruned
- quantized
language:
- en
- de
- ar
- fr
- es
- it
- pt
- nl
- pl
- tr
- ru
---
# ClickBook Gemma 4 E2B β€” European + Arabic, IQ4_XS
The **smallest** ClickBook on-device reading model, and the most thoroughly
tested. Vocabulary pruned to Latin, Cyrillic and Arabic: eleven European
languages plus Arabic.
**1.666 GB.**
A reader taps a word in a book; the model explains it in the sense that sentence
gives it, writes fresh examples, translates the word, and translates the passage.
## Which of the three to use
| model | size | languages | score |
|---|---:|---|---:|
| **this one** | **1.666 GB** | 11 β€” Latin, Cyrillic, Arabic | **78.4** |
| [multi](https://huggingface.co/Clickbook/ClickBook-Gemma-4-E2B-multi-IQ4_XS) | 1.844 GB | 18 β€” adds CJK, Korean, Hindi, Tamil, Thai, Hebrew | 79.2 |
| [allscripts](https://huggingface.co/Clickbook/ClickBook-Gemma-4-E2B-allscripts-IQ4_XS) | 1.950 GB | every script | 63.6 |
**Take this one if you serve only these eleven languages.** It is 178 MB smaller
than `multi` at the same quality, and these are the languages with the most
benchmark evidence behind them.
**Take `multi` if you need any Asian or Indic language.** The 178 MB it costs buys
seven more languages at no measured quality cost β€” and note that on Android both
files exceed Play's 1.5 GB asset-pack limit anyway, so the smaller file does not
simplify packaging.
## Languages
| language | reading | answering | evidence |
|---|:--:|:--:|---|
| English, German, Arabic | yes | yes | 90-item benchmark, **two seeds** |
| French | yes | not yet | 183 items, older build: 86.3 |
| Portuguese | yes | not yet | 183 items: 85.7 |
| Spanish, Italian | yes | not yet | 183 items: 81.3 |
| Russian | yes | not yet | 183 items: 81.2 |
| Dutch | yes | not yet | 183 items: 79.2 |
| Polish | yes | not yet | 183 items: 76.7 |
| Turkish | yes | not yet | 183 items: **69.3 β€” weakest** |
"Answering" means the model can produce the TRANSLATION and CONTEXT tabs in that
language. Those templates exist for English, German and Arabic only; the other
eight can be read *from*, with answers in one of those three.
Turkish is the weakest language in the set by a clear margin β€” agglutinative
morphology fragments hardest under a pruned vocabulary.
## Evaluation
90 held-out tapped words in English, German and Arabic, graded 0–100 by an LLM
judge against a rubric containing a reference sense.
| build | vocabulary | size | score |
|---|---:|---:|---:|
| same weights at f16 | 231,955 | 8.676 GB | 80.5 |
| multi (18 languages) | 231,955 | 1.844 GB | 79.2 |
| **this model** | 180,850 | **1.666 GB** | **79.0** / 77.7 (two seeds) |
| unpruned vocabulary | 262,144 | 1.950 GB | 63.6 |
**This is the only build measured at two seeds**, giving 78.4 Β± 0.7 β€” worth
knowing, because single-seed differences of a point or so between these builds are
inside that noise. The apparent 0.2 gap to `multi` is not meaningful; the 15.6 gap
to the unpruned build is.
Quantisation costs 1.3 points against f16 for a 4.7Γ— smaller file. Pruning further
β€” to this build's 180,850 tokens β€” costs nothing measurable.
## Usage
```bash
llama-server -m ClickBook-Gemma-4-E2B-eu-ar-ru-IQ4_XS.gguf -c 2048 -ngl 99 --jinja
```
```jsonc
{
"messages": [{ "role": "user", "content": "<prompt from prompts.json>" }],
"temperature": 0.1, "top_k": 40, "top_p": 0.9, "repeat_penalty": 1.05,
"max_tokens": 111,
"chat_template_kwargs": { "enable_thinking": false } // REQUIRED
}
```
### `enable_thinking: false` is required
Without it the model reasons before answering, spends the whole token budget in
`reasoning_content`, and returns **empty `content` with `finish_reason: "length"`**.
That is indistinguishable from a broken model. Raise all caps to 1000 first if you
want reasoning deliberately.
`prompts.json` here is filtered to this build's eleven languages. The wider file
shipped with `multi` includes Chinese, Japanese, Korean, Hindi, Tamil, Thai and
Hebrew β€” sending those prompts to **this** model would hand it text it has no
tokens for.
Per-tab caps: MEANING 111, EXAMPLE 222, TRANSLATION 111, CONTEXT 444. Measured on
the benchmark, 0 of 360 panels reached them.
## Limitations
- **Not a chat model.** Tuned for four narrow tasks driven by the supplied prompts.
- Failures concentrate on polysemous common words β€” *bank*, *charge*, *Schloss* β€”
15 of 90 items.
- No on-device dictionary is bundled. An external sense resource measured +12.7
points on this benchmark but is not included.
- Turkish is materially weaker than the rest.
## License and provenance
**Apache License 2.0**, matching the base model,
[`google/gemma-4-E2B-it`](https://huggingface.co/google/gemma-4-E2B-it). Google
also publishes a [Gemma 4 license page](https://ai.google.dev/gemma/docs/gemma_4_license),
linked from the upstream card.
**Modifications**, as Apache 2.0 requires derivative works to state:
1. **Vocabulary pruned** from 262,144 to 180,850 tokens, retaining Latin, Cyrillic
and Arabic and removing CJK, Hangul, Devanagari, Thai, Hebrew, Greek and other
scripts the product does not serve.
2. **Quantised** to IQ4_XS with Q2_K token embeddings under an importance matrix.
No weights were fine-tuned, distilled or retrained.
Gemma is a trademark of Google LLC. This is an independent derivative, not
endorsed by or affiliated with Google.