Instructions to use evalengine/unbound-e4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use evalengine/unbound-e4b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="evalengine/unbound-e4b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("evalengine/unbound-e4b")
model = AutoModelForImageTextToText.from_pretrained("evalengine/unbound-e4b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use evalengine/unbound-e4b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "evalengine/unbound-e4b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "evalengine/unbound-e4b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/evalengine/unbound-e4b

SGLang

How to use evalengine/unbound-e4b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "evalengine/unbound-e4b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "evalengine/unbound-e4b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "evalengine/unbound-e4b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "evalengine/unbound-e4b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use evalengine/unbound-e4b with Docker Model Runner:
```
docker model run hf.co/evalengine/unbound-e4b
```

johnsonchromia commited on 3 days ago

Commit

c06dcd1

verified ·

1 Parent(s): d32fd44

README: fill in E4B-2 bench numbers; note desktop-only GGUFs

Browse files

Files changed (1) hide show

README.md +31 -17

README.md CHANGED Viewed

@@ -26,46 +26,60 @@ Uncensored variant of `google/gemma-4-E4B-it` from the [**Chromia**](https://x.c
 team — the *larger* sibling of [`evalengine/unbound-e2b`](https://huggingface.co/evalengine/unbound-e2b),
 more capable on knowledge-heavy and reasoning tasks while still fitting on a
 modern laptop. This repo holds the merged HF weights; for the **on-device GGUF builds**
-(Ollama / llama.cpp / LM Studio / [wllama](https://github.com/ngxson/wllama) in-browser), see
 [`evalengine/unbound-e4b-GGUF`](https://huggingface.co/evalengine/unbound-e4b-GGUF).
 ## What this is for
 Same use cases as Unbound E2B — offline / security research / unrestricted
-coding / private workflows — but trading ~2× the parameters (and ~2× the on-disk
-size) for stronger capability. Pick E4B when you have the RAM / VRAM headroom
-and want a noticeably smarter on-device model; pick E2B when you need it to
-fit on a phone or a constrained edge device.
-Base capability is preserved close to `gemma-4-E4B-it`.
 ## Benchmarks (vs base `gemma-4-E4B-it`)
 | Axis | Base | Unbound E4B | Δ |
 |---|---|---|---|
-| Refusal rate (AdvBench 520) | [TBD] | **[TBD]** | **[TBD]** |
-| Useful-compliance rate | [TBD] | **[TBD]** | [TBD] |
-| Hallucination rate | [TBD] | [TBD] | [TBD] |
-| Coherence on benign prompts | 1.0 | [TBD] | [TBD] |
-| TruthfulQA mc2 (lm-eval, `--limit 100`) | [TBD] | [TBD] | [TBD] |
-| MMLU (lm-eval, `--limit 100`) | [TBD] | [TBD] | [TBD] |
-| GSM8K (lm-eval, `--limit 100`) | [TBD] | [TBD] | [TBD] |
-| KL divergence vs base | 0 | [TBD] | (SFT-expected) |
 ## Recommended sampling
-Same guidance as Unbound E2B:
 - **Creative writing / open-ended / general chat** → Gemma defaults:
   `temperature=1.0, top_p=0.95, top_k=64`.
-- **Factual or brand/identity questions** → lower `temperature` to ~0.3–0.5
   for sharper recall.
 - **llama.cpp**: pass `--jinja` for proper chat-template handling.
 - **Gemma 4 thinking mode** is on by default. Set `enable_thinking: false`
   in the chat-template kwargs for shorter/faster replies.
 ## Run on-device (GGUF)
 ```bash
 ollama pull hf.co/evalengine/unbound-e4b-GGUF
 ollama run  hf.co/evalengine/unbound-e4b-GGUF

 team — the *larger* sibling of [`evalengine/unbound-e2b`](https://huggingface.co/evalengine/unbound-e2b),
 more capable on knowledge-heavy and reasoning tasks while still fitting on a
 modern laptop. This repo holds the merged HF weights; for the **on-device GGUF builds**
+(Ollama / llama.cpp / LM Studio), see
 [`evalengine/unbound-e4b-GGUF`](https://huggingface.co/evalengine/unbound-e4b-GGUF).
 ## What this is for
 Same use cases as Unbound E2B — offline / security research / unrestricted
+coding / private workflows — but trading ~2× the parameters (and ~2× the
+on-disk size) for noticeably stronger capability. Pick E4B when you have the
+RAM / VRAM headroom and want a sharper on-device model; pick E2B when you
+need to fit on a phone or a constrained edge device.
+Base capability is preserved close to `gemma-4-E4B-it`, so it also doubles
+as a general-purpose ~4B chat model.
 ## Benchmarks (vs base `gemma-4-E4B-it`)
 | Axis | Base | Unbound E4B | Δ |
 |---|---|---|---|
+| Refusal rate (AdvBench 520, LLM judge) | 98.08% | **2.69%** | **−95.4 pts** |
+| Useful-compliance rate | 0.96% | **43.46%** | +42.5 pts |
+| Hallucination rate (on harmful prompts) | 1.35% | 14.81% | +13.5 pts |
+| Coherence on benign prompts | 1.0 | 1.0 | 0 |
+| TruthfulQA mc2 (lm-eval, `--limit 100`) | 0.4394 | **0.4823** | +4.3 pts |
+| MMLU (lm-eval, `--limit 100`, 61 subtasks avg) | ~0.425 | 0.3891 | −3.6 pts |
+| GSM8K (lm-eval, flexible-extract) | 0.74 (`--limit 200`) | 0.60 (`--limit 100`) | regression mostly limit-noise; see notes |
+| SimpleQA correct rate | — | 2.0% | (post-abliteration: model rarely declines) |
+| KL divergence vs base | 0 | 2.99 | (SFT-expected) |
+**Compared to Unbound E2B (current ship):** E4B is **+19 pp useful_compliance**
+(43% vs 24%), **−7 pp hallucination** (15% vs 22%), **3.3× the GSM8K math
+score**, and ships with a **cleaner KL** (2.99 vs 3.80). The headline refusal
+rate is essentially the same (~2.7%).
 ## Recommended sampling
 - **Creative writing / open-ended / general chat** → Gemma defaults:
   `temperature=1.0, top_p=0.95, top_k=64`.
+- **Factual or brand/identity questions** → drop `temperature` to ~0.3–0.5
   for sharper recall.
 - **llama.cpp**: pass `--jinja` for proper chat-template handling.
 - **Gemma 4 thinking mode** is on by default. Set `enable_thinking: false`
   in the chat-template kwargs for shorter/faster replies.
+Some edge-case prompts may deflect on the first ask; a re-ask or strategic
+re-phrasing usually gets through.
 ## Run on-device (GGUF)
+Q4_K_M / Q6_K / Q8_0 split files at
+[`evalengine/unbound-e4b-GGUF`](https://huggingface.co/evalengine/unbound-e4b-GGUF) —
+note that E4B's per-layer-input embedding tensor exceeds 2 GB in every quant
+we ship, so these are **desktop runtimes only** (no wllama). For an
+in-browser build use [`evalengine/unbound-e2b-GGUF`](https://huggingface.co/evalengine/unbound-e2b-GGUF).
 ```bash
 ollama pull hf.co/evalengine/unbound-e4b-GGUF
 ollama run  hf.co/evalengine/unbound-e4b-GGUF