Instructions to use evalengine/unbound-e2b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use evalengine/unbound-e2b-gguf with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="evalengine/unbound-e2b-gguf", filename="mmproj-unbound-e2b.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use evalengine/unbound-e2b-gguf with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M # Run inference directly in the terminal: llama-cli -hf evalengine/unbound-e2b-gguf:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M # Run inference directly in the terminal: llama-cli -hf evalengine/unbound-e2b-gguf:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf evalengine/unbound-e2b-gguf:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf evalengine/unbound-e2b-gguf:Q4_K_M
Use Docker
docker model run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use evalengine/unbound-e2b-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "evalengine/unbound-e2b-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "evalengine/unbound-e2b-gguf", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
- Ollama
How to use evalengine/unbound-e2b-gguf with Ollama:
ollama run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
- Unsloth Studio new
How to use evalengine/unbound-e2b-gguf with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for evalengine/unbound-e2b-gguf to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for evalengine/unbound-e2b-gguf to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for evalengine/unbound-e2b-gguf to start chatting
- Pi new
How to use evalengine/unbound-e2b-gguf with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "evalengine/unbound-e2b-gguf:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use evalengine/unbound-e2b-gguf with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf evalengine/unbound-e2b-gguf:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default evalengine/unbound-e2b-gguf:Q4_K_M
Run Hermes
hermes
- Docker Model Runner
How to use evalengine/unbound-e2b-gguf with Docker Model Runner:
docker model run hf.co/evalengine/unbound-e2b-gguf:Q4_K_M
- Lemonade
How to use evalengine/unbound-e2b-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull evalengine/unbound-e2b-gguf:Q4_K_M
Run and chat with the model
lemonade run user.unbound-e2b-gguf-Q4_K_M
List all available models
lemonade list
README: add mmproj (vision) section + disclaimer + with/without-vision usage
Browse files
README.md
CHANGED
|
@@ -95,11 +95,54 @@ ollama run hf.co/evalengine/unbound-e2b-GGUF
|
|
| 95 |
./llama-cli -m unbound-e2b-Q4_K_M-00001-of-00003.gguf -p "your prompt"
|
| 96 |
```
|
| 97 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
## Run in the browser (wllama)
|
| 99 |
|
| 100 |
[wllama](https://github.com/ngxson/wllama) is a WebAssembly port of llama.cpp
|
| 101 |
that runs entirely in the browser — no server, no install. Use Q2_K, Q3_K_M,
|
| 102 |
-
Q4_K_M, or Q6_K (Q8_0 has a tensor above the 2 GB ArrayBuffer limit)
|
|
|
|
|
|
|
| 103 |
|
| 104 |
```js
|
| 105 |
import { Wllama } from '@wllama/wllama';
|
|
|
|
| 95 |
./llama-cli -m unbound-e2b-Q4_K_M-00001-of-00003.gguf -p "your prompt"
|
| 96 |
```
|
| 97 |
|
| 98 |
+
## Vision / image input (optional)
|
| 99 |
+
|
| 100 |
+
Gemma 4 E2B ships a vision tower; we extracted it as `mmproj-unbound-e2b.gguf`
|
| 101 |
+
(942 MB) in this repo. Pair it with any of the LM quants above to enable
|
| 102 |
+
image-to-text inference.
|
| 103 |
+
|
| 104 |
+
> **Disclaimer.** The vision encoder is **Google's original weights, unchanged**.
|
| 105 |
+
> Unbound's abliteration + SFT-heal only touched the *language model* — the
|
| 106 |
+
> vision tower was frozen during training. Practical consequences:
|
| 107 |
+
>
|
| 108 |
+
> - The LM is uncensored, so it will discuss whatever it *sees* directly.
|
| 109 |
+
> - But the vision encoder still has Google's original alignment baked into
|
| 110 |
+
> visual feature extraction. It may down-weight or distort features for
|
| 111 |
+
> content classes Google's base model was tuned to suppress.
|
| 112 |
+
> - We have **not benchmarked the visual axis** (no measured refusal rate /
|
| 113 |
+
> coherence / hallucination on image inputs). Treat vision as a preview
|
| 114 |
+
> feature, not a flagship one.
|
| 115 |
+
|
| 116 |
+
### Run with vision (llama.cpp `llama-mtmd-cli`)
|
| 117 |
+
|
| 118 |
+
```bash
|
| 119 |
+
./llama-mtmd-cli \
|
| 120 |
+
-m unbound-e2b-Q4_K_M-00001-of-00003.gguf \
|
| 121 |
+
--mmproj mmproj-unbound-e2b.gguf \
|
| 122 |
+
--image path/to/your/image.png \
|
| 123 |
+
-p "What is in this image?"
|
| 124 |
+
```
|
| 125 |
+
|
| 126 |
+
`llama-gemma3-cli` works the same way and is Gemma-specific.
|
| 127 |
+
|
| 128 |
+
### Run text-only (no `--mmproj`)
|
| 129 |
+
|
| 130 |
+
```bash
|
| 131 |
+
./llama-cli -m unbound-e2b-Q4_K_M-00001-of-00003.gguf -p "your prompt"
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
The LM quants work standalone — you do **not** need `mmproj-unbound-e2b.gguf`
|
| 135 |
+
unless you want image input. Ollama / LM Studio's standard text chat works
|
| 136 |
+
out of the box; the mmproj file is only loaded when you point a multimodal
|
| 137 |
+
runtime at it.
|
| 138 |
+
|
| 139 |
## Run in the browser (wllama)
|
| 140 |
|
| 141 |
[wllama](https://github.com/ngxson/wllama) is a WebAssembly port of llama.cpp
|
| 142 |
that runs entirely in the browser — no server, no install. Use Q2_K, Q3_K_M,
|
| 143 |
+
Q4_K_M, or Q6_K (Q8_0 has a tensor above the 2 GB ArrayBuffer limit).
|
| 144 |
+
Browser inference is **text-only** for this model (wllama doesn't currently
|
| 145 |
+
load `mmproj` for vision):
|
| 146 |
|
| 147 |
```js
|
| 148 |
import { Wllama } from '@wllama/wllama';
|