Instructions to use PollardWeights/Nanbeige4.3-33B-Pollard with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PollardWeights/Nanbeige4.3-33B-Pollard with Trellis:

# No code snippets available yet for this library.

# To use this model, check the repository files and the library's documentation.

# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.js

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use PollardWeights/Nanbeige4.3-33B-Pollard with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS
# Run inference directly in the terminal:
llama cli -hf PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS
# Run inference directly in the terminal:
llama cli -hf PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS
# Run inference directly in the terminal:
./build/bin/llama cli -hf PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Use Docker

docker model run hf.co/PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Llama new
LM Studio
Jan

vLLM

How to use PollardWeights/Nanbeige4.3-33B-Pollard with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PollardWeights/Nanbeige4.3-33B-Pollard"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PollardWeights/Nanbeige4.3-33B-Pollard",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Ollama
How to use PollardWeights/Nanbeige4.3-33B-Pollard with Ollama:
```
ollama run hf.co/PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS
```
Unsloth Desktop

How to use PollardWeights/Nanbeige4.3-33B-Pollard with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use PollardWeights/Nanbeige4.3-33B-Pollard with Docker Model Runner:
```
docker model run hf.co/PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS
```

Lemonade

How to use PollardWeights/Nanbeige4.3-33B-Pollard with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Run and chat with the model

lemonade run user.Nanbeige4.3-33B-Pollard-IQ2_XXS

List all available models

lemonade list

Hermes Agent

How to use PollardWeights/Nanbeige4.3-33B-Pollard with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use PollardWeights/Nanbeige4.3-33B-Pollard with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "PollardWeights/Nanbeige4.3-33B-Pollard:IQ2_XXS" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Nanbeige4.3-33B -- Pollard

Pollard shrank this model: 66.24 GB (f16) -> 8.52 GB -- 87% smaller, 7.8x down.

The smallest rung here; larger, higher-fidelity rungs are listed below.

format this model's size

f16 66.24 GB

Q8_0 ~35.11 GB

Q6_K 27.17 GB

Q4_K_M ~19.21 GB

PollardMix (this repo's IQ1_KT) 8.52 GB

format	this model's size
f16	66.24 GB
Q8_0	~35.11 GB
Q6_K	27.17 GB
Q4_K_M	~19.21 GB
PollardMix (this repo's IQ1_KT)	8.52 GB

Pollard builds of Nanbeige/Nanbeige4.3-33B made with Pollard Weights -- a ladder of measured-allocation quants (bits placed by per-layer sensitivity, not a uniform crush).

Standard GGUF -- runs in stock llama.cpp / ik_llama.cpp, Ollama, LM Studio, except where noted. IQ1_KT needs ik_llama.cpp: its allocation puts ik_llama-only atoms on the tensors it protects. The rest run anywhere.

Model details


Parameter count	~33.1B
Architecture	`llama`
Input support	text
imatrix	yes -- see calibration
Perplexity measured	yes -- table below

Which file should I choose?

Every rung is the same weights, sized to a different RAM budget by the measured allocation. Pick the largest one that fits your machine with room for context:

~29.2 GB RAM / VRAM -> Q6_K (27.17 GB).
~22.5 GB RAM / VRAM -> IQ4_XS (20.50 GB).
~19.3 GB RAM / VRAM -> IQ3_S (17.30 GB).
~15.2 GB RAM / VRAM -> IQ2_XXS (13.23 GB).
~10.5 GB RAM / VRAM -> IQ1_KT (8.52 GB). (ik_llama.cpp)

Available files

file	PPL	size	Mean KLD	Top-1 agree	runs in	Tool-ready	verified	notes
`Nanbeige4.3-33B-Pollard-IQ1_KT.gguf`	3.1454	8.52 GB	0.382	81.87%	ik_llama	yes (90%)	yes	smallest
`Nanbeige4.3-33B-Pollard-IQ2_XXS.gguf`	2.7847	13.23 GB	0.186	89.27%	any llama.cpp	yes (97%)	yes	recommended default
`Nanbeige4.3-33B-Pollard-IQ3_S.gguf`	2.8114	17.30 GB	0.0456	94.62%	any llama.cpp	yes (97%)	yes
`Nanbeige4.3-33B-Pollard-IQ4_XS.gguf`	2.8415	20.50 GB	0.0222	96.21%	any llama.cpp	yes (97%)	yes
`Nanbeige4.3-33B-Pollard-Q6_K.gguf`	2.8196	27.17 GB	0.0083	97.75%	any llama.cpp	no (62%)	yes	largest

Top-1 agree = share of tokens where the rung's most-likely token is the same one the f16 would have picked (llama-perplexity's Same top p for GGUF, the same logit comparison in pollard-verify for the other lanes). Higher is closer to the original model.

Verified = the rung passed pollard-verify --gate: it is the complete model (every tower, projector and processor file of the source), it loads, and it passes the coherence and runtime checks. Its KLD / top-1, tool-call and retention numbers are measured and shown here as they are; "below band" names a number outside the gate file's band for its bit width -- reported, not blocking.

Tool-ready = the rung still calls tools (pollard-toolcall): served with llama-server --jinja, asked 60 tool probes (single call, choose among eight tools, no call needed, nested arguments) through OpenAI-style tools, and scored on the calls that server parses -- shown as success relative to the reference (the f16 for GGUF, the bf16 original for the other lanes) with a 95% paired-bootstrap CI, or as a bare success rate when no reference run was recorded. "cross-backend" = the reference ran on a different server than the rung; "not measured" = this lane's server could not run where the rung was built; "error" = request failures prevented a valid capability measurement.

Retention vs f16

What each rung keeps of the f16 on real work -- quantized score / f16 score on the same tasks, the same settings, the same harness (100% = no loss). Measured against our own f16, so the rungs compare directly; it is not comparable to a retention figure published on a different suite.

file	Tool calls
`Nanbeige4.3-33B-Pollard-Q6_K.gguf`	62% valid
`Nanbeige4.3-33B-Pollard-IQ4_XS.gguf`	97% valid
`Nanbeige4.3-33B-Pollard-IQ3_S.gguf`	97% valid
`Nanbeige4.3-33B-Pollard-IQ2_XXS.gguf`	97% valid
`Nanbeige4.3-33B-Pollard-IQ1_KT.gguf`	90% valid

Tool calls = the rung's tool-call success as a share of the f16's (pollard-toolcall); "valid" when no f16 run was recorded. Decisions = share of typed decisions identical to the f16's.

Memory at context

Nanbeige4.3 keeps full attention on all 52 layers (8 KV heads x 128 dims), so the KV cache grows linearly with context: 213 KB per token at f16. At long context it, not the weights, decides what fits. The numbers are from pollard-calc (decimal GB), at llama.cpp's cache-type sizes.

KV cache alone:

Context	KV f16	KV q8_0	KV q4_0
32K	7.0 GB	3.7 GB	2.0 GB
128K	27.9 GB	14.8 GB	7.9 GB
256K	55.8 GB	29.7 GB	15.7 GB
512K	111.7 GB	59.3 GB	31.4 GB

Weights + KV cache (RAM + VRAM total; add ~1-2 GB for compute buffers):

KV at f16:

Rung	File	32K	128K	256K	512K
IQ1_KT	8.5 GB	15.5 GB	36.4 GB	64.4 GB	120.2 GB
IQ2_XXS	13.2 GB	20.2 GB	41.2 GB	69.1 GB	124.9 GB
IQ3_S	17.3 GB	24.3 GB	45.2 GB	73.1 GB	129.0 GB
IQ4_XS	20.5 GB	27.5 GB	48.4 GB	76.3 GB	132.2 GB
Q6_K	27.2 GB	34.2 GB	55.1 GB	83.0 GB	138.8 GB

KV at q8_0:

Rung	File	32K	128K	256K	512K
IQ1_KT	8.5 GB	12.2 GB	23.4 GB	38.2 GB	67.8 GB
IQ2_XXS	13.2 GB	16.9 GB	28.1 GB	42.9 GB	72.6 GB
IQ3_S	17.3 GB	21.0 GB	32.1 GB	47.0 GB	76.6 GB
IQ4_XS	20.5 GB	24.2 GB	35.3 GB	50.2 GB	79.8 GB
Q6_K	27.2 GB	30.9 GB	42.0 GB	56.8 GB	86.5 GB

KV at q4_0:

Rung	File	32K	128K	256K	512K
IQ1_KT	8.5 GB	10.5 GB	16.4 GB	24.2 GB	39.9 GB
IQ2_XXS	13.2 GB	15.2 GB	21.1 GB	28.9 GB	44.6 GB
IQ3_S	17.3 GB	19.3 GB	25.1 GB	33.0 GB	48.7 GB
IQ4_XS	20.5 GB	22.5 GB	28.4 GB	36.2 GB	51.9 GB
Q6_K	27.2 GB	29.1 GB	35.0 GB	42.9 GB	58.6 GB

Quantized KV in llama.cpp needs flash attention, e.g. llama-server -hf PollardWeights/Nanbeige4.3-33B-Pollard:<TAG> --jinja -c 131072 -fa on -ctk q8_0 -ctv q8_0.

Download a specific file

pip install -U "huggingface_hub[cli]"
hf download PollardWeights/Nanbeige4.3-33B-Pollard \
  --include "Nanbeige4.3-33B-Pollard-IQ1_KT.gguf" --local-dir ./

How to run

IQ1_KT is built on ik_llama-only atoms, so it runs with ik_llama.cpp:

llama-cli    -m Nanbeige4.3-33B-Pollard-IQ1_KT.gguf -ngl 99 -p "Explain why the sky is blue."
llama-server -m Nanbeige4.3-33B-Pollard-IQ1_KT.gguf -ngl 99

For stock llama.cpp, Ollama or LM Studio, use Q6_K instead:

llama-server -hf PollardWeights/Nanbeige4.3-33B-Pollard:Q6_K
llama-cli    -m Nanbeige4.3-33B-Pollard-Q6_K.gguf -ngl 99 -p "Explain why the sky is blue."

Tool calling

The chat template writes tool calls in the qwen3_coder syntax (detected by rendering it with pollard-modelkind):

<tool_call>
<function=get_weather>
<parameter=city>
Paris
</parameter>
<parameter=days>
7
</parameter>
</function>
</tool_call>

Serve with:

llama.cpp (GGUF): llama-server -m Nanbeige4.3-33B-Pollard-IQ1_KT.gguf --jinja -- the model's own template renders the tools and llama.cpp parses the calls (specialised: Qwen3-Coder); pass tools in /v1/chat/completions.
vLLM: vllm serve <the original weights> --enable-auto-tool-choice --tool-call-parser qwen3_coder

Use it in your harness

Any agent harness that speaks the OpenAI, Anthropic or Responses API can drive this model through llama-server. Pollard Weights' pollard serve picks the runtime this rung needs, sizes the context to the harness and prints its exact config (--write merges it in, after backing the original up):

pollard serve PollardWeights/Nanbeige4.3-33B-Pollard:IQ1_KT --harness claude-code      # or omp, codex, opencode, aider, ...

harness	calls	min. context	setup
Oh My Pi (`omp`)	`/v1/chat/completions`	32k	`~/.omp/agent/models.yml`: provider `api: openai-completions`, `baseUrl: http://127.0.0.1:8080/v1`, `auth: none`
Claude Code	`/v1/messages`	64k	`ANTHROPIC_BASE_URL=http://127.0.0.1:8080` `ANTHROPIC_AUTH_TOKEN=pollard` `ANTHROPIC_MODEL=Nanbeige4.3-33B-Pollard-IQ1_KT`
Codex CLI	`/v1/responses`	32k	`~/.codex/config.toml`: `[model_providers.pollard]` `base_url = "http://127.0.0.1:8080/v1"`, `wire_api = "responses"`
OpenCode	`/v1/chat/completions`	32k	`opencode.json`: provider `npm: @ai-sdk/openai-compatible`, `options.baseURL: http://127.0.0.1:8080/v1`, `limit.context`
Cline / Roo Code	`/v1/chat/completions`	32k	API Provider "OpenAI Compatible", Base URL `http://127.0.0.1:8080/v1`
Aider	`/v1/chat/completions`	16k	`OPENAI_API_BASE=http://127.0.0.1:8080/v1 aider --model openai/Nanbeige4.3-33B-Pollard-IQ1_KT`
Continue	`/v1/chat/completions`	32k	`config.yaml`: `provider: llama.cpp`, `apiBase: http://127.0.0.1:8080`
Goose	`/v1/chat/completions`	32k	`GOOSE_PROVIDER=openai` `OPENAI_HOST=http://127.0.0.1:8080`
OpenHands	`/v1/chat/completions`	32k	model `openai/Nanbeige4.3-33B-Pollard-IQ1_KT`, base URL `http://host.docker.internal:8080/v1`
Open WebUI	`/v1/chat/completions`	8k	Admin Settings -> Connections -> OpenAI API: `http://127.0.0.1:8080/v1`
Ollama	own runtime	8k	`ollama run hf.co/PollardWeights/Nanbeige4.3-33B-Pollard:Q6_K`, then `/set parameter num_ctx 8192` or more
LM Studio / Jan	own runtime	8k	load the GGUF, set the context length to 8192 or more

Without Pollard, start the llama-server from ik_llama.cpp yourself (this rung needs it) with a context the harness fits in and the model's own template:

llama-server -m Nanbeige4.3-33B-Pollard-IQ1_KT.gguf -c 65536 --jinja --alias Nanbeige4.3-33B-Pollard-IQ1_KT -np 1 --port 8080

Too small a context fails silently: the harness's system prompt and tool schemas are truncated and the model looks like it ignores its tools. Claude Code needs 64k.

imatrix (calibration)

The importance matrix (Nanbeige4.3-33B.imatrix, included) was computed on the pollard-calib 'auto' profile (sources, revisions and licences in the shipped .sources.json).

The exact corpus is included as release-calib.txt, so the allocation can be reproduced rather than taken on trust.

ARM / AVX

llama.cpp repacks weights into an interleaved layout at load time for faster inference on ARM and AVX machines -- no special file needed, online repacking covers these quants. The old Q4_0_4_4/4_8/8_8 variants are not required.

Errata

IQ1_KT carries ik_llama-only atoms and needs ik_llama.cpp to run; stock llama.cpp rejects any ggml type above 42 outright. Checked with pollard-ggufcheck, from the files' tensor types rather than their names.
Measured allocation places bits by per-layer sensitivity under a size budget.
Single machine; replication invited.

Credits & license

Base model: Nanbeige/Nanbeige4.3-33B (Nanbeige)
Quantization tooling: llama.cpp (ggml-org)
Method + tooling: Pollard Weights -- measure first, no claim before a number.
License: apache-2.0, inherited from the base model.

Built with Pollard Weights -- frontier models, small hardware, no compromise.

Downloads last month: 1,312

GGUF

Model size

33B params

Architecture

llama

Hardware compatibility

2-bit

3-bit

4-bit

6-bit

View +1 variant

Model tree for PollardWeights/Nanbeige4.3-33B-Pollard

Base model

Nanbeige/Nanbeige4.3-33B-Base

Finetuned

Nanbeige/Nanbeige4.3-33B

Quantized

(99)

this model

PollardWeights
/

Nanbeige4.3-33B-Pollard

Nanbeige4.3-33B -- Pollard

Pollard shrank this model: 66.24 GB (f16) -> 8.52 GB -- 87% smaller, 7.8x down.

Model details

Which file should I choose?

Available files

Retention vs f16

Memory at context

Download a specific file

How to run

Tool calling

Use it in your harness

imatrix (calibration)

ARM / AVX

Errata

Credits & license

Model tree for PollardWeights/Nanbeige4.3-33B-Pollard