Instructions to use Rewnozom/Rewnozom-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Rewnozom/Rewnozom-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Rewnozom/Rewnozom-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Rewnozom/Rewnozom-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Rewnozom/Rewnozom-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Rewnozom/Rewnozom-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Rewnozom/Rewnozom-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Rewnozom/Rewnozom-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Rewnozom/Rewnozom-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Rewnozom/Rewnozom-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Rewnozom/Rewnozom-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Rewnozom/Rewnozom-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Rewnozom/Rewnozom-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Rewnozom/Rewnozom-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Rewnozom/Rewnozom-GGUF:Q4_K_M

Ollama
How to use Rewnozom/Rewnozom-GGUF with Ollama:
```
ollama run hf.co/Rewnozom/Rewnozom-GGUF:Q4_K_M
```
Unsloth Desktop

How to use Rewnozom/Rewnozom-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Rewnozom/Rewnozom-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Rewnozom/Rewnozom-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Rewnozom/Rewnozom-GGUF with Docker Model Runner:
```
docker model run hf.co/Rewnozom/Rewnozom-GGUF:Q4_K_M
```

Lemonade

How to use Rewnozom/Rewnozom-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Rewnozom/Rewnozom-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Rewnozom-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Rewnozom/Rewnozom-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Rewnozom/Rewnozom-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Rewnozom/Rewnozom-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Rewnozom/Rewnozom-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Rewnozom/Rewnozom-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Rewnozom/Rewnozom-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Rewnozom-GGUF

GGUF conversion of Rewnozom/Rewnozom, derived from Qwen/Qwen2.5-7B-Instruct-1M.

Original model: Rewnozom/Rewnozom
Converted model: Rewnozom/Rewnozom-GGUF
Format: GGUF
Runtime targets: llama.cpp, Ollama, LM Studio, Jan, and other GGUF loaders

Personal recommendation: start with the two hybrid MX profiles, MX-k_quants and MX-legacy-quants, because they keep the sensitive tensors higher while leaving the rest on a compact base quant.

Available Quantizations

Only quantizations with an actual .gguf file in this workspace are listed.

Quant	File	Approx size
`F32`	`F32/Rewnozom-1M-LR.F32.gguf`	28.38 GB
`F16`	`F16/Rewnozom-1M-LR.F16.gguf`	14.19 GB
`BF16`	`BF16/Rewnozom-1M-LR.BF16.gguf`	14.19 GB
`MX-k_quants`	`MX-k_quants/Rewnozom-1M-LR.Q4_K_M.k_quants_hybrid_q4km_q5_q6.gguf`	4.62 GB
`MX-legacy-quants`	`MX-legacy-quants/Rewnozom-1M-LR.Q4_0.legacy_hybrid_q40_q50_q80.gguf`	5.05 GB
`MXFP4_MOE`	`extra_current_llama_quants/MXFP4_MOE/Rewnozom-1M-LR.MXFP4_MOE.gguf`	7.54 GB
`Q1_0`	`extra_current_llama_quants/Q1_0/Rewnozom-1M-LR.Q1_0.gguf`	1.35 GB
`Q2_0`	`extra_current_llama_quants/Q2_0/Rewnozom-1M-LR.Q2_0.gguf`	2.42 GB
`Q2_K`	`k_quants/Q2_K/Rewnozom-1M-LR.Q2_K.gguf`	2.81 GB
`Q3_K_L`	`k_quants/Q3_K_L/Rewnozom-1M-LR.Q3_K_L.gguf`	3.81 GB
`Q3_K_M`	`k_quants/Q3_K_M/Rewnozom-1M-LR.Q3_K_M.gguf`	3.55 GB
`Q3_K_S`	`k_quants/Q3_K_S/Rewnozom-1M-LR.Q3_K_S.gguf`	3.25 GB
`Q4_K_M`	`k_quants/Q4_K_M/Rewnozom-1M-LR.Q4_K_M.gguf`	4.36 GB
`Q4_K_S`	`k_quants/Q4_K_S/Rewnozom-1M-LR.Q4_K_S.gguf`	4.15 GB
`Q5_K_M`	`k_quants/Q5_K_M/Rewnozom-1M-LR.Q5_K_M.gguf`	5.07 GB
`Q5_K_S`	`k_quants/Q5_K_S/Rewnozom-1M-LR.Q5_K_S.gguf`	4.95 GB
`Q6_K`	`k_quants/Q6_K/Rewnozom-1M-LR.Q6_K.gguf`	5.82 GB
`Q4_0`	`legacy_quants/Q4_0/Rewnozom-1M-LR.Q4_0.gguf`	4.13 GB
`Q4_1`	`legacy_quants/Q4_1/Rewnozom-1M-LR.Q4_1.gguf`	4.54 GB
`Q5_0`	`legacy_quants/Q5_0/Rewnozom-1M-LR.Q5_0.gguf`	4.95 GB
`Q5_1`	`legacy_quants/Q5_1/Rewnozom-1M-LR.Q5_1.gguf`	5.36 GB
`Q8_0`	`legacy_quants/Q8_0/Rewnozom-1M-LR.Q8_0.gguf`	7.54 GB
`TQ1_0`	`t_quants/TQ1_0/Rewnozom-1M-LR.TQ1_0.gguf`	1.99 GB
`TQ2_0`	`t_quants/TQ2_0/Rewnozom-1M-LR.TQ2_0.gguf`	2.28 GB

Quantization Groups

Group	Purpose
`F32`	Full float32 GGUF reference conversion. Highest precision, largest file.
`F16`	Float16 GGUF reference conversion. Good baseline for further quantization.
`BF16`	BFloat16 GGUF reference conversion for runtimes that prefer BF16.
`extra_current_llama_quants`	Additional current llama.cpp-compatible quant types.
`k_quants`	K-quant family, usually the best default family for local use.
`legacy_quants`	Older GGUF quant family for compatibility and comparison.
`t_quants`	Ternary/low-bit quant family for very small local deployments.
`ollama_modelfiles`	Generated Ollama Modelfiles using the base system prompt from `sp.md`.

Recommended Starting Points

Use case	Quant
Personal recommendation, K-family hybrid	`MX-k_quants`
Personal recommendation, legacy hybrid	`MX-legacy-quants`
Best quality among compact K-quants	`Q6_K`
Balanced default	`Q4_K_M`
Smaller memory footprint	`Q3_K_M` or `Q3_K_S`
Very small local test	`Q2_K`, `Q2_0`, `TQ2_0`, or `Q1_0`
Legacy compatibility check	`Q4_0`, `Q4_1`, `Q5_0`, `Q5_1`, `Q8_0`

Actual quality and speed depend on runtime, CPU/GPU offload, context length, and prompt workload. Validate the target quant against your real tasks before using it as a default.

llama.cpp

Run directly from a local GGUF file:

llama-cli -m k_quants/Q4_K_M/Rewnozom-1M-LR.Q4_K_M.gguf \
  -p "Review this implementation plan for missing constraints."

Start an OpenAI-compatible local server:

llama-server -m k_quants/Q4_K_M/Rewnozom-1M-LR.Q4_K_M.gguf

Dataset Context

The model is associated with a synthetic reasoning/control-plane dataset family covering:

boolean CSP logic
branch-dependent task DAGs
ordering and plan repair
multi-hop forward inference
request decomposition
context relevance
deterministic state transitions
memory lifecycle
retrieval/navigation policy
executor routing
tool execution
permissions
multi-agent orchestration
result validation
retry/escalation
composite execution kernel behavior

The dataset design uses deterministic formal worlds, double oracle checks, structural dedupe, and machine-verifiable answers rather than synthetic prose chain-of-thought.

Limitations

GGUF quantization changes numerical behavior compared with the source model.
Lower-bit quants trade quality for memory and speed.
Permission enforcement, destructive actions, and state mutation should remain controlled by deterministic application logic.