Instructions to use homerquan/mn-context-engine-model-v-Q4_K_M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use homerquan/mn-context-engine-model-v-Q4_K_M with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="homerquan/mn-context-engine-model-v-Q4_K_M",
	filename="mn-context-engine-model-v3.Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use homerquan/mn-context-engine-model-v-Q4_K_M with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

Use Docker

docker model run hf.co/homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

LM Studio
Jan

vLLM

How to use homerquan/mn-context-engine-model-v-Q4_K_M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "homerquan/mn-context-engine-model-v-Q4_K_M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "homerquan/mn-context-engine-model-v-Q4_K_M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

Ollama
How to use homerquan/mn-context-engine-model-v-Q4_K_M with Ollama:
```
ollama run hf.co/homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M
```

Unsloth Studio new

How to use homerquan/mn-context-engine-model-v-Q4_K_M with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for homerquan/mn-context-engine-model-v-Q4_K_M to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for homerquan/mn-context-engine-model-v-Q4_K_M to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for homerquan/mn-context-engine-model-v-Q4_K_M to start chatting

Pi new

How to use homerquan/mn-context-engine-model-v-Q4_K_M with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use homerquan/mn-context-engine-model-v-Q4_K_M with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use homerquan/mn-context-engine-model-v-Q4_K_M with Docker Model Runner:
```
docker model run hf.co/homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M
```

Lemonade

How to use homerquan/mn-context-engine-model-v-Q4_K_M with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull homerquan/mn-context-engine-model-v-Q4_K_M:Q4_K_M

Run and chat with the model

lemonade run user.mn-context-engine-model-v-Q4_K_M-Q4_K_M

List all available models

lemonade list

mn-context-engine-model-v-Q4_K_M

mn-context-engine-model-v-Q4_K_M is the portable GGUF Q4_K_M build of mn-context-engine-model-v3, the production merged context-compression model for Membrane / MirrorNeuron.

Author: Homer Quan

Website: https://www.mirrorneuron.io

Artifact

File: mn-context-engine-model-v3.Q4_K_M.gguf
Format: GGUF
Quantization: Q4_K_M
Size: 1,915,305,856 bytes, about 1.8 GB
SHA-256: a0e8d8b136174ff4ce6fcb0a5a73e7a204a423a38d0042546c167fac69d27fa4

This is about 69% smaller than the merged Hugging Face safetensors model.

Intended Use

Use this GGUF model with recent llama.cpp-compatible runtimes for local context-compression inference. GGUF is portable across Apple Silicon Metal, NVIDIA CUDA, and CPU fallback builds when the runtime has SmolLM3 support.

For production Membrane deployments, use deterministic cleanup, restoration, privacy redaction, and graph repair after model generation when exact protected-fact preservation is contractual.

Benchmark Summary

Evaluation used Membrane's 100-case mock context-compression suite. Mean ratio is compressed_tokens / original_tokens, so lower is more compressed.

The Q4_K_M benchmark was run locally on Apple Silicon through llama.cpp's OpenAI-compatible server with Metal acceleration. Token counts for this GGUF path used the benchmark's estimated token counter because the Hugging Face tokenizer was not loaded in the llama.cpp server path.

Method	Quality	Fact Recall	Hard Constraints	Pinned	Source Refs	Ratio	Total Time
SmolLM3 v2 LoRA llm_only	0.882	0.942	1.000	0.750	0.698	0.496	3053.4s
SmolLM3 v3 DPO llm_only	0.864	0.916	1.000	0.713	0.627	0.518	1693.3s
SmolLM3 v3 DPO Q4_K_M llm_only	0.843	0.879	0.950	0.693	0.554	0.532	581.7s

Q4_K_M speed telemetry:

Runtime	Mean Latency	P50	P95	Generated Tokens/s	Input Tokens/s
llama.cpp server, Apple M4 Pro Metal	5.81s	5.19s	14.58s	70.4	126.6

Full benchmark reports are included under benchmark/.

llama.cpp Usage

llama-cli \
  -m mn-context-engine-model-v3.Q4_K_M.gguf \
  -ngl 99 \
  -p "Compress this context while preserving protected facts:"

-ngl 99 asks llama.cpp to place model layers on the available accelerator. On Mac this uses Metal when llama.cpp was built with Metal support. On NVIDIA this uses CUDA when llama.cpp was built with CUDA support.

Source

Source model: homerquan/mn-context-engine-model-v3
Base model: HuggingFaceTB/SmolLM3-3B
Quantization tool: llama.cpp, GGUF Q4_K_M

Limitations

Q4_K_M is smaller and faster, but has lower protected-fact/source-ref recall than the full merged model in the current benchmark.
Exact source-reference and pinned-term retention should remain backed by deterministic validation and repair.
It was evaluated on Membrane's deterministic mock-context suite; external workloads should be re-benchmarked.

Downloads last month: 31

GGUF

Model size

3B params

Architecture

smollm3

Hardware compatibility

4-bit

Model tree for homerquan/mn-context-engine-model-v-Q4_K_M

Base model

HuggingFaceTB/SmolLM3-3B-Base

Finetuned

HuggingFaceTB/SmolLM3-3B

Quantized

(72)

this model