Instructions to use Omega-Factory/Qwen3-Coder-Next-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Omega-Factory/Qwen3-Coder-Next-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Omega-Factory/Qwen3-Coder-Next-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Ollama
How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Ollama:
```
ollama run hf.co/Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
```
Unsloth Desktop

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Docker Model Runner:
```
docker model run hf.co/Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
```

Lemonade

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-Coder-Next-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

jhollingsworth commited on 14 days ago

Commit

0525744

verified ·

1 Parent(s): ed6b2d5

Upload receipts/Q4_K_M/llama-runtime.json with huggingface_hub

Browse files

Files changed (1) hide show

receipts/Q4_K_M/llama-runtime.json +1 -0

receipts/Q4_K_M/llama-runtime.json ADDED Viewed

	@@ -0,0 +1 @@

+ {"llama_runtime_closure": {"schema_version": "omega.llama-linked-runtime.v1", "scope": "staged_llama_and_linked_dependencies", "executable_sha256": "c36a46d071be74447afad1ce9afce2e623a9b870509ea31d92a0aecbb628cf15", "ld_library_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain:/usr/local/cuda/lib64:/usr/local/cuda-12/lib64", "staged_libraries": ["/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-base.so", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-base.so.0", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-base.so.0.20.2", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cpu.so", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cpu.so.0", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cpu.so.0.20.2", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cuda.so", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cuda.so.0", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cuda.so.0.20.2", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml.so", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml.so.0", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml.so.0.20.2", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-common.so", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-common.so.0", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-common.so.0.1.2", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-quantize-impl.so", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-server-impl.so", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama.so", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama.so.0", "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama.so.0.1.2"], "libraries": [{"load_path": "/lib/x86_64-linux-gnu/libc.so.6", "resolved_path": "/usr/lib/x86_64-linux-gnu/libc.so.6", "sha256": "5955dead1a55f545cf9cf34a40b2eb65deb84ea503ac467a266d061073315fa7"}, {"load_path": "/lib/x86_64-linux-gnu/libcuda.so.1", "resolved_path": "/usr/lib/x86_64-linux-gnu/libcuda.so.595.91.07", "sha256": "4839b5da17cd8f58a8e9c57e97c9b61f3c41be7934479f48afc983416b2492c3"}, {"load_path": "/lib/x86_64-linux-gnu/libdl.so.2", "resolved_path": "/usr/lib/x86_64-linux-gnu/libdl.so.2", "sha256": "15ba6ee5171eb72f8317c22e2c18ac1a1174e6e274fa458be17550764ef8e993"}, {"load_path": "/lib/x86_64-linux-gnu/libgcc_s.so.1", "resolved_path": "/usr/lib/x86_64-linux-gnu/libgcc_s.so.1", "sha256": "39c9bb846d8491ec41e89b7d34b824e7b6e7ff37d7b9de549305a15c2f7a6cf7"}, {"load_path": "/lib/x86_64-linux-gnu/libgomp.so.1", "resolved_path": "/usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0", "sha256": "92da9020425dac4bac9737cc8f5f74d9df382e349a4514415e897e5f6740da9c"}, {"load_path": "/lib/x86_64-linux-gnu/libm.so.6", "resolved_path": "/usr/lib/x86_64-linux-gnu/libm.so.6", "sha256": "3d448cb402aa7973d3a58f1fefbbe6ac856989702858c9b6ae4128193b0083bf"}, {"load_path": "/lib/x86_64-linux-gnu/libnccl.so.2", "resolved_path": "/usr/lib/x86_64-linux-gnu/libnccl.so.2.21.5", "sha256": "9429e7f8bd83508fd1dc7d94945bb9a1ac68ad1bc41cf0d222f45ba3e3658da2"}, {"load_path": "/lib/x86_64-linux-gnu/libpthread.so.0", "resolved_path": "/usr/lib/x86_64-linux-gnu/libpthread.so.0", "sha256": "fdfd7d83a335ba12bd73e857f95a798dae0355f37935231966fd088647978e23"}, {"load_path": "/lib/x86_64-linux-gnu/librt.so.1", "resolved_path": "/usr/lib/x86_64-linux-gnu/librt.so.1", "sha256": "fa2b83203da2d25fb690140e8bc2da0c034532a045eec9fb8264849bad04f219"}, {"load_path": "/lib/x86_64-linux-gnu/libstdc++.so.6", "resolved_path": "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", "sha256": "41c4cd10be11160dec736958a59eb0553357ad77008e9ba56250d4fd8698b8ec"}, {"load_path": "/lib64/ld-linux-x86-64.so.2", "resolved_path": "/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2", "sha256": "595bcdc306999711a5a9c787e5d01618f94eb92c4c09a178e875a324f1072d89"}, {"load_path": "/usr/local/cuda/lib64/libcublas.so.12", "resolved_path": "/usr/local/cuda-12.4/targets/x86_64-linux/lib/libcublas.so.12.4.5.8", "sha256": "4cc45526449a95d3985389785e985b7832460ea3db178bc9e9963be2111fbee8"}, {"load_path": "/usr/local/cuda/lib64/libcublasLt.so.12", "resolved_path": "/usr/local/cuda-12.4/targets/x86_64-linux/lib/libcublasLt.so.12.4.5.8", "sha256": "44a813aa2da08830f9083f81d0eb73f1ae4052a4d9b0b0de480a8f6cd9eb3078"}, {"load_path": "/usr/local/cuda/lib64/libcudart.so.12", "resolved_path": "/usr/local/cuda-12.4/targets/x86_64-linux/lib/libcudart.so.12.4.127", "sha256": "8774224f5b11a73b15d074a3fcce7327322c5c4cfdfd924d6a826779eec968fe"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-base.so", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-base.so", "sha256": "695e8b95551f27e675616f808c1382126e9d608d30a0c606152135c2b396864c"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-base.so.0", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-base.so.0", "sha256": "695e8b95551f27e675616f808c1382126e9d608d30a0c606152135c2b396864c"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-base.so.0.20.2", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-base.so.0.20.2", "sha256": "695e8b95551f27e675616f808c1382126e9d608d30a0c606152135c2b396864c"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cpu.so", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cpu.so", "sha256": "f7a7f9606bf19c49469b7ef104fe5ea7980d2cdbae01037a27b778f204438495"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cpu.so.0", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cpu.so.0", "sha256": "f7a7f9606bf19c49469b7ef104fe5ea7980d2cdbae01037a27b778f204438495"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cpu.so.0.20.2", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cpu.so.0.20.2", "sha256": "f7a7f9606bf19c49469b7ef104fe5ea7980d2cdbae01037a27b778f204438495"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cuda.so", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cuda.so", "sha256": "806e6924170fac510ab611ad89d379941693f9dc8cc87e5fd9249357129be4b4"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cuda.so.0", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cuda.so.0", "sha256": "806e6924170fac510ab611ad89d379941693f9dc8cc87e5fd9249357129be4b4"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cuda.so.0.20.2", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml-cuda.so.0.20.2", "sha256": "806e6924170fac510ab611ad89d379941693f9dc8cc87e5fd9249357129be4b4"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml.so", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml.so", "sha256": "16a4ea57eaf54cfc5ff85791fe55c0448e1e6477c36f39a8349a4e69e0a9e55b"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml.so.0", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml.so.0", "sha256": "16a4ea57eaf54cfc5ff85791fe55c0448e1e6477c36f39a8349a4e69e0a9e55b"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml.so.0.20.2", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libggml.so.0.20.2", "sha256": "16a4ea57eaf54cfc5ff85791fe55c0448e1e6477c36f39a8349a4e69e0a9e55b"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-common.so", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-common.so", "sha256": "08f4ee4c0ba4adc5371f3ed1954443453601ee9f99fd6bbd6ed768468f83cadc"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-common.so.0", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-common.so.0", "sha256": "08f4ee4c0ba4adc5371f3ed1954443453601ee9f99fd6bbd6ed768468f83cadc"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-common.so.0.1.2", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-common.so.0.1.2", "sha256": "08f4ee4c0ba4adc5371f3ed1954443453601ee9f99fd6bbd6ed768468f83cadc"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-quantize-impl.so", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-quantize-impl.so", "sha256": "df70aa4b36cc813562a806db4596b843d0131779c6d7ac7a3b5229e8e3e5ed08"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-server-impl.so", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama-server-impl.so", "sha256": "878260762266b5d71e545acac95cb23748cf435d3c6be6f33965da0f4eae3ebc"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama.so", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama.so", "sha256": "4f08e7b940c4e41a9ae96ab8f22043b5ca37e10ba14ad28c2fb815d57b479629"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama.so.0", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama.so.0", "sha256": "4f08e7b940c4e41a9ae96ab8f22043b5ca37e10ba14ad28c2fb815d57b479629"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama.so.0.1.2", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libllama.so.0.1.2", "sha256": "4f08e7b940c4e41a9ae96ab8f22043b5ca37e10ba14ad28c2fb815d57b479629"}, {"load_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libmtmd.so.0", "resolved_path": "/workspace/mym-runs/qwen3_8b5d58c7dcd3414196318115b6bea339/toolchain/libmtmd.so.0", "sha256": "1c94ec816a9a375487b804a644a62d8bc9702495c3c19003888011a18dd7325e"}]}, "runtime_closure_sha256": "0ebfb12ae158d04d407736847c7629cb3f59968146d0b4fd7310576388765f4b", "runtime_identity_scope": "staged_llama_and_linked_dependencies", "backend": "llama_cpp", "artifact_sha256": "c6a4c85791cf23606278e3131a0d723d3f720c75d51f4624d99a33e7cff1d6db", "executable_sha256": "c36a46d071be74447afad1ce9afce2e623a9b870509ea31d92a0aecbb628cf15", "pid": 1066061, "terminated": true, "model_meta": {"vocab_type": 2, "n_vocab": 151936, "n_ctx": 262144, "n_ctx_train": 262144, "n_embd": 2048, "n_params": 79674391296, "size": 48483058688, "ftype": "Q4_K - Medium"}, "gpu_offload_observed": null, "request_count": 1201, "request_timeout_s": 120, "log_sha256": "e0402c63757a8edc58c3f82f863887b9ddd923a2930cb1cc05f70de69fd5dd3d"}