Instructions to use QuantPasture/GLM-4.7-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use QuantPasture/GLM-4.7-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="QuantPasture/GLM-4.7-GGUF",
	filename="GLM-4.7-IQ2_M/GLM-4.7-IQ2_M-00001-of-00004.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use QuantPasture/GLM-4.7-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M

Use Docker

docker model run hf.co/QuantPasture/GLM-4.7-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use QuantPasture/GLM-4.7-GGUF with Ollama:
```
ollama run hf.co/QuantPasture/GLM-4.7-GGUF:Q4_K_M
```

Unsloth Studio

How to use QuantPasture/GLM-4.7-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for QuantPasture/GLM-4.7-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for QuantPasture/GLM-4.7-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for QuantPasture/GLM-4.7-GGUF to start chatting

How to use QuantPasture/GLM-4.7-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "QuantPasture/GLM-4.7-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use QuantPasture/GLM-4.7-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default QuantPasture/GLM-4.7-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use QuantPasture/GLM-4.7-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf QuantPasture/GLM-4.7-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "QuantPasture/GLM-4.7-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use QuantPasture/GLM-4.7-GGUF with Docker Model Runner:
```
docker model run hf.co/QuantPasture/GLM-4.7-GGUF:Q4_K_M
```

Lemonade

How to use QuantPasture/GLM-4.7-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull QuantPasture/GLM-4.7-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.GLM-4.7-GGUF-Q4_K_M

List all available models

lemonade list

GLM-4.7-GGUF / glm-4.7-quantization-data.csv

AesSedai

Super-squash branch 'main' using huggingface_hub

e479c6f 2 months ago

Raw

History Blame Contribute Delete

3.18 kB

	model_name,file_size_gb,bpw,Mean KLD_mean,0.1% KLD,0.1% Δp,1.0% KLD,1.0% Δp,10.0% KLD,10.0% Δp,25.0% Δp,5.0% KLD,5.0% Δp,75.0% Δp,90.0% KLD,90.0% Δp,95.0% KLD,95.0% Δp,99.0% KLD,99.0% Δp,99.9% KLD,99.9% Δp,"Cor(ln(PPL(Q)), ln(PPL(base)))",Device 0,Device 1,Maximum KLD,Maximum Δp,Mean KLD_std,Mean PPL(Q)-PPL(base)_mean,Mean PPL(Q)-PPL(base)_std,Mean PPL(Q)/PPL(base)_mean,Mean PPL(Q)/PPL(base)_std,Mean PPL(Q)_mean,Mean PPL(Q)_std,Mean PPL(base)_mean,Mean PPL(base)_std,Mean ln(PPL(Q)/PPL(base))_mean,Mean ln(PPL(Q)/PPL(base))_std,Mean Δp_mean,Mean Δp_std,Median KLD,Median Δp,Minimum KLD,Minimum Δp,RMS Δp_mean,RMS Δp_std,Same top p_mean,Same top p_std,file_path,file_size_gib,ggml_cuda_init,llama_context,llama_kv_cache,llama_memory_breakdown_print,llama_model_loader,llama_params_fit,llama_params_fit_impl,llama_perf_context_print,load,load_tensors,print_info,system_info
	GLM-4.7-IQ2_M (aes_sedai),115.01922418688001,2.57,0.194644,-0.0,-97.937,3e-06,-74.478,0.000168,-11.616,-1.861,3.6e-05,-25.426,0.104,0.42937,3.117,0.789266,8.873,2.50372,26.194,6.182775,56.202,96.27,30908.6,30908.6,10.104371,98.761,0.003154,1.142702,0.050966,1.131615,0.005621,9.82488,0.179312,8.682178,0.157064,0.123646,0.004967,-3.003,0.09,0.058937,-0.038,-0.000268,-99.842,14.575,0.205,79.092,0.257,kld/GLM-4.7/ddh0-imatrix-v2/aes_sedai/GLM-4.7-IQ2_M.md,107.12,2.0,3155121851.0,190.0,-106821106809011.0,-389.0,0.44,146561024.0,97.0,0.9713,18799.6,1024.0,4.848568601128313e+47
	GLM-4.7-IQ4_XS (aes_sedai),177.46804867072,3.96,0.043752,-3e-06,-70.087,0.0,-27.147,3.1e-05,-3.361,-0.408,7e-06,-7.514,0.129,0.082512,1.907,0.156206,4.843,0.589023,16.334,2.273483,45.008,99.17,30908.6,30908.6,9.705799,94.087,0.001071,0.184265,0.020816,1.021223,0.002385,8.866443,0.160719,8.682178,0.157064,0.021001,0.002336,-0.586,0.044,0.011393,-0.003,-0.00043,-97.16,6.907,0.156,89.98,0.19,kld/GLM-4.7/ddh0-imatrix-v2/aes_sedai/GLM-4.7-IQ4_XS.md,165.28,2.0,3155121851.0,190.0,-165004164992011.0,-490.0,1.1,101451024.0,97.0,0.9713,113309.93,1024.0,4.848568601128313e+47
	GLM-4.7-Q4_K_M (aes_sedai),225.23882242048003,5.03,0.017262,-4e-06,-43.804,-0.0,-14.276,1.3e-05,-1.787,-0.181,3e-06,-4.033,0.114,0.031493,1.468,0.057026,3.302,0.215352,10.683,1.048207,31.669,99.65,30908.6,30908.6,6.079566,96.033,0.000585,0.064609,0.013197,1.007442,0.001517,8.746787,0.158456,8.682178,0.157064,0.007414,0.001506,-0.192,0.027,0.00419,-0.0,-0.00013,-96.787,4.251,0.124,93.293,0.158,kld/GLM-4.7/ddh0-imatrix-v2/aes_sedai/GLM-4.7-Q4_K_M.md,209.77,2.0,3155121851.0,190.0,-210060210049011.0,-590.0,0.41,86271024.0,97.0,0.9713,114828.68,1024.0,4.848568601128313e+47
	GLM-4.7-Q5_K_M (aes_sedai),268.5965172736,6.0,0.011578,-4e-06,-27.058,-0.0,-8.82,8e-06,-1.247,-0.113,2e-06,-2.81,0.104,0.017811,1.169,0.032285,2.716,0.13033,8.279,0.782634,22.754,99.75,30908.6,30908.6,7.857873,97.731,0.000687,0.0002,0.011092,1.000023,0.001278,8.682378,0.157101,8.682178,0.157064,2.3e-05,0.001278,-0.029,0.019,0.002398,0.0,-0.000536,-97.54,2.983,0.124,95.134,0.136,kld/GLM-4.7/ddh0-imatrix-v2/aes_sedai/GLM-4.7-Q5_K_M.md,250.15,2.0,3155121851.0,190.0,-250945250933011.0,-690.0,0.44,72491024.0,97.0,0.9713,116206.81,1024.0,4.848568601128313e+47