How to use Goraint/QwQ-32b-MLX-AWQ-4bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("Goraint/QwQ-32b-MLX-AWQ-4bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True)
How to use Goraint/QwQ-32b-MLX-AWQ-4bit with Pi:
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Goraint/QwQ-32b-MLX-AWQ-4bit"
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Goraint/QwQ-32b-MLX-AWQ-4bit" } ] } } }
# Start Pi in your project directory: pi
How to use Goraint/QwQ-32b-MLX-AWQ-4bit with OpenClaw:
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Goraint/QwQ-32b-MLX-AWQ-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
openclaw agent --local --agent main --message "Hello from Hugging Face"
How to use Goraint/QwQ-32b-MLX-AWQ-4bit with MLX LM:
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "Goraint/QwQ-32b-MLX-AWQ-4bit"
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "Goraint/QwQ-32b-MLX-AWQ-4bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Goraint/QwQ-32b-MLX-AWQ-4bit", "messages": [ {"role": "user", "content": "Hello"} ] }'
How to use Goraint/QwQ-32b-MLX-AWQ-4bit with Hermes Agent:
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Goraint/QwQ-32b-MLX-AWQ-4bit
hermes
This model is a quantized version of QwQ3-32B converted to AWQ Q4 format using the mlx library for efficient inference. It retains the core capabilities of QwQ 32b while optimizing for resource constraints.
Chat template
Files info
Quantized
Base model
