Instructions to use DoppelReflEx/QWQ-32B-Dawnwhisper with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DoppelReflEx/QWQ-32B-Dawnwhisper with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DoppelReflEx/QWQ-32B-Dawnwhisper")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DoppelReflEx/QWQ-32B-Dawnwhisper")
model = AutoModelForCausalLM.from_pretrained("DoppelReflEx/QWQ-32B-Dawnwhisper")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use DoppelReflEx/QWQ-32B-Dawnwhisper with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DoppelReflEx/QWQ-32B-Dawnwhisper"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DoppelReflEx/QWQ-32B-Dawnwhisper",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DoppelReflEx/QWQ-32B-Dawnwhisper

SGLang

How to use DoppelReflEx/QWQ-32B-Dawnwhisper with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DoppelReflEx/QWQ-32B-Dawnwhisper" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DoppelReflEx/QWQ-32B-Dawnwhisper",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DoppelReflEx/QWQ-32B-Dawnwhisper" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DoppelReflEx/QWQ-32B-Dawnwhisper",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DoppelReflEx/QWQ-32B-Dawnwhisper with Docker Model Runner:
```
docker model run hf.co/DoppelReflEx/QWQ-32B-Dawnwhisper
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

QWQ-32B-Dawnwhisper

Use Qwen2.5-32B tokenizer. It should be better with normal roleplay instead of reasoning roleplay, I guess.

Like many people said: Tiny Deepseek R1 at home if you don't have too good specs. 16GB Vram card could run IQ3 variants very well.

After quick test, this merge perform very good result and strong capability in roleplay. Nothing more. Thank you for using my merge model.

GGUF (Thank mradermacher and his team, especially nicoboss)

Static - Imatrix

Setting

Please use ChatML template

Reasoning is not necessary to turn on, but a nice feature to enable if you want to 'boost' your experience when roleplaying and multitasking (but more time consuming :/).

Reasoning token is <thinking> </thinking>. You could search how to enable thinking mode in internet. Note that in silly tarven, you should turn off Always add character's name to prompt in Context Formatting and Include names Never in Instruct Template.

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: trashpanda-org/QwQ-32B-Snowdrop-v0
    parameters:
      density: 0.9
      weight: 1
  - model: ArliAI/QwQ-32B-ArliAI-RpR-v3
    parameters:
      density: 0.8
      weight: 0.8
merge_method: dare_ties
base_model: Qwen/QwQ-32B
parameters:
  normalize: true
  rescale: true
tokenizer_source: Qwen/Qwen2.5-32B-Instruct
dtype: bfloat16