Instructions to use XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro")
model = AutoModelForMultimodalLM.from_pretrained("XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro

SGLang

How to use XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro with Docker Model Runner:
```
docker model run hf.co/XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro
```

Qwopus3.6-27B-Coder GPTQ-Pro

These models are built and maintained on rented GPU compute. If you want to show some appreciation, a follow on X or a coffee helps keep the releases coming.

Follow @xreyrobert Buy me a coffee

This is a GPTQ-Pro 4-bit quantization of Jackrong/Qwopus3.6-27B-Coder.

It is a deployment artifact, not a new fine-tune. The goal is to make the Qwopus3.6 27B Coder checkpoint practical for local GPTQ-compatible coding-agent serving while preserving the source model's agent and tool-use oriented chat template.

Among the three artifacts in this publication batch, this is currently the strongest local coding-agent candidate: it ties the previous Qwopus3.6-27B-v2 GPTQ-Pro artifact on the Terminal-Bench Smoke24 score while using less wall time and fewer generated tokens.

Source And Credits

Source model:

Jackrong/Qwopus3.6-27B-Coder

Quantization tooling and reference recipe:

Thanks to Jackrong for the Qwopus3.6 models and to groxaxo for GPTQ-Pro and the Qwen3.6 GPTQ-Pro recipe this run was aligned with.

Artifact Summary

Field	Value
Source model	`Jackrong/Qwopus3.6-27B-Coder`
Architecture	`Qwen3_5ForConditionalGeneration`
Model type	`qwen3_5`
Tensor files	`6`
Safetensors size	`17.63 GiB`
Indexed tensors	`2423`
Quantized `qweight` tensors	`408`
`mtp.*` tensors in index	`true`
vision/visual tensors in index	`true`
Index metadata size matches shards	`true`

This upload includes an MTP-aware GPTQ patch shard:

model-mtp-aware-gptq.safetensors
MTP_AWARE_GPTQ_PATCH.json

That means the artifact has MTP tensors present and quantized MTP linears, but it does not yet mean speculative decoding is a recommended serving mode. See the MTP status notes below.

Quantization Recipe

Setting	Value
Method	GPTQ-Pro / GPTQModel
Quantizer	`gptqmodel:6.1.0-dev`
Bits	`4`
Group size	`128`
Symmetric quantization	`true`
Desc act	`false`
True sequential	`true`
Calibration dataset	WikiText
Calibration samples	`256`
Calibration sequence length	`2048`
MSE	`2.0`
Damp percent	`0.05`
Damp auto increment	`0.01`
FOEM alpha	`0.25`
FOEM beta	`0.2`
FOEM device	`auto`
Dense VRAM strategy	`exclusive`
MoE VRAM strategy	`exclusive`
Disk offload	`true`
Pack implementation	`cpu`

MTP-aware patch metadata:

Field	Value
Patch type	`mtp-aware-gptq-pro-core`
MTP bits	`4`
MTP group size	`128`
MTP calibration samples	`256`
MTP calibration length	`2048`
Quantized MTP key count	`32`

Quantized MTP modules:

mtp.fc
mtp.layers.0.self_attn.q_proj
mtp.layers.0.self_attn.k_proj
mtp.layers.0.self_attn.v_proj
mtp.layers.0.self_attn.o_proj
mtp.layers.0.mlp.gate_proj
mtp.layers.0.mlp.up_proj
mtp.layers.0.mlp.down_proj

Intended Serving Shape

This checkpoint is intended for text-only vLLM serving as a local coding-agent model.

Recommended starting point:

vllm serve XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro \
  --served-model-name qwopus3.6-27b-coder-gptq-pro \
  --language-model-only \
  --dtype float16 \
  --quantization gptq_marlin \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --max-num-seqs 1 \
  --kv-cache-dtype fp8_e5m2 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --enable-prefix-caching \
  --gpu-memory-utilization 0.95 \
  --trust-remote-code

Serving context for the published Smoke24/vLLM measurements:

The Smoke24/vLLM numbers were collected on an internal llm-residency vLLM deployment. The custom image recipe is not published yet, so this card does not present that image as a public reproduction target. The stable serving knobs captured from the run are listed for context.

Field	Value
Nomad job profile	`vllm-qwopus36-coder`
Served model name	`qwopus3.6-27b-coder-gptq-pro-foem-4bit-g128-ns256`
Critical flags	`--dtype float16`, `--quantization gptq_marlin`, `--kv-cache-dtype fp8_e5m2`, `--reasoning-parser qwen3`, `--tool-call-parser qwen3_coder`, `--max-model-len 131072`; `--max-num-batched-tokens` was not set explicitly.

For initial production-style testing, keep speculative decoding off until you have validated MTP behavior with your exact vLLM version and workload.

Public vLLM Reproducibility

This artifact has a public reproducibility path on the unmodified upstream vLLM OpenAI image:

image: docker.io/vllm/vllm-openai:nightly-7a1eb8ac2ec4ea69338c51dc7afd4b15010abfa8
vLLM version observed in validation: 0.20.1rc1.dev16+g7a1eb8ac2
GPU class: single RTX 3090 24 GB / Ampere
--enforce-eager was not used
no local sleep/wake patch or localhost/*sleepwake* image is required for the validation below

Validated serving shape:

context: --max-model-len 131072
--language-model-only, --dtype float16, --quantization gptq_marlin
--kv-cache-dtype fp8_e5m2, --enable-prefix-caching, --max-num-seqs 1
--max-cudagraph-capture-size 32, --gpu-memory-utilization 0.95
--reasoning-parser qwen3, --tool-call-parser qwen3_coder
--enable-sleep-mode was included in the validation command

Startup note: this dense 27B Coder profile can fail the first cold start after torch compile/profiling with a pessimistic KV-cache check. The public validation passed on the second start when reusing persistent vLLM/Nomad-style cache directories such as TORCHINDUCTOR_CACHE_DIR=/data/vllm-qwopus36-coder/torch_compile_cache and VLLM_CACHE_ROOT=/data/vllm-qwopus36-coder. Treat startup retry plus persistent compile cache as part of the serving recipe.

Validation And Benchmarks

Completed artifact checks:

Local shard index inspection completed before upload.
Remote file list verified after upload.
Remote model.safetensors.index.json verified after upload.
Index metadata total size matches the local safetensor shards.
The remote artifact contains the expected safetensor shards.

Terminal-Bench 2.0 Smoke24 result and associated vLLM serving measurements. This Smoke24 run used max_model_len=131072 for apples-to-apples comparison with the other local models in this publication batch:

Run	Score	Success rate	Wall-time	Output tokens	Observed decode	LLM API time
`qwopus3.6-27b-coder-gptq-pro-foem-4bit-g128-ns256`	`16/24`	`66.7%`	`218.8m`	`202.2k`	`38.9 tok/s`	`86.7m`

Smoke24 is a fixed 24-task Terminal-Bench 2.0 comparison corpus, not a full Terminal-Bench leaderboard run.

In this local harness, the coder artifact:

tied Qwopus3.6-27B-v2-GPTQ-Pro-v1 on solved tasks at 16/24;
had the fastest wall time among the compared local runs at 218.8m;
emitted the fewest output tokens among the compared local runs at 202.2k;
had the lowest LLM API time among the 16/24 Smoke24 runs in this local batch.

Task list and harness shape:

benchmarks/terminal-bench-2.0/smoke24_task_list_20260616.md

MTP And Vision Status

The artifact contains mtp.* tensors.
The MTP large linears listed above were quantized with an MTP-aware GPTQ-Pro core capture path.
MTP speculative decoding is not yet published as the recommended serving mode for this artifact; validate it separately before relying on it.
Vision/visual tensors are present because of the source checkpoint structure, but this release is positioned and validated as text-only.

Limitations

Experimental quantization.
Terminal-Bench Smoke24 is a small local comparison corpus, not a full benchmark submission.
The coder Smoke24 result is assembled from a smoke12 run plus a missing12 complement run over the same fixed 24-task corpus.
MTP tensors are present, but speculative decoding is not yet a supported recommendation.
Vision tensors are present, but vision behavior has not been validated.
Loader behavior may vary across vLLM, Transformers, GPTQModel, and GPTQ-Marlin versions.