Instructions to use maci0/Qwopus3.6-27B-Coder-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use maci0/Qwopus3.6-27B-Coder-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="maci0/Qwopus3.6-27B-Coder-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("maci0/Qwopus3.6-27B-Coder-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("maci0/Qwopus3.6-27B-Coder-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use maci0/Qwopus3.6-27B-Coder-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "maci0/Qwopus3.6-27B-Coder-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maci0/Qwopus3.6-27B-Coder-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/maci0/Qwopus3.6-27B-Coder-NVFP4

SGLang

How to use maci0/Qwopus3.6-27B-Coder-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "maci0/Qwopus3.6-27B-Coder-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maci0/Qwopus3.6-27B-Coder-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "maci0/Qwopus3.6-27B-Coder-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maci0/Qwopus3.6-27B-Coder-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use maci0/Qwopus3.6-27B-Coder-NVFP4 with Docker Model Runner:
```
docker model run hf.co/maci0/Qwopus3.6-27B-Coder-NVFP4
```

maci0 commited on 24 days ago

Commit

5804f84

verified ·

1 Parent(s): a0fabf9

card: DATASHEET design language

Browse files

Files changed (1) hide show

README.md +4 -46

README.md CHANGED Viewed

@@ -23,47 +23,8 @@ tags:
   - coder
 ---
-<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; border: 1px solid #cbd5e1; border-radius: 16px; box-shadow: 0 10px 15px -3px rgba(0,0,0,0.08); overflow: hidden; margin-bottom: 26px;">
-  <div style="background: linear-gradient(135deg, #7c3aed 0%, #4c1d95 100%); padding: 26px; color: white;">
-    <div style="font-size: 11px; font-weight: 800; letter-spacing: 1.5px; text-transform: uppercase; color: #ddd6fe; margin-bottom: 12px;">◆ <a href="https://huggingface.co/spaces/maci0/rogue-quants" style="color: #ddd6fe; text-decoration: none;">Rogue Quants</a> · NVFP4</div>
-    <div style="display: flex; align-items: center; justify-content: space-between; flex-wrap: wrap; gap: 10px;">
-      <h1 style="margin: 0; font-size: 25px; font-weight: 800; display: flex; align-items: center; gap: 12px; color: white; border: none;">🪐 Qwopus3.6-27B-Coder · NVFP4</h1>
-      <span style="background: #10b981; color: white; font-size: 11px; font-weight: 800; padding: 5px 12px; border-radius: 20px; text-transform: uppercase; letter-spacing: 0.5px;">💻 Coder</span>
-    </div>
-    <p style="margin: 10px 0 0 0; font-size: 14px; color: #ddd6fe; font-weight: 500;">27B coder vision-language &middot; agentic + tool-calling &middot; thinking &middot; GPTQ NVFP4 W4A4</p>
-  </div>
-  <div style="display: flex; gap: 8px; flex-wrap: wrap; padding: 13px 24px; background: #faf5ff; border-bottom: 1px solid #e9d5ff;">
-    <span style="background: #f3e8ff; color: #6b21a8; font-size: 11px; font-weight: 700; padding: 4px 11px; border-radius: 20px; border: 1px solid #e9d5ff;">⚙️ NVFP4 &middot; W4A4</span>
-    <span style="background: #f3e8ff; color: #6b21a8; font-size: 11px; font-weight: 700; padding: 4px 11px; border-radius: 20px; border: 1px solid #e9d5ff;">💾 ~18 GB</span>
-    <span style="background: #f3e8ff; color: #6b21a8; font-size: 11px; font-weight: 700; padding: 4px 11px; border-radius: 20px; border: 1px solid #e9d5ff;">📉 PPL 6.63</span>
-    <span style="background: #f3e8ff; color: #6b21a8; font-size: 11px; font-weight: 700; padding: 4px 11px; border-radius: 20px; border: 1px solid #e9d5ff;">📐 256K context</span>
-    <span style="background: #f3e8ff; color: #6b21a8; font-size: 11px; font-weight: 700; padding: 4px 11px; border-radius: 20px; border: 1px solid #e9d5ff;">🚀 vLLM &middot; Blackwell</span>
-    <span style="background: #ede9fe; color: #5b21b6; font-size: 11px; font-weight: 700; padding: 4px 11px; border-radius: 20px; border: 1px solid #ddd6fe;">💻 Coder</span>
-    <span style="background: #dcfce7; color: #166534; font-size: 11px; font-weight: 700; padding: 4px 11px; border-radius: 20px; border: 1px solid #bbf7d0;">🛠️ Tool-calling</span>
-  </div>
-  <div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); gap: 14px; padding: 20px 24px; background: #ffffff;">
-    <div style="border: 1px solid #e9d5ff; padding: 14px; border-radius: 10px; background: #faf5ff; text-align: center;">
-      <span style="font-size: 11px; font-weight: 800; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">Size on disk</span>
-      <span style="font-size: 24px; font-weight: 900; color: #4c1d95; display: block; line-height: 1;">18 GB</span>
-      <span style="font-size: 12px; color: #64748b; font-weight: 600;">vs 55.6 GB bf16 (~33%)</span>
-    </div>
-    <div style="border: 1px solid #e9d5ff; padding: 14px; border-radius: 10px; background: #faf5ff; text-align: center;">
-      <span style="font-size: 11px; font-weight: 800; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">wikitext-2 PPL</span>
-      <span style="font-size: 24px; font-weight: 900; color: #4c1d95; display: block; line-height: 1;">6.63</span>
-      <span style="font-size: 12px; color: #64748b; font-weight: 600;">near-lossless vs bf16</span>
-    </div>
-    <div style="border: 1px solid #e9d5ff; padding: 14px; border-radius: 10px; background: #faf5ff; text-align: center;">
-      <span style="font-size: 11px; font-weight: 800; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">Context</span>
-      <span style="font-size: 24px; font-weight: 900; color: #4c1d95; display: block; line-height: 1;">256K</span>
-      <span style="font-size: 12px; color: #64748b; font-weight: 600;">262144 tokens</span>
-    </div>
-    <div style="border: 1px solid #e9d5ff; padding: 14px; border-radius: 10px; background: #faf5ff; text-align: center;">
-      <span style="font-size: 11px; font-weight: 800; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">Scheme</span>
-      <span style="font-size: 24px; font-weight: 900; color: #4c1d95; display: block; line-height: 1;">NVFP4</span>
-      <span style="font-size: 12px; color: #64748b; font-weight: 600;">W4A4 &middot; GPTQ + MSE</span>
-    </div>
-  </div>
-</div>
 **TL;DR:** Qwopus3.6-27B-Coder, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 18 GB, wikitext-2 PPL 6.63, 256K agentic coder.
@@ -83,7 +44,7 @@ this build: 6.63.
 - About 18 GB on disk versus about 55.6 GB for the bf16 source (about 33%).
 - Built for vLLM on NVIDIA Blackwell, where both the 4-bit weight and 4-bit activation
   paths are accelerated. On pre-Blackwell GPUs vLLM runs it weight-only.
-- Loading and generation verified in vLLM v0.23.0 on an NVIDIA GB10 (Blackwell, sm_121).
 ## Fidelity
@@ -213,7 +174,4 @@ the base model.
 - Base model: [Jackrong](https://huggingface.co/Jackrong)
 - Quantization tooling: [llm-compressor](https://github.com/vllm-project/llm-compressor) / [compressed-tensors](https://github.com/neuralmagic/compressed-tensors)
-<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; border: 1px solid #e9d5ff; border-left: 5px solid #7c3aed; border-radius: 0 12px 12px 0; background: #faf5ff; padding: 16px 20px; margin-top: 28px; font-size: 13px; color: #334155; line-height: 1.6;">
-  Part of <a href="https://huggingface.co/spaces/maci0/rogue-quants" style="color: #6d28d9; font-weight: 700; text-decoration: none;">🎲 Rogue Quants</a>, a set of NVFP4 (W4A4) quants for vLLM on Blackwell. See the full <a href="https://huggingface.co/collections/maci0/nvfp4-quants-gb10-blackwell-6a446fc03174db196e436339" style="color: #6d28d9; font-weight: 700; text-decoration: none;">NVFP4 Quants collection</a>.
-  <br>Built on NVIDIA GB10 (Blackwell, sm_121) with llm-compressor · GPTQ + MSE · shared fused-layer scales.
-</div>

   - coder
 ---
+<div style="font-family:ui-monospace,SFMono-Regular,Menlo,monospace;max-width:760px;border:1px solid #cec9ba;background:#f2efe6;color:#14130f;padding:24px 26px;position:relative;border-radius:2px;margin-bottom:26px;"><div style="display:flex;justify-content:space-between;align-items:flex-start;gap:12px;"><span style="font-size:12px;font-weight:700;letter-spacing:0.02em;">RQ-27B-CODER</span></div><div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:25px;font-weight:800;letter-spacing:-0.02em;line-height:1.12;margin:16px 0 6px;color:#14130f;">Qwopus3.6-27B-Coder · NVFP4</div><div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:14px;color:#4a4740;margin-bottom:16px;">27B agentic coder VL · tool-calling · thinking-mode reasoning.</div><table style="width:100%;border-collapse:collapse;font-size:13px;font-variant-numeric:tabular-nums;"><tr><td style="padding:6px 0;border-top:1.5px solid #14130f;border-bottom:1px solid #cec9ba;color:#6f6b60;text-transform:uppercase;letter-spacing:0.05em;font-size:11px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;">Format</td><td style="padding:6px 0;border-top:1.5px solid #14130f;border-bottom:1px solid #cec9ba;text-align:right;font-weight:700;color:#14130f;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;">NVFP4 · W4A4</td></tr><tr><td style="padding:6px 0;border-bottom:1px solid #cec9ba;color:#6f6b60;text-transform:uppercase;letter-spacing:0.05em;font-size:11px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;">Size</td><td style="padding:6px 0;border-bottom:1px solid #cec9ba;text-align:right;font-weight:700;color:#14130f;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;">18 GB</td></tr><tr><td style="padding:6px 0;border-bottom:1px solid #cec9ba;color:#6f6b60;text-transform:uppercase;letter-spacing:0.05em;font-size:11px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;">Perplexity</td><td style="padding:6px 0;border-bottom:1px solid #cec9ba;text-align:right;font-weight:700;color:#14130f;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;">6.63</td></tr><tr><td style="padding:6px 0;color:#6f6b60;text-transform:uppercase;letter-spacing:0.05em;font-size:11px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;">Context</td><td style="padding:6px 0;text-align:right;font-weight:700;color:#14130f;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;">256K</td></tr></table></div>
 **TL;DR:** Qwopus3.6-27B-Coder, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 18 GB, wikitext-2 PPL 6.63, 256K agentic coder.
 - About 18 GB on disk versus about 55.6 GB for the bf16 source (about 33%).
 - Built for vLLM on NVIDIA Blackwell, where both the 4-bit weight and 4-bit activation
   paths are accelerated. On pre-Blackwell GPUs vLLM runs it weight-only.
+- Loading and generation verified in vLLM on an NVIDIA GB10 (Blackwell, sm_121).
 ## Fidelity
 - Base model: [Jackrong](https://huggingface.co/Jackrong)
 - Quantization tooling: [llm-compressor](https://github.com/vllm-project/llm-compressor) / [compressed-tensors](https://github.com/neuralmagic/compressed-tensors)
+<div style="font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:12px;color:#6f6b60;border-top:1.5px solid #14130f;padding-top:14px;margin-top:30px;">Part of <a href="https://huggingface.co/spaces/maci0/rogue-quants" style="color:#b5231c;font-weight:700;text-decoration:none;">Rogue Quants</a> — NVFP4 component datasheets &middot; <a href="https://huggingface.co/collections/maci0/nvfp4-quants-gb10-blackwell-6a446fc03174db196e436339" style="color:#b5231c;font-weight:700;text-decoration:none;">collection</a>. Fabricated on GB10 (Blackwell) with llm-compressor.</div>