knoopx/ThinkingCap-Qwen3.6-27B-NInfer

ThinkingCap-Qwen3.6-27B

A single-file NInfer .ninfer artifact set for ThinkingCap-Qwen3.6-27B — a token-efficient / efficient-thinking multimodal (image+video→text) 27B finetune of Qwen/Qwen3.6-27B. This card documents two derived NInfer artifacts, each a self-contained single binary that the NInfer engine serves directly:

• thinkingcap_qwen3_6_27b_nvfp4.ninfer — NVFP4 (W4A4 mixed-precision) artifact, 16.55 GiB
• thinkingcap_qwen3_6_27b.ninfer — groupwise-int4 (on-the-fly Q4/Q5/Q6/W8) artifact, 16.07 GiB

Both are byte-level containers (raw-bytes-v1 encoding) produced by the NInfer converter toolchain, not Hugging Face pytorch_model.bin / safetensors weights. They are the serving artifacts for the model on the NInfer runtime.

Model architecture

• Architecture: Qwen3_5ForConditionalGeneration (model_type: qwen3_5), a 27B dense multimodal (image+video→text) finetune of Qwen/Qwen3.6-27B. It is a fully dense feed-forward transformer: every decoder layer uses a dense SwiGLU MLP with no routing.
• Text: 64 decoder layers on a hybrid attention schedule — 16 full-attention layers interleaved with 48 recurrent layers (short convolution, width 4) on a full-attention interval of 4; hidden size 5120, intermediate 17408, output matrix 248,320 rows (tokenizer domain 248,077), 24 query heads / 4 KV heads (head dim 256), 1 MTP draft layer, 262,144 max position embeddings
• Vision: 27-layer ViT (hidden 1152, intermediate 4304, 16 heads), 16×16 spatial patch, 2-frame temporal patch, 2×2 spatial merge
• Positional: RoPE θ = 10,000,000 with mRoPE section [11, 11, 10]
• Upstream base: bottlecapai/ThinkingCap-Qwen3.6-27B (itself a finetune of Qwen/Qwen3.6-27B)

Artifacts

1. NVFP4 — thinkingcap_qwen3_6_27b_nvfp4.ninfer

Quantization scheme: NVFP4 W4A4 mixed-precision. The compressed-tensors NVFP4 recipe (nvfp4-pack-quantized) stores packed 4-bit FP4 (E2M1) weights together with per-tensor group scales, with input_activations: tensor_group so activations are also 4-bit-quantized at inference. This is a true W4A4 path — unlike a W4A16 build, where only the weights are 4-bit (Q4) but activations and GEMM inputs stay at 16-bit, keeping the matmul precision at 16-bit. W4A4 trades some quality headroom for lower memory and (on FP4-capable hardware) lower compute, and is the configuration that mirrors the upstream vLLM-quantized NVFP4 checkpoints.

Dual-source conversion. This artifact is built from two fixed source roles in one pass:

1. Base BF16 checkpoint — the full-precision source (bottlecapai/ThinkingCap-Qwen3.6-27B, rev 6a9e13bd), preflight of 1,118 recipes / 1,199 BF16 tensors across 3 shards.
2. vLLM-quantized NVFP4 checkpoint — the NVFP4 source (sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4, rev 9b5389d4), preflight of 379 linear_nvfp4 + 117 linear_bf16 lines with dtypes U8×379 / F8_E4M3×379 / F32×758.

The converter fuses the two: NVFP4-packed lines take their 4-bit weight bytes + scales from the vLLM NVFP4 source, while the remaining 16-bit and resource tensors are sourced from the base BF16 tree. The result is a single NInfer container carrying both the NVFP4 linear layers and the BF16/FP32 remainder.

Converter module + recipe: tools.convert.qwen3_6_27b.convert_nvfp4, RECIPE_ID = qwen3_6_27b_nvfp4-v1 (converter revision 19e80bb).

Conversion command:

python3 -m tools.convert.qwen3_6_27b.convert_nvfp4 \
  --model <bf16-source-dir> \
  --nvfp4-model <nvfp4-source-dir> \
  --out out/thinkingcap_qwen3_6_27b_nvfp4.ninfer \
  --device cpu

CLI arguments (from the module's argparse): --model (base BF16 dir, required), --nvfp4-model (NVFP4 dir, required), --out (required), --device (default cuda; the conversion run used cpu), plus report-only --out-basename, --base-repo, and --nvfp4-repo.

Artifact stats (from thinkingcap_qwen3_6_27b_nvfp4.ninfer.conversion.json):

field	value
identity	qwen3.6-27b / weights_id nvfp4
target_key	qwen3_6_27b
recipe_id	qwen3_6_27b_nvfp4-v1
objects	1307 (1301 tensors + 6 resources)
artifact bytes	18,324,064,000 (16.55 GiB)
tensor_bytes	18,310,883,160
resource_bytes	12,833,441
alignment_bytes	138,503
md5	24a3d90b8aae9ea4f5296061439bd625
elapsed	69.54 s

Format histogram (per-tensor format counts, summing to 1301 tensors):

format	count
BF16	591
FP32	343
NVFP4	247
Q4G64_F16S	55
Q5G64_F16S	54
W8G32_F16S	9
Q6G64_F16S	1
I32	1
total tensors	1301

Plus 6 resources (raw-bytes-v1 encoding). Layouts: blockscale-k16-m128x4-v1 ×247 (the NVFP4 blockscale line), contiguous-le-v1 ×935, row-split-k128-v1 ×119.

NVFP4 source: sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4 (the --nvfp4-model input to the conversion; rev 9b5389d4)

---

2. Groupwise-int4 — thinkingcap_qwen3_6_27b.ninfer

Quantization scheme: on-the-fly groupwise quantization from a single BF16 source. Rather than consuming a pre-quantized checkpoint, the converter reads the base BF16 checkpoint and quantizes each linear layer at conversion time into a per-group 4/5/6-bit + W8 format selected by tensor size/role — Q4G64_F16S (4-bit, group-64), Q5G64_F16S (5-bit, group-64), Q6G64_F16S (6-bit, group-64), and W8G32_F16S (8-bit, group-32). Larger/more-sensitive linears get higher-bit groupwise formats; smaller ones get 4-bit, trading memory for quality. Everything else stays BF16/FP32. This is a W4A16-class build: weights are groupwise-quantized but activations remain 16-bit, in contrast to the W4A4 NVFP4 artifact above.

Converter module + recipe: tools.convert.qwen3_6_27b.convert, RECIPE_ID = qwen3_6_27b-v2 (converter revision 843284b).

Conversion command:

python -m tools.convert.qwen3_6_27b.convert \
  --model <bf16-source-dir> \
  --out out/thinkingcap_qwen3_6_27b.ninfer \
  --device cuda

CLI arguments (from the module's argparse): --model (BF16 dir, required), --out (required), --device (default cuda; the conversion run resolved to cpu).

Artifact stats (from thinkingcap_qwen3_6_27b.ninfer.conversion.json):

field	value
identity	qwen3.6-27b / weights_id groupwise-int
target_key	qwen3_6_27b
recipe_id	qwen3_6_27b-v2
objects	1124 (1118 tensors + 6 resources)
artifact bytes	17,495,365,888 (16.07 GiB)
tensor_bytes	17,482,342,304
resource_bytes	12,833,441
alignment_bytes	14,015
md5	8c08f2f50ca93cab2e13508365d65147
elapsed	188.55 s

Format histogram (per-tensor format counts, summing to 1118 tensors):

format	count
BF16	582
Q5G64_F16S	246
Q4G64_F16S	183
FP32	96
W8G32_F16S	7
Q6G64_F16S	3
I32	1
total tensors	1118

Plus 6 resources (raw-bytes-v1 encoding). Layouts: contiguous-le-v1 ×679, row-split-k128-v1 ×439.

Source: bottlecapai/ThinkingCap-Qwen3.6-27B (BF16 base, rev 6a9e13bd; preflight 1,118 recipes / 1,199 BF16 tensors / 3 shards).

Serving

Both artifacts are served by the NInfer engine:

--model-id thinkingcap-27b

Known issues

• No KL-divergence or perplexity benchmark has been run on either artifact. Quality of the W4A4 NVFP4 and the groupwise-int4 builds has not been measured against the BF16 reference on a held-out corpus; the only validation performed is structural (format histogram, object/tensor/resource counts, and byte/alignment integrity from the converter's own accounting). Do not treat these artifacts as numerically validated for downstream quality claims.

References

• Base checkpoint: bottlecapai/ThinkingCap-Qwen3.6-27B
• NVFP4 source: sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4 (the --nvfp4-model input to the conversion; rev 9b5389d4)
• Converter toolchain: tools.convert.qwen3_6_27b (modules convert / convert_nvfp4), NInfer source tree.