mattbucci/Qwen3-Coder-30B-A3B-REAP-AWQ

Qwen3-Coder-30B-A3B REAP AWQ (4-bit)

In-house REAP-pruned and AWQ-quantized variant of Qwen/Qwen3-Coder-30B-A3B-Instruct, built end-to-end from the upstream BF16 base.

• REAP prune (Router-aware Expert Pruning, arxiv:2510.13999): 128 → 96 experts per layer (~25% experts dropped). Saliency S_{L,E} = Σ_t (gate_t_E × ||down_proj_E(x_t)||₂) accumulated over 1024 code-mix calibration samples; top-96 experts per layer kept.
• AWQ calibration: GPTQ W4A16, group_size=128, 1024 samples × 2048 tokens with moe_calibrate_all_experts=True so every expert sees every token (eliminates the rare-routed-expert zero-scale failure mode).
• Calibration mix: 40% code (evol-codealpaca-v1), 25% thinking traces (AM-Thinking-v1-Distilled), 20% math (NuminaMath-CoT), 15% chat (UltraChat).
• MoE router gates + shared expert kept BF16/FP16 (in ignore list); only the per-expert Linears are INT4-packed.
• Disk size: 13 GB (vs 60+ GB BF16). 7 safetensors shards.

Architecture

Field	Value
Class	Qwen3MoeForCausalLM
Hidden layers	48
Experts	96 per layer (pruned from 128)
Active experts per token	8
Hidden size	2048
Quant format	AWQ 4-bit, group_size=128

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "mattbucci/Qwen3-Coder-30B-A3B-REAP-AWQ",
    torch_dtype="bfloat16",
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained("mattbucci/Qwen3-Coder-30B-A3B-REAP-AWQ")

messages = [{"role": "user", "content": "Write a Python function that returns the nth Fibonacci number using memoization."}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=300, temperature=0.1)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

SGLang serving

The model uses Qwen3-Coder's tool-call XML format (<function=NAME>...):

python -m sglang.launch_server \
    --model-path mattbucci/Qwen3-Coder-30B-A3B-REAP-AWQ \
    --quantization moe_wna16 \
    --dtype bfloat16 \
    --tool-call-parser qwen3_coder \
    --tensor-parallel-size 2 \
    --context-length 32768 \
    --disable-cuda-graph

Functional checks

• Code generation: clean (memoized Fibonacci, idiomatic Python) — finish_reason: stop.
• Basic Q&A: PASS.

Notes

• REAP keeps 96 of 128 experts per layer; per-layer survivor lists are not uniform — different experts may be dropped in different layers. The pruned BF16 base is available privately on request; this AWQ pack is the recommended runtime artifact.
• Calibration uses moe_calibrate_all_experts=True which forces every token through every kept expert during the Hessian collection phase, removing the rare-routed-expert blind spot that caused zero-scale failures in earlier MoE AWQ ships.

License

Apache 2.0, inherited from Qwen/Qwen3-Coder-30B-A3B-Instruct.