Instructions to use mdomina/Qwen3-Coder-Next-Cyber-k32 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mdomina/Qwen3-Coder-Next-Cyber-k32 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mdomina/Qwen3-Coder-Next-Cyber-k32")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("mdomina/Qwen3-Coder-Next-Cyber-k32")
model = AutoModelForCausalLM.from_pretrained("mdomina/Qwen3-Coder-Next-Cyber-k32")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mdomina/Qwen3-Coder-Next-Cyber-k32 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mdomina/Qwen3-Coder-Next-Cyber-k32"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mdomina/Qwen3-Coder-Next-Cyber-k32",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mdomina/Qwen3-Coder-Next-Cyber-k32

SGLang

How to use mdomina/Qwen3-Coder-Next-Cyber-k32 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mdomina/Qwen3-Coder-Next-Cyber-k32" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mdomina/Qwen3-Coder-Next-Cyber-k32",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mdomina/Qwen3-Coder-Next-Cyber-k32" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mdomina/Qwen3-Coder-Next-Cyber-k32",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mdomina/Qwen3-Coder-Next-Cyber-k32 with Docker Model Runner:
```
docker model run hf.co/mdomina/Qwen3-Coder-Next-Cyber-k32
```

Qwen3-Coder-Next-Cyber-k32

Continued-pretraining (CPT) of Qwen/Qwen3-Coder-Next-Base for the cybersecurity domain, using Expert Upcycling + freeze instead of full-model CPT.

The original 80B backbone is frozen and left untouched; the domain knowledge is injected into newly added experts, so general/coding ability is preserved by construction (zero-forgetting) while cyber capacity is added on top.

⚠️ Research artifact. This is a domain-adapted base model (knowledge injection), not an instruction/chat-tuned assistant. Downstream SFT/alignment is expected before use. Post-CPT benchmark evaluation is pending.

What it is


Base	Qwen3-Coder-Next-Base (80B total / ~3B active, hybrid Gated DeltaNet + full Attention + MoE, 512 experts top-10 + 1 shared, 256k context, Apache-2.0)
Method	Expert Upcycling (Amazon, arXiv:2604.19835) + parameter freeze
Experts	512 → 544 (k=32 cyber-relevant experts cloned per layer, 48 layers)
Total params	84.5B (80B frozen base + 32 added experts/layer)
Trainable	only {new experts 512–543, router, shared expert} = 5.04B of 84.5B — the original 80B is frozen
CPT tokens	~200M (physical shuffled blend), ~3.2 epochs
Final train loss	0.68

Method in one paragraph

Starting from the frozen 80B base, we clone k=32 existing experts per layer (selected by utility ‖g‖² on cyber data) to create fresh capacity (512→544 experts). During CPT only the new experts, the router, and the shared expert receive gradients (5.04B trainable). Because the original 512 experts and all attention/backbone weights are frozen, the base model's general and coding skills cannot be eroded. An anti-leak load-balancing term (coef·relu(target − frac_new)) prevents the new experts from being ignored by the router (capacity death): at the end of CPT the new experts carry frac_new ≈ 0.146 of routed mass (target 0.118, uniform 0.059), i.e. they are alive and actively used.

Which experts carry the cyber capacity: in every layer, expert indices 512–543 (the 32 appended experts) are the newly trained cyber experts; indices 0–511 are the frozen original experts. Each new expert is a warm-init clone of one of 32 high-utility base experts (same 32 per layer). Router num_experts_per_tok stays 10 (+1 shared).

Training data

A physical, globally pre-shuffled blend (~200M tokens) mixing curated cyber corpora with filtered cyber web text:

Tier	Share	Content
web	55%	filtered cybersecurity web text (CyberFineWeb-clean)
depth	25%	curated in-depth cyber (MITRE ATT&CK, CWE/OWASP, HackTricks, CVE writeups, …)
compute-foundations	7%	RFC networking, Linux/Windows internals, IAM, crypto/NIST
reference	5%	reference docs
code	5%	security-relevant code
general	3%	general text (anti-catastrophic-forgetting ballast)

Training setup

Framework: Megatron-core 0.18 via ms-swift (megatron pt)
Hardware: 4× A100-SXM4-80GB, pipeline-parallel (PP=4)
Hyperparams: lr 1e-4 → 1e-5 cosine, warmup 2%, grad-clip 1.0, micro-batch 8, global-batch 32, seq-len 256, 24,400 steps (~37h)
Freeze: grouped-GEMM experts kept as separate params; regex isolates the 32 new experts + router + shared

Intended use & limitations

Intended: a domain-adapted base for building cybersecurity assistants (threat hunting / blue-team and red-team knowledge), as a starting point for downstream SFT/RL. Teaching offensive technique is included deliberately — in prior experiments it improved blue-team/defensive performance.

Limitations:

Base model — not instruction-tuned; expect raw completion behavior beyond the base's own chat template.
CPT used a short sequence length (256), so very long-range document structure was under-trained.
Quantitative cyber benchmarks (MMLU-cyber, held-out tasks) are not yet run — the frac_new and loss signals show the mechanism worked, but downstream capability gains are still to be measured.

Responsible use

This model encodes offensive-security knowledge for authorized security testing, defensive research, and education. Do not use it to attack systems you do not own or lack explicit permission to test. Users are responsible for complying with applicable law.

License

Apache-2.0, inherited from the base model (Qwen3-Coder-Next-Base).

Citation / lineage

Base: Qwen3-Coder-Next-Base (Qwen team)
Upcycling method: Expert Upcycling, arXiv:2604.19835
CPT recipe & full engineering notes: kalithos-cybersec (recipes/cpt/RESULT-CPT-V2-2026-07-09.md)

Downloads last month: 416

Safetensors

Model size

85B params

Tensor type

BF16

Model tree for mdomina/Qwen3-Coder-Next-Cyber-k32

Base model

Qwen/Qwen3-Coder-Next-Base

Finetuned

(2)

this model

Paper for mdomina/Qwen3-Coder-Next-Cyber-k32

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts

Paper • 2604.19835 • Published Apr 21 • 21