Instructions to use OliviaRossi/Qwopus-KAT-Coder-35B-Merged with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OliviaRossi/Qwopus-KAT-Coder-35B-Merged with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OliviaRossi/Qwopus-KAT-Coder-35B-Merged")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("OliviaRossi/Qwopus-KAT-Coder-35B-Merged")
model = AutoModelForMultimodalLM.from_pretrained("OliviaRossi/Qwopus-KAT-Coder-35B-Merged", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OliviaRossi/Qwopus-KAT-Coder-35B-Merged with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OliviaRossi/Qwopus-KAT-Coder-35B-Merged"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OliviaRossi/Qwopus-KAT-Coder-35B-Merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OliviaRossi/Qwopus-KAT-Coder-35B-Merged

SGLang

How to use OliviaRossi/Qwopus-KAT-Coder-35B-Merged with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OliviaRossi/Qwopus-KAT-Coder-35B-Merged" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OliviaRossi/Qwopus-KAT-Coder-35B-Merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OliviaRossi/Qwopus-KAT-Coder-35B-Merged" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OliviaRossi/Qwopus-KAT-Coder-35B-Merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OliviaRossi/Qwopus-KAT-Coder-35B-Merged with Docker Model Runner:
```
docker model run hf.co/OliviaRossi/Qwopus-KAT-Coder-35B-Merged
```

Self-speculative Multi-Token Prediction and native vision input, inherited from the Qwopus3.6 / Qwen3.6-35B-A3B lineage — see capabilities and setup requirements below.

Contents — At a Glance · Executive Summary · MTP & Vision · Fusion Architecture · Intended Use & Limitations · Hardware Sizing · Quickstart · Sampling · Prompt & Chat Templates · Lineage · Citation · License

At a Glance


Total parameters	~35B
Active parameters / token	~3B (8 of 256 routed experts + 1 shared expert)
Layers	40
Attention	Hybrid GatedDeltaNet linear attention
Context window	32,768+ tokens
Format	`safetensors` (BF16 / FP16, ~70 GB)
Ideal hardware	Single 80 GB GPU (A100/H100), 2x 48 GB GPUs, or 128 GB+ unified memory (Apple Silicon)
Languages	English, Chinese
License	Apache 2.0

↑ back to top

Executive Summary

Qwopus-KAT-Coder-35B is a Mixture-of-Experts (MoE) coding model built by merging two 35B fine-tunes:

Jackrong/Qwopus3.6-35B-A3B-Coder — strong mathematical code synthesis, multi-step reasoning, and low-drift logic planning.
Kwaipilot/KAT-Coder-V2.5-Dev — repository-level SWE-agent tuning, native terminal tool-calling, and automated issue resolution.

The result pairs algorithmic reasoning with autonomous environment execution: an unquantized, full-precision safetensors checkpoint engineered for high-throughput inference using vLLM, SGLang, Transformers, or TGI.

On this merge: this is a community SLERP fusion of the two base models above. No independent benchmark numbers are published for the merged checkpoint — treat capability claims as inherited from the parent models rather than separately verified.

↑ back to top

MTP & Vision Capabilities

Both parent lines trace back to Qwen3.6-35B-A3B (Alibaba Cloud), which natively ships a vision encoder and a multi-token-prediction (MTP) head. This merge inherits access to both — with two important asterisks explained below.

Multi-token prediction (MTP)

Self-speculative decoding lets the model draft several tokens per forward pass instead of one using its internal MTP head. Native framework support (e.g., vLLM or custom speculative execution pipelines) leverages this structure for faster generation.

Realistic speedup: community testing on Qwen3.6-35B-A3B MTP builds reports roughly 1.4–2.2× faster generation with no accuracy loss — a wide range depending on prompt type and acceptance rate, not a fixed multiplier.
Caveat specific to this merge: the Qwopus3.6 parent lineage has MTP-equipped variants, but Kwaipilot/KAT-Coder-V2.5-Dev does not ship a native MTP head. Whether the merged checkpoint's MTP head survived SLERP fusion cleanly hasn't been independently verified — benchmark MTP speculative execution against standard decoding on your own workload, and fall back to standard autoregressive generation if draft-acceptance rates look poor.

Vision / multimodal input

The underlying architecture supports image input for tasks like UI-screenshot-to-code, turning an architecture diagram into scaffolding, or tracing a GUI bug screenshot back to the likely frontend cause.

Setup requirement this needs: vision requires processing multimodal inputs via standard Hugging Face AutoProcessor or vLLM vision execution pipelines. Ensure your inference script initializes both the processor and vision components included in the repository config.

↑ back to top

The Fusion Architecture

Highlight	Description
Active-parameter efficiency (A3B)	35B total parameters, ~3B active per token (8 of 256 routed experts + 1 shared expert selected). Targets 70B-class capability at 3B-class inference speed.
Hybrid GatedDeltaNet linear attention	Linear recurrence layers interleaved with periodic standard self-attention, keeping memory scaling flat over long context (32k+ tokens).
SLERP MoE consolidation	Weights fused in spherical coordinate space (α = 0.5) across expert projections and attention matrices, preserving expert specialization while limiting representation collapse.

↑ back to top

Intended Use & Limitations

Intended use

Full-precision native deployment on multi-GPU nodes or high-VRAM accelerators (vLLM, SGLang, Transformers).
Local, self-hosted coding assistance: generation, multi-file refactors, and agentic tool-calling workflows.
Developers who want a private alternative to hosted coding assistants without relying on quantization loss.
Image-to-code and UI-to-code workflows using native vision processing pipelines.

Out of scope

Direct execution on low-memory consumer GPUs (e.g., 24 GB single GPUs) without quantization. For 24 GB VRAM setups, use the GGUF Repo.
Safety-critical or production deployment decisions made without human review.
General-purpose assistant use outside software engineering — the merge specifically targets coding and agentic-tool workflows, not general chat quality.

Known limitations

No independent evaluation suite has been run on the merged checkpoint; see the note in the Executive Summary.
Merge behavior is sensitive to the SLERP interpolation coefficient (α = 0.5 here) and may diverge from either parent model in ways that haven't been formally characterized.
MTP-head behavior after the merge is unverified, since only one of the two parents ships a native MTP head — see the caveat above before relying on speculative-decoding speedups.
Inherits any biases, factual errors, or failure modes present in the two base models and in their own training data.

↑ back to top

Hardware Sizing & Precision

Precision	Weights Size	Minimum VRAM / RAM	Primary Target Setup	Recommendation
`BF16` / `FP16`	~70 GB	80 GB VRAM / 128 GB RAM	1x A100/H100 (80GB), 2x RTX 6000 Ada / A6000, Mac Studio (128GB+)	⭐ Native unquantized default
`Q4_K_M` (GGUF)	~20.4 GB	24 GB VRAM / 32 GB RAM	Single RTX 3090 / 4090	See GGUF Repository

↑ back to top

Quickstart Guide

1. High-Performance Serving with `vLLM`

Launch an OpenAI-compatible server:

vllm serve OliviaRossi/Qwopus-KAT-Coder-35B-Merged \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --trust-remote-code \
  --port 8080

2. Python Inference with `transformers`

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OliviaRossi/Qwopus-KAT-Coder-35B-Merged"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

messages = [
    {"role": "system", "content": "You are an autonomous AI coding assistant."},
    {"role": "user", "content": "Write a production-grade Async Task Queue in Python with Redis backend."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=2048,
    temperature=0.7,
    top_p=0.95,
    top_k=20,
    do_sample=True
)

print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

3. Agentic IDE setup (Cline / Roo-Code / Continue.dev)

Point your agent extension to your local vLLM OpenAI server instance:

{
  "apiProvider": "openai",
  "apiBaseUrl": "http://localhost:8080/v1",
  "apiKey": "local",
  "modelId": "OliviaRossi/Qwopus-KAT-Coder-35B-Merged",
  "contextWindow": 32768,
  "maxTokens": 4096
}

↑ back to top

Recommended Sampling Hyperparameters

Generic low-temperature advice (0.1–0.2) doesn't hold for this model family: on Qwen3.6-35B-A3B-based MoE models, very low temperature combined with a repetition penalty above ~1.05 tends to push the model into loop degradation instead of preventing it. The settings below reflect current community guidance for this lineage specifically.

temperature: 0.7           # 0.6-1.0 is the working range for Qwen3.6-35B-A3B; lower settings can loop
top_p: 0.95
top_k: 20
min_p: 0.00                # keep at 0.00 so MoE expert activation isn't over-filtered
presence_penalty: 1.5       # effective for low-drift reasoning and clean tool-call output
repeat_penalty: 1.0         # stay at or below 1.05 — higher compounds with low temp to degrade output
context_window: 32768

↑ back to top

Prompt Template & Chat Template Options

Default template (ChatML)

<|im_start|>system
You are an autonomous AI coding assistant. You analyze complex codebases, write clean and efficient code, debug issues systematically, and produce complete, working implementations.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

When using transformers, use tokenizer.apply_chat_template() to automatically handle template formatting.

Community chat-template alternatives

If you hit tool-calling parse errors or want stricter multi-turn formatting on vLLM or transformers, these are two active community templates built for Qwen-based coding models:

peculiar-ragdoll/Qwen-Sharp-Chat-Templates — a maintained fork of froggeric's fixed Qwen chat templates, tuned for terser multi-turn output.
Jackrong/Qwen-Tool-Calling-Template-Fix — targets JSON tool-call parsing specifically, from the same author as the Qwopus3.6 base model.

↑ back to top

Acknowledgments & Lineage

Role	Model	Link
Ultimate base architecture	Qwen3.6-35B-A3B (Alibaba Cloud) — source of the native vision encoder and MTP head	—
Qwopus base fine-tune	`Jackrong/Qwopus3.6-35B-A3B-Coder`	huggingface.co
KAT-Coder base fine-tune	`Kwaipilot/KAT-Coder-V2.5-Dev`	huggingface.co
Merged base weights	`OliviaRossi/Qwopus-KAT-Coder-35B-Merged`	huggingface.co
Merge method	SLERP, α = 0.50	—
Quantized alternative	`OliviaRossi/Qwopus-KAT-Coder-35B-Merged-GGUF`	huggingface.co
Community chat templates	peculiar-ragdoll, Jackrong	linked in Prompt Template & Chat Template Options

↑ back to top

Citation

If this merge is useful in your work, you can cite the repository directly:

@misc{qwopus-kat-coder-35b-merged,
  title  = {Qwopus-KAT-Coder-35B-Merged},
  author = {OliviaRossi},
  year   = {2026},
  note   = {SLERP merge of Jackrong/Qwopus3.6-35B-A3B-Coder and Kwaipilot/KAT-Coder-V2.5-Dev},
  url    = {https://huggingface.co/OliviaRossi/Qwopus-KAT-Coder-35B-Merged}
}

Please also credit the two base models above, since their training is what this merge builds on.

↑ back to top

License

Released under the Apache 2.0 license. Usage is also subject to the licenses of the underlying base models listed above — check each source repository before redistributing derivative weights.

Built for developers who want a private, self-hosted coding agent — no API keys, no rate limits, no data leaving your machine.

Downloads last month: 274

Safetensors

Model size

68B params

Tensor type

BF16

Model tree for OliviaRossi/Qwopus-KAT-Coder-35B-Merged

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

unsloth/Qwen3.6-35B-A3B

Adapter

Jackrong/Qwopus3.6-35B-A3B-v1

Adapter

Jackrong/Qwopus3.6-35B-A3B-Coder

Finetuned

(3)

this model

At a Glance

Executive Summary

MTP & Vision Capabilities

Multi-token prediction (MTP)

Vision / multimodal input

The Fusion Architecture

Intended Use & Limitations

Hardware Sizing & Precision

Quickstart Guide

1. High-Performance Serving with vLLM

2. Python Inference with transformers

Recommended Sampling Hyperparameters

Prompt Template & Chat Template Options

Default template (ChatML)

Community chat-template alternatives

Acknowledgments & Lineage

Citation

License

Model tree for OliviaRossi/Qwopus-KAT-Coder-35B-Merged

1. High-Performance Serving with `vLLM`

2. Python Inference with `transformers`