cyburn/Ornith-1.0-35B-int4-AutoRound

Ornith-1.0-35B — INT4 AutoRound

INT4 quantization of deepreinforce-ai/Ornith-1.0-35B using spark-auto-round.

Model Details

Property	Value
Base model	deepreinforce-ai/Ornith-1.0-35B
Architecture	Qwen3.5 MoE (256 experts, 8 active)
Parameters	35B
Context window	196,608 tokens
Quantization	INT4, group_size=128, symmetric
Packing format	auto_round:auto_gptq
AutoRound version	0.14.3
Calibration	512 samples, 1000 iters

Shared expert gate layers are kept in FP16 for accuracy.

Capabilities

• Agentic coding — tool-calling with qwen3_xml format (OpenAI-compatible)
• Reasoning — built-in chain-of-thought via <think>...</think> blocks
• Multimodal — image + text inputs
• Long context — up to 196K tokens

Benchmarks (base model, BF16)

Benchmark	Score
SWE-bench Verified	75.6%
SWE-bench Pro	50.4%
Terminal-Bench 2.1	64.2%
NL2Repo	34.6%
Claw-eval Average	69.8%

Usage

vLLM

vllm serve cyburn/Ornith-1.0-35B-int4-AutoRound \
  --served-model-name qwen/qwen3.5 \
  --load-format instanttensor \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.80 \
  --kv-cache-dtype fp8 \
  --max-model-len 196608 \
  --max-num-batched-tokens 16384 \
  --attention-backend FLASHINFER \
  --quantization compressed-tensors \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --chat-template chat_template.jinja \
  --generation-config auto \
  --trust-remote-code

Requires vLLM ≥ 0.19.1.

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "cyburn/Ornith-1.0-35B-int4-AutoRound",
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("cyburn/Ornith-1.0-35B-int4-AutoRound")

Requires transformers ≥ 5.8.1.

Quantization

Produced with spark-auto-round v0.14.3:

bits=4, group_size=128, sym=True, iters=1000, nsamples=512
packing_format=auto_round:auto_gptq

License

MIT — same as the base model.