Joseph007fu/Ornith-1.0-35B-MTP2-MTPLX

Ornith-1.0-35B MTP2 — High-Acceptance Q8 Sidecar for MTPLX

A trained, standalone high-acceptance Q8 affine MTP2 sidecar for Ornith-1.0-35B, optimized and validated on Apple Silicon with MTPLX 2.4.2.

This repository contains the adjusted MTP weights, runtime contract, benchmark summaries, and a bundle assembler. It intentionally does not redistribute the 22-36 GB trunk weights. Download a compatible trunk separately and combine it with this sidecar.

Main contribution: MTP2 acceptance optimization

The main result is not a new trunk quantization. It is the adjustment of an existing Qwen3.5-compatible MTP initialization to Ornith UD-Q8 hidden states and output behavior, with special attention to both the first and second speculative tokens.

Same-contract comparison against the borrowed MTP initialization:

Metric	Borrowed MTP	Trained MTP2	Change
Aggregate acceptance	71.66%	86.29%	+14.63 pp
D1 acceptance	78.57%	87.18%	+8.61 pp
D2 acceptance	56.39%	84.88%	+28.49 pp
MTP throughput	101.31 tok/s	116.10-116.67 tok/s	+14.79-15.36 tok/s
Token-weighted speedup	1.193x	1.368-1.372x	+0.175-0.179x
MTP-induced validator regressions vs AR	1	0	fixed in tested suite

The largest improvement is at D2. This matters because a nominal depth-2 MTP head is only useful when the second drafted token is accepted often enough to offset its additional forward cost.

The reliability result is equally important. The borrowed MTP introduced one new validation failure relative to paired AR; the trained MTP2 introduced zero under the same eight-task Hermes contract. The complete 164-task HumanEval run also recorded zero AR-pass-to-MTP-fail transitions. This is a scoped, observed-zero-regression result, not a claim that the model can never make an error.

The comparison above measures the merged training candidate. We separately reloaded the exact standalone sidecar published here and repeated the same three-run contract. That distribution-ready package produced 114.83-115.75 tok/s, or 1.361-1.367x over its paired AR runs, while preserving 86.29% aggregate, 87.18% D1, 84.88% D2, and zero induced validator regressions. We report these package numbers separately so the downloadable artifact is not credited with the slightly faster in-memory candidate result.

Trunk compatibility and application status

Combination	Status	AR tok/s	MTP2 tok/s	MTP2 vs AR	Quality result
UD-Q8 trunk + this Q8 MTP2	Verified / recommended	84.65	115.21	1.361-1.367x	HumanEval 125/164; zero AR-pass to MTP-fail
UD-Q4 trunk + this Q8 MTP2	Experimental / not primary-ready	96.26	127.37	1.312-1.326x	21/32 vs Q4 AR 22/32; one regression

After the MTP2 acceptance work was complete, the same sidecar was tested with two trunk precisions. The Q4 trunk was about 13.7% faster than Q8 AR in our same-host test, but its 32-task HumanEval score fell from Q8 AR's 26/32 to 22/32. The cross-precision Q4+Q8-MTP2 combination then fell to 21/32 and reproduced two Hermes-format regressions. It is published for research and reproduction, not as the recommended primary model.

Files

• mtp.safetensors - 897,467,628-byte trained Q8/group-64 MTP2 sidecar.
• mtp_config.json - architecture, quantization, and trunk profiles.
• mtplx_runtime.json - tested runtime contract and benchmark settings.
• artifact-manifest.json - provenance, pinned revisions, hashes, and claims.
• scripts/assemble_mtplx_bundle.py - fail-closed local bundle assembler.
• TECHNICAL_REPORT.md - methodology, results, limitations, and decision.
• reports/*.json - compact, path-sanitized benchmark summaries.

Recommended combination: UD-Q8

hf download Brooooooklyn/Ornith-1.0-35B-UD-Q8_K_XL-mlx \
  --revision 1aa23d99400067a00d72ad0f47bb0c9ef6126480 \
  --local-dir ~/Models/Ornith-1.0-35B-UD-Q8_K_XL-mlx

hf download Joseph007fu/Ornith-1.0-35B-MTP2-MTPLX \
  --local-dir ~/Models/Ornith-1.0-35B-MTP2-MTPLX

python ~/Models/Ornith-1.0-35B-MTP2-MTPLX/scripts/assemble_mtplx_bundle.py \
  --profile q8-verified \
  --trunk ~/Models/Ornith-1.0-35B-UD-Q8_K_XL-mlx \
  --sidecar-dir ~/Models/Ornith-1.0-35B-MTP2-MTPLX \
  --output ~/Models/Ornith-1.0-35B-UD-Q8-MTP2-MTPLX

Inspect before serving:

mtplx inspect-model ~/Models/Ornith-1.0-35B-UD-Q8-MTP2-MTPLX \
  --require-mtp --json

The accepted test contract used greedy AR and draft sampling, MTP depth 2, Q8/group-64 affine MTP and draft LM heads, unquantized KV cache, capture_commit, linear-gdn-from-conv-tape, margin 0.9, pre-D2 margin 2.0, and minimum speculative depth 0.

Experimental combination: UD-Q4

To reproduce the faster but lower-quality cross-precision experiment, download Brooooooklyn/Ornith-1.0-35B-UD-Q4_K_XL-mlx at revision 1fbf6f036940ce58694904c1035d84a32fa54994, then run the assembler with --profile q4-experimental.

The assembler prints and records a NO-GO warning and places a public-release blocker in the assembled Q4 runtime contract, so strict MTPLX inspection will not promote it as verified. This Q8 MTP head was optimized against the Q8 trunk distribution; it is not a Q4-specific MTP head.

Important claim boundaries

• MTP speculative decoding does not make the trunk intrinsically smarter.
• The Q8 result means no observed regression under the named test contract; it is not a bit-for-bit equivalence claim.
• The Q4 result is an exploratory cross-precision measurement, not an accepted release configuration.
• Throughput is host-, runtime-, prompt-, and thermal-state dependent.
• Q4 KV-cache quantization was not used in these measurements.
• MTP3 was tested separately and intentionally not packaged.
• This is a sidecar release. Standard mlx-lm alone will not automatically use these speculative weights; use a compatible MTPLX runtime.

Provenance and licensing

• Ornith-1.0-35B: MIT, revision 5df2ed3f675c7beaa490328cc70bb573b65fb660.
• UD-Q8 trunk: MIT, revision 1aa23d99400067a00d72ad0f47bb0c9ef6126480.
• UD-Q4 trunk: MIT, revision 1fbf6f036940ce58694904c1035d84a32fa54994.
• Initial MTP graft: wang-yang/Ornith-1.0-35B-MTPLX, MIT model repository, revision 5ad47122e7d28827bc3430e8216c248fd32aae13.
• The graft traces to Qwen/Qwen3.5-35B-A3B, Apache-2.0, revision 59d61f3ce65a6d9863b86d2e96597125219dc754.
• This release materially changes the initial graft through Ornith-Q8 teacher capture, MTP2 distillation, LoRA merge, and affine Q8 deployment export.

See NOTICE and licenses/ for retained notices. The release is distributed under Apache-2.0 to preserve the upstream Qwen obligations clearly.

中文摘要

建議使用 UD-Q8 主幹 + 本 Q8 MTP2 sidecar。同條件三輪測試約為 1.361-1.367x，完整 HumanEval 為 AR 119/164、MTP2 125/164，沒有 觀察到 AR 通過但 MTP2 失敗的題目。主要成果是把整體接受率從 71.66% 提升至 86.29%，D2 從 56.39% 提升至 84.88%；同一組 Hermes 驗證中， 原始借用 MTP 新增的驗證退步由 1 項降為 0 項。

UD-Q4 組合雖然速度較快，但 32 題 HumanEval 明顯低於 Q8，且接上現有 Q8 MTP2 後仍有一題回歸，因此只建議作研究或低風險快速模式，不建議 取代 Q8 成為主力。