Qwen3.5-4B-M8

3-model DARE-TIES variant of Qwen3.5-4B, distilling Claude Opus 4.6 reasoning style from two community fine-tunes onto the official base.

The M8 recipe was the winner of a 13-variant ablation study (M1–M8 + sub-variants) measuring HumanEval and MBPP across different importance signals (Fisher, LRP, hybrid), gating mechanisms (PR #682 turbo head, mergetime detectors), and merge methods (DARE-TIES, OmniMerge v2, mergekit ex-LRP).

M8 is the simplest production-recommended config: it matches the best balanced merges on HE+MBPP without requiring expensive Fisher / LRP precomputation. It uses only magnitude-based importance combined with all gating mechanisms.

Source	Role
`Qwen/Qwen3.5-4B`	base
`Jackrong/Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2`	reasoning teacher (weight 0.55)
`Crownelius/Crow-4B-Opus-4.6-Distill-Heretic_Qwen3.5`	uncensored teacher (weight 0.45)

M8 Recipe

method:        omnimerge_v2
features:      obim, darex, emr
density:       0.53
weights:       0.55 / 0.45 (jackrong / crow)
darex-q:       0.75
importance:    magnitude (no Fisher, no LRP)
--pr682-turbo: critical-layer protect (norm/embed/head/bias forced density=1.0)
               + skip on shape mismatch
--m7-detector --m7-layer-aware:
               mergetime detector with layer-aware norm clamping
               (τ_norm=0.02 for attn, 0.10 for MLP)
               + sign-flip gate + consensus bonus
seed:          42

Run via:

python scripts/dare_ties_merge.py \
    --base Qwen/Qwen3.5-4B \
    --source Jackrong/Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2 \
    --source Crownelius/Crow-4B-Opus-4.6-Distill-Heretic_Qwen3.5 \
    --output ./m8 \
    --method omnimerge_v2 --v2-features obim,darex,emr \
    --weights 0.55,0.45 --density 0.53 --darex-q 0.75 \
    --pr682-turbo \
    --m7-detector --m7-layer-aware \
    --skip-patterns "model.visual,mtp.layers" \
    --seed 42 --device cuda

Evaluation

All evaluations: lm-evaluation-harness, llama.cpp llama-server with Q6_K quantization, --reasoning-format deepseek --reasoning-budget 8192, /v1/completions raw endpoint, greedy decoding (temperature=0.0, top_p=1.0), max_gen_toks=2048, --parallel 2, fence-strip rescore.

Headline numbers (Q6_K)

Model	HumanEval	MBPP
`Qwen/Qwen3.5-4B` (base)	48.78	45.20
`Jackrong/...Distilled-v2` (source 1)	56.10	47.40
`Crownelius/Crow-4B-...` (source 2)	53.05	50.40
`Qwen3.5-4B-M8` (this model)	54.27	51.40

M8 retains ≈97% of source 1's HE and beats both sources on MBPP.

Full ablation matrix

13 variants tested. All use same base + 2 sources, same eval pipeline.

Variant	HE	MBPP	Recipe
M1	51.22	47.00	DARE-TIES baseline
M2	52.44	49.40	OmniMerge v2 (d=0.53, w=0.55/0.45)
M2-turbo	53.66	49.80	M2 + critical-layer protect (`--pr682-turbo`)
M3-redo	57.93	48.80	M2 + Fisher importance
M4-v2 (mergekit)	51.83	51.80	mergekit ex-LRP, real LRP signal
M5 (OMv2_LRP)	53.05	51.40	OMv2 + LRP
M6-hybrid (mergekit)	51.83	51.80	mergekit ex-LRP + Fisher@attn / LRP@mlp
M4v2-dare	54.88	51.80	OMv2 + OBIM + LRP
M6-dare	53.66	51.00	OMv2 + OBIM + hybrid signal
M7	59.76	44.20	M2-turbo + mergetime detector (uniform clamp τ=0.02)
M7v2	53.66	51.20	M7 + layer-aware clamp (MLP τ=0.10)
M7v3	54.88	49.00	M7 mid-strength uniform (τ=0.05, flip=50)
M8 (this)	54.27	51.40	M7v2 + `--pr682-turbo` + magnitude (no Fisher/LRP)

Why M8

Importance signals (Fisher / LRP / hybrid) contribute ≈zero on this base/source pair at density 0.53. M4-v2 and M6-hybrid produced byte-identical merged tensors at d=0.7 — top-70% importance masks coincide regardless of signal source.
Layer-aware mergetime gating is what moves MBPP from 49 → 51. M7v2 shows it; M8 confirms the gain holds without any importance signal.
Critical-layer protection (--pr682-turbo) keeps norm / embed / head / bias at density 1.0, preventing per-token / per-vocab structure from being shredded by sparsification.
M7's HE 59.76 / MBPP 44.20 is not the production pick: the uniform τ=0.02 clamp damages MBPP. Layer-aware clamping (τ_attn=0.02, τ_mlp=0.10, M7v2 + M8) recovers MBPP at small HE cost.

So M8 is the stable recipe: matches M4v2-dare on MBPP (51.40 vs 51.80) and is within 0.6 pp on HE (54.27 vs 54.88), while requiring no external signal precomputation.

Files

model-*.safetensors — BF16 merged weights
config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja, merges.txt, vocab.json — Qwen3.5 tokenizer and config (unchanged from base)

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "ManniX-ITA/Qwen3.5-4B-M8",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained("ManniX-ITA/Qwen3.5-4B-M8")

msgs = [{"role": "user", "content": "Write a Python function that returns the n-th Fibonacci number."}]
inputs = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True).to(model.device)
out = model.generate(inputs, max_new_tokens=1024, do_sample=False)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

llama.cpp (Q6_K)

llama-server -m Qwen3.5-4B-M8-Q6_K.gguf \
    --port 8099 -c 32768 -ngl 99 --no-warmup \
    --reasoning-format deepseek --reasoning-budget 8192 \
    --parallel 2 --cache-type-k q8_0 --cache-type-v q8_0

Upstream contributions

Two patches submitted to mergekit PR #682 during this study:

Comment #4361070820 — mergekit/merge_methods/lrp.py silently fell back to magnitude when given .safetensors LRP score files (torch.load raises on safetensors). Added a safetensors loader branch.
Comment #4361416737 — mergekit/architecture/auto.py _wi() checked only layer 0 for optionality, breaking hybrid architectures (Qwen3.5 alternating self_attn / linear_attn). Fixed to check all layer instantiations.

These two patches together unblock mergekit-yaml end-to-end on Qwen3.5 hybrid models with the LRP merge method.