Darwin-Chimera-4B-MoE-2E-PoC (private)

2-expert MoE upcycling PoC: Qwen3.5-4B (expert0) + Reasoning-Fix (expert1), fused as one Qwen3.5-MoE with a trained router (frozen experts).

What it proves

  • Two parents' FFNs preserved 100% as frozen experts (trace intact, single-parent non-reducible).
  • Router trained (163K params) โ†’ both experts genuinely used (~49-56%), coherent generation.
  • Honest result: neutral vs base on accuracy (parents too similar) โ€” this is a mechanism/originality PoC, not a performance model. Real gains come from divergent parents + RFT-edge (see Darwin-V9-Chimera-4B-RFT).

Build/verify scripts: chimera_build.py / chimera_router_train.py / chimera_verify.py.

Downloads last month
16
Safetensors
Model size
9B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support