Darwin-Chimera-4B-MoE-2E-PoC (private)
2-expert MoE upcycling PoC: Qwen3.5-4B (expert0) + Reasoning-Fix (expert1), fused as one Qwen3.5-MoE with a trained router (frozen experts).
What it proves
- Two parents' FFNs preserved 100% as frozen experts (trace intact, single-parent non-reducible).
- Router trained (163K params) โ both experts genuinely used (~49-56%), coherent generation.
- Honest result: neutral vs base on accuracy (parents too similar) โ this is a mechanism/originality PoC, not a performance model. Real gains come from divergent parents + RFT-edge (see Darwin-V9-Chimera-4B-RFT).
Build/verify scripts: chimera_build.py / chimera_router_train.py / chimera_verify.py.
- Downloads last month
- 16