| --- |
| license: apache-2.0 |
| tags: |
| - merge |
| - not-an-original-model |
| --- |
| |
| # Mini-Merge-35B-A3B |
|
|
| This is a model **merge**, not an original training run. It's a plain element-wise |
| weighted average of two existing open-weight checkpoints (a ~40-line safetensors script; |
| no mergekit, since its catalog doesn't know this architecture). No pretraining, no |
| post-training of our own. The exact recipe ships as `merge.yaml` next to the weights. |
|
|
| ## Ingredients |
| - `nex-agi/Nex-N2-mini` |
| - `Qwen/Qwen3.6-35B-A3B` |
|
|
| Merge variant: `mini-b0.2`. Full pipeline and commands: |
| https://github.com/PixelML/merge-theater |
|
|
| ## Benchmarks (same lm-eval tasks, parents vs merge) |
|
|
| | task | mini-b0.2 | Nex-N2-mini | Qwen3.6-35B-A3B | |
| |---|---|---|---| |
| | arc_challenge | 62.4 | **62.5** | 54.6 | |
| | gsm8k | **90.1** | 89.2 | 29.3 | |
| | hellaswag | **84.0** | 83.5 | 83.5 | |
| | humaneval | 66.5 | **72.0** | 61.0 | |
| | mbpp | 63.8 | 54.6 | **66.2** | |
| | winogrande | 73.3 | 73.6 | **73.7** | |
| |
| Aggregate (mean over shared tasks): |
| - mini-b0.2: 73.36 |
| - Nex-N2-mini: 72.59 |
| - Qwen3.6-35B-A3B: 61.37 |
| |
| A merge topping a parent by a point or two mostly reflects eval-format luck, not new |
| capability, so read it that way. The numbers reproduce with the command below. |
| |
| ## Reproduce |
| ```bash |
| modal run modal_app.py::sweep --weights-b 0.2,0.3,0.4 |
| ``` |
| |
| ## License |
| Both parents (`nex-agi/Nex-N2-mini` and `Qwen/Qwen3.6-35B-A3B`) are Apache-2.0, so this merge is released under Apache-2.0. |
| The merge used a small original script, not mergekit, so no LGPL applies. This card must |
| travel with the weights. |
| |
| ## Credit |
| Built by Claude Code (Anthropic's agent) running autonomously: the merge, the eval |
| pipeline, the ratio sweep, and this card. |
| |
