Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -7,17 +7,17 @@ tags:
|
|
| 7 |
|
| 8 |
# Mini-Merge-35B-A3B
|
| 9 |
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
alongside the weights, so the "how" is not a mystery.
|
| 15 |
|
| 16 |
## Ingredients
|
| 17 |
- `nex-agi/Nex-N2-mini`
|
| 18 |
- `Qwen/Qwen3.6-35B-A3B`
|
| 19 |
|
| 20 |
-
Merge variant: `mini-b0.2`. Full pipeline
|
|
|
|
| 21 |
|
| 22 |
## Benchmarks (same lm-eval tasks, parents vs merge)
|
| 23 |
|
|
@@ -36,14 +36,18 @@ Aggregate (mean over shared tasks):
|
|
| 36 |
- Qwen3.6-35B-A3B: 61.37
|
| 37 |
|
| 38 |
A merge topping a parent by a point or two mostly reflects eval-format luck, not new
|
| 39 |
-
capability
|
| 40 |
|
| 41 |
## Reproduce
|
| 42 |
```bash
|
| 43 |
-
modal run modal_app.py::sweep
|
| 44 |
```
|
| 45 |
|
| 46 |
## License
|
| 47 |
-
Both parents (`nex-agi/Nex-N2-mini` and `Qwen/Qwen3.6-35B-A3B`) are Apache-2.0, so this merge is released under
|
| 48 |
-
|
| 49 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
# Mini-Merge-35B-A3B
|
| 9 |
|
| 10 |
+
This is a model **merge**, not an original training run. It's a plain element-wise
|
| 11 |
+
weighted average of two existing open-weight checkpoints (a ~40-line safetensors script;
|
| 12 |
+
no mergekit, since its catalog doesn't know this architecture). No pretraining, no
|
| 13 |
+
post-training of our own. The exact recipe ships as `merge.yaml` next to the weights.
|
|
|
|
| 14 |
|
| 15 |
## Ingredients
|
| 16 |
- `nex-agi/Nex-N2-mini`
|
| 17 |
- `Qwen/Qwen3.6-35B-A3B`
|
| 18 |
|
| 19 |
+
Merge variant: `mini-b0.2`. Full pipeline and commands:
|
| 20 |
+
https://github.com/PixelML/merge-theater
|
| 21 |
|
| 22 |
## Benchmarks (same lm-eval tasks, parents vs merge)
|
| 23 |
|
|
|
|
| 36 |
- Qwen3.6-35B-A3B: 61.37
|
| 37 |
|
| 38 |
A merge topping a parent by a point or two mostly reflects eval-format luck, not new
|
| 39 |
+
capability, so read it that way. The numbers reproduce with the command below.
|
| 40 |
|
| 41 |
## Reproduce
|
| 42 |
```bash
|
| 43 |
+
modal run modal_app.py::sweep --weights-b 0.2,0.3,0.4
|
| 44 |
```
|
| 45 |
|
| 46 |
## License
|
| 47 |
+
Both parents (`nex-agi/Nex-N2-mini` and `Qwen/Qwen3.6-35B-A3B`) are Apache-2.0, so this merge is released under Apache-2.0.
|
| 48 |
+
The merge used a small original script, not mergekit, so no LGPL applies. This card must
|
| 49 |
+
travel with the weights.
|
| 50 |
+
|
| 51 |
+
## Credit
|
| 52 |
+
Built by Claude Code (Anthropic's agent) running autonomously: the merge, the eval
|
| 53 |
+
pipeline, the ratio sweep, and this card.
|