seanphan commited on
Commit
59b5dba
·
verified ·
1 Parent(s): b6cc033

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +15 -11
README.md CHANGED
@@ -7,17 +7,17 @@ tags:
7
 
8
  # Mini-Merge-35B-A3B
9
 
10
- **This is a model *merge*, not an original training run.** It was produced by a plain
11
- element-wise weighted average of two existing open-weight checkpoints -- a ~40-line
12
- safetensors script, no mergekit (its catalog doesn't know this architecture). Zero
13
- pretraining, zero post-training of our own. The exact recipe ships as `merge.yaml`
14
- alongside the weights, so the "how" is not a mystery.
15
 
16
  ## Ingredients
17
  - `nex-agi/Nex-N2-mini`
18
  - `Qwen/Qwen3.6-35B-A3B`
19
 
20
- Merge variant: `mini-b0.2`. Full pipeline: the merge-theater repo.
 
21
 
22
  ## Benchmarks (same lm-eval tasks, parents vs merge)
23
 
@@ -36,14 +36,18 @@ Aggregate (mean over shared tasks):
36
  - Qwen3.6-35B-A3B: 61.37
37
 
38
  A merge topping a parent by a point or two mostly reflects eval-format luck, not new
39
- capability. Read accordingly; the numbers are reproducible with the command below.
40
 
41
  ## Reproduce
42
  ```bash
43
- modal run modal_app.py::sweep # or ::rematch for the 397B build
44
  ```
45
 
46
  ## License
47
- Both parents (`nex-agi/Nex-N2-mini` and `Qwen/Qwen3.6-35B-A3B`) are Apache-2.0, so this merge is released under
48
- **Apache-2.0**. The merge used a small original script, not mergekit, so no LGPL applies.
49
- This card must travel with the weights.
 
 
 
 
 
7
 
8
  # Mini-Merge-35B-A3B
9
 
10
+ This is a model **merge**, not an original training run. It's a plain element-wise
11
+ weighted average of two existing open-weight checkpoints (a ~40-line safetensors script;
12
+ no mergekit, since its catalog doesn't know this architecture). No pretraining, no
13
+ post-training of our own. The exact recipe ships as `merge.yaml` next to the weights.
 
14
 
15
  ## Ingredients
16
  - `nex-agi/Nex-N2-mini`
17
  - `Qwen/Qwen3.6-35B-A3B`
18
 
19
+ Merge variant: `mini-b0.2`. Full pipeline and commands:
20
+ https://github.com/PixelML/merge-theater
21
 
22
  ## Benchmarks (same lm-eval tasks, parents vs merge)
23
 
 
36
  - Qwen3.6-35B-A3B: 61.37
37
 
38
  A merge topping a parent by a point or two mostly reflects eval-format luck, not new
39
+ capability, so read it that way. The numbers reproduce with the command below.
40
 
41
  ## Reproduce
42
  ```bash
43
+ modal run modal_app.py::sweep --weights-b 0.2,0.3,0.4
44
  ```
45
 
46
  ## License
47
+ Both parents (`nex-agi/Nex-N2-mini` and `Qwen/Qwen3.6-35B-A3B`) are Apache-2.0, so this merge is released under Apache-2.0.
48
+ The merge used a small original script, not mergekit, so no LGPL applies. This card must
49
+ travel with the weights.
50
+
51
+ ## Credit
52
+ Built by Claude Code (Anthropic's agent) running autonomously: the merge, the eval
53
+ pipeline, the ratio sweep, and this card.