SeaWolf-AI commited on
Commit
91e6efb
ยท
verified ยท
1 Parent(s): 6c73c4a

Card in English: lead with Chimera value/significance; remove base_model (pointed to private repo -> 401 for visitors)

Browse files
Files changed (1) hide show
  1. README.md +38 -35
README.md CHANGED
@@ -1,6 +1,5 @@
1
  ---
2
  license: gemma
3
- base_model: FINAL-Bench/Darwin-V9-Chimera-4B-SFT
4
  tags:
5
  - chimera
6
  - darwin
@@ -17,49 +16,53 @@ library_name: transformers
17
 
18
  # Darwin-4B-Chimera
19
 
20
- **๋น„๋“œ๋ž˜ํ”„ํŠธ(VIDRAFT) Chimera ๊ธฐ์ˆ ๋กœ ๋งŒ๋“  4B ํ•œ๊ตญ์–ด ์ถ”๋ก  ๋ชจ๋ธ.**
21
 
22
- ์ž‘์€ ๋ชจ๋ธ์ด ์ž๊ธฐ ํฌ๊ธฐ๋ฅผ ๋„˜์–ด์„œ๊ฒŒ ๋งŒ๋“œ๋Š” ๊ฒƒ์ด ๋ชฉํ‘œ์ž…๋‹ˆ๋‹ค. ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ๋Š˜๋ฆฌ๋Š” ๋Œ€์‹ , ์„œ๋กœ ๋‹ค๋ฅธ ๊ฐ•์ ์„ **ํ•ฉ์ฒด**ํ•˜๊ณ  ์Šค์Šค๋กœ **์ •์ œ**ํ•˜๊ฒŒ ํ–ˆ์Šต๋‹ˆ๋‹ค.
23
 
24
  ---
25
 
26
- ## Chimera๋ž€
27
 
28
- **Chimera**๋Š” ์„œ๋กœ ๋‹ค๋ฅธ ๊ณ„์—ดยท๊ฐ•์ ์„ ๊ฐ€์ง„ ๋ชจ๋ธ์˜ ๊ตฌ์„ฑ์š”์†Œ๋ฅผ **ํ•˜๋‚˜์˜ ๋ชจ๋ธ๋กœ ํ•ฉ์ฒด**์‹œํ‚ค๋Š” ๋น„๋“œ๋ž˜ํ”„ํŠธ์˜ ๋ชจ๋ธ ์ง„ํ™” ๊ธฐ์ˆ ์ž…๋‹ˆ๋‹ค.
29
 
30
- - ๊ฐ ๋ถ€๋ชจ์˜ ์ง€์‹์„ **์†์ƒ ์—†์ด ๋ณด์กด**ํ•œ ์ฑ„ ํ•ฉ์นฉ๋‹ˆ๋‹ค.
31
- - ๋‹จ์ผ ๋ถ€๋ชจ๋กœ๋Š” ๋„๋‹ฌํ•  ์ˆ˜ ์—†๋Š” ์กฐํ•ฉ์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค โ€” ๊ฒฐ๊ณผ๋ฌผ์€ ์–ด๋А ํ•œ ๋ถ€๋ชจ๋กœ๋„ ํ™˜์›๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.
32
- - **์ถ”๊ฐ€ ์‚ฌ์ „ํ•™์Šต ์—†์ด** ์„ธ๋Œ€๋ฅผ ์ด์–ด๊ฐ‘๋‹ˆ๋‹ค.
33
 
34
- ์ด ๋ชจ๋ธ์€ Chimera 4B ๊ณ„๋ณด์˜ ๊ฒ€์ฆ๋œ ๊ฐœ์ฒด์— ๋น„๋“œ๋ž˜ํ”„ํŠธ ์ž์ฒด **RFT(๊ฐ•ํ™” ๋ฏธ์„ธ์กฐ์ •)** ํŒŒ์ดํ”„๋ผ์ธ์„ ์ ์šฉํ•ด, ํ•œ๊ตญ์–ด ์ง€์‹ ์ถ”๋ก ์„ ๋Œ์–ด์˜ฌ๋ฆฐ ๊ฒฐ๊ณผ๋ฌผ์ž…๋‹ˆ๋‹ค.
 
35
 
36
- ---
37
-
38
- ## ์„ฑ๋Šฅ
39
 
40
- **KMMLU 6๊ณผ๋ชฉ ยท 240๋ฌธํ•ญ held-out ยท greedy ๋””์ฝ”๋”ฉ**
 
41
 
42
- | ๋ชจ๋ธ | KMMLU | vs ๊ธฐ์ค€์„  |
43
- |---|---|---|
44
- | ๊ธฐ์ค€์„  (Chimera 4B) | 42.9% | โ€” |
45
- | **Darwin-4B-Chimera (๋ณธ ๋ชจ๋ธ)** | **48.3%** | **+5.4%p** |
46
 
47
- ๊ฐ™์€ ๊ฐ€์ค‘์น˜ยท๊ฐ™์€ ํ•˜๋„ค์Šค์—์„œ ์ •์ œ ๋‹จ๊ณ„๋งŒ ๋‹ฌ๋ฆฌํ•ด ์ธก์ •ํ–ˆ์Šต๋‹ˆ๋‹ค. 6๊ณผ๋ชฉ ์ค‘ ๋‹ค์ˆ˜์—์„œ ์ƒ์Šนํ–ˆ์Šต๋‹ˆ๋‹ค.
 
48
 
49
  ---
50
 
51
- ## ์ •์งํ•œ ๋ฒ”์œ„ (Honest scope)
 
 
 
 
 
 
 
 
 
52
 
53
- ์ˆ˜์น˜๋ฅผ ๋ถ€ํ’€๋ฆฌ์ง€ ์•Š๊ธฐ ์œ„ํ•ด ํ•œ๊ณ„๋ฅผ ๋จผ์ € ๋ฐํž™๋‹ˆ๋‹ค.
54
 
55
- - **ํ‰๊ฐ€ ์…‹์ด ์ž‘์Šต๋‹ˆ๋‹ค.** 240๋ฌธํ•ญ ๊ธฐ์ค€ ํ‘œ์ค€์˜ค์ฐจ ์•ฝ ยฑ1.4%p์ž…๋‹ˆ๋‹ค. ๋” ํฐ ์…‹์—์„œ์˜ ์žฌํ™•์ธ์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.
56
- - **KMMLU 6๊ณผ๋ชฉ ํ•œ์ •** ์ˆ˜์น˜์ด๋ฉฐ, ์ „ ๋„๋ฉ”์ธ ์ผ๋ฐ˜ํ™”๋ฅผ ๋œปํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.
57
- - **4B ๊ทœ๋ชจ** ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ํ”Œ๋ž˜๊ทธ์‹ญ(Darwin-398B ๋“ฑ)๊ณผ ์ง์ ‘ ๋น„๊ต ๋Œ€์ƒ์ด ์•„๋‹™๋‹ˆ๋‹ค.
58
- - **greedy ๋””์ฝ”๋”ฉ** ๊ธฐ์ค€์ž…๋‹ˆ๋‹ค. ๋””์ฝ”๋”ฉยทํ”„๋กฌํ”„ํŠธ๊ฐ€ ๋ฐ”๋€Œ๋ฉด ๊ฒฐ๊ณผ๊ฐ€ ๋‹ฌ๋ผ์งˆ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
59
 
60
  ---
61
 
62
- ## ์‚ฌ์šฉ
63
 
64
  ```python
65
  from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -76,22 +79,22 @@ print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
76
 
77
  ---
78
 
79
- ## ๋ฐฉ๋ฒ•๋ก  ๊ณต๊ฐœ ๋ฒ”์œ„
80
 
81
- Chimera ํ•ฉ์ฒด ๊ตฌ์„ฑ๊ณผ RFT ์ •์ œ์˜ ๋‚ด๋ถ€ ์„ค๊ณ„(๋ฐ์ดํ„ฐ ์„ ๋ณ„ ๊ธฐ์ค€ยทํ•™์Šต ์„ค์ •ยท๋ผ์šฐํŒ… ๋ฐฉ์‹ ๋“ฑ)๋Š” **๋น„๋“œ๋ž˜ํ”„ํŠธ ์˜์—…๊ธฐ๋ฐ€**์ž…๋‹ˆ๋‹ค.
82
 
83
- > **๊ฒฐ๊ณผ๋Š” ๊ณต๊ฐœํ•˜๊ณ , ๋ ˆ์‹œํ”ผ๋Š” ๊ณต๊ฐœํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.**
84
 
85
- ๋ชจ๋ธ ๊ฐ€์ค‘์น˜์™€ ์ธก์ • ์ˆ˜์น˜๋Š” ๊ณต๊ฐœ๋˜์–ด ์žˆ์œผ๋ฏ€๋กœ, ๋ˆ„๊ตฌ๋‚˜ ์œ„ ๊ฒฐ๊ณผ๋ฅผ ์ง์ ‘ ์žฌํ˜„ยท๊ฒ€์ฆํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
86
 
87
  ---
88
 
89
- ## ๊ด€๋ จ ๋งํฌ
90
 
91
- - **Darwin ํŒจ๋ฐ€๋ฆฌ** โ€” https://huggingface.co/collections/FINAL-Bench/darwin-family
92
- - **์ง„ํ™”์  ๋ณ‘ํ•ฉ ๋ฐฉ๋ฒ•๋ก  ๋…ผ๋ฌธ** โ€” arXiv [2605.14386](https://arxiv.org/abs/2605.14386)
93
- - **๋น„๋“œ๋ž˜ํ”„ํŠธ** โ€” https://vidraft.net
94
 
95
- ## ๋ผ์ด์„ ์Šค
96
 
97
- `gemma` โ€” ํ•ฉ์ฒด ๊ณ„๋ณด์— Gemma ํŒŒ์ƒ ๊ตฌ์„ฑ์š”์†Œ๊ฐ€ ํฌํ•จ๋˜์–ด Gemma ๋ผ์ด์„ ์Šค๋ฅผ ๋”ฐ๋ฆ…๋‹ˆ๋‹ค. ์‚ฌ์šฉ ์ „ [Gemma Terms of Use](https://ai.google.dev/gemma/terms)๋ฅผ ํ™•์ธํ•˜์‹ญ์‹œ์˜ค.
 
1
  ---
2
  license: gemma
 
3
  tags:
4
  - chimera
5
  - darwin
 
16
 
17
  # Darwin-4B-Chimera
18
 
19
+ **A 4B Korean-reasoning model built with VIDRAFT's Chimera technology.**
20
 
21
+ Most models get better by getting bigger. This one got better by **combining** โ€” and then refining itself.
22
 
23
  ---
24
 
25
+ ## Why Chimera matters
26
 
27
+ Merging models usually means losing something. Average two networks and you get a compromise: each parent's edge is blunted, and the child is worse than both at what each did best. That is why "model merging" has a reputation as a cheap trick.
28
 
29
+ Chimera is built on the opposite premise.
 
 
30
 
31
+ ### 1. The parents survive intact
32
+ Chimera fuses components from models of **different families and different strengths** while preserving what each parent knew. Knowledge is carried over, not averaged away. What you merge in is what you keep.
33
 
34
+ ### 2. The result cannot be reduced to any single parent
35
+ A Chimera individual is not "model A with a bit of B." It sits at a point in model space that **no parent reaches alone**, and it cannot be recovered from any one of them. That non-reducibility is the entire point โ€” it is what makes a lineage an asset rather than a re-skin.
 
36
 
37
+ ### 3. No additional pretraining
38
+ A new generation is a **fusion plus an evaluation**, not a training run measured in GPU-months. Iteration collapses from months to days โ€” which means the space of viable combinations actually gets explored instead of theorized about.
39
 
40
+ ### 4. It compounds
41
+ Chimera individuals become parents. Strengths accumulate across generations: parent โ†’ child โ†’ grandchild. Capability is **grown**, not purchased.
 
 
42
 
43
+ ### 5. It answers the real constraint
44
+ Frontier capability is gated by capital: tens of thousands of GPUs, months of training, hundreds of millions of dollars. Chimera attacks that gate directly โ€” **structural evolution instead of brute-force scale**. It is how a small team competes on method rather than on budget.
45
 
46
  ---
47
 
48
+ ## What that buys you at 4B
49
+
50
+ This model is a late-generation Chimera individual, refined further by VIDRAFT's own reinforcement fine-tuning pipeline.
51
+
52
+ **KMMLU ยท 6 subjects ยท 240 held-out items ยท greedy decoding**
53
+
54
+ | Model | KMMLU | ฮ” |
55
+ |---|---|---|
56
+ | Chimera 4B baseline | 42.9% | โ€” |
57
+ | **Darwin-4B-Chimera** | **48.3%** | **+5.4pp** |
58
 
59
+ **+5.4pp on Korean knowledge reasoning with zero parameter growth** โ€” 4.02B before, 4.02B after. The gain came from what the model learned from itself, not from more weights. Same architecture, same size, same inference cost: strictly better.
60
 
61
+ That matters because 4B is the size that actually ships. It runs on a single consumer GPU, on-premise, inside an air-gapped network โ€” the places where frontier APIs cannot go.
 
 
 
62
 
63
  ---
64
 
65
+ ## Usage
66
 
67
  ```python
68
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
79
 
80
  ---
81
 
82
+ ## What is open, and what is not
83
 
84
+ The **weights are open** and the **numbers are stated with the exact evaluation setup**, so anyone can reproduce the result above rather than take our word for it.
85
 
86
+ The internal design of Chimera fusion and of the refinement pipeline โ€” component selection, data selection criteria, training configuration, routing โ€” is **VIDRAFT proprietary**.
87
 
88
+ > **Results are open. The recipe is not.**
89
 
90
  ---
91
 
92
+ ## Links
93
 
94
+ - **Darwin family** โ€” https://huggingface.co/collections/FINAL-Bench/darwin-family
95
+ - **Method paper (evolutionary merging)** โ€” arXiv [2605.14386](https://arxiv.org/abs/2605.14386)
96
+ - **VIDRAFT** โ€” https://vidraft.net
97
 
98
+ ## License
99
 
100
+ `gemma` โ€” the Chimera lineage includes Gemma-derived components, so the Gemma license applies. Review the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) before use.