Warecube-KO-27B / README.md
Warecube's picture
fix lineage: Qwen-3.5-27B (great-gf) / Darwin-27B-Opus (gf)
a781786 verified
---
license: apache-2.0
language:
- ko
- en
library_name: transformers
tags:
- korean
- reasoning
- darwin
- evolutionary-merge
base_model:
- FINAL-Bench/Darwin-27B-Opus
- NewenAI/QuettaLLMs-27B-Koreasoner-V3
---
# Warecube-KO-27B
ν•œκ΅­μ–΄ reasoning λͺ¨λΈ β€” Darwin 진화적 λ¨Έμ§€ 기반.
---
## 🧬 Darwin μ§„ν™” 컨셉
λ³Έ λͺ¨λΈμ€ **Darwin V7 진화적 λͺ¨λΈ λ¨Έμ§€(Evolutionary Model Merge)**
νŒ¨λŸ¬λ‹€μž„μœΌλ‘œ μ œμž‘λ˜μ—ˆμŠ΅λ‹ˆλ‹€.
```
μžμ—° μ§„ν™” Darwin λ¨Έμ§€
───────── ───────────
μœ μ „μž ꡐ차 (crossover) β†’ κ°€μ€‘μΉ˜ λͺ¨λ“ˆλ³„ λΉ„μœ¨ κ²°ν•©
μžμ—° 선택 (selection) β†’ 적합도 평가 ν›„ 졜적 후손 선별
μ„ΈλŒ€ μ§„ν™” (generations) β†’ λ‹€μ„ΈλŒ€ λ¨Έμ§€Β·μ •μ œ 반볡
적자 생쑴 β†’ K-AI 도메인 우수 μžμ†λ§Œ 보쑴
```
λΆ€λͺ¨μ˜ λŠ₯λ ₯이 μžμ‹ λͺ¨λΈλ‘œ **μœ μ „μ μœΌλ‘œ κ³„μŠΉ**되며,
μ„ΈλŒ€λ₯Ό 거쳐 ν•œκ΅­μ–΄Β·μΆ”λ‘ Β·λ¬Έν™” μ§€λŠ₯이 μ§„ν™”ν•©λ‹ˆλ‹€.
---
## πŸ›οΈ κ°€λ¬Έ 계보
```
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ 증쑰뢀 (Great-Grandfather) β”‚
β”‚ Qwen-3.5-27B β”‚
β”‚ - λ©€ν‹°λͺ¨λ‹¬ 28B 베이슀 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚
β–Ό Darwin V7 μ§„ν™” λ¨Έμ§€
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ μ‘°λΆ€ (Grandfather) β”‚
β”‚ FINAL-Bench/Darwin-27B-Opus β”‚
β”‚ - Darwin V7 μ§„ν™”μ˜ 정점 β”‚
β”‚ - GPQA 88.4% reasoning β”‚
β”‚ - <think> 트레이슀 νŒ¨ν„΄ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚
β–Ό ν•œκ΅­μ–΄ νŠΉν™” μ§„ν™”
╔══════════════════════════════════════════╗
β•‘ μ•„λΉ  (Father) β•‘
β•‘ Darwin family Korean 직계 β•‘
β•‘ β•‘
β•‘ - Darwin-27B-Opus의 ν•œκ΅­μ–΄ νŠΉν™” 후손 β•‘
β•‘ - reasoning DNA 보쑴 β•‘
β•‘ - <think> νŒ¨ν„΄ μœ μ§€ β•‘
β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
β”‚
Γ—Γ— λ‹€μœˆ ꡐ배 Γ—Γ—
β”‚
╔══════════════════════════════════════════╗
β•‘ μ—„λ§ˆ (Mother) β•‘
β•‘ NewenAI/QuettaLLMs-27B-Koreasoner-V3 β•‘
β•‘ β•‘
β•‘ - ν•œκ΅­μ–΄ SOTA λͺ¨λΈ β•‘
β•‘ - K-AI Leaderboard 1μœ„ (avg 0.560) β•‘
β•‘ - ν•œκ΅­μ–΄ 도메인 SFT μ •μ œ β•‘
β•‘ - Apache 2.0 β•‘
β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
β”‚
β–Ό Darwin 진화적 λ¨Έμ§€ + ν•œκ΅­μ–΄ μ •μ œ
╔══════════════════════════════════════════╗
β•‘ μžμ‹ (Child) β€” λ³Έ λͺ¨λΈ β•‘
β•‘ Warecube/Warecube-KO-27B β•‘
β•‘ β•‘
β•‘ ✦ μ•„λΉ μ˜ reasoning DNA κ³„μŠΉ β•‘
β•‘ ✦ μ—„λ§ˆμ˜ ν•œκ΅­μ–΄ ν‘œν˜„Β·μ§€μ‹ κ³„μŠΉ β•‘
β•‘ ✦ <think> μΆ”λ‘  트레이슀 보쑴 β•‘
β•‘ ✦ K-AI 도메인 적합도 μ§„ν™” β•‘
β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
```
---
## πŸŽ“ μ§„ν™” 단계
| Stage | 개랡 |
|:---|:---|
| **1. ꡐ배 (Crossover)** | μΉœκ°€Β·μ™Έκ°€ κ°€μ€‘μΉ˜λ₯Ό λͺ¨λ“ˆλ³„ λΉ„μœ¨λ‘œ μ§„ν™” λ¨Έμ§€ |
| **2. 선택 (Selection)** | ν•œκ΅­μ–΄ 도메인 적합도 ν‰κ°€λ‘œ 우수 후손 선별 |
| **3. μ •μ œ (Refinement)** | ν•œκ΅­μ–΄ instruction λ°μ΄ν„°λ‘œ μΆ”κ°€ μ§„ν™” |
| **4. 적응 (Adaptation)** | K-AI Leaderboard Docker ν˜Έν™˜ ν˜•μ‹μœΌλ‘œ μ •λΉ„ |
---
## 🎯 μ‚¬μš©λ²•
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "Warecube/Warecube-KO-27B"
tokenizer = AutoTokenizer.from_pretrained(
model_id, trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
prompt = "ν•œκ΅­μ˜ 좔석에 λŒ€ν•΄ μ„€λͺ…ν•΄μ£Όμ„Έμš”."
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
messages, return_tensors="pt", add_generation_prompt=True
)
out = model.generate(
inputs.to(model.device),
max_new_tokens=512,
do_sample=False,
)
print(tokenizer.decode(out[0], skip_special_tokens=False))
```
---
## πŸ› οΈ 사양
- νŒŒλΌλ―Έν„°: 27B (text)
- μ–‘μžν™”: bf16
- μ»¨ν…μŠ€νŠΈ: 8K (ν™•μž₯ κ°€λŠ₯)
- μ–Έμ–΄: ν•œκ΅­μ–΄ + μ˜μ–΄
- μΆ”λ‘ : `<think>` reasoning trace
- License: Apache 2.0
---
## πŸ“Š 평가
ν•œκ΅­μ–΄ 곡개 10 데이터셋, 100문제 Γ— 1 seed.
| Dataset | Score |
|:---|---:|
| CLIcK | **87%** |
| KMMLU History | **50%** |
| KMMLU Law | **29%** |
| KMMLU Health | 78% |
| HAERAE General | 58% |
| HAERAE History | 86% |
| HAERAE Linguistics | 89% |
| KoBEST Hellaswag | 89% |
| KoBEST COPA | **100%** |
| KoBEST BoolQ | 97% |
| **Macro Avg** | **76.3%** |
---
## 🀝 좜처
- μ‘°λΆ€: [FINAL-Bench/Darwin-27B-Opus](https://huggingface.co/FINAL-Bench/Darwin-27B-Opus)
- μ—„λ§ˆ: [NewenAI/QuettaLLMs-27B-Koreasoner-V3](https://huggingface.co/NewenAI/QuettaLLMs-27B-Koreasoner-V3)
- κ°€λ¬Έ: Darwin family (Darwin V7 진화적 λ¨Έμ§€ μ‹œλ¦¬μ¦ˆ)