---
license: apache-2.0
base_model:
  - FINAL-Bench/Darwin-27B-KR
tags:
  - qwen3.5
  - korean
  - sft
  - reasoning
  - thinking
  - darwin
  - k-ai
language:
  - ko
  - en
  - ja
  - zh
  - multilingual
pipeline_tag: text-generation
library_name: transformers
---

# Darwin-27B-KR-V2

> **Qwen3.5 Hybrid Architecture | ~26B Params | Thinking Mode | 262K Context | BF16 | Apache 2.0**

---

## Model Overview

**Darwin-27B-KR-V2** is the next evolution of [FINAL-Bench/Darwin-27B-KR](https://huggingface.co/FINAL-Bench/Darwin-27B-KR), enhanced with targeted Korean SFT (Supervised Fine-Tuning) for K-AI Leaderboard optimization.

Built on VIDRAFT's Darwin evolutionary merge lineage, this model inherits strong chain-of-thought reasoning from Darwin-27B-Opus and further refines Korean language capabilities through carefully curated training data targeting MuSR (Multi-Step Reasoning), KMMLU-Pro (Korean domain knowledge), and Metacognitive evaluation.

### Key Features

- **Darwin lineage** — Evolutionary merge backbone from VIDRAFT's Darwin-27B-Opus
- **K-AI targeted SFT** — 1,027 curated Korean reasoning & knowledge pairs
- **Thinking mode** — `<think>` tag based step-by-step reasoning
- **262K context** — Ultra-long document processing
- **BF16** — Memory-efficient (~48GB)
- **Apache 2.0** — Free for commercial use

---

## Training

| Item | Details |
|---|---|
| **Base Model** | [FINAL-Bench/Darwin-27B-KR](https://huggingface.co/FINAL-Bench/Darwin-27B-KR) |
| **Method** | LoRA SFT (rank=64, alpha=128) + Merge |
| **Data** | 1,027 Korean SFT pairs (MuSR 428 + KMMLU-Pro 500 + Metacognitive 99) |
| **Epochs** | 2 |
| **Learning Rate** | 2e-5 (cosine schedule) |
| **Effective Batch** | 16 |
| **Target Modules** | q/k/v/o_proj, gate/up/down_proj (1.17% trainable) |
| **Hardware** | 8x NVIDIA B200 (183GB each) |
| **Training Time** | ~25 minutes |
| **Final Loss** | 0.66 |
| **Precision** | BF16 |

### SFT Data Composition

| Source | Count | Description |
|---|---|---|
| **MuSR (Korean)** | 428 | Multi-step reasoning: causal, temporal, spatial, counterfactual |
| **KMMLU-Pro** | 500 | Korean domain knowledge: law, economics, science, history, medicine |
| **Metacognitive** | 99 | Self-correcting reasoning with TICOS framework |
| **Total** | **1,027** | All pairs include `<think>` reasoning tags |

---

## Model Specifications

| Property | Value |
|---|---|
| **Architecture** | Qwen3.5 (GatedDeltaNet Hybrid Attention, 64-layer) |
| **Parameters** | ~26B |
| **Hidden Size** | 5120 |
| **Layers** | 64 |
| **Context Length** | 262,144 tokens |
| **Precision** | BF16 (~48GB) |
| **Vocab Size** | 248,320 |
| **Thinking** | Supported (`<think>` tags) |
| **License** | Apache 2.0 |

---

## VRAM Requirements

| Setup | VRAM | Notes |
|---|---|---|
| BF16 (native) | ~48 GB | Single H100/B200 or 2x A100 |
| 4-bit quantized | ~14 GB | Single RTX 4090 |
| 8-bit quantized | ~26 GB | Single A6000 |

---

## Usage

> **Requirements**: `transformers >= 4.57.0`

### Transformers

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("FINAL-Bench/Darwin-27B-KR-V2")
model = AutoModelForCausalLM.from_pretrained(
    "FINAL-Bench/Darwin-27B-KR-V2",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "대한민국 헌법재판소의 역할과 권한에 대해 설명해주세요."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=4096, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

### vLLM

```bash
vllm serve FINAL-Bench/Darwin-27B-KR-V2 \
    --enforce-eager \
    --max-model-len 32768 \
    --dtype bfloat16
```

---

## Lineage

```
Qwen/Qwen3.5-27B
    |
    v
FINAL-Bench/Darwin-27B-Opus (evolutionary merge by VIDRAFT)
    |
    v
FINAL-Bench/Darwin-27B-KR (Korean-specialized variant)
    |
    v
FINAL-Bench/Darwin-27B-KR-V2 (this model, + K-AI targeted SFT)
```

---

## Acknowledgements

- [VIDRAFT / FINAL-Bench](https://huggingface.co/FINAL-Bench) — Darwin evolutionary merge system
- [Qwen Team](https://huggingface.co/Qwen) — Qwen3.5 architecture

---

## Citation

```bibtex
@misc{darwin_27b_kr_v2_2026,
  title        = {Darwin-27B-KR-V2: Korean-Enhanced Reasoning Model},
  author       = {VIDRAFT},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/FINAL-Bench/Darwin-27B-KR-V2}}
}
```