---
license: apache-2.0
base_model:
- Qwen/Qwen3-4B
language:
- ko
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- darwin
- darwin-chimera
- attention-healing
- sliding-window
- qwen3
- vidraft
- research-checkpoint
---

# Darwin-Chimera-4B-Gen1 (Backbone · Research)

> ⚠️ **Generation-1 backbone — a research checkpoint, not a product.** Private repo.
> This is **a Qwen3-4B derivative**, not a from-scratch model. We state this explicitly.

## What this is

`Darwin-Chimera-4B-Gen1` is the first-generation **adapter backbone** of the Darwin-Chimera
line. We take **Qwen/Qwen3-4B** and re-wire only its **attention** via VIDRAFT
attention-healing, while **freezing the FFN, embeddings, and lm_head**, and convert the
attention to a **sliding-window** configuration.

The purpose is to verify that a VIDRAFT-healed attention circuit can sit on a frozen
knowledge core — the foundation for Generation-2 (FFN cross-breeding with other models).

## Honest weight fingerprint (vs Qwen/Qwen3-4B)

Measured relative change `||A−B|| / ||A||` against the original Qwen3-4B:

| Component | Relative change | Note |
|-----------|:---:|------|
| FFN (mlp) | **0.000%** | frozen — identical to Qwen3-4B |
| embed / lm_head | **0.000%** | frozen — identical |
| attention (self_attn) | **3.0%** mean (7.5% max) | healed |
| layernorm | 0.04% | minimal |
| config (hidden/inter/layers/vocab) | identical | only `sliding_window=4096` added |

→ At the weight level this checkpoint is **clearly a Qwen3-4B derivative**. We make **no**
claim of independence or from-scratch training. Knowledge/FFN is 100% Qwen3-4B.

## Training

- **Method**: attention-only healing (self_attn + per-layer norms trainable; FFN/embed/lm_head frozen)
- **Attention**: full → sliding window (4096), 5:1 sliding:full layer ratio
- **Tokens**: ~3B (Korean-centric annealing mix)
- **Base**: Qwen/Qwen3-4B (Apache 2.0)

## Evaluation (base, zero-shot — reference only)

- **Generation**: 6/6 domains coherent (Korean / English / science / code / math / biology), no gibberish
- **KMMLU** (6 subjects, 240Q, zero-shot, greedy): **27.1%** vs Qwen3-4B base **13.3%** (same protocol, +13.8pp)
- Absolute KMMLU is low because this is a **base (non-instruct) checkpoint**; instruction-following
  and benchmark quality are expected to come from a later SFT stage. The comparison above is a
  same-condition relative measurement, not an absolute SOTA claim.

## Intended use

- Backbone for **Darwin-Chimera Generation-2** (cross-architecture FFN cross-breeding research)
- Research and experimentation only. **Not for production.**

## License & attribution

Apache 2.0, inherited from **Qwen/Qwen3-4B**. Built on Qwen/Qwen3-4B.