Darwin-760B-RSI

763B Mixture-of-Experts · DeepSeek-V4.1-Flash lineage · GPQA Diamond 92.93 in a single sample · the parent's accuracy with 26% fewer output tokens · Darwin layer evolution

reasoning · MoE 384 experts · 763B parameters · Korean + English + Chinese · efficient thinking · Darwin evolution

The largest model of the Darwin family: 92.93% on GPQA Diamond from a single sample, and the parent's accuracy at about three quarters of its answer length.


🧬 Darwin: evolve the parent, keep what works

Darwin treats a strong open model as a parent. It measures where the parent is weak and changes exactly those parts, instead of re-training everything and risking what already works.

  • Diagnose before you change. Every Darwin generation starts from a measured map of the parent.
  • Change little, precisely. Darwin-760B-RSI replaces 7 of the parent's 40 decoder layers; everything else is the parent, byte for byte.
  • Measured, not claimed. Every candidate was measured against the parent on the same items under identical settings, and only a child that held the parent's accuracy could ship.

Lineage

Role
Parent deepseek-ai/DeepSeek-V4.1-Flash-1017 (revision 08c8657f0229) 763B MoE · MIT License
Darwin layer evolution decoder layers 23, 26, 27, 28, 30, 31, 32 chosen by a layer-selection search over layers 23–36; each layer's weights are an unchanged copy of an MIT-licensed checkpoint of the same architecture (attribution in NOTICE)
Preserved the other 33 decoder layers · router · engram tables · embeddings · tokenizer · chat template byte-identical to the parent
Format routed experts MXFP4 (4.25 bit) · dense FP8 no tensor arithmetic, no re-quantization

The child has exactly the parent's tensors, shapes and parameter count (763,205,315,794 parameters).


🏆 Results

Benchmark Score Setting
GPQA Diamond (198) 92.93 single sample · temperature 1.0 · reasoning_effort 100 · 65,536-token context
Overall, 6 benchmarks (1,200 items) 93.2 (parent 92.5) greedy · reasoning_effort 75 · 32,768-token context, parent re-measured in the same run
Output length 74% of the parent same 1,200 items · 1,553.5 vs 2,098.6 tokens per answer

Same run, parent vs Darwin-760B-RSI

Measured by VIDRAFT on a 1,200-item set, with the parent re-measured in the same run under identical settings: thinking on (reasoning_effort 75), greedy decoding (temperature 0, seed 1234), context 32,768, vLLM 0.34.0, tensor parallel 4 on 4 x H200.

Benchmark Items Darwin-760B-RSI (%) DeepSeek-V4.1-Flash-1017 (%)
Overall 1,200 93.2 92.5
GPQA Diamond 198 88.9 86.4
MMLU-Pro 280 90.0 89.6
BBH 270 96.3 94.8
MATH (level 5) 258 93.8 93.4
AIME 2025 30 93.3 90.0
HumanEval 164 98.2 100.0
Mean output tokens per answer 1,200 1,553.5 2,098.6
  • Why GPQA Diamond reads 88.9 here and 92.93 above: this table is a like-for-like comparison with the parent, so both models run with greedy decoding, reasoning_effort 75 and a 32,768-token context. The leaderboard entry (92.93) uses the full thinking setting: reasoning_effort 100, sampling at temperature 1.0 and a 65,536-token context. With more room to think, the same model answers more GPQA questions correctly.
  • Paired comparison on all 1,200 items: +0.8 points, 95% bootstrap interval [−0.2, +1.8], McNemar p = 0.20: the parent's accuracy at 26% fewer output tokens.
  • Answers that reached the 32,768-token context: 0.8% for Darwin-760B-RSI, 1.6% for the parent.

Held-out confirmation and Chinese axes

A second run on items not used for the layer search, parent and child on the same items under the same settings (context 32,768, reasoning_effort 75, greedy):

Set Items Darwin-760B-RSI DeepSeek-V4.1-Flash-1017 Output tokens (child / parent)
Held-out English reasoning (GPQA main non-Diamond, MMLU-Pro, BBH, MATH 5) 1,200 89.92% 90.25% 80%
Chinese factual QA (Chinese SimpleQA, F-score) 300 73.7 71.8 70%
Chinese long context (LongBench lsht + passage retrieval, 8k to 26k tokens) 100 82% 82% -
Hard instruction following (3 to 5 constraints) 200 85.0% 82.5% -

No set shows a statistically significant difference from the parent (paired bootstrap, 95%), while answers are 20 to 30% shorter.

Evaluation protocol

Setting GPQA Diamond (leaderboard entry) Parent-vs-child runs
Samples per question 1 1
Sampling temperature 1.0 · top_p 0.95 · top_k 20 · seed 2000 greedy (temperature 0) · seed 1234
Thinking reasoning_effort 100 reasoning_effort 75
Context 65,536 tokens 32,768 tokens
Prompt "Answer the following multiple choice question … 'Answer: $LETTER' … Think step by step before answering." (choices shuffled once with a fixed seed) same family of official-style prompts
Engine vLLM 0.34.0 · tensor parallel 4 · 4 x H200 · engram CPU offload · DSpark speculative decoding same

All numbers are self-measured and reproducible with the settings above.


⚙️ Specifications

Architecture Mixture-of-Experts (DeepSeek-V4.1-Flash), 40 decoder layers, hidden 5,120
Experts 384 routed (6 active per token) + 1 shared expert per layer
Parameters 763,205,315,794
Precision routed experts MXFP4 (4.25 bit) · dense FP8 · engram tables FP8
Size on disk about 510 GB (48 safetensors shards)
Minimum tested hardware 4 x H200 (engram tables offloaded to CPU memory)

🚀 Quickstart

Serving with vLLM 0.34.0 (4 × H200)

export VLLM_ENGINE_READY_TIMEOUT_S=3600
vllm serve FINAL-Bench/Darwin-760B-RSI \
  --served-model-name Darwin-760B-RSI \
  --tensor-parallel-size 4 \
  --tokenizer-mode deepseek_v41 \
  --engram-config '{"cpu_offload":true}' \
  --max-model-len 65536 \
  --max-num-seqs 64 \
  --max-num-batched-tokens 8192 \
  --kernel-config '{"enable_flashinfer_autotune":false}' \
  --language-model-only \
  --reasoning-parser deepseek_v41 \
  --speculative-config '{"method":"dspark","num_speculative_tokens":5}' \
  --gpu-memory-utilization 0.90

Chat Completions (OpenAI-compatible)

from openai import OpenAI
c = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
r = c.chat.completions.create(model="Darwin-760B-RSI",
    messages=[{"role": "user", "content": "How many primes are there below 100?"}],
    temperature=1.0, top_p=0.95, extra_body={"top_k": 20, "chat_template_kwargs": {"thinking": True, "reasoning_effort": 100}})
print(r.choices[0].message.content)

Tip: this is a thinking model. Give it room: a context of 64K tokens or more is recommended for hard reasoning.


🔗 Related Darwin Models


📚 Citation

@misc{darwin760b_rsi_2026,
  title  = {Darwin-760B-RSI: Layer-Level Darwin Evolution of a 763B Mixture-of-Experts Reasoning Model},
  author = {FINAL-Bench / Darwin Research Team},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/FINAL-Bench/Darwin-760B-RSI}},
  note   = {GPQA Diamond 92.93 \% (single sample)}
}

@misc{darwin_family_2026,
  title  = {Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning},
  author = {Kim, Taebong and Hong, Youngsik and Kim, Minsik and Choi, Sunyoung and Jang, Jaewon and Shin, Junghoon},
  year   = {2026},
  eprint = {2605.14386},
  archivePrefix = {arXiv}
}

📜 License

MIT License (see LICENSE, the parent's MIT license file, included unchanged). Third-party attribution for every weight shard is in NOTICE.

🏢 About

Built by VIDRAFT · evaluated with FINAL-Bench.

This model is part of the Darwin Family.


한국어 요약

Darwin-760B-RSI 는 다윈 가족에서 가장 큰 모델입니다. 아버지 모델 deepseek-ai/DeepSeek-V4.1-Flash-1017(7,632억 파라미터, MIT)을 바탕으로, 40개 층 가운데 7개 층(23, 26, 27, 28, 30, 31, 32)을 다윈 층 선택 탐색으로 골라 바꿨고 나머지는 아버지와 바이트까지 같습니다.

  • GPQA Diamond 92.93% (198문항, 1번 풀기, 생각 강도 100·문맥 65,536). 아래 아버지 비교표의 88.9 는 두 모델을 같은 조건(그리디·생각 강도 75·문맥 32,768)으로 맞춘 값입니다
  • 아버지와 같은 조건 1,200문항에서 정확도 93.2% (아버지 92.5%), 답 길이는 26% 짧음
  • 학습에 쓰지 않은 확인용 1,200문항과 중국어 환각·긴 문맥·어려운 지시에서도 아버지와 같은 수준을 지키면서 답이 20~30% 짧음
Downloads last month
-
Safetensors
Model size
763B params
Tensor type
BF16
·
F32
·
F8_E4M3
·
I8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for FINAL-Bench/Darwin-760B-RSI

Evaluation results

  • Accuracy (single sample, 65K context) on GPQA Diamond
    self-reported
    92.930