card: add insights cross-links (on-device / quantization)

README.md

@@ -1,307 +1,310 @@
----
-license: apache-2.0
-tags:
-  - jgos
-  - ourbox
-  - darwin
-  - darwin-platform
-  - evolutionary-merge
-  - ffn-merge
-  - model-breeding
-  - korean
-  - korean-specialized
-  - reasoning
-  - advanced-reasoning
-  - chain-of-thought
-  - thinking
-  - qwen3.6
-  - qwen
-  - moe
-  - mixture-of-experts
-  - multi-token-prediction
-  - multilingual
-  - gpqa
-  - benchmark
-  - open-source
-  - apache-2.0
-  - vidraft
-  - eval-results
-language:
-  - ko
-  - en
-  - zh
-  - ja
-  - de
-  - fr
-  - es
-  - ru
-  - ar
-  - multilingual
-pipeline_tag: text-generation
-library_name: transformers
-model-index:
-- name: Ourbox-35B-JGOS
-  results:
-  - task:
-      type: question-answering
-      name: Question Answering
-    dataset:
-      name: GPQA Diamond
-      type: Idavidrein/gpqa
-      config: gpqa_diamond
-    metrics:
-    - type: accuracy
-      value: 86.36
-      name: Accuracy
----
-
-> ### 📱 Run it on your phone or a GPU-less PC → **POCKET**  ·  🚀 **[Try it live (CPU chat)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU)**
-> VIDRAFT's on-device family: a 35B model that runs on **iPhone** and on **CPU with no GPU** — stock `llama.cpp`, no fork.
->
-> [![Live demo](https://img.shields.io/badge/🤗_Space-POCKET_CPU_chat-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) [![Collection](https://img.shields.io/badge/📚-POCKET_collection-5dbf9a)](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6) [![35B](https://img.shields.io/badge/POCKET--35B-GGUF-243456)](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [![KR MLX](https://img.shields.io/badge/POCKET--KR-iPhone-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) [![EN](https://img.shields.io/badge/POCKET--EN-GGUF-185fa5)](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF)
-
->
-
-
-# Ourbox-35B-JGOS: Korean-Specialized, Darwin-Evolved 35B-A3B Reasoning MoE — 86.36% on GPQA Diamond
-
-<p align="center">
-  <a href="https://huggingface.co/FINAL-Bench/Ourbox-35B-JGOS"><img src="https://img.shields.io/badge/⭐_GPQA_Diamond-86.36%25_Ourbox--35B--JGOS-gold?style=for-the-badge" alt="GPQA"></a>
-  <a href="https://huggingface.co/FINAL-Bench/Ourbox-35B-JGOS"><img src="https://img.shields.io/badge/🇰🇷_Korean-Specialized-red?style=for-the-badge" alt="Korean"></a>
-</p>
-
-<p align="center">
-  <a href="https://huggingface.co/FINAL-Bench/Darwin-36B-Opus"><img src="https://img.shields.io/badge/🧬_Sibling-Darwin--36B--Opus_(88.4%25)-blue?style=for-the-badge" alt="Sibling"></a>
-  <a href="https://huggingface.co/spaces/FINAL-Bench/Leaderboard"><img src="https://img.shields.io/badge/🏆_FINAL_Bench-Leaderboard-green?style=for-the-badge" alt="FINAL Bench"></a>
-  <a href="https://huggingface.co/datasets/Idavidrein/gpqa"><img src="https://img.shields.io/badge/📊_Benchmark-GPQA_Diamond-orange?style=for-the-badge" alt="GPQA"></a>
-</p>
-
-> Darwin-Evolved on Qwen3.6-35B-A3B | 35B total / ~3B active | 🇰🇷 Korean-Specialized | Thinking Mode | Hybrid Linear/Full Attention | Multi-Token Prediction | 262K Context | BF16 | Apache 2.0
-> **Darwin FFN-level evolutionary merge → Korean specialization → 86.36% on GPQA Diamond (majority-of-8+)**
-
----
-
-## Abstract
-
-**Ourbox-35B-JGOS** is a 35-billion-parameter mixture-of-experts (MoE) reasoning model produced by the **Darwin** evolutionary breeding platform (FINAL-Bench / VIDRAFT_LAB). Rather than retraining from scratch, Darwin recombines the **feed-forward (FFN / MoE expert) tensors** of the **Qwen3.6-35B-A3B** backbone with those of additional specialized donor models, then **evolves** the merged descendant toward a target objective — here, **Korean-language specialization**.
-
-Because the merge operates at the **expert-FFN level**, Ourbox inherits complementary domain and language competencies from multiple sources while preserving the backbone's hybrid-attention topology and 262K long-context behavior. The result is a Korean-specialized reasoning model that remains highly competitive on English graduate-level science: on **GPQA Diamond** (198 questions across physics, chemistry, biology), Ourbox-35B-JGOS scores **86.36% (171/198)** under a majority-of-8+ protocol. On Hugging Face's live GPQA leaderboard this **improves on its own Qwen3.6-35B-A3B backbone (86.0)** by +0.36 points and edges past **GLM-5.1 (86.2)** and **GLM-5 (86.0)** — with only **~3B active parameters**.
-
----
-
-## GPQA Diamond Leaderboard — Hugging Face `Idavidrein/gpqa` (2026-07-11)
-
-Ourbox-35B-JGOS on the **official Hugging Face GPQA Diamond leaderboard** (`Idavidrein/gpqa`, base-model view, 50 models). FINAL-Bench models in **bold**:
-
-| # | Model | GPQA Diamond |
-|---|---|---|
-| 1 | zai-org/GLM-5.2 | 91.2 |
-| 2 | **FINAL-Bench/Darwin-398B-JGOS** | 90.9 |
-| 3 | moonshotai/Kimi-K2.6 | 90.5 |
-| 4 | tencent/Hy3 | 90.4 |
-| 5 | deepseek-ai/DeepSeek-V4-Pro | 90.1 |
-| 6 | **FINAL-Bench/Darwin-28B-REASON** | 89.39 |
-| 7 | Qwen/Qwen3.5-397B-A17B | 88.4 |
-| 8 | **FINAL-Bench/Darwin-36B-Opus** | 88.4 |
-| 9 | **FINAL-Bench/Darwin-60B-DUO** | 88.38 |
-| 10 | inclusionAI/Ring-2.6-1T | 88.27 |
-| 11 | deepseek-ai/DeepSeek-V4-Flash | 88.1 |
-| 12 | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B (NVFP4) | 87.9 |
-| 13 | zai-org/GLM-4.7-FP8 | 87.88 |
-| 14 | Qwen/Qwen3.6-27B | 87.8 |
-| 15 | moonshotai/Kimi-K2.5 | 87.6 |
-| 16 | moonshotai/Kimi-K2.5 *(source)* | 87.37 |
-| 17 | tencent/Hy3-preview | 87.2 |
-| 18 | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B (BF16) | 87.0 |
-| 19 | **FINAL-Bench/Darwin-27B-Opus** | 86.9 |
-| 20 | Qwen/Qwen3.5-122B-A10B | 86.6 |
-| **★ 21** | **FINAL-Bench/Ourbox-35B-JGOS** 🇰🇷 | **86.36** |
-| 22 | zai-org/GLM-5.1 | 86.2 |
-| 23 | zai-org/GLM-5 | 86.0 |
-| 24 | Qwen/Qwen3.6-35B-A3B *(Ourbox backbone)* | 86.0 |
-| 25 | **FINAL-Bench/Darwin-31B-Opus** | 85.9 |
-
-The FINAL-Bench Darwin family dominates the upper board — **5 of the 20 models ranked above Ourbox are Darwin models** (Darwin-398B-JGOS #2, Darwin-28B-REASON #6, Darwin-36B-Opus #8, Darwin-60B-DUO #9, Darwin-27B-Opus #19). At **86.36%**, Ourbox-35B-JGOS ranks **#21 of 50** on the live leaderboard and — most notably — **improves on its own Qwen3.6-35B-A3B backbone (86.0, #24) by +0.36 points**, confirming that the Darwin FFN-merge and Korean specialization *added* capability rather than eroding it. It also edges past **GLM-5.1 (86.2, #22)** and **GLM-5 (86.0, #23)** while activating only ~3B parameters.
-
-> Ranks reflect the live leaderboard as of 2026-07-11 (which counts quantized/duplicate entries); positions shift as it updates. Ourbox-35B-JGOS is **live and listed at #21**.
-
----
-
-## What Is Darwin?
-
-**Darwin** is the evolutionary model-breeding platform developed by FINAL-Bench / VIDRAFT_LAB. Rather than allocating further compute to gradient optimization, Darwin treats trained checkpoints as a **genetic pool** and discovers high-performing descendants through principled recombination of their weight tensors — with a particular focus on the **FFN / MoE expert** subspace, where domain and language competence is concentrated.
-
-At a high level, the platform performs:
-
-1. **Per-tensor compatibility analysis** across the backbone and donor models to identify which FFN experts and components transfer cleanly and which require weighted recombination.
-2. **FFN-level merge & evolution** — the descendant's expert tensors are assembled from the pool and iteratively evolved toward a target objective (Korean specialization for Ourbox).
-3. **Verification** via a multi-phase scientific benchmark before release.
-
-Specific algorithmic details of the Darwin engine are proprietary to FINAL-Bench. All Darwin models are released under the base model's open-source license (Apache 2.0).
-
-**JGOS** is the reasoning-model line built with Darwin; **Ourbox** is its Korean-specialized 35B-A3B member.
-
----
-
-## Evolution Process
-
-Ourbox-35B-JGOS is bred, not trained:
-
-- **Backbone**: [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen) — the foundation MoE, contributing its hybrid-attention topology (¾ linear + ¼ full), 256-expert routing, MTP head, and 262K context.
-- **FFN donors**: additional specialized models whose **feed-forward / expert tensors** are recombined into the backbone by the Darwin engine, contributing complementary domain and Korean-language competence.
-- **Evolution objective**: Korean specialization — the evolutionary selection biases the merged expert population toward stronger Korean reasoning and generation, while structural and long-context behavior is inherited intact from the backbone.
-
-The merge operates **without gradient optimization on the final assembly**; a deployable bfloat16 checkpoint is produced by the Darwin pipeline directly.
-
----
-
-## 🇰🇷 Korean Specialization
-
-Ourbox-35B-JGOS is specialized for **Korean**. The Darwin evolutionary process selects and recombines FFN experts to strengthen Korean-language reasoning, comprehension, and generation — targeting natural Korean output, robust handling of Korean scientific/technical text, and reduced character-level corruption on large Korean inputs.
-
-Crucially, this specialization does **not** come at the cost of general capability: the model's **86.36% GPQA Diamond** (in English) improves on its own Qwen3.6-35B-A3B backbone (86.0) on Hugging Face's live leaderboard, evidence that the FFN-merge preserved scientific reasoning depth while adding Korean strength. The model remains fully multilingual (Korean-first), with English, Chinese, Japanese, and other languages inherited from the backbone.
-
----
-
-## Architecture
-
-Ourbox-35B-JGOS retains the full Qwen3.6-35B-A3B architecture (`qwen3_5_moe` codebase):
-
-| | |
-|---|---|
-| Foundation | Qwen3.6-35B-A3B (`Qwen3_5MoeForCausalLM`) |
-| Breeding platform | Darwin (FFN-level evolutionary merge) |
-| Total parameters | ~35 B |
-| Active parameters | ~3 B (top-8 of 256 routed experts per layer) |
-| Layers | 40 |
-| Hidden size | 2048 |
-| Attention | **Hybrid** — 30 linear-attention + 10 full-attention layers (`full_attention_interval = 4`) |
-| Full-attention heads | 16 Q / 2 KV (GQA), head dim 256, partial rotary 0.25 |
-| Linear attention | Gated-DeltaNet style — 16 key heads × 128, 32 value heads × 128, conv kernel 4 |
-| Experts per layer | 256 routed (top-8) + 1 shared, expert intermediate 512 |
-| Multi-Token Prediction | 1 MTP layer (`mtp_num_hidden_layers = 1`) |
-| Context length | 262,144 tokens |
-| Vocabulary | 248,320 |
-| RoPE | θ = 1e7, interleaved mRoPE, sections [11, 11, 10] |
-| Dtype | bfloat16 |
-| Checkpoint size | ~69 GB (2 shards) |
-| License | Apache 2.0 |
-
-The **hybrid attention** design (¾ linear + ¼ full) gives near-linear KV-cache scaling across the 262K window, and the **Multi-Token Prediction** head provides a built-in draft for speculative decoding.
-
----
-
-## GPQA Diamond Evaluation
-
-### Methodology
-
-Ourbox-35B-JGOS was evaluated on all **198 GPQA Diamond** questions using a two-pass **majority-of-8+** protocol (identical to sibling FINAL-Bench reasoning models, for cross-model comparability):
-
-**Pass 1 — Greedy baseline**
-- All 198 questions, deterministic decoding (`do_sample=False`)
-- Up to 5,120 new tokens per question (full `<think>` trajectories)
-- Standard multiple-choice prompt format
-
-**Pass 2 — Stochastic majority vote with tiebreaker**
-- Each question is answered by **8 independent stochastic generations** (`temperature=0.7`, `max_tokens=5120`); the majority answer is taken
-- Where the 8-vote margin is inconclusive (e.g. 3:3 / 3:4 / 4:4), an additional **16-vote tiebreaker** round (`temperature=0.5`) resolves the answer
-
-The final answer for each question is extracted after the `</think>` delimiter.
-
-### Result
-
-| Metric | Value |
-|---|---|
-| Correct | **171 / 198** |
-| **GPQA Diamond accuracy (maj@8+)** | **86.36%** |
-
-Evaluated against the [`Idavidrein/gpqa`](https://huggingface.co/datasets/Idavidrein/gpqa) `gpqa_diamond` split. The majority-of-8+ protocol surfaces answers that greedy decoding leaves subdominant — a pattern characteristic of well-formed chain-of-thought models — carrying Ourbox above its Qwen3.6-35B-A3B backbone (86.0) and past GLM-5.1 (86.2) on graduate-level science.
-
----
-
-## Usage
-
-```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
-import torch
-
-tok = AutoTokenizer.from_pretrained("FINAL-Bench/Ourbox-35B-JGOS", trust_remote_code=True)
-model = AutoModelForCausalLM.from_pretrained(
-    "FINAL-Bench/Ourbox-35B-JGOS",
-    torch_dtype=torch.bfloat16,
-    device_map="auto",
-    trust_remote_code=True,
-)
-
-messages = [
-    {"role": "user", "content": "상대론적 운동에너지 식을 유도해줘."}
-]
-text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-inputs = tok(text, return_tensors="pt").to(model.device)
-outputs = model.generate(**inputs, max_new_tokens=5120, temperature=0.6, do_sample=True)
-print(tok.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
-```
-
-### Answer extraction for evaluations
-
-This is a **thinking model** — responses always begin with a `<think>` reasoning trace. For benchmarks, extract the final answer after `</think>`:
-
-```python
-response = tok.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
-idx = response.rfind("</think>")
-answer_part = response[idx + len("</think>"):].strip() if idx >= 0 else response
-```
-
-### Recommended settings
-
-- **Temperature**: 0.6–0.7 for reasoning / majority voting; 0.0 for greedy deterministic
-- **max_new_tokens**: ≥5120 to accommodate full `<think>` trajectories
-- **Chat template**: assistant turn opens with `<think>` when `apply_chat_template(add_generation_prompt=True)` is used
-
----
-
-## VRAM Requirements
-
-| Precision | VRAM | Recommended GPU |
-|---|---|---|
-| bf16 (full) | ~72 GB | 1× H100 80GB / 1× B200 |
-| 8-bit | ~40 GB | 1× A100 40GB+ / 1× L40S |
-| 4-bit | ~22 GB | 1× RTX 4090 / 1× A10 |
-
----
-
-## Key Findings
-
-1. **Korean specialization without capability loss.** Darwin's FFN-level merge adds Korean-language strength while retaining **86.36% GPQA Diamond** — above the model's own Qwen3.6-35B-A3B backbone (86.0). Specialization and general reasoning are not a zero-sum trade under expert-level recombination.
-
-2. **Specialization improves on the backbone.** On Hugging Face's live GPQA Diamond leaderboard, Ourbox (86.36) exceeds its own Qwen3.6-35B-A3B backbone (86.0) and edges past GLM-5.1 (86.2) and GLM-5 (86.0) — the Darwin FFN-merge added Korean capability without eroding scientific reasoning, at ~3B active parameters.
-
-3. **Breeding beats retraining for specialization.** A deployable, Korean-specialized 35B checkpoint is produced by evolutionary FFN recombination — no full-model gradient training on the final assembly — demonstrating Darwin as an efficient route to targeted, high-capability models.
-
----
-
-## References
-
-- Rein et al., *GPQA: A Graduate-Level Google-Proof Q&A Benchmark*, 2024. [dataset](https://huggingface.co/datasets/Idavidrein/gpqa)
-- Qwen Team, *Qwen3.6 Technical Report*, 2026.
-
----
-
-## Built By
-
-**FINAL-Bench / VIDRAFT_LAB** — Darwin evolutionary breeding platform, JGOS Korean-specialized reasoning line.
-Backbone weights by the Qwen Team (Qwen3.6-35B-A3B). Released under Apache 2.0.
-
----
-
-## Citation
-
-```bibtex
-@misc{ourbox-35b-jgos,
-  title   = {Ourbox-35B-JGOS: Korean-Specialized, Darwin-Evolved 35B-A3B Reasoning MoE},
-  author  = {FINAL-Bench and VIDRAFT_LAB},
-  year    = {2026},
-  url     = {https://huggingface.co/FINAL-Bench/Ourbox-35B-JGOS},
-  note    = {Qwen3.6-35B-A3B backbone, Darwin FFN-level evolutionary merge, Korean-specialized, 86.36% GPQA Diamond (maj@8+)}
-}
-```
+---
+license: apache-2.0
+tags:
+  - jgos
+  - ourbox
+  - darwin
+  - darwin-platform
+  - evolutionary-merge
+  - ffn-merge
+  - model-breeding
+  - korean
+  - korean-specialized
+  - reasoning
+  - advanced-reasoning
+  - chain-of-thought
+  - thinking
+  - qwen3.6
+  - qwen
+  - moe
+  - mixture-of-experts
+  - multi-token-prediction
+  - multilingual
+  - gpqa
+  - benchmark
+  - open-source
+  - apache-2.0
+  - vidraft
+  - eval-results
+language:
+  - ko
+  - en
+  - zh
+  - ja
+  - de
+  - fr
+  - es
+  - ru
+  - ar
+  - multilingual
+pipeline_tag: text-generation
+library_name: transformers
+model-index:
+- name: Ourbox-35B-JGOS
+  results:
+  - task:
+      type: question-answering
+      name: Question Answering
+    dataset:
+      name: GPQA Diamond
+      type: Idavidrein/gpqa
+      config: gpqa_diamond
+    metrics:
+    - type: accuracy
+      value: 86.36
+      name: Accuracy
+---
+
+> ### 📱 Run it on your phone or a GPU-less PC → **POCKET**  ·  🚀 **[Try it live (CPU chat)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU)**
+> VIDRAFT's on-device family: a 35B model that runs on **iPhone** and on **CPU with no GPU** — stock `llama.cpp`, no fork.
+>
+> [![Live demo](https://img.shields.io/badge/🤗_Space-POCKET_CPU_chat-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) [![Collection](https://img.shields.io/badge/📚-POCKET_collection-5dbf9a)](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6) [![35B](https://img.shields.io/badge/POCKET--35B-GGUF-243456)](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [![KR MLX](https://img.shields.io/badge/POCKET--KR-iPhone-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) [![EN](https://img.shields.io/badge/POCKET--EN-GGUF-185fa5)](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF)
+
+>
+
+
+# Ourbox-35B-JGOS: Korean-Specialized, Darwin-Evolved 35B-A3B Reasoning MoE — 86.36% on GPQA Diamond
+
+<p align="center">
+  <a href="https://huggingface.co/FINAL-Bench/Ourbox-35B-JGOS"><img src="https://img.shields.io/badge/⭐_GPQA_Diamond-86.36%25_Ourbox--35B--JGOS-gold?style=for-the-badge" alt="GPQA"></a>
+  <a href="https://huggingface.co/FINAL-Bench/Ourbox-35B-JGOS"><img src="https://img.shields.io/badge/🇰🇷_Korean-Specialized-red?style=for-the-badge" alt="Korean"></a>
+</p>
+
+<p align="center">
+  <a href="https://huggingface.co/FINAL-Bench/Darwin-36B-Opus"><img src="https://img.shields.io/badge/🧬_Sibling-Darwin--36B--Opus_(88.4%25)-blue?style=for-the-badge" alt="Sibling"></a>
+  <a href="https://huggingface.co/spaces/FINAL-Bench/Leaderboard"><img src="https://img.shields.io/badge/🏆_FINAL_Bench-Leaderboard-green?style=for-the-badge" alt="FINAL Bench"></a>
+  <a href="https://huggingface.co/datasets/Idavidrein/gpqa"><img src="https://img.shields.io/badge/📊_Benchmark-GPQA_Diamond-orange?style=for-the-badge" alt="GPQA"></a>
+</p>
+
+> Darwin-Evolved on Qwen3.6-35B-A3B | 35B total / ~3B active | 🇰🇷 Korean-Specialized | Thinking Mode | Hybrid Linear/Full Attention | Multi-Token Prediction | 262K Context | BF16 | Apache 2.0
+> **Darwin FFN-level evolutionary merge → Korean specialization → 86.36% on GPQA Diamond (majority-of-8+)**
+
+---
+
+## Abstract
+
+**Ourbox-35B-JGOS** is a 35-billion-parameter mixture-of-experts (MoE) reasoning model produced by the **Darwin** evolutionary breeding platform (FINAL-Bench / VIDRAFT_LAB). Rather than retraining from scratch, Darwin recombines the **feed-forward (FFN / MoE expert) tensors** of the **Qwen3.6-35B-A3B** backbone with those of additional specialized donor models, then **evolves** the merged descendant toward a target objective — here, **Korean-language specialization**.
+
+Because the merge operates at the **expert-FFN level**, Ourbox inherits complementary domain and language competencies from multiple sources while preserving the backbone's hybrid-attention topology and 262K long-context behavior. The result is a Korean-specialized reasoning model that remains highly competitive on English graduate-level science: on **GPQA Diamond** (198 questions across physics, chemistry, biology), Ourbox-35B-JGOS scores **86.36% (171/198)** under a majority-of-8+ protocol. On Hugging Face's live GPQA leaderboard this **improves on its own Qwen3.6-35B-A3B backbone (86.0)** by +0.36 points and edges past **GLM-5.1 (86.2)** and **GLM-5 (86.0)** — with only **~3B active parameters**.
+
+---
+
+## GPQA Diamond Leaderboard — Hugging Face `Idavidrein/gpqa` (2026-07-11)
+
+Ourbox-35B-JGOS on the **official Hugging Face GPQA Diamond leaderboard** (`Idavidrein/gpqa`, base-model view, 50 models). FINAL-Bench models in **bold**:
+
+| # | Model | GPQA Diamond |
+|---|---|---|
+| 1 | zai-org/GLM-5.2 | 91.2 |
+| 2 | **FINAL-Bench/Darwin-398B-JGOS** | 90.9 |
+| 3 | moonshotai/Kimi-K2.6 | 90.5 |
+| 4 | tencent/Hy3 | 90.4 |
+| 5 | deepseek-ai/DeepSeek-V4-Pro | 90.1 |
+| 6 | **FINAL-Bench/Darwin-28B-REASON** | 89.39 |
+| 7 | Qwen/Qwen3.5-397B-A17B | 88.4 |
+| 8 | **FINAL-Bench/Darwin-36B-Opus** | 88.4 |
+| 9 | **FINAL-Bench/Darwin-60B-DUO** | 88.38 |
+| 10 | inclusionAI/Ring-2.6-1T | 88.27 |
+| 11 | deepseek-ai/DeepSeek-V4-Flash | 88.1 |
+| 12 | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B (NVFP4) | 87.9 |
+| 13 | zai-org/GLM-4.7-FP8 | 87.88 |
+| 14 | Qwen/Qwen3.6-27B | 87.8 |
+| 15 | moonshotai/Kimi-K2.5 | 87.6 |
+| 16 | moonshotai/Kimi-K2.5 *(source)* | 87.37 |
+| 17 | tencent/Hy3-preview | 87.2 |
+| 18 | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B (BF16) | 87.0 |
+| 19 | **FINAL-Bench/Darwin-27B-Opus** | 86.9 |
+| 20 | Qwen/Qwen3.5-122B-A10B | 86.6 |
+| **★ 21** | **FINAL-Bench/Ourbox-35B-JGOS** 🇰🇷 | **86.36** |
+| 22 | zai-org/GLM-5.1 | 86.2 |
+| 23 | zai-org/GLM-5 | 86.0 |
+| 24 | Qwen/Qwen3.6-35B-A3B *(Ourbox backbone)* | 86.0 |
+| 25 | **FINAL-Bench/Darwin-31B-Opus** | 85.9 |
+
+The FINAL-Bench Darwin family dominates the upper board — **5 of the 20 models ranked above Ourbox are Darwin models** (Darwin-398B-JGOS #2, Darwin-28B-REASON #6, Darwin-36B-Opus #8, Darwin-60B-DUO #9, Darwin-27B-Opus #19). At **86.36%**, Ourbox-35B-JGOS ranks **#21 of 50** on the live leaderboard and — most notably — **improves on its own Qwen3.6-35B-A3B backbone (86.0, #24) by +0.36 points**, confirming that the Darwin FFN-merge and Korean specialization *added* capability rather than eroding it. It also edges past **GLM-5.1 (86.2, #22)** and **GLM-5 (86.0, #23)** while activating only ~3B parameters.
+
+> Ranks reflect the live leaderboard as of 2026-07-11 (which counts quantized/duplicate entries); positions shift as it updates. Ourbox-35B-JGOS is **live and listed at #21**.
+
+---
+
+## What Is Darwin?
+
+**Darwin** is the evolutionary model-breeding platform developed by FINAL-Bench / VIDRAFT_LAB. Rather than allocating further compute to gradient optimization, Darwin treats trained checkpoints as a **genetic pool** and discovers high-performing descendants through principled recombination of their weight tensors — with a particular focus on the **FFN / MoE expert** subspace, where domain and language competence is concentrated.
+
+At a high level, the platform performs:
+
+1. **Per-tensor compatibility analysis** across the backbone and donor models to identify which FFN experts and components transfer cleanly and which require weighted recombination.
+2. **FFN-level merge & evolution** — the descendant's expert tensors are assembled from the pool and iteratively evolved toward a target objective (Korean specialization for Ourbox).
+3. **Verification** via a multi-phase scientific benchmark before release.
+
+Specific algorithmic details of the Darwin engine are proprietary to FINAL-Bench. All Darwin models are released under the base model's open-source license (Apache 2.0).
+
+**JGOS** is the reasoning-model line built with Darwin; **Ourbox** is its Korean-specialized 35B-A3B member.
+
+---
+
+## Evolution Process
+
+Ourbox-35B-JGOS is bred, not trained:
+
+- **Backbone**: [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen) — the foundation MoE, contributing its hybrid-attention topology (¾ linear + ¼ full), 256-expert routing, MTP head, and 262K context.
+- **FFN donors**: additional specialized models whose **feed-forward / expert tensors** are recombined into the backbone by the Darwin engine, contributing complementary domain and Korean-language competence.
+- **Evolution objective**: Korean specialization — the evolutionary selection biases the merged expert population toward stronger Korean reasoning and generation, while structural and long-context behavior is inherited intact from the backbone.
+
+The merge operates **without gradient optimization on the final assembly**; a deployable bfloat16 checkpoint is produced by the Darwin pipeline directly.
+
+---
+
+## 🇰🇷 Korean Specialization
+
+Ourbox-35B-JGOS is specialized for **Korean**. The Darwin evolutionary process selects and recombines FFN experts to strengthen Korean-language reasoning, comprehension, and generation — targeting natural Korean output, robust handling of Korean scientific/technical text, and reduced character-level corruption on large Korean inputs.
+
+Crucially, this specialization does **not** come at the cost of general capability: the model's **86.36% GPQA Diamond** (in English) improves on its own Qwen3.6-35B-A3B backbone (86.0) on Hugging Face's live leaderboard, evidence that the FFN-merge preserved scientific reasoning depth while adding Korean strength. The model remains fully multilingual (Korean-first), with English, Chinese, Japanese, and other languages inherited from the backbone.
+
+---
+
+## Architecture
+
+Ourbox-35B-JGOS retains the full Qwen3.6-35B-A3B architecture (`qwen3_5_moe` codebase):
+
+| | |
+|---|---|
+| Foundation | Qwen3.6-35B-A3B (`Qwen3_5MoeForCausalLM`) |
+| Breeding platform | Darwin (FFN-level evolutionary merge) |
+| Total parameters | ~35 B |
+| Active parameters | ~3 B (top-8 of 256 routed experts per layer) |
+| Layers | 40 |
+| Hidden size | 2048 |
+| Attention | **Hybrid** — 30 linear-attention + 10 full-attention layers (`full_attention_interval = 4`) |
+| Full-attention heads | 16 Q / 2 KV (GQA), head dim 256, partial rotary 0.25 |
+| Linear attention | Gated-DeltaNet style — 16 key heads × 128, 32 value heads × 128, conv kernel 4 |
+| Experts per layer | 256 routed (top-8) + 1 shared, expert intermediate 512 |
+| Multi-Token Prediction | 1 MTP layer (`mtp_num_hidden_layers = 1`) |
+| Context length | 262,144 tokens |
+| Vocabulary | 248,320 |
+| RoPE | θ = 1e7, interleaved mRoPE, sections [11, 11, 10] |
+| Dtype | bfloat16 |
+| Checkpoint size | ~69 GB (2 shards) |
+| License | Apache 2.0 |
+
+The **hybrid attention** design (¾ linear + ¼ full) gives near-linear KV-cache scaling across the 262K window, and the **Multi-Token Prediction** head provides a built-in draft for speculative decoding.
+
+---
+
+## GPQA Diamond Evaluation
+
+### Methodology
+
+Ourbox-35B-JGOS was evaluated on all **198 GPQA Diamond** questions using a two-pass **majority-of-8+** protocol (identical to sibling FINAL-Bench reasoning models, for cross-model comparability):
+
+**Pass 1 — Greedy baseline**
+- All 198 questions, deterministic decoding (`do_sample=False`)
+- Up to 5,120 new tokens per question (full `<think>` trajectories)
+- Standard multiple-choice prompt format
+
+**Pass 2 — Stochastic majority vote with tiebreaker**
+- Each question is answered by **8 independent stochastic generations** (`temperature=0.7`, `max_tokens=5120`); the majority answer is taken
+- Where the 8-vote margin is inconclusive (e.g. 3:3 / 3:4 / 4:4), an additional **16-vote tiebreaker** round (`temperature=0.5`) resolves the answer
+
+The final answer for each question is extracted after the `</think>` delimiter.
+
+### Result
+
+| Metric | Value |
+|---|---|
+| Correct | **171 / 198** |
+| **GPQA Diamond accuracy (maj@8+)** | **86.36%** |
+
+Evaluated against the [`Idavidrein/gpqa`](https://huggingface.co/datasets/Idavidrein/gpqa) `gpqa_diamond` split. The majority-of-8+ protocol surfaces answers that greedy decoding leaves subdominant — a pattern characteristic of well-formed chain-of-thought models — carrying Ourbox above its Qwen3.6-35B-A3B backbone (86.0) and past GLM-5.1 (86.2) on graduate-level science.
+
+---
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+tok = AutoTokenizer.from_pretrained("FINAL-Bench/Ourbox-35B-JGOS", trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    "FINAL-Bench/Ourbox-35B-JGOS",
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True,
+)
+
+messages = [
+    {"role": "user", "content": "상대론적 운동에너지 식을 유도해줘."}
+]
+text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tok(text, return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=5120, temperature=0.6, do_sample=True)
+print(tok.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+### Answer extraction for evaluations
+
+This is a **thinking model** — responses always begin with a `<think>` reasoning trace. For benchmarks, extract the final answer after `</think>`:
+
+```python
+response = tok.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
+idx = response.rfind("</think>")
+answer_part = response[idx + len("</think>"):].strip() if idx >= 0 else response
+```
+
+### Recommended settings
+
+- **Temperature**: 0.6–0.7 for reasoning / majority voting; 0.0 for greedy deterministic
+- **max_new_tokens**: ≥5120 to accommodate full `<think>` trajectories
+- **Chat template**: assistant turn opens with `<think>` when `apply_chat_template(add_generation_prompt=True)` is used
+
+---
+
+## VRAM Requirements
+
+| Precision | VRAM | Recommended GPU |
+|---|---|---|
+| bf16 (full) | ~72 GB | 1× H100 80GB / 1× B200 |
+| 8-bit | ~40 GB | 1× A100 40GB+ / 1× L40S |
+| 4-bit | ~22 GB | 1× RTX 4090 / 1× A10 |
+
+---
+
+## Key Findings
+
+1. **Korean specialization without capability loss.** Darwin's FFN-level merge adds Korean-language strength while retaining **86.36% GPQA Diamond** — above the model's own Qwen3.6-35B-A3B backbone (86.0). Specialization and general reasoning are not a zero-sum trade under expert-level recombination.
+
+2. **Specialization improves on the backbone.** On Hugging Face's live GPQA Diamond leaderboard, Ourbox (86.36) exceeds its own Qwen3.6-35B-A3B backbone (86.0) and edges past GLM-5.1 (86.2) and GLM-5 (86.0) — the Darwin FFN-merge added Korean capability without eroding scientific reasoning, at ~3B active parameters.
+
+3. **Breeding beats retraining for specialization.** A deployable, Korean-specialized 35B checkpoint is produced by evolutionary FFN recombination — no full-model gradient training on the final assembly — demonstrating Darwin as an efficient route to targeted, high-capability models.
+
+---
+
+## References
+
+- Rein et al., *GPQA: A Graduate-Level Google-Proof Q&A Benchmark*, 2024. [dataset](https://huggingface.co/datasets/Idavidrein/gpqa)
+- Qwen Team, *Qwen3.6 Technical Report*, 2026.
+
+---
+
+## Built By
+
+**FINAL-Bench / VIDRAFT_LAB** — Darwin evolutionary breeding platform, JGOS Korean-specialized reasoning line.
+Backbone weights by the Qwen Team (Qwen3.6-35B-A3B). Released under Apache 2.0.
+
+---
+
+## Citation
+
+```bibtex
+@misc{ourbox-35b-jgos,
+  title   = {Ourbox-35B-JGOS: Korean-Specialized, Darwin-Evolved 35B-A3B Reasoning MoE},
+  author  = {FINAL-Bench and VIDRAFT_LAB},
+  year    = {2026},
+  url     = {https://huggingface.co/FINAL-Bench/Ourbox-35B-JGOS},
+  note    = {Qwen3.6-35B-A3B backbone, Darwin FFN-level evolutionary merge, Korean-specialized, 86.36% GPQA Diamond (maj@8+)}
+}
+```
+
+## Learn more
+- On-device, sovereign LLMs without a GPU: [Can you run a large LLM without a GPU?](https://vidraft.net/insights/on-device-llm-without-gpu.html)