| --- |
| title: DarwinX |
| emoji: 🧬 |
| colorFrom: blue |
| colorTo: indigo |
| sdk: static |
| app_file: index.html |
| pinned: false |
| license: cc-by-4.0 |
| short_description: Evolving Agent Harnesses Through Natural Selection |
| --- |
| |
| # DarwinX: Evolving Agent Harnesses Through Natural Selection |
|
|
| Project page for [arXiv:2608.07545](https://arxiv.org/abs/2608.07545). |
|
|
| DarwinX treats agent self-evolution as selection over a *population* of harnesses with the base |
| model frozen. Across four benchmarks, one loop adds ~17 points on average: |
|
|
| | Benchmark | Result | Gain | What it isolates | |
| | --- | --- | --- | --- | |
| | Terminal-Bench 2.1 (avg@5) | 84.7% | +7.7 on matched base | in-domain evolution | |
| | TerminalWorld (held-out) | 68.3% | +7.3 | disjoint held-out task split | |
| | WebArena-Infinity (audit-clean pass@1) | 93.0% | +49.5 pp | synthetic → real intent shift | |
| | SWE-bench Verified (zero-shot transfer) | 84.2% | +3.4 | cross-benchmark transfer | |
|
|
| ## Page sections |
|
|
| `Abstract` · `Method` · `Results at a glance` · `Benchmark detail` · `Ablation: what evolution |
| changes` · `Limitations` · `BibTeX` |
|
|
| The ablation section reproduces the paper's framing: it is an *exploratory attribution, not a |
| per-skill causal ablation*, since the skills were co-selected rather than independently randomized. |
| The limitations section is carried over from the paper's Discussion. |
|
|
| ## Interactive figures |
|
|
| Eight figures are interactive, and each is driven by real run artifacts or the paper's own plotting |
| scripts rather than illustrative numbers. `tools/build_data.py` regenerates `assets/data.js` from the |
| original files, so nothing is hand-transcribed. |
|
|
| **1. TerminalWorld merge explorer.** Toggle any subset of the four evolved specialists and watch |
| which of the 41 held-out tasks the selection covers. Built from the `per_task_results` arrays of the |
| five evaluation runs, joined on `task_id` (the run files list the tasks in different orders, so |
| position-based joining would silently misalign them). |
|
|
| It surfaces something the paper states only in aggregate. The four specialists solve 24/25/26/27 |
| tasks and their **union is 29**, but the harness recombination actually produced solves **28**: |
|
|
| | | tasks | |
| | --- | --- | |
| | best single specialist (D) | 27/41 | |
| | union of all four | 29/41 | |
| | realized merge | 28/41 | |
| | solved by merge, by no specialist | `tw_448247` | |
| | solved by some specialist, not by merge | `tw_449421`, `tw_498533` | |
| | solved by all four | 21/41 | |
| | solved by none | 12/41 | |
|
|
| So recombination is not a free set union: it adds a capability no parent had and loses two. These are |
| single attempts and one task is worth 2.4 points, so individual flips sit inside the noise band — |
| the page says so next to the widget. |
|
|
| **2. TB2.1 cluster explorer.** Per-cluster base vs. evolved avg@5, sortable by gain, base rate, |
| cluster size, or name, with exact rates on hover. Numbers from `notes/TB21_RESULTS.md` (paired |
| protocol, 88 tasks). Deltas are carried over as reported rather than recomputed, because the source |
| rounds them from unrounded rates. |
|
|
| **3. WebArena-Infinity evolution curve.** All 37 evaluated variants with a best-so-far envelope, |
| from `notes/tw_dynamics.json` → `wai_adaptive_scores`. Toggle either series. |
|
|
| **4. Headline four-benchmark panels.** Replaces the right half of the teaser. From |
| `scripts/gen_summary_figure.py`. The reason to make this one interactive is that the paper's figure |
| has to *state* its truncation in the caption: on a shared axis WebArena-Infinity's +49.5 flattens the |
| two terminal benchmarks into slivers, so each panel is scaled to its own range. Here the reader can |
| switch between per-panel and shared 0–100 axes and see both framings. The best-prior-agent bar can be |
| hidden, since those systems use different models and effort settings and are context rather than a |
| controlled comparison. Per-panel labels carry the exceptions: on SWE-bench Verified the grey bar is |
| the fix-skill reference, not an unevolved Monet, and there is no prior-agent bar. |
|
|
| **5. TerminalWorld specialist bars.** From `notes/tw_dynamics.json` → `tw_heldout`. Same axis |
| question in miniature: the six arms span 58.5–68.3, so a 0–100 track renders them near-identical. |
| Defaults to a 55–70 axis with the full axis one click away and the truncation named in both notes. |
| Spec. D and the Claude Code reference are both 27/41, so D's bar landing exactly on the dashed |
| reference line is a built-in check that the axis transform is right. |
|
|
| **6. TB2.1 compute.** From `scripts/gen_tb21_compute.py` (medians over clean attempts). Switch |
| between turns and tokens; the bars share one scale across both task groups so the newly-solved vs. |
| already-solved contrast is not rescaled away. |
|
|
| **7. WAI invalid-trajectory composition.** From `scripts/gen_wai_invalid_composition.py`. Break the |
| 293 → 17 collapse down by application or by mechanism. Both rows sit on a shared absolute scale by |
| default, which makes the "after" row a near-invisible sliver — that *is* the finding; normalizing |
| each row to its own total then shows what the remainder consists of. `build_data.py` asserts both |
| decompositions still total 293 and 17. |
|
|
| **8. WAI audit dumbbells.** From `scripts/gen_wai_audit_by_app.py`. One line per application from raw |
| pass@1 to audited pass@1, so line length reads directly as how much a harness was leaning on |
| trajectories the audit rejects. Sortable by audit loss. `build_data.py` cross-checks the audited |
| DarwinX column against the per-application table rendered elsewhere on the page. |
|
|
| Three figures stay static because they are conceptual diagrams with no underlying data: the teaser's |
| selection schematic, the method overview, and the per-generation operators. The archive lineage tree |
| also stays static for a different reason — its node/edge data lives in a `state.db` on the cluster, |
| not on this machine, so there is nothing truthful to make interactive yet. |
|
|
| Tables are click-to-sort. Numeric columns open descending, text columns A–Z, and `Overall` rows stay |
| pinned to the bottom. |
|
|
| ## Local preview |
|
|
| ```bash |
| python3 -m http.server 8000 |
| # open http://localhost:8000 |
| ``` |
|
|
| ## Regenerate and test |
|
|
| ```bash |
| python3 tools/build_data.py # rebuild assets/data.js from the run artifacts |
| # then, with the server running, open: |
| # http://localhost:8000/tools/interaction_test.html |
| # it drives every widget with real click events and writes PASS/FAIL into the page title |
| ``` |
|
|
| Headless run: |
|
|
| ```bash |
| python3 -m http.server 8000 & |
| "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --headless --disable-gpu \ |
| --virtual-time-budget=6000 --dump-dom http://localhost:8000/tools/interaction_test.html \ |
| | grep -o '<title>.*</title>' |
| ``` |
|
|
| ## Layout |
|
|
| ``` |
| index.html the page: inline CSS + vanilla JS, no build step, no external |
| requests, no web fonts, no chart library |
| assets/data.js generated — real numbers behind the interactive figures |
| assets/*.png web-sized figures (max 2000px wide, 256-color) — 736 KB total |
| assets/full/*.png full-resolution originals for the three dense diagrams, |
| linked from the page via click-to-zoom |
| tools/build_data.py regenerates assets/data.js from the original run artifacts |
| tools/interaction_test.html click-driven test suite for the interactive figures (49 checks) |
| ``` |
|
|
| | Figure | Content | |
| | --- | --- | |
| | `teaser.png` | four-benchmark teaser (full figure, click-to-zoom target) | |
| | `teaser_schematic.png` | teaser's left half — the selection schematic shown on the page | |
| | `overview.png` | method overview | |
| | `population.png` | per-generation operators (mutation, classification, merge) | |
| | `tb21_category_gain.png` | per-cluster TB2.1 gain | |
| | `tb21_compute.png` | per-task compute: newly-solved vs already-solved | |
| | `tw_heldout_specialists.png` | held-out generalization vs archive diversity | |
| | `wai_evolution_curve.png` | best-so-far screening score | |
| | `wai_tree.png` | archive lineage tree | |
| | `wai_invalid_composition.png` | invalid trajectories, 293 → 17 | |
| | `wai_audit_by_app.png` | raw vs audited pass@1 per application | |
|
|
| Result figures are rendered from the paper's PDF sources at 200 DPI, then palette-reduced without |
| dithering (RMSE vs. truecolor < 0.5%, which is imperceptible for line art but 4.7x smaller). |
|
|
| ## Hosting |
|
|
| The frontmatter above configures this directory as a Hugging Face **static Space**. The same files |
| work unchanged as a GitHub Pages site or behind any static file server. |
|
|
| One thing to change when hosting: `og:image` and `twitter:image` are relative paths, which most |
| crawlers resolve but X/Twitter wants absolute. Replace them with the deployed absolute URL to get a |
| large social card. |
|
|
