title: DarwinX
emoji: 🧬
colorFrom: blue
colorTo: indigo
sdk: static
app_file: index.html
pinned: false
license: cc-by-4.0
short_description: Evolving Agent Harnesses Through Natural Selection
DarwinX: Evolving Agent Harnesses Through Natural Selection
Project page for arXiv:2608.07545.
DarwinX treats agent self-evolution as selection over a population of harnesses with the base model frozen. Across four benchmarks, one loop adds ~17 points on average:
| Benchmark | Result | Gain | What it isolates |
|---|---|---|---|
| Terminal-Bench 2.1 (avg@5) | 84.7% | +7.7 on matched base | in-domain evolution |
| TerminalWorld (held-out) | 68.3% | +7.3 | disjoint held-out task split |
| WebArena-Infinity (audit-clean pass@1) | 93.0% | +49.5 pp | synthetic → real intent shift |
| SWE-bench Verified (zero-shot transfer) | 84.2% | +3.4 | cross-benchmark transfer |
Page sections
Abstract · Method · Results at a glance · Benchmark detail · Ablation: what evolution changes · Limitations · BibTeX
The ablation section reproduces the paper's framing: it is an exploratory attribution, not a per-skill causal ablation, since the skills were co-selected rather than independently randomized. The limitations section is carried over from the paper's Discussion.
Interactive figures
Eight figures are interactive, and each is driven by real run artifacts or the paper's own plotting
scripts rather than illustrative numbers. tools/build_data.py regenerates assets/data.js from the
original files, so nothing is hand-transcribed.
1. TerminalWorld merge explorer. Toggle any subset of the four evolved specialists and watch
which of the 41 held-out tasks the selection covers. Built from the per_task_results arrays of the
five evaluation runs, joined on task_id (the run files list the tasks in different orders, so
position-based joining would silently misalign them).
It surfaces something the paper states only in aggregate. The four specialists solve 24/25/26/27 tasks and their union is 29, but the harness recombination actually produced solves 28:
| tasks | |
|---|---|
| best single specialist (D) | 27/41 |
| union of all four | 29/41 |
| realized merge | 28/41 |
| solved by merge, by no specialist | tw_448247 |
| solved by some specialist, not by merge | tw_449421, tw_498533 |
| solved by all four | 21/41 |
| solved by none | 12/41 |
So recombination is not a free set union: it adds a capability no parent had and loses two. These are single attempts and one task is worth 2.4 points, so individual flips sit inside the noise band — the page says so next to the widget.
2. TB2.1 cluster explorer. Per-cluster base vs. evolved avg@5, sortable by gain, base rate,
cluster size, or name, with exact rates on hover. Numbers from notes/TB21_RESULTS.md (paired
protocol, 88 tasks). Deltas are carried over as reported rather than recomputed, because the source
rounds them from unrounded rates.
3. WebArena-Infinity evolution curve. All 37 evaluated variants with a best-so-far envelope,
from notes/tw_dynamics.json → wai_adaptive_scores. Toggle either series.
4. Headline four-benchmark panels. Replaces the right half of the teaser. From
scripts/gen_summary_figure.py. The reason to make this one interactive is that the paper's figure
has to state its truncation in the caption: on a shared axis WebArena-Infinity's +49.5 flattens the
two terminal benchmarks into slivers, so each panel is scaled to its own range. Here the reader can
switch between per-panel and shared 0–100 axes and see both framings. The best-prior-agent bar can be
hidden, since those systems use different models and effort settings and are context rather than a
controlled comparison. Per-panel labels carry the exceptions: on SWE-bench Verified the grey bar is
the fix-skill reference, not an unevolved Monet, and there is no prior-agent bar.
5. TerminalWorld specialist bars. From notes/tw_dynamics.json → tw_heldout. Same axis
question in miniature: the six arms span 58.5–68.3, so a 0–100 track renders them near-identical.
Defaults to a 55–70 axis with the full axis one click away and the truncation named in both notes.
Spec. D and the Claude Code reference are both 27/41, so D's bar landing exactly on the dashed
reference line is a built-in check that the axis transform is right.
6. TB2.1 compute. From scripts/gen_tb21_compute.py (medians over clean attempts). Switch
between turns and tokens; the bars share one scale across both task groups so the newly-solved vs.
already-solved contrast is not rescaled away.
7. WAI invalid-trajectory composition. From scripts/gen_wai_invalid_composition.py. Break the
293 → 17 collapse down by application or by mechanism. Both rows sit on a shared absolute scale by
default, which makes the "after" row a near-invisible sliver — that is the finding; normalizing
each row to its own total then shows what the remainder consists of. build_data.py asserts both
decompositions still total 293 and 17.
8. WAI audit dumbbells. From scripts/gen_wai_audit_by_app.py. One line per application from raw
pass@1 to audited pass@1, so line length reads directly as how much a harness was leaning on
trajectories the audit rejects. Sortable by audit loss. build_data.py cross-checks the audited
DarwinX column against the per-application table rendered elsewhere on the page.
Three figures stay static because they are conceptual diagrams with no underlying data: the teaser's
selection schematic, the method overview, and the per-generation operators. The archive lineage tree
also stays static for a different reason — its node/edge data lives in a state.db on the cluster,
not on this machine, so there is nothing truthful to make interactive yet.
Tables are click-to-sort. Numeric columns open descending, text columns A–Z, and Overall rows stay
pinned to the bottom.
Local preview
python3 -m http.server 8000
# open http://localhost:8000
Regenerate and test
python3 tools/build_data.py # rebuild assets/data.js from the run artifacts
# then, with the server running, open:
# http://localhost:8000/tools/interaction_test.html
# it drives every widget with real click events and writes PASS/FAIL into the page title
Headless run:
python3 -m http.server 8000 &
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --headless --disable-gpu \
--virtual-time-budget=6000 --dump-dom http://localhost:8000/tools/interaction_test.html \
| grep -o '<title>.*</title>'
Layout
index.html the page: inline CSS + vanilla JS, no build step, no external
requests, no web fonts, no chart library
assets/data.js generated — real numbers behind the interactive figures
assets/*.png web-sized figures (max 2000px wide, 256-color) — 736 KB total
assets/full/*.png full-resolution originals for the three dense diagrams,
linked from the page via click-to-zoom
tools/build_data.py regenerates assets/data.js from the original run artifacts
tools/interaction_test.html click-driven test suite for the interactive figures (49 checks)
| Figure | Content |
|---|---|
teaser.png |
four-benchmark teaser (full figure, click-to-zoom target) |
teaser_schematic.png |
teaser's left half — the selection schematic shown on the page |
overview.png |
method overview |
population.png |
per-generation operators (mutation, classification, merge) |
tb21_category_gain.png |
per-cluster TB2.1 gain |
tb21_compute.png |
per-task compute: newly-solved vs already-solved |
tw_heldout_specialists.png |
held-out generalization vs archive diversity |
wai_evolution_curve.png |
best-so-far screening score |
wai_tree.png |
archive lineage tree |
wai_invalid_composition.png |
invalid trajectories, 293 → 17 |
wai_audit_by_app.png |
raw vs audited pass@1 per application |
Result figures are rendered from the paper's PDF sources at 200 DPI, then palette-reduced without dithering (RMSE vs. truecolor < 0.5%, which is imperceptible for line art but 4.7x smaller).
Hosting
The frontmatter above configures this directory as a Hugging Face static Space. The same files work unchanged as a GitHub Pages site or behind any static file server.
One thing to change when hosting: og:image and twitter:image are relative paths, which most
crawlers resolve but X/Twitter wants absolute. Replace them with the deployed absolute URL to get a
large social card.