File size: 8,936 Bytes
6efe2fe 8f03abe 6efe2fe | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 | <!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Explore RoleRouteBench live multi-agent routing trajectories, JEV reference results, and the public/private isolation protocol.">
<meta name="theme-color" content="#08111f">
<title>RoleRouteBench — Dynamic role routing, made inspectable</title>
<link rel="stylesheet" href="styles.css">
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "Dataset",
"name": "RoleRouteBench",
"alternateName": "RoutingBench V4-beta",
"description": "A reproducible multi-domain benchmark for dynamic role routing in multi-agent systems, with 82 live trajectories across four domains.",
"identifier": "https://doi.org/10.57967/hf/11430",
"url": "https://huggingface.co/datasets/TrangBui/RoleRouteBench",
"version": "4.0.0-beta.2",
"creator": {"@type": "Person", "name": "Trang Bui"},
"license": "https://huggingface.co/datasets/TrangBui/RoleRouteBench/blob/main/LICENSE_DATA",
"sameAs": [
"https://doi.org/10.57967/hf/11430",
"https://huggingface.co/datasets/TrangBui/RoleRouteBench",
"https://github.com/BuiThiThanhTrang/RACO_MAS"
],
"keywords": ["multi-agent systems", "dynamic routing", "agent orchestration", "tool use", "benchmark", "live trajectories"]
}
</script>
</head>
<body>
<a class="skip-link" href="#main">Skip to content</a>
<header class="site-header">
<a class="brand" href="#top" aria-label="RoleRouteBench home">
<span class="brand-mark">RR</span>
<span>RoleRouteBench</span>
</a>
<nav aria-label="Primary navigation">
<a href="#results">Results</a>
<a href="#explorer">Explorer</a>
<a href="#isolation">Isolation</a>
<a href="#submit">Submit</a>
</nav>
<a class="header-link" href="https://huggingface.co/datasets/TrangBui/RoleRouteBench">Dataset ↗</a>
</header>
<main id="main">
<section class="hero" id="top">
<div class="hero-copy">
<p class="eyebrow">Frozen environment · live consequences · multi-step tasks</p>
<h1>Dynamic role routing,<br><span>made inspectable.</span></h1>
<p class="hero-lede">RoleRouteBench evaluates whether a decision model can coordinate specialized executors across evolving task states—not merely whether the final answer happens to be correct.</p>
<div class="hero-actions">
<a class="button primary" href="#explorer">Explore trajectories</a>
<a class="button secondary" href="https://doi.org/10.57967/hf/11430">DOI 10.57967/hf/11430 ↗</a>
</div>
</div>
<div class="route-card" aria-label="Illustrative routing trajectory">
<div class="route-card-head"><span>packet / live state</span><span class="live-dot">LIVE</span></div>
<div class="route-step"><span class="step-index">01</span><div><strong>Inspect evidence</strong><small>state changed</small></div><span class="status success">success</span></div>
<div class="route-line"></div>
<div class="route-step"><span class="step-index">02</span><div><strong>Author candidate</strong><small>artifact created</small></div><span class="status success">success</span></div>
<div class="route-line warning"></div>
<div class="route-step"><span class="step-index">03</span><div><strong>Execute candidate</strong><small>failure preserved</small></div><span class="status failed">failed</span></div>
<div class="route-line"></div>
<div class="route-step"><span class="step-index">04</span><div><strong>Diagnose & repair</strong><small>new routed action</small></div><span class="status success">success</span></div>
</div>
</section>
<section class="stats" aria-label="Benchmark summary">
<div><strong>82</strong><span>live trajectories</span></div>
<div><strong>4</strong><span>task domains</span></div>
<div><strong>568</strong><span>primary decisions</span></div>
<div><strong>33</strong><span>specialized roles</span></div>
</section>
<section class="section why">
<div class="section-kicker">Why RoleRouteBench?</div>
<div class="why-grid">
<h2>Final task success cannot tell you whether a router coordinated well.</h2>
<div class="why-copy">
<p>Outcome quality also depends on the frozen executor. RoleRouteBench therefore retains every routing decision, failed action, state transition, budget event, and independent evaluator result.</p>
<p>Competing routers see the same public state, role pool, adapters, executors, budgets, and evaluator. The environment stays fixed; only the routing policy changes.</p>
</div>
</div>
</section>
<section class="section" id="results">
<div class="section-heading">
<div><div class="section-kicker">Reference system</div><h2>JEV 1.13 results</h2></div>
<p>Reference signals for environment validation—not an oracle and not a composite leaderboard score.</p>
</div>
<div class="table-wrap">
<table>
<thead><tr><th>Domain</th><th>Packets</th><th>Trajectory pass</th><th>Trusted task pass</th><th>Primary decisions</th><th>Budget-suppressed</th></tr></thead>
<tbody id="results-body"><tr><td colspan="6">Loading reference results…</td></tr></tbody>
</table>
</div>
</section>
<section class="section explorer-section" id="explorer">
<div class="section-heading">
<div><div class="section-kicker">Trajectory explorer</div><h2>Follow each routed consequence</h2></div>
<p>Filter public compact traces. Raw prompts and executor outputs are not included.</p>
</div>
<div class="filters" aria-label="Trajectory filters">
<label>Domain<select id="domain-filter"><option value="">All domains</option></select></label>
<label>Packet<select id="packet-filter"></select></label>
<label>Role<select id="role-filter"><option value="">All roles</option></select></label>
<label>Event status<select id="status-filter"><option value="">All statuses</option></select></label>
</div>
<div class="packet-summary" id="packet-summary" aria-live="polite"></div>
<div class="trajectory" id="trajectory" aria-live="polite"></div>
</section>
<section class="section isolation-section" id="isolation">
<div class="section-heading">
<div><div class="section-kicker">Evaluation boundary</div><h2>Public/private isolation by construction</h2></div>
<p>“Private” means evaluator-only under the benchmark protocol; the artifacts remain downloadable for transparent reproduction.</p>
</div>
<div class="isolation-grid">
<div class="boundary public-boundary">
<span class="boundary-label">ROUTER + EXECUTOR VISIBLE</span>
<h3>Public state</h3>
<ul><li>Task and public artifacts</li><li>Full role pool plus STOP</li><li>Prior observable consequences</li><li>Remaining call budget</li></ul>
</div>
<div class="boundary-flow" aria-hidden="true"><span>route</span><b>→</b><span>execute</span><b>→</b><span>observe</span></div>
<div class="boundary private-boundary">
<span class="boundary-label">EVALUATOR ONLY</span>
<h3>Reference state</h3>
<ul><li>Gold patches and proofs</li><li>Answer workbooks and oracles</li><li>Trusted evaluator results</li><li>Track A labels</li></ul>
</div>
</div>
<p class="boundary-note">The evaluator runs only after termination. It does not feed private artifacts back into the trajectory.</p>
</section>
<section class="section submit-section" id="submit">
<div>
<div class="section-kicker">Community evaluation</div>
<h2>Bring your own router.</h2>
<p>JEV is the reference system. Submit reproducible results from another decision model under the frozen environment—without sending us your model.</p>
</div>
<div class="submit-actions">
<a class="button primary" href="https://github.com/BuiThiThanhTrang/RACO_MAS/issues/new?template=router-result.yml">Submit router results ↗</a>
<a class="button secondary" href="https://github.com/BuiThiThanhTrang/RACO_MAS/blob/main/puppeteer/CONTRIBUTING_ROUTER_RESULTS.md">Read the protocol ↗</a>
</div>
</section>
</main>
<footer>
<div><strong>RoleRouteBench</strong><span>Version 4.0.0-beta.2 · DOI 10.57967/hf/11430</span></div>
<div class="footer-links"><a href="https://huggingface.co/datasets/TrangBui/RoleRouteBench">Dataset</a><a href="https://github.com/BuiThiThanhTrang/RACO_MAS">Code</a><a href="https://doi.org/10.57967/hf/11430">DOI</a></div>
</footer>
<script src="app.js" defer></script>
</body>
</html>
|