| <!doctype html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <meta name="description" content="InferenceNet research leaderboard: single-pass generation, DeepAgents and an independent DeepSeek Harness result on 1,000 econometric replication tasks."> |
| <meta name="color-scheme" content="dark"> |
| <title>InferenceNet · Agent & Harness Leaderboard</title> |
| <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='6' fill='%230c1219'/%3E%3Ctext x='5' y='23' font-family='sans-serif' font-size='21' fill='%2378dbc0'%3EiN%3C/text%3E%3C/svg%3E"> |
| <link rel="stylesheet" href="./leaderboard.css"> |
| <script src="./leaderboard.js" defer></script> |
| </head> |
| <body> |
| <a class="skip" href="#rankings">Skip to rankings</a> |
| <main class="page"> |
| <nav class="topbar" aria-label="Main navigation"> |
| <a class="brand" href="https://easonai-5589.github.io/inferencenet-challenge/" target="_blank" rel="noopener noreferrer"><span class="brand-mark" aria-hidden="true">iN</span>InferenceNet</a> |
| <div class="nav-links"> |
| <a href="https://huggingface.co/datasets/CamoAiLab/InferenceNet" target="_blank" rel="noopener noreferrer" data-i18n="dataset">Dataset ↗</a> |
| <a href="https://huggingface.co/spaces/CamoAiLab/InferenceNet-Leaderboard/tree/main/results" target="_blank" rel="noopener noreferrer" data-i18n="archive">Results archive ↗</a> |
| <button id="language" type="button" aria-label="Switch to Chinese">中文</button> |
| </div> |
| </nav> |
|
|
| <header class="hero"> |
| <p class="eyebrow"><span class="dot" aria-hidden="true"></span><span data-i18n="edition">RESEARCH LEADERBOARD · SEPTEMBER 2026</span></p> |
| <h1 data-i18n="title">Models and harnesses.<br><span>Real research, reproduced.</span></h1> |
| <p class="intro" data-i18n="intro">Explore single-pass agents, DeepAgents and a separate DeepSeek Harness result on real econometric replication tasks.</p> |
| <div class="facts" aria-label="Dataset coverage"> |
| <span><strong id="model-count">—</strong><span data-i18n="models">requested model aliases</span></span> |
| <span><strong id="group-count">—</strong><span data-i18n="groups">experiment groups</span></span> |
| <span><strong id="task-count">—</strong><span data-i18n="tasks">tasks per group</span></span> |
| <span><strong id="record-count">—</strong><span data-i18n="records">task records</span></span> |
| <span class="update" id="published-date"></span> |
| </div> |
| </header> |
|
|
| <section class="protocols" aria-label="Experimental settings"> |
| <div><p class="protocol-tag baseline"><span aria-hidden="true">A</span><strong data-i18n="single">Single-pass Agent</strong></p><p data-i18n="singleDesc">One code-generation call, followed by execution. No iterative agent loop.</p></div> |
| <div><p class="protocol-tag harness"><span aria-hidden="true">B</span><strong data-i18n="deepagents">DeepAgents</strong></p><p data-i18n="harnessDesc">DeepAgents plans, uses tools and revises code. Up to 6 model calls, plus final execution.</p></div> |
| <div><p class="protocol-tag independent"><span aria-hidden="true">C</span><strong data-i18n="dsh">DeepSeek Harness · separate run</strong></p><p data-i18n="dshDesc">Official DSH SDK, requested v4-pro API alias. A fixed technical recovery selection; underlying model weights are unverified.</p></div> |
| </section> |
|
|
| <section id="rankings" aria-labelledby="rankings-title"> |
| <div class="section-head"><div><p class="eyebrow" data-i18n="resultsLabel">01 / RESULTS</p><h2 id="rankings-title" data-i18n="rankings">Agent & Harness leaderboard</h2></div><span class="badge" data-i18n="research">Research results · provisional</span></div> |
| <p class="scope-note" data-i18n="scope">Locally scored, archived results. Official scorer parity is not yet verified. Different budgets and run protocols mean the lift is descriptive, not an equal-cost comparison.</p> |
| <aside id="independent-notice" class="independent-notice" hidden><strong data-i18n="dshNoticeTitle">DSH is an independent result.</strong><p data-i18n="dshNotice">The request used deepseek-v4-pro; the underlying model weights were not verified. The fixed 1,000-task selection includes 87 technical replacement attempts (27 with changed input-schema prompts), with 30 task outcomes still unknown. It is excluded from the paired DeepAgents lift.</p><a id="dsh-protocol" data-i18n="dshProtocol" target="_blank" rel="noopener noreferrer">Read run protocol & evidence ↗</a></aside> |
|
|
| <div class="toolbar"> |
| <div class="view-switch" role="group" aria-label="Leaderboard view"> |
| <button type="button" data-view="compare" aria-pressed="false" data-i18n="compare">Baseline / DeepAgents pairs</button> |
| <button type="button" data-view="baseline" aria-pressed="false" data-i18n="single">Single-pass Agent</button> |
| <button type="button" data-view="harness" aria-pressed="true" data-i18n="harness">All harness runs</button> |
| </div> |
| <a class="download" href="./leaderboard.csv" download data-i18n="download">Download CSV ↓</a> |
| </div> |
| <div class="filters"> |
| <label class="metric-control"><span data-i18n="metric">Metric</span><select id="metric"> |
| <optgroup label="Local paper profile" data-group="local"> |
| <option value="perfect" data-i18n="perfect">Full replication · local</option> |
| </optgroup> |
| <optgroup label="HF-style profile · parity unverified" data-group="hf"> |
| <option value="partial_replication" data-i18n="partial_replication">Partial replication · HF-style</option> |
| <option value="compilation_success" data-i18n="compilation_success">Execution success · HF-style</option> |
| <option value="coefficient_direction" data-i18n="coefficient_direction">Coefficient direction · HF-style</option> |
| <option value="significance_level" data-i18n="significance_level">Significance level · HF-style</option> |
| </optgroup> |
| </select></label> |
| <label id="sort-control"><span data-i18n="sort">Sort by</span><select id="sort"><option value="deepagents" data-i18n="harnessScore">DeepAgents score</option><option value="baseline" data-i18n="singleScore">Single-pass score</option><option value="gain" data-i18n="lift">Lift</option></select></label> |
| <label class="search-control"><span data-i18n="search">Find a model</span><input id="search" type="search" placeholder="Search models…" autocomplete="off"></label> |
| </div> |
| <p id="metric-description" class="metric-description"></p> |
| <p id="load-state" role="status" class="load-state">Loading archived results…</p> |
| <div id="error" class="error" role="alert" hidden><p data-i18n="error">Results could not be loaded. Please retry or open the results archive.</p><button type="button" id="retry" data-i18n="retry">Retry</button></div> |
| <div id="table-wrap" class="table-wrap" tabindex="0" aria-label="Scrollable leaderboard" hidden> |
| <table><caption id="caption" class="sr-only"></caption><thead id="table-head"></thead><tbody id="table-body"></tbody></table> |
| </div> |
| <p id="empty" class="load-state" hidden data-i18n="empty">No matching models.</p> |
| <div class="table-foot"><span id="result-count" aria-live="polite"></span><span data-i18n="denominator">Every score uses all 1,000 tasks. Unknown and invalid outcomes earn no successes.</span></div> |
| </section> |
|
|
| <section class="method" aria-labelledby="method-title"> |
| <div class="section-head"><div><p class="eyebrow" data-i18n="methodLabel">02 / HOW TO READ THIS</p><h2 id="method-title" data-i18n="methodTitle">A comparison you can inspect.</h2></div><a href="./legacy.html" data-i18n="historical">Historical leaderboard ↗</a></div> |
| <div class="method-grid"> |
| <article><h3 data-i18n="fixedTasks">Fixed task set</h3><p data-i18n="fixedDesc">The same Selected_1000 task IDs and dataset revision are used for every group. Each score is successes ÷ 1,000; failed or unknown tasks stay in the denominator.</p></article> |
| <article><h3 data-i18n="separateMetrics">Two scoring profiles</h3><p data-i18n="profilesDesc">Full replication is the local paper metric. The four HF-style metrics reproduce the published descriptions locally; they are not yet verified against the official scorer.</p></article> |
| <article><h3 data-i18n="budgetTitle">Read lift with the budgets</h3><p data-i18n="budgetDesc">Baseline: 1 model call. DeepAgents and DSH: up to 6 model calls and 4 trial tool calls per attempt. DSH adds 87 recovery attempts; full environment and reasoning parity is unverified. DSH has no paired lift; these are not equal-cost causal estimates.</p></article> |
| </div> |
| <details class="evidence"><summary data-i18n="evidence">Evidence status & metric definitions</summary> |
| <p data-i18n="unknownDesc">“Metric unknown” means the selected metric cannot be assessed, including failed execution or unavailable outputs. It is different from a task whose final status is unknown. Neither is dropped or counted as a success.</p> |
| <ul id="definitions"></ul> |
| <div class="evidence-table-wrap"><table class="evidence-table"><caption data-i18n="evidenceCaption">Status from each accepted archive</caption><thead><tr><th data-i18n="model">Model / API alias</th><th data-i18n="protocol">Protocol</th><th data-i18n="evidenceCol">Evidence</th></tr></thead><tbody id="evidence-body"></tbody></table></div> |
| <p data-i18n="verification">HF-style counts are checked against archived task flags. Full-replication counts for Sol and Opus come from accepted summaries; all other displayed runs, including DSH, have per-task local flags. This refresh does not rerun or rescore any experiment.</p> |
| <p data-i18n="archiveNote">Per-run archive files remain frozen, including their original publication metadata. This page is a new display derived from those archives. GPT-5.5 and unarchived experiments are not included in this edition.</p> |
| </details> |
| </section> |
| <footer><span>InferenceNet <span class="muted">/</span> <span data-i18n="footer">Econometric replication, measured.</span></span><div><a id="revision-link" href="https://huggingface.co/spaces/CamoAiLab/InferenceNet-Leaderboard/tree/main/results" target="_blank" rel="noopener noreferrer" data-i18n="source">Source snapshot ↗</a><a href="./leaderboard-data.json" data-i18n="json">Data JSON ↗</a><a href="https://easonai-5589.github.io/inferencenet-challenge/" target="_blank" rel="noopener noreferrer" data-i18n="project">Project ↗</a></div></footer> |
| </main> |
| <noscript><p>This leaderboard needs JavaScript. <a href="./leaderboard.csv">Download the results CSV</a> or <a href="https://huggingface.co/spaces/CamoAiLab/InferenceNet-Leaderboard/tree/main/results">browse the archives</a>.</p></noscript> |
| </body> |
| </html> |
|
|
