Display DSH as an independent harness result with verified metrics

Add a separate DSH arm while preserving all baseline and DeepAgents archives.

The fixed 1000-task selection includes 87 preselected technical supplements; all 1087 attempts and unknown usage are retained. The API alias is deepseek-v4-pro; served weights and strict official protocol parity are unverified. Full replication: 37.2%. HF-style metrics: 69.9%, 55.6%, 68.3%, 59.8%.

Publication reads sealed scores without model calls or rescoring. The display preserves existing paired comparisons and links to a pinned archive commit.

README.md

@@ -6,22 +6,38 @@ colorTo: blue
 sdk: static
 pinned: false
 license: apache-2.0
-short_description: Single-pass agents and DeepAgents on econometric replication
+short_description: Single-pass, DeepAgents and DSH replication results
 ---
 
 # InferenceNet Agent & Harness Leaderboard
 
-The September 2026 research leaderboard displays **six models, twelve groups,
-and 12,000 archived task records**. Switch between a paired comparison, a
-single-pass Agent ranking, and an Agent + Harness ranking. Every group uses
-the same 1,000 Selected_1000 task IDs.
+The September 2026 research leaderboard displays **six requested model aliases,
+thirteen groups, and 13,000 archived task records**. Switch between the original
+baseline / DeepAgents paired comparison, a single-pass Agent ranking, and a
+harness ranking that also includes one independent DSH run. Every group uses
+the same 1,000 Selected_1000 task IDs. An API alias does not establish the
+identity of the underlying model weights.
 
 - **Single-pass Agent:** one model call to generate code, followed by execution.
-- **Agent + Harness:** DeepAgents with up to six model calls and four trial tool
+- **DeepAgents:** up to six model calls and four trial tool
   calls, followed by final execution.
+- **DeepSeek Harness (DSH):** official SDK 0.1.5rc1, requested
+  `deepseek-v4-pro` API alias; displayed separately as
+  **DeepSeek API [v4-pro alias] + DSH**. Its underlying model weights are
+  **unverified**, and it is never used in the baseline / DeepAgents paired lift.
 
 Models: GPT-5.6 Sol, Claude Opus 4.8, Kimi K3, Gemini 3.1 Pro Preview,
-Qwen3.7-Max, and DeepSeek V4 Pro. GPT-5.5 is outside this edition.
+Qwen3.7-Max, and the historical DeepSeek V4 Pro entries. GPT-5.5 is outside this edition.
+
+The DSH [run protocol](./results/deepseek-v4-pro/dsh/README.md) documents its
+fixed selection of 1,000 results: 913 original attempts and 87 technical
+replacement attempts (60 infrastructure failures and 27 input-schema failures).
+The 27 schema recoveries changed the input-schema prompt. All 1,087 attempts
+remain in the cost accounting; selection is not best-of. The final selection
+has 699 scored, 271 failed and 30 uncertain task outcomes. Matching per-attempt
+numeric caps do not establish equal total run budgets: the DSH result includes
+87 additional attempts. Full execution-environment and reasoning-setting
+parity with the prior DeepAgents run has not been verified.
 
 ## Reading the results
 
@@ -43,21 +59,24 @@ See each run's configuration before making cross-model claims.
 
 Kimi retains 5 baseline and 4 harness task-status unknowns. Qwen3.7-Max harness
 retains 6 task-status unknowns and 1 invalid-evidence record (task 453), which is
-not sealed. Historical Sol/Opus interruption counts are shown as originally
+not sealed. DSH retains 30 task-status unknowns, with no invalid-evidence
+records in the fixed selection. Historical Sol/Opus interruption counts are shown as originally
 exported. An archive of 1,000 task IDs does not imply 1,000 definitive outcomes.
 
 ## Data and reproducibility
 
 - [`leaderboard-data.json`](./leaderboard-data.json): displayed data, metric
   profiles, counts, evidence status and source-file SHA-256 hashes.
-- [`leaderboard.csv`](./leaderboard.csv): 60 displayed metric rows; six models ×
-  two arms × five metrics. One metric profile is explicit on every row.
+- [`leaderboard.csv`](./leaderboard.csv): 65 displayed metric rows; thirteen
+  groups × five metrics. One metric profile is explicit on every row.
 - [`results/`](./results): frozen per-task archives, configurations and summaries.
 - [`legacy.html`](./legacy.html): original leaderboard with a historical banner.
   Its original [`results.csv`](./results.csv) is unchanged and is not mixed into
   the new comparison.
 
-Source archive commit: `731111974cf4d1a3c7a4f848a899f0b0708dd12b`.
+The exact source archive commit is recorded as `source_revision` in
+[`leaderboard-data.json`](./leaderboard-data.json); every displayed evidence
+link is pinned to that commit.
 Dataset: [CamoAiLab/InferenceNet](https://huggingface.co/datasets/CamoAiLab/InferenceNet),
 revision `59f9512a38e594528807744214a60ee00367434e`, task list
 `Selected_1000/1000_new.csv`.
@@ -71,10 +90,11 @@ To rebuild the display exports from those archives:
 
 ```sh
 python build_leaderboard.py --source . --output . \
-  --source-revision 731111974cf4d1a3c7a4f848a899f0b0708dd12b
+  --source-revision ARCHIVE_COMMIT_SHA --published-date 2026-09-20
 ```
 
-The builder verifies all task populations and 56 metric counts against archived
+Use the 40-character archive commit SHA from `leaderboard-data.json` above.
+The builder verifies all task populations and 61 metric counts against archived
 flags. The four Sol/Opus local full-replication summaries have no per-task local
 flags in the archive and are preserved from their accepted summaries. This is a
 display refresh, not a fresh inference run, native artifact audit, or rescore.

build_leaderboard.py

@@ -7,6 +7,7 @@ import argparse
 import csv
 import hashlib
 import json
+from datetime import date
 from pathlib import Path
 
 NAMES = {
@@ -18,30 +19,45 @@ NAMES = {
     "deepseek-v4-pro": "DeepSeek V4 Pro",
 }
 METRICS = ("compilation_success", "partial_replication", "coefficient_direction", "significance_level")
+PAIRED_ARMS = ("baseline", "deepagents")
+DSH_ENTRY = ("deepseek-v4-pro", "dsh")
+DSH_NAME = "DeepSeek API [v4-pro alias] + DSH"
+DSH_NOTICE = (
+    "Requested API alias: deepseek-v4-pro. Underlying model weights are unverified. "
+    "This technical recovery result is a separate harness entry, not a paired DeepAgents gain."
+)
 
 
-def build(source, output, revision):
+def build(source, output, revision, published_date=None):
     assert len(revision) == 40 and all(c in "0123456789abcdef" for c in revision)
-    index = json.loads((source / "results/index.json").read_text())
+    index = json.loads((source / "results/index.json").read_text(encoding="utf-8"))
     entries = index["entries"]
-    assert {(e["model"], e["arm"]) for e in entries} == {(m, a) for m in NAMES for a in ("baseline", "deepagents")}
-    assert len(entries) == 12
+    pairs = {(e["model"], e["arm"]) for e in entries}
+    original_pairs = {(m, a) for m in NAMES for a in PAIRED_ARMS}
+    assert original_pairs <= pairs <= original_pairs | {DSH_ENTRY}
+    assert len(entries) == len(pairs), "Duplicate archive entry"
     models = {m: {"id": m, "name": name, "arms": {}} for m, name in NAMES.items()}
     population = None
     dataset_revision = None
     source_files = {}
+    checked_metric_count = accepted_summary_count = records_count = 0
     for entry in entries:
         path = Path(entry["path"])
         assert path == Path("results") / entry["model"] / entry["arm"]
-        summary = json.loads((source / path / "summary.json").read_text())
-        config = json.loads((source / path / "config.json").read_text())
-        records = [json.loads(line) for line in (source / path / "results.jsonl").read_text().splitlines()]
+        summary = json.loads((source / path / "summary.json").read_text(encoding="utf-8"))
+        config = json.loads((source / path / "config.json").read_text(encoding="utf-8"))
+        records = [json.loads(line) for line in (source / path / "results.jsonl").read_text(encoding="utf-8").splitlines()]
         ids = {r["task_id"] for r in records}
         assert len(records) == len(ids) == entry["rows"] == summary["result"]["expected_count"] == 1000
         assert ids == set(config["dataset"]["task_ids"])
         assert all(r["model"] == entry["model"] and r["arm"] == entry["arm"] for r in records)
         assert (summary["model"], summary["arm"]) == (entry["model"], entry["arm"])
         assert summary["official_parity_verified"] is False
+        independent = (entry["model"], entry["arm"]) == DSH_ENTRY
+        if independent:
+            assert config["model_alias"] == "deepseek-v4-pro"
+            assert config["underlying_model_weights_verified"] is False
+            assert summary["evidence_class"] == "technical_recovery_view"
         if population is None:
             population = ids
             dataset_revision = summary["dataset_revision"]
@@ -55,10 +71,15 @@ def build(source, output, revision):
             assert abs(m["rate"] - count / n) < 1e-12
             field = "local_perfect" if local else metric
             has_flags = all(field in r for r in records)
+            if independent:
+                assert has_flags, "Every DSH displayed metric requires an archived task flag"
             if not local or has_flags:
                 assert all(r[field] is None or isinstance(r[field], bool) for r in records)
                 assert sum(r[field] is True for r in records) == count
                 assert sum(r[field] is None for r in records) == unknown
+                checked_metric_count += 1
+            else:
+                accepted_summary_count += 1
             # Sol/Opus local flags were not exported: preserve their accepted
             # summary counts, without claiming a fresh per-task recomputation.
             metrics[metric] = {
@@ -77,29 +98,37 @@ def build(source, output, revision):
             "interruption_count": summary["result"].get("interruption_count"),
             "evidence_class": summary["evidence_class"],
         }
+        if independent:
+            arm.update(display_name=DSH_NAME, identity_notice=DSH_NOTICE, paired_comparison=False,
+                       model_alias=config["model_alias"], underlying_model_weights_verified=False,
+                       comparison_group="independent", protocol_url=(path / "README.md").as_posix())
         models[entry["model"]]["arms"][entry["arm"]] = arm
+        records_count += len(records)
         for name in ("summary.json", "results.jsonl", "config.json"):
             rel = (path / name).as_posix()
             source_files[rel] = hashlib.sha256((source / rel).read_bytes()).hexdigest()
     data = {
         "schema_version": 1, "kind": "research_leaderboard",
-        "published_date": "2026-09-17", "source_repository": "CamoAiLab/InferenceNet-Leaderboard",
+        "published_date": published_date or date.today().isoformat(), "source_repository": "CamoAiLab/InferenceNet-Leaderboard",
         "source_revision": revision, "dataset_revision": dataset_revision,
         "task_list": "Selected_1000/1000_new.csv", "tasks_per_group": 1000,
-        "groups": 12, "records": 12000, "official_parity_verified": False,
+        "groups": len(entries), "records": records_count, "model_aliases": len(models),
+        "paired_groups": len(original_pairs), "independent_groups": len(pairs-original_pairs),
+        "paired_arms": list(PAIRED_ARMS), "official_parity_verified": False,
         "models": list(models.values()), "source_sha256": source_files,
     }
     output.mkdir(parents=True, exist_ok=True)
-    (output / "leaderboard-data.json").write_text(json.dumps(data, indent=2) + "\n")
-    with (output / "leaderboard.csv").open("w", newline="") as f:
+    (output / "leaderboard-data.json").write_text(json.dumps(data, indent=2) + "\n", encoding="utf-8")
+    with (output / "leaderboard.csv").open("w", newline="", encoding="utf-8") as f:
         writer = csv.writer(f)
-        writer.writerow(["model", "arm", "harness", "metric_profile", "metric", "successes", "denominator", "unknown", "score_percent", "official_parity_verified", "source_revision"])
+        writer.writerow(["model", "arm", "harness", "metric_profile", "metric", "successes", "denominator", "unknown", "score_percent", "official_parity_verified", "source_revision", "display_name", "comparison_group", "underlying_model_weights_verified"])
         for model in data["models"]:
             for arm_id, arm in model["arms"].items():
                 for key, metric in arm["metrics"].items():
-                    writer.writerow([model["id"], arm_id, arm["harness"], metric["profile"], key, metric["count"], 1000, metric["unknown"], metric["score"], False, revision])
-    print("Verified: 6 models, 12 arms, 12,000 unique-per-arm records; 60 displayed metric summaries.")
-    print("56 metric summaries checked against task flags; 4 historical local summaries preserved as accepted.")
+                    writer.writerow([model["id"], arm_id, arm["harness"], metric["profile"], key, metric["count"], 1000, metric["unknown"], metric["score"], False, revision, arm.get("display_name", model["name"]), arm.get("comparison_group", "historical_pair"), arm.get("underlying_model_weights_verified", "")])
+    print(f"Verified: {len(models)} requested model aliases, {len(entries)} arms, {records_count:,} records; {len(entries)*5} displayed metric summaries.")
+    print(f"{checked_metric_count} metric summaries checked against task flags; {accepted_summary_count} historical local summaries preserved as accepted.")
+    return data
 
 
 if __name__ == "__main__":
@@ -107,5 +136,6 @@ if __name__ == "__main__":
     parser.add_argument("--source", type=Path, default=Path("."))
     parser.add_argument("--output", type=Path, default=Path("."))
     parser.add_argument("--source-revision", required=True)
+    parser.add_argument("--published-date", help="Publication date, YYYY-MM-DD; defaults to today")
     args = parser.parse_args()
-    build(args.source, args.output, args.source_revision)
+    build(args.source, args.output, args.source_revision, args.published_date)

index.html

@@ -3,7 +3,7 @@
 <head>
   <meta charset="utf-8">
   <meta name="viewport" content="width=device-width, initial-scale=1">
-  <meta name="description" content="InferenceNet research leaderboard: six models compared with single-pass generation and a DeepAgents harness on 1,000 econometric replication tasks.">
+  <meta name="description" content="InferenceNet research leaderboard: single-pass generation, DeepAgents and an independent DeepSeek Harness result on 1,000 econometric replication tasks.">
   <meta name="color-scheme" content="dark">
   <title>InferenceNet · Agent & Harness Leaderboard</title>
   <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='6' fill='%230c1219'/%3E%3Ctext x='5' y='23' font-family='sans-serif' font-size='21' fill='%2378dbc0'%3EiN%3C/text%3E%3C/svg%3E">
@@ -24,30 +24,33 @@
 
     <header class="hero">
       <p class="eyebrow"><span class="dot" aria-hidden="true"></span><span data-i18n="edition">RESEARCH LEADERBOARD · SEPTEMBER 2026</span></p>
-      <h1 data-i18n="title">Same models.<br><span>Two ways to solve.</span></h1>
-      <p class="intro" data-i18n="intro">Compare single-pass agents with a DeepAgents harness on real econometric replication tasks.</p>
+      <h1 data-i18n="title">Models and harnesses.<br><span>Real research, reproduced.</span></h1>
+      <p class="intro" data-i18n="intro">Explore single-pass agents, DeepAgents and a separate DeepSeek Harness result on real econometric replication tasks.</p>
       <div class="facts" aria-label="Dataset coverage">
-        <span><strong id="model-count">—</strong><span data-i18n="models">models</span></span>
+        <span><strong id="model-count">—</strong><span data-i18n="models">requested model aliases</span></span>
         <span><strong id="group-count">—</strong><span data-i18n="groups">experiment groups</span></span>
         <span><strong id="task-count">—</strong><span data-i18n="tasks">tasks per group</span></span>
-        <span class="update" data-i18n="updated">Published 17 Sep 2026</span>
+        <span><strong id="record-count">—</strong><span data-i18n="records">task records</span></span>
+        <span class="update" id="published-date"></span>
       </div>
     </header>
 
     <section class="protocols" aria-label="Experimental settings">
       <div><p class="protocol-tag baseline"><span aria-hidden="true">A</span><strong data-i18n="single">Single-pass Agent</strong></p><p data-i18n="singleDesc">One code-generation call, followed by execution. No iterative agent loop.</p></div>
-      <div><p class="protocol-tag harness"><span aria-hidden="true">B</span><strong data-i18n="harness">Agent + Harness</strong></p><p data-i18n="harnessDesc">DeepAgents plans, uses tools and revises code. Up to 6 model calls, plus final execution.</p></div>
+      <div><p class="protocol-tag harness"><span aria-hidden="true">B</span><strong data-i18n="deepagents">DeepAgents</strong></p><p data-i18n="harnessDesc">DeepAgents plans, uses tools and revises code. Up to 6 model calls, plus final execution.</p></div>
+      <div><p class="protocol-tag independent"><span aria-hidden="true">C</span><strong data-i18n="dsh">DeepSeek Harness · separate run</strong></p><p data-i18n="dshDesc">Official DSH SDK, requested v4-pro API alias. A fixed technical recovery selection; underlying model weights are unverified.</p></div>
     </section>
 
     <section id="rankings" aria-labelledby="rankings-title">
       <div class="section-head"><div><p class="eyebrow" data-i18n="resultsLabel">01 / RESULTS</p><h2 id="rankings-title" data-i18n="rankings">Agent & Harness leaderboard</h2></div><span class="badge" data-i18n="research">Research results · provisional</span></div>
       <p class="scope-note" data-i18n="scope">Locally scored, archived results. Official scorer parity is not yet verified. Different budgets and run protocols mean the lift is descriptive, not an equal-cost comparison.</p>
+      <aside id="independent-notice" class="independent-notice" hidden><strong data-i18n="dshNoticeTitle">DSH is an independent result.</strong><p data-i18n="dshNotice">The request used deepseek-v4-pro; the underlying model weights were not verified. The fixed 1,000-task selection includes 87 technical replacement attempts (27 with changed input-schema prompts), with 30 task outcomes still unknown. It is excluded from the paired DeepAgents lift.</p><a id="dsh-protocol" data-i18n="dshProtocol" target="_blank" rel="noopener noreferrer">Read run protocol & evidence ↗</a></aside>
 
       <div class="toolbar">
         <div class="view-switch" role="group" aria-label="Leaderboard view">
-          <button type="button" data-view="compare" aria-pressed="true" data-i18n="compare">Side by side</button>
+          <button type="button" data-view="compare" aria-pressed="false" data-i18n="compare">Baseline / DeepAgents pairs</button>
           <button type="button" data-view="baseline" aria-pressed="false" data-i18n="single">Single-pass Agent</button>
-          <button type="button" data-view="deepagents" aria-pressed="false" data-i18n="harness">Agent + Harness</button>
+          <button type="button" data-view="harness" aria-pressed="true" data-i18n="harness">All harness runs</button>
         </div>
         <a class="download" href="./leaderboard.csv" download data-i18n="download">Download CSV ↓</a>
       </div>
@@ -63,7 +66,7 @@
             <option value="significance_level" data-i18n="significance_level">Significance level · HF-style</option>
           </optgroup>
         </select></label>
-        <label id="sort-control"><span data-i18n="sort">Sort by</span><select id="sort"><option value="deepagents" data-i18n="harnessScore">Harness score</option><option value="baseline" data-i18n="singleScore">Single-pass score</option><option value="gain" data-i18n="lift">Lift</option></select></label>
+        <label id="sort-control"><span data-i18n="sort">Sort by</span><select id="sort"><option value="deepagents" data-i18n="harnessScore">DeepAgents score</option><option value="baseline" data-i18n="singleScore">Single-pass score</option><option value="gain" data-i18n="lift">Lift</option></select></label>
         <label class="search-control"><span data-i18n="search">Find a model</span><input id="search" type="search" placeholder="Search models…" autocomplete="off"></label>
       </div>
       <p id="metric-description" class="metric-description"></p>
@@ -81,13 +84,13 @@
       <div class="method-grid">
         <article><h3 data-i18n="fixedTasks">Fixed task set</h3><p data-i18n="fixedDesc">The same Selected_1000 task IDs and dataset revision are used for every group. Each score is successes ÷ 1,000; failed or unknown tasks stay in the denominator.</p></article>
         <article><h3 data-i18n="separateMetrics">Two scoring profiles</h3><p data-i18n="profilesDesc">Full replication is the local paper metric. The four HF-style metrics reproduce the published descriptions locally; they are not yet verified against the official scorer.</p></article>
-        <article><h3 data-i18n="budgetTitle">Read lift with the budgets</h3><p data-i18n="budgetDesc">Baseline: 1 model call. DeepAgents: up to 6 model calls and 4 trial tool calls. Recovery amendments, time limits and run protocols differ; these are not equal-cost causal estimates.</p></article>
+        <article><h3 data-i18n="budgetTitle">Read lift with the budgets</h3><p data-i18n="budgetDesc">Baseline: 1 model call. DeepAgents and DSH: up to 6 model calls and 4 trial tool calls per attempt. DSH adds 87 recovery attempts; full environment and reasoning parity is unverified. DSH has no paired lift; these are not equal-cost causal estimates.</p></article>
       </div>
       <details class="evidence"><summary data-i18n="evidence">Evidence status & metric definitions</summary>
         <p data-i18n="unknownDesc">“Metric unknown” means the selected metric cannot be assessed, including failed execution or unavailable outputs. It is different from a task whose final status is unknown. Neither is dropped or counted as a success.</p>
         <ul id="definitions"></ul>
-        <div class="evidence-table-wrap"><table class="evidence-table"><caption data-i18n="evidenceCaption">Status from each accepted archive</caption><thead><tr><th data-i18n="model">Model</th><th data-i18n="single">Single-pass Agent</th><th data-i18n="harness">Agent + Harness</th></tr></thead><tbody id="evidence-body"></tbody></table></div>
-        <p data-i18n="verification">HF-style counts are checked against archived task flags. Full-replication counts for Sol and Opus come from accepted summaries; the other four models also have per-task local flags. This refresh does not rerun or rescore any experiment.</p>
+        <div class="evidence-table-wrap"><table class="evidence-table"><caption data-i18n="evidenceCaption">Status from each accepted archive</caption><thead><tr><th data-i18n="model">Model / API alias</th><th data-i18n="protocol">Protocol</th><th data-i18n="evidenceCol">Evidence</th></tr></thead><tbody id="evidence-body"></tbody></table></div>
+        <p data-i18n="verification">HF-style counts are checked against archived task flags. Full-replication counts for Sol and Opus come from accepted summaries; all other displayed runs, including DSH, have per-task local flags. This refresh does not rerun or rescore any experiment.</p>
         <p data-i18n="archiveNote">Per-run archive files remain frozen, including their original publication metadata. This page is a new display derived from those archives. GPT-5.5 and unarchived experiments are not included in this edition.</p>
       </details>
     </section>

leaderboard-data.json

@@ -1,785 +1,860 @@
-{
-  "schema_version": 1,
-  "kind": "research_leaderboard",
-  "published_date": "2026-09-17",
-  "source_repository": "CamoAiLab/InferenceNet-Leaderboard",
-  "source_revision": "731111974cf4d1a3c7a4f848a899f0b0708dd12b",
-  "dataset_revision": "59f9512a38e594528807744214a60ee00367434e",
-  "task_list": "Selected_1000/1000_new.csv",
-  "tasks_per_group": 1000,
-  "groups": 12,
-  "records": 12000,
-  "official_parity_verified": false,
-  "models": [
-    {
-      "id": "gpt-5.6-sol",
-      "name": "GPT-5.6 Sol",
-      "arms": {
-        "baseline": {
-          "path": "results/gpt-5.6-sol/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 338,
-              "denominator": 1000,
-              "unknown": 445,
-              "failure": 217,
-              "score": 33.8,
-              "profile": "local-paper-v1",
-              "verification": "accepted_summary"
-            },
-            "compilation_success": {
-              "count": 557,
-              "denominator": 1000,
-              "unknown": 3,
-              "failure": 440,
-              "score": 55.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 430,
-              "denominator": 1000,
-              "unknown": 444,
-              "failure": 126,
-              "score": 43.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 537,
-              "denominator": 1000,
-              "unknown": 444,
-              "failure": 19,
-              "score": 53.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 485,
-              "denominator": 1000,
-              "unknown": 444,
-              "failure": 71,
-              "score": 48.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-13T17:49:16.537733+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": null,
-          "invalid_evidence_count": 0,
-          "interruption_count": 3,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/gpt-5.6-sol/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 518,
-              "denominator": 1000,
-              "unknown": 95,
-              "failure": 387,
-              "score": 51.8,
-              "profile": "local-paper-v1",
-              "verification": "accepted_summary"
-            },
-            "compilation_success": {
-              "count": 910,
-              "denominator": 1000,
-              "unknown": 10,
-              "failure": 80,
-              "score": 91.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 699,
-              "denominator": 1000,
-              "unknown": 92,
-              "failure": 209,
-              "score": 69.9,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 879,
-              "denominator": 1000,
-              "unknown": 92,
-              "failure": 29,
-              "score": 87.9,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 787,
-              "denominator": 1000,
-              "unknown": 93,
-              "failure": 120,
-              "score": 78.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-13T17:49:16.537733+00:00",
-          "complete": true,
-          "harness": "DeepAgents",
-          "model_call_cap": 6,
-          "task_unknown_count": null,
-          "invalid_evidence_count": 0,
-          "interruption_count": 10,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "claude-opus-4-8",
-      "name": "Claude Opus 4.8",
-      "arms": {
-        "baseline": {
-          "path": "results/claude-opus-4-8/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 184,
-              "denominator": 1000,
-              "unknown": 538,
-              "failure": 278,
-              "score": 18.4,
-              "profile": "local-paper-v1",
-              "verification": "accepted_summary"
-            },
-            "compilation_success": {
-              "count": 467,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 533,
-              "score": 46.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 305,
-              "denominator": 1000,
-              "unknown": 537,
-              "failure": 158,
-              "score": 30.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 437,
-              "denominator": 1000,
-              "unknown": 537,
-              "failure": 26,
-              "score": 43.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 370,
-              "denominator": 1000,
-              "unknown": 537,
-              "failure": 93,
-              "score": 37.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-13T17:50:43.870921+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": null,
-          "invalid_evidence_count": 0,
-          "interruption_count": 0,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/claude-opus-4-8/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 392,
-              "denominator": 1000,
-              "unknown": 174,
-              "failure": 434,
-              "score": 39.2,
-              "profile": "local-paper-v1",
-              "verification": "accepted_summary"
-            },
-            "compilation_success": {
-              "count": 834,
-              "denominator": 1000,
-              "unknown": 4,
-              "failure": 162,
-              "score": 83.4,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 621,
-              "denominator": 1000,
-              "unknown": 170,
-              "failure": 209,
-              "score": 62.1,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 793,
-              "denominator": 1000,
-              "unknown": 170,
-              "failure": 37,
-              "score": 79.3,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 696,
-              "denominator": 1000,
-              "unknown": 172,
-              "failure": 132,
-              "score": 69.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-13T17:50:43.870921+00:00",
-          "complete": true,
-          "harness": "DeepAgents",
-          "model_call_cap": 6,
-          "task_unknown_count": null,
-          "invalid_evidence_count": 0,
-          "interruption_count": 4,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "kimi-k3",
-      "name": "Kimi K3",
-      "arms": {
-        "baseline": {
-          "path": "results/kimi-k3/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 230,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 289,
-              "score": 23.0,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 526,
-              "denominator": 1000,
-              "unknown": 5,
-              "failure": 469,
-              "score": 52.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 366,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 153,
-              "score": 36.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 494,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 25,
-              "score": 49.4,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 431,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 88,
-              "score": 43.1,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T15:28:00.953066+00:00",
-          "complete": false,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": 5,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/kimi-k3/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 390,
-              "denominator": 1000,
-              "unknown": 159,
-              "failure": 451,
-              "score": 39.0,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 867,
-              "denominator": 1000,
-              "unknown": 4,
-              "failure": 129,
-              "score": 86.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 635,
-              "denominator": 1000,
-              "unknown": 156,
-              "failure": 209,
-              "score": 63.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 804,
-              "denominator": 1000,
-              "unknown": 156,
-              "failure": 40,
-              "score": 80.4,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 709,
-              "denominator": 1000,
-              "unknown": 157,
-              "failure": 134,
-              "score": 70.9,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T15:28:00.953066+00:00",
-          "complete": false,
-          "harness": "DeepAgents 0.7.13",
-          "model_call_cap": 6,
-          "task_unknown_count": 4,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "gemini-3.1-pro-preview",
-      "name": "Gemini 3.1 Pro Preview",
-      "arms": {
-        "baseline": {
-          "path": "results/gemini-3.1-pro-preview/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 246,
-              "denominator": 1000,
-              "unknown": 462,
-              "failure": 292,
-              "score": 24.6,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 543,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 457,
-              "score": 54.3,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 366,
-              "denominator": 1000,
-              "unknown": 460,
-              "failure": 174,
-              "score": 36.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 506,
-              "denominator": 1000,
-              "unknown": 460,
-              "failure": 34,
-              "score": 50.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 437,
-              "denominator": 1000,
-              "unknown": 460,
-              "failure": 103,
-              "score": 43.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T15:28:00.953066+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/gemini-3.1-pro-preview/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 391,
-              "denominator": 1000,
-              "unknown": 155,
-              "failure": 454,
-              "score": 39.1,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 850,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 150,
-              "score": 85.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 632,
-              "denominator": 1000,
-              "unknown": 153,
-              "failure": 215,
-              "score": 63.2,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 811,
-              "denominator": 1000,
-              "unknown": 153,
-              "failure": 36,
-              "score": 81.1,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 718,
-              "denominator": 1000,
-              "unknown": 154,
-              "failure": 128,
-              "score": 71.8,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T15:28:00.953066+00:00",
-          "complete": true,
-          "harness": "DeepAgents 0.7.13",
-          "model_call_cap": 6,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "qwen3.7-max",
-      "name": "Qwen3.7-Max",
-      "arms": {
-        "baseline": {
-          "path": "results/qwen3.7-max/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 198,
-              "denominator": 1000,
-              "unknown": 519,
-              "failure": 283,
-              "score": 19.8,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 486,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 514,
-              "score": 48.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 315,
-              "denominator": 1000,
-              "unknown": 518,
-              "failure": 167,
-              "score": 31.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 454,
-              "denominator": 1000,
-              "unknown": 518,
-              "failure": 28,
-              "score": 45.4,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 396,
-              "denominator": 1000,
-              "unknown": 518,
-              "failure": 86,
-              "score": 39.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T17:18:47.373930+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/qwen3.7-max/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 248,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 271,
-              "score": 24.8,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 523,
-              "denominator": 1000,
-              "unknown": 7,
-              "failure": 470,
-              "score": 52.3,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 379,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 140,
-              "score": 37.9,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 488,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 31,
-              "score": 48.8,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 433,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 86,
-              "score": 43.3,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T17:18:47.373930+00:00",
-          "complete": false,
-          "harness": "DeepAgents 0.7.13",
-          "model_call_cap": 6,
-          "task_unknown_count": 6,
-          "invalid_evidence_count": 1,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "deepseek-v4-pro",
-      "name": "DeepSeek V4 Pro",
-      "arms": {
-        "baseline": {
-          "path": "results/deepseek-v4-pro/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 129,
-              "denominator": 1000,
-              "unknown": 723,
-              "failure": 148,
-              "score": 12.9,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 281,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 719,
-              "score": 28.1,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 195,
-              "denominator": 1000,
-              "unknown": 723,
-              "failure": 82,
-              "score": 19.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 260,
-              "denominator": 1000,
-              "unknown": 723,
-              "failure": 17,
-              "score": 26.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 236,
-              "denominator": 1000,
-              "unknown": 723,
-              "failure": 41,
-              "score": 23.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T17:18:47.373930+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/deepseek-v4-pro/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 183,
-              "denominator": 1000,
-              "unknown": 648,
-              "failure": 169,
-              "score": 18.3,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 352,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 648,
-              "score": 35.2,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 270,
-              "denominator": 1000,
-              "unknown": 648,
-              "failure": 82,
-              "score": 27.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 338,
-              "denominator": 1000,
-              "unknown": 648,
-              "failure": 14,
-              "score": 33.8,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 308,
-              "denominator": 1000,
-              "unknown": 648,
-              "failure": 44,
-              "score": 30.8,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T17:18:47.373930+00:00",
-          "complete": true,
-          "harness": "DeepAgents 0.7.13",
-          "model_call_cap": 6,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    }
-  ],
-  "source_sha256": {
-    "results/gpt-5.6-sol/baseline/summary.json": "6d6ab1aaffe754e08f9656dcb5f51fcd5c772e60660dc34d86cfc21f3c5c03d7",
-    "results/gpt-5.6-sol/baseline/results.jsonl": "3b12160382043e27531ff5cb3e23bccc9a8f7807e4121a82ffa13df28f65a859",
-    "results/gpt-5.6-sol/baseline/config.json": "90ec4e0781b57498fef52de31eacf5c8ec9424549643865a7140a1ccdb04c1d4",
-    "results/gpt-5.6-sol/deepagents/summary.json": "821ca6e55a66006c7fb958951a7859a9dccde29d87de8415d5c3f07f523563c5",
-    "results/gpt-5.6-sol/deepagents/results.jsonl": "378cdec8e7a880f091ba442649d1c034997acefdc623898b60a063baad29d7be",
-    "results/gpt-5.6-sol/deepagents/config.json": "a3e24175f2395cffbbcd6d9d3c89689bd3a3b22dd4ce61bb78d5da33c6b6cc1b",
-    "results/claude-opus-4-8/baseline/summary.json": "7b5b06bd9b3d7b47952b1dc424a01fcdf04ddccca0b3f64d04132358d6380cdc",
-    "results/claude-opus-4-8/baseline/results.jsonl": "b9d6bd51c9eb81a634e78e422c86da16f655d97da87d3f5afa4c9aa0f96809f4",
-    "results/claude-opus-4-8/baseline/config.json": "8058afcd14c34d666277c0f32ac5962398b10618df5b1aa0e13cd9862344dc56",
-    "results/claude-opus-4-8/deepagents/summary.json": "f86da9c76f8becb75b4970c486160013209a6aa4239923018b217e55f1b0a614",
-    "results/claude-opus-4-8/deepagents/results.jsonl": "3bef3b9850f24b9cb0b5806efd8425c4e4cc650735a6b28b45befdd55a6ec28a",
-    "results/claude-opus-4-8/deepagents/config.json": "12215dfcba5d2d692268a305364514a03ff7579094b2d4af12f493383b8e0f21",
-    "results/kimi-k3/baseline/summary.json": "2edd2e96f3b448ceceb6189edb7f23c737b245c00c9820728c69f5237feb34b7",
-    "results/kimi-k3/baseline/results.jsonl": "0502ae8892dd6c66ee92890a83c678527cff6c3396aa4c13faaadcf41e1857c1",
-    "results/kimi-k3/baseline/config.json": "7191c406e8695a5cb760ee8fa99c4fb0dd71bca59347acc4d6d33fe0630be081",
-    "results/kimi-k3/deepagents/summary.json": "81c8cabaa58acc65ef6afba792241b70951b666ab90c3f7c25194b61b0d9a228",
-    "results/kimi-k3/deepagents/results.jsonl": "bdab99f8f1d81885f74326cf9c8c8e57b9bb8a8d616bfbb1b5a2f6bf32b737ca",
-    "results/kimi-k3/deepagents/config.json": "312c9260c01d5988878e4f2b88da55a16362ee994e8217c63b15ba772da3857c",
-    "results/gemini-3.1-pro-preview/baseline/summary.json": "a8ba5100a28f1f0aeae6949eb3669c7717c844c4e2eee6fa42ded351aef6b5c7",
-    "results/gemini-3.1-pro-preview/baseline/results.jsonl": "2f1e2019157d105efee7ba311a49eba99abe4a7b84f0a4b4e0ac174e2d56cdba",
-    "results/gemini-3.1-pro-preview/baseline/config.json": "ff9884d2095695a1f40f5fc573982b9a49fd53d41dca2a6fc36d04dc81f10998",
-    "results/gemini-3.1-pro-preview/deepagents/summary.json": "bf9ce721c94103f91b0ca576f717f4f5dca7cc9965601658a65fc6d2a15b7045",
-    "results/gemini-3.1-pro-preview/deepagents/results.jsonl": "c2824f8277b283a6dd60118b6e6a0cb0419e4c8b9193de15a7bcf7aaaad24181",
-    "results/gemini-3.1-pro-preview/deepagents/config.json": "f09d9d99d26328ae2f54e9173b22f69d68f7d7bb50c3cc5e352c996b98dd57f9",
-    "results/qwen3.7-max/baseline/summary.json": "e506814d33789673fcd6b8e6869cdcbd52371cd3d0455ce53ae27e1098a52214",
-    "results/qwen3.7-max/baseline/results.jsonl": "4924e929d67956f9deb6c319605042d09f40b12f7bd2f18baa0464b8f7c9851a",
-    "results/qwen3.7-max/baseline/config.json": "9f553440e741863e82f32fec11470affe6f0dbffa03c668c647bd63d8f73227f",
-    "results/qwen3.7-max/deepagents/summary.json": "a5dcb2297ccc31a921e3f4eeb025ed81a764e16556a12068733cfe3bfe0bdbe3",
-    "results/qwen3.7-max/deepagents/results.jsonl": "e009f8c5c88b05896836330822afbda33af463c1f264f9adb9bbf5b094e61dfa",
-    "results/qwen3.7-max/deepagents/config.json": "3862d438d381ca3e57b1bf68921ef01a8c38727902152ada2c6e1fcf7b70cc77",
-    "results/deepseek-v4-pro/baseline/summary.json": "c51e4f1de78eba8f30e4a59276fb7821132cd103f0ae3f9d04c75c98344f2e34",
-    "results/deepseek-v4-pro/baseline/results.jsonl": "7827218f6f18f39988e8c439f0c93cab611c04d158d8704aa50c9274cd438837",
-    "results/deepseek-v4-pro/baseline/config.json": "4cef524f37a325029ea4ad325b163f72f9cf725f507f20cd76424d06e71e24ad",
-    "results/deepseek-v4-pro/deepagents/summary.json": "231e4264b0403e12788136d01af465f86fe9ba3c3ab992510925a7ef4c2a40e9",
-    "results/deepseek-v4-pro/deepagents/results.jsonl": "a006c8b0b80064295faeb3aa7395a2408f0208677853448816b361bf67e39bb6",
-    "results/deepseek-v4-pro/deepagents/config.json": "400c28e4008d5241bf50bc71c2674861846d6b6d2e7da21f2a7602a60ecc0ae9"
-  }
-}
+{
+  "schema_version": 1,
+  "kind": "research_leaderboard",
+  "published_date": "2026-09-20",
+  "source_repository": "CamoAiLab/InferenceNet-Leaderboard",
+  "source_revision": "6ea7c5684664bfe5c6e3d282a5e28f68244acd9e",
+  "dataset_revision": "59f9512a38e594528807744214a60ee00367434e",
+  "task_list": "Selected_1000/1000_new.csv",
+  "tasks_per_group": 1000,
+  "groups": 13,
+  "records": 13000,
+  "model_aliases": 6,
+  "paired_groups": 12,
+  "independent_groups": 1,
+  "paired_arms": [
+    "baseline",
+    "deepagents"
+  ],
+  "official_parity_verified": false,
+  "models": [
+    {
+      "id": "gpt-5.6-sol",
+      "name": "GPT-5.6 Sol",
+      "arms": {
+        "baseline": {
+          "path": "results/gpt-5.6-sol/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 338,
+              "denominator": 1000,
+              "unknown": 445,
+              "failure": 217,
+              "score": 33.8,
+              "profile": "local-paper-v1",
+              "verification": "accepted_summary"
+            },
+            "compilation_success": {
+              "count": 557,
+              "denominator": 1000,
+              "unknown": 3,
+              "failure": 440,
+              "score": 55.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 430,
+              "denominator": 1000,
+              "unknown": 444,
+              "failure": 126,
+              "score": 43.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 537,
+              "denominator": 1000,
+              "unknown": 444,
+              "failure": 19,
+              "score": 53.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 485,
+              "denominator": 1000,
+              "unknown": 444,
+              "failure": 71,
+              "score": 48.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-13T17:49:16.537733+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": null,
+          "invalid_evidence_count": 0,
+          "interruption_count": 3,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/gpt-5.6-sol/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 518,
+              "denominator": 1000,
+              "unknown": 95,
+              "failure": 387,
+              "score": 51.8,
+              "profile": "local-paper-v1",
+              "verification": "accepted_summary"
+            },
+            "compilation_success": {
+              "count": 910,
+              "denominator": 1000,
+              "unknown": 10,
+              "failure": 80,
+              "score": 91.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 699,
+              "denominator": 1000,
+              "unknown": 92,
+              "failure": 209,
+              "score": 69.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 879,
+              "denominator": 1000,
+              "unknown": 92,
+              "failure": 29,
+              "score": 87.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 787,
+              "denominator": 1000,
+              "unknown": 93,
+              "failure": 120,
+              "score": 78.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-13T17:49:16.537733+00:00",
+          "complete": true,
+          "harness": "DeepAgents",
+          "model_call_cap": 6,
+          "task_unknown_count": null,
+          "invalid_evidence_count": 0,
+          "interruption_count": 10,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "claude-opus-4-8",
+      "name": "Claude Opus 4.8",
+      "arms": {
+        "baseline": {
+          "path": "results/claude-opus-4-8/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 184,
+              "denominator": 1000,
+              "unknown": 538,
+              "failure": 278,
+              "score": 18.4,
+              "profile": "local-paper-v1",
+              "verification": "accepted_summary"
+            },
+            "compilation_success": {
+              "count": 467,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 533,
+              "score": 46.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 305,
+              "denominator": 1000,
+              "unknown": 537,
+              "failure": 158,
+              "score": 30.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 437,
+              "denominator": 1000,
+              "unknown": 537,
+              "failure": 26,
+              "score": 43.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 370,
+              "denominator": 1000,
+              "unknown": 537,
+              "failure": 93,
+              "score": 37.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-13T17:50:43.870921+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": null,
+          "invalid_evidence_count": 0,
+          "interruption_count": 0,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/claude-opus-4-8/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 392,
+              "denominator": 1000,
+              "unknown": 174,
+              "failure": 434,
+              "score": 39.2,
+              "profile": "local-paper-v1",
+              "verification": "accepted_summary"
+            },
+            "compilation_success": {
+              "count": 834,
+              "denominator": 1000,
+              "unknown": 4,
+              "failure": 162,
+              "score": 83.4,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 621,
+              "denominator": 1000,
+              "unknown": 170,
+              "failure": 209,
+              "score": 62.1,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 793,
+              "denominator": 1000,
+              "unknown": 170,
+              "failure": 37,
+              "score": 79.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 696,
+              "denominator": 1000,
+              "unknown": 172,
+              "failure": 132,
+              "score": 69.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-13T17:50:43.870921+00:00",
+          "complete": true,
+          "harness": "DeepAgents",
+          "model_call_cap": 6,
+          "task_unknown_count": null,
+          "invalid_evidence_count": 0,
+          "interruption_count": 4,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "kimi-k3",
+      "name": "Kimi K3",
+      "arms": {
+        "baseline": {
+          "path": "results/kimi-k3/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 230,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 289,
+              "score": 23.0,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 526,
+              "denominator": 1000,
+              "unknown": 5,
+              "failure": 469,
+              "score": 52.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 366,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 153,
+              "score": 36.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 494,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 25,
+              "score": 49.4,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 431,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 88,
+              "score": 43.1,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T15:28:00.953066+00:00",
+          "complete": false,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": 5,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/kimi-k3/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 390,
+              "denominator": 1000,
+              "unknown": 159,
+              "failure": 451,
+              "score": 39.0,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 867,
+              "denominator": 1000,
+              "unknown": 4,
+              "failure": 129,
+              "score": 86.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 635,
+              "denominator": 1000,
+              "unknown": 156,
+              "failure": 209,
+              "score": 63.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 804,
+              "denominator": 1000,
+              "unknown": 156,
+              "failure": 40,
+              "score": 80.4,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 709,
+              "denominator": 1000,
+              "unknown": 157,
+              "failure": 134,
+              "score": 70.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T15:28:00.953066+00:00",
+          "complete": false,
+          "harness": "DeepAgents 0.7.13",
+          "model_call_cap": 6,
+          "task_unknown_count": 4,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "gemini-3.1-pro-preview",
+      "name": "Gemini 3.1 Pro Preview",
+      "arms": {
+        "baseline": {
+          "path": "results/gemini-3.1-pro-preview/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 246,
+              "denominator": 1000,
+              "unknown": 462,
+              "failure": 292,
+              "score": 24.6,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 543,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 457,
+              "score": 54.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 366,
+              "denominator": 1000,
+              "unknown": 460,
+              "failure": 174,
+              "score": 36.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 506,
+              "denominator": 1000,
+              "unknown": 460,
+              "failure": 34,
+              "score": 50.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 437,
+              "denominator": 1000,
+              "unknown": 460,
+              "failure": 103,
+              "score": 43.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T15:28:00.953066+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/gemini-3.1-pro-preview/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 391,
+              "denominator": 1000,
+              "unknown": 155,
+              "failure": 454,
+              "score": 39.1,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 850,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 150,
+              "score": 85.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 632,
+              "denominator": 1000,
+              "unknown": 153,
+              "failure": 215,
+              "score": 63.2,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 811,
+              "denominator": 1000,
+              "unknown": 153,
+              "failure": 36,
+              "score": 81.1,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 718,
+              "denominator": 1000,
+              "unknown": 154,
+              "failure": 128,
+              "score": 71.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T15:28:00.953066+00:00",
+          "complete": true,
+          "harness": "DeepAgents 0.7.13",
+          "model_call_cap": 6,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "qwen3.7-max",
+      "name": "Qwen3.7-Max",
+      "arms": {
+        "baseline": {
+          "path": "results/qwen3.7-max/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 198,
+              "denominator": 1000,
+              "unknown": 519,
+              "failure": 283,
+              "score": 19.8,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 486,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 514,
+              "score": 48.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 315,
+              "denominator": 1000,
+              "unknown": 518,
+              "failure": 167,
+              "score": 31.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 454,
+              "denominator": 1000,
+              "unknown": 518,
+              "failure": 28,
+              "score": 45.4,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 396,
+              "denominator": 1000,
+              "unknown": 518,
+              "failure": 86,
+              "score": 39.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T17:18:47.373930+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/qwen3.7-max/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 248,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 271,
+              "score": 24.8,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 523,
+              "denominator": 1000,
+              "unknown": 7,
+              "failure": 470,
+              "score": 52.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 379,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 140,
+              "score": 37.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 488,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 31,
+              "score": 48.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 433,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 86,
+              "score": 43.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T17:18:47.373930+00:00",
+          "complete": false,
+          "harness": "DeepAgents 0.7.13",
+          "model_call_cap": 6,
+          "task_unknown_count": 6,
+          "invalid_evidence_count": 1,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "deepseek-v4-pro",
+      "name": "DeepSeek V4 Pro",
+      "arms": {
+        "baseline": {
+          "path": "results/deepseek-v4-pro/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 129,
+              "denominator": 1000,
+              "unknown": 723,
+              "failure": 148,
+              "score": 12.9,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 281,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 719,
+              "score": 28.1,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 195,
+              "denominator": 1000,
+              "unknown": 723,
+              "failure": 82,
+              "score": 19.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 260,
+              "denominator": 1000,
+              "unknown": 723,
+              "failure": 17,
+              "score": 26.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 236,
+              "denominator": 1000,
+              "unknown": 723,
+              "failure": 41,
+              "score": 23.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T17:18:47.373930+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/deepseek-v4-pro/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 183,
+              "denominator": 1000,
+              "unknown": 648,
+              "failure": 169,
+              "score": 18.3,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 352,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 648,
+              "score": 35.2,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 270,
+              "denominator": 1000,
+              "unknown": 648,
+              "failure": 82,
+              "score": 27.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 338,
+              "denominator": 1000,
+              "unknown": 648,
+              "failure": 14,
+              "score": 33.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 308,
+              "denominator": 1000,
+              "unknown": 648,
+              "failure": 44,
+              "score": 30.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T17:18:47.373930+00:00",
+          "complete": true,
+          "harness": "DeepAgents 0.7.13",
+          "model_call_cap": 6,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "dsh": {
+          "path": "results/deepseek-v4-pro/dsh",
+          "metrics": {
+            "perfect": {
+              "count": 372,
+              "denominator": 1000,
+              "unknown": 305,
+              "failure": 323,
+              "score": 37.2,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 699,
+              "denominator": 1000,
+              "unknown": 30,
+              "failure": 271,
+              "score": 69.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 556,
+              "denominator": 1000,
+              "unknown": 302,
+              "failure": 142,
+              "score": 55.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 683,
+              "denominator": 1000,
+              "unknown": 302,
+              "failure": 15,
+              "score": 68.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 598,
+              "denominator": 1000,
+              "unknown": 304,
+              "failure": 98,
+              "score": 59.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-19T18:18:48.070003+00:00",
+          "complete": false,
+          "harness": "DeepSeek Harness SDK 0.1.5rc1",
+          "model_call_cap": 6,
+          "task_unknown_count": 30,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "technical_recovery_view",
+          "display_name": "DeepSeek API [v4-pro alias] + DSH",
+          "identity_notice": "Requested API alias: deepseek-v4-pro. Underlying model weights are unverified. This technical recovery result is a separate harness entry, not a paired DeepAgents gain.",
+          "paired_comparison": false,
+          "model_alias": "deepseek-v4-pro",
+          "underlying_model_weights_verified": false,
+          "comparison_group": "independent",
+          "protocol_url": "results/deepseek-v4-pro/dsh/README.md"
+        }
+      }
+    }
+  ],
+  "source_sha256": {
+    "results/gpt-5.6-sol/baseline/summary.json": "6d6ab1aaffe754e08f9656dcb5f51fcd5c772e60660dc34d86cfc21f3c5c03d7",
+    "results/gpt-5.6-sol/baseline/results.jsonl": "3b12160382043e27531ff5cb3e23bccc9a8f7807e4121a82ffa13df28f65a859",
+    "results/gpt-5.6-sol/baseline/config.json": "90ec4e0781b57498fef52de31eacf5c8ec9424549643865a7140a1ccdb04c1d4",
+    "results/gpt-5.6-sol/deepagents/summary.json": "821ca6e55a66006c7fb958951a7859a9dccde29d87de8415d5c3f07f523563c5",
+    "results/gpt-5.6-sol/deepagents/results.jsonl": "378cdec8e7a880f091ba442649d1c034997acefdc623898b60a063baad29d7be",
+    "results/gpt-5.6-sol/deepagents/config.json": "a3e24175f2395cffbbcd6d9d3c89689bd3a3b22dd4ce61bb78d5da33c6b6cc1b",
+    "results/claude-opus-4-8/baseline/summary.json": "7b5b06bd9b3d7b47952b1dc424a01fcdf04ddccca0b3f64d04132358d6380cdc",
+    "results/claude-opus-4-8/baseline/results.jsonl": "b9d6bd51c9eb81a634e78e422c86da16f655d97da87d3f5afa4c9aa0f96809f4",
+    "results/claude-opus-4-8/baseline/config.json": "8058afcd14c34d666277c0f32ac5962398b10618df5b1aa0e13cd9862344dc56",
+    "results/claude-opus-4-8/deepagents/summary.json": "f86da9c76f8becb75b4970c486160013209a6aa4239923018b217e55f1b0a614",
+    "results/claude-opus-4-8/deepagents/results.jsonl": "3bef3b9850f24b9cb0b5806efd8425c4e4cc650735a6b28b45befdd55a6ec28a",
+    "results/claude-opus-4-8/deepagents/config.json": "12215dfcba5d2d692268a305364514a03ff7579094b2d4af12f493383b8e0f21",
+    "results/kimi-k3/baseline/summary.json": "2edd2e96f3b448ceceb6189edb7f23c737b245c00c9820728c69f5237feb34b7",
+    "results/kimi-k3/baseline/results.jsonl": "0502ae8892dd6c66ee92890a83c678527cff6c3396aa4c13faaadcf41e1857c1",
+    "results/kimi-k3/baseline/config.json": "7191c406e8695a5cb760ee8fa99c4fb0dd71bca59347acc4d6d33fe0630be081",
+    "results/kimi-k3/deepagents/summary.json": "81c8cabaa58acc65ef6afba792241b70951b666ab90c3f7c25194b61b0d9a228",
+    "results/kimi-k3/deepagents/results.jsonl": "bdab99f8f1d81885f74326cf9c8c8e57b9bb8a8d616bfbb1b5a2f6bf32b737ca",
+    "results/kimi-k3/deepagents/config.json": "312c9260c01d5988878e4f2b88da55a16362ee994e8217c63b15ba772da3857c",
+    "results/gemini-3.1-pro-preview/baseline/summary.json": "a8ba5100a28f1f0aeae6949eb3669c7717c844c4e2eee6fa42ded351aef6b5c7",
+    "results/gemini-3.1-pro-preview/baseline/results.jsonl": "2f1e2019157d105efee7ba311a49eba99abe4a7b84f0a4b4e0ac174e2d56cdba",
+    "results/gemini-3.1-pro-preview/baseline/config.json": "ff9884d2095695a1f40f5fc573982b9a49fd53d41dca2a6fc36d04dc81f10998",
+    "results/gemini-3.1-pro-preview/deepagents/summary.json": "bf9ce721c94103f91b0ca576f717f4f5dca7cc9965601658a65fc6d2a15b7045",
+    "results/gemini-3.1-pro-preview/deepagents/results.jsonl": "c2824f8277b283a6dd60118b6e6a0cb0419e4c8b9193de15a7bcf7aaaad24181",
+    "results/gemini-3.1-pro-preview/deepagents/config.json": "f09d9d99d26328ae2f54e9173b22f69d68f7d7bb50c3cc5e352c996b98dd57f9",
+    "results/qwen3.7-max/baseline/summary.json": "e506814d33789673fcd6b8e6869cdcbd52371cd3d0455ce53ae27e1098a52214",
+    "results/qwen3.7-max/baseline/results.jsonl": "4924e929d67956f9deb6c319605042d09f40b12f7bd2f18baa0464b8f7c9851a",
+    "results/qwen3.7-max/baseline/config.json": "9f553440e741863e82f32fec11470affe6f0dbffa03c668c647bd63d8f73227f",
+    "results/qwen3.7-max/deepagents/summary.json": "a5dcb2297ccc31a921e3f4eeb025ed81a764e16556a12068733cfe3bfe0bdbe3",
+    "results/qwen3.7-max/deepagents/results.jsonl": "e009f8c5c88b05896836330822afbda33af463c1f264f9adb9bbf5b094e61dfa",
+    "results/qwen3.7-max/deepagents/config.json": "3862d438d381ca3e57b1bf68921ef01a8c38727902152ada2c6e1fcf7b70cc77",
+    "results/deepseek-v4-pro/baseline/summary.json": "c51e4f1de78eba8f30e4a59276fb7821132cd103f0ae3f9d04c75c98344f2e34",
+    "results/deepseek-v4-pro/baseline/results.jsonl": "7827218f6f18f39988e8c439f0c93cab611c04d158d8704aa50c9274cd438837",
+    "results/deepseek-v4-pro/baseline/config.json": "4cef524f37a325029ea4ad325b163f72f9cf725f507f20cd76424d06e71e24ad",
+    "results/deepseek-v4-pro/deepagents/summary.json": "231e4264b0403e12788136d01af465f86fe9ba3c3ab992510925a7ef4c2a40e9",
+    "results/deepseek-v4-pro/deepagents/results.jsonl": "a006c8b0b80064295faeb3aa7395a2408f0208677853448816b361bf67e39bb6",
+    "results/deepseek-v4-pro/deepagents/config.json": "400c28e4008d5241bf50bc71c2674861846d6b6d2e7da21f2a7602a60ecc0ae9",
+    "results/deepseek-v4-pro/dsh/summary.json": "62936f76b1ca0de5e761d1ea1f894343695503ce934bf0213dc9fde1ea72a782",
+    "results/deepseek-v4-pro/dsh/results.jsonl": "b12427d5f4476351e571e4b29f61e0463540a1701bec51e642753d717dbc1df2",
+    "results/deepseek-v4-pro/dsh/config.json": "df322f13761e06dc78cb553c5dac347ce4361599937af4017a902c56a7df9d77"
+  }
+}

leaderboard.css

@@ -3,3 +3,5 @@
 @media(max-width:760px){.page{padding:0 20px}.topbar{min-height:76px}.nav-links{gap:12px}.nav-links a{font-size:11px}.brand{font-size:18px}.brand-mark{display:none}.hero{padding-top:34px}h1{letter-spacing:-1.3px}.intro{font-size:15px}.facts{gap:18px}.facts .update{width:100%;margin-left:0}.protocols{gap:18px}.protocols>div+div{padding-left:18px}.protocols p+p{font-size:12px}.protocol-tag{font-size:12px}h2{font-size:22px}.section-head{align-items:flex-start;flex-direction:column;gap:12px}.toolbar{align-items:flex-start;flex-direction:column}.view-switch{width:100%}.view-switch button{flex:1;font-size:11px;padding:8px}.filters{flex-wrap:wrap;gap:12px}.metric-control{width:100%;min-width:0!important}.filters label:not(.metric-control){flex:1}.search-control{margin-left:0;width:auto}.metric-description{padding-bottom:16px}.method-grid{grid-template-columns:1fr;gap:20px}.method{padding-top:36px}.table-foot{flex-direction:column;gap:5px}.table-foot>span:last-child{text-align:left}.table-wrap td{padding:17px 14px}.method .section-head{gap:14px}footer{flex-direction:column;gap:12px}}
 tbody th[scope=row]{background:transparent;color:var(--text);border-bottom-color:#23313b}tbody tr:last-child th{border-bottom:none}
 @media(prefers-reduced-motion:no-preference){button,a{transition:background .12s,color .12s}}
+.protocols{grid-template-columns:repeat(3,1fr);gap:24px}.protocols>div+div{padding-left:24px}.independent{color:var(--amber)}.independent .fill{background:var(--amber)}.independent-notice{margin-top:18px;padding:16px 20px;border:1px solid #4b422e;border-left:3px solid var(--amber);border-radius:6px;background:#1c1e1b;font-size:12px;color:var(--muted)}.independent-notice strong{color:var(--amber)}.independent-notice p{margin:5px 0 7px;max-width:960px}.independent-notice a{color:var(--amber)}.run-protocol{display:block;font-size:10px;margin-top:5px}.model-name{white-space:normal;max-width:355px}.model-meta{flex-wrap:wrap}.facts{gap:24px}.facts .update{font-size:11px}
+@media(max-width:760px){.protocols{grid-template-columns:1fr;gap:18px}.protocols>div+div{border-left:0;border-top:1px solid var(--line);padding:18px 0 0}.facts{gap:14px 24px}.independent-notice{padding:14px}.view-switch button{flex-basis:100%}.model-cell{min-width:225px}.model-name{max-width:275px}}

leaderboard.csv

@@ -1,61 +1,66 @@
-model,arm,harness,metric_profile,metric,successes,denominator,unknown,score_percent,official_parity_verified,source_revision
-gpt-5.6-sol,baseline,none,local-paper-v1,perfect,338,1000,445,33.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,baseline,none,hf-leaderboard-v1,compilation_success,557,1000,3,55.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,baseline,none,hf-leaderboard-v1,partial_replication,430,1000,444,43.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,baseline,none,hf-leaderboard-v1,coefficient_direction,537,1000,444,53.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,baseline,none,hf-leaderboard-v1,significance_level,485,1000,444,48.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,local-paper-v1,perfect,518,1000,95,51.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,compilation_success,910,1000,10,91.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,partial_replication,699,1000,92,69.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,coefficient_direction,879,1000,92,87.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,significance_level,787,1000,93,78.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,local-paper-v1,perfect,184,1000,538,18.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,hf-leaderboard-v1,compilation_success,467,1000,0,46.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,hf-leaderboard-v1,partial_replication,305,1000,537,30.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,hf-leaderboard-v1,coefficient_direction,437,1000,537,43.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,hf-leaderboard-v1,significance_level,370,1000,537,37.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,local-paper-v1,perfect,392,1000,174,39.2,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,compilation_success,834,1000,4,83.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,partial_replication,621,1000,170,62.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,coefficient_direction,793,1000,170,79.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,significance_level,696,1000,172,69.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,local-paper-v1,perfect,230,1000,481,23.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,hf-leaderboard-v1,compilation_success,526,1000,5,52.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,hf-leaderboard-v1,partial_replication,366,1000,481,36.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,hf-leaderboard-v1,coefficient_direction,494,1000,481,49.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,hf-leaderboard-v1,significance_level,431,1000,481,43.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,390,1000,159,39.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,867,1000,4,86.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,635,1000,156,63.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,804,1000,156,80.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,709,1000,157,70.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,local-paper-v1,perfect,246,1000,462,24.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,compilation_success,543,1000,0,54.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,partial_replication,366,1000,460,36.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,coefficient_direction,506,1000,460,50.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,significance_level,437,1000,460,43.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,391,1000,155,39.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,850,1000,0,85.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,632,1000,153,63.2,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,811,1000,153,81.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,718,1000,154,71.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,local-paper-v1,perfect,198,1000,519,19.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,hf-leaderboard-v1,compilation_success,486,1000,0,48.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,hf-leaderboard-v1,partial_replication,315,1000,518,31.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,hf-leaderboard-v1,coefficient_direction,454,1000,518,45.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,hf-leaderboard-v1,significance_level,396,1000,518,39.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,248,1000,481,24.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,523,1000,7,52.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,379,1000,481,37.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,488,1000,481,48.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,433,1000,481,43.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,local-paper-v1,perfect,129,1000,723,12.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,hf-leaderboard-v1,compilation_success,281,1000,0,28.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,hf-leaderboard-v1,partial_replication,195,1000,723,19.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,hf-leaderboard-v1,coefficient_direction,260,1000,723,26.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,hf-leaderboard-v1,significance_level,236,1000,723,23.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,183,1000,648,18.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,352,1000,0,35.2,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,270,1000,648,27.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,338,1000,648,33.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,308,1000,648,30.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
+model,arm,harness,metric_profile,metric,successes,denominator,unknown,score_percent,official_parity_verified,source_revision,display_name,comparison_group,underlying_model_weights_verified
+gpt-5.6-sol,baseline,none,local-paper-v1,perfect,338,1000,445,33.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,baseline,none,hf-leaderboard-v1,compilation_success,557,1000,3,55.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,baseline,none,hf-leaderboard-v1,partial_replication,430,1000,444,43.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,baseline,none,hf-leaderboard-v1,coefficient_direction,537,1000,444,53.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,baseline,none,hf-leaderboard-v1,significance_level,485,1000,444,48.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,local-paper-v1,perfect,518,1000,95,51.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,compilation_success,910,1000,10,91.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,partial_replication,699,1000,92,69.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,coefficient_direction,879,1000,92,87.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,significance_level,787,1000,93,78.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+claude-opus-4-8,baseline,none,local-paper-v1,perfect,184,1000,538,18.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,baseline,none,hf-leaderboard-v1,compilation_success,467,1000,0,46.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,baseline,none,hf-leaderboard-v1,partial_replication,305,1000,537,30.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,baseline,none,hf-leaderboard-v1,coefficient_direction,437,1000,537,43.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,baseline,none,hf-leaderboard-v1,significance_level,370,1000,537,37.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,local-paper-v1,perfect,392,1000,174,39.2,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,compilation_success,834,1000,4,83.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,partial_replication,621,1000,170,62.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,coefficient_direction,793,1000,170,79.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,significance_level,696,1000,172,69.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+kimi-k3,baseline,none,local-paper-v1,perfect,230,1000,481,23.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,baseline,none,hf-leaderboard-v1,compilation_success,526,1000,5,52.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,baseline,none,hf-leaderboard-v1,partial_replication,366,1000,481,36.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,baseline,none,hf-leaderboard-v1,coefficient_direction,494,1000,481,49.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,baseline,none,hf-leaderboard-v1,significance_level,431,1000,481,43.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,390,1000,159,39.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,867,1000,4,86.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,635,1000,156,63.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,804,1000,156,80.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,709,1000,157,70.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+gemini-3.1-pro-preview,baseline,none,local-paper-v1,perfect,246,1000,462,24.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,compilation_success,543,1000,0,54.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,partial_replication,366,1000,460,36.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,coefficient_direction,506,1000,460,50.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,significance_level,437,1000,460,43.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,391,1000,155,39.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,850,1000,0,85.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,632,1000,153,63.2,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,811,1000,153,81.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,718,1000,154,71.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+qwen3.7-max,baseline,none,local-paper-v1,perfect,198,1000,519,19.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,baseline,none,hf-leaderboard-v1,compilation_success,486,1000,0,48.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,baseline,none,hf-leaderboard-v1,partial_replication,315,1000,518,31.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,baseline,none,hf-leaderboard-v1,coefficient_direction,454,1000,518,45.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,baseline,none,hf-leaderboard-v1,significance_level,396,1000,518,39.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,248,1000,481,24.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,523,1000,7,52.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,379,1000,481,37.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,488,1000,481,48.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,433,1000,481,43.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+deepseek-v4-pro,baseline,none,local-paper-v1,perfect,129,1000,723,12.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,baseline,none,hf-leaderboard-v1,compilation_success,281,1000,0,28.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,baseline,none,hf-leaderboard-v1,partial_replication,195,1000,723,19.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,baseline,none,hf-leaderboard-v1,coefficient_direction,260,1000,723,26.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,baseline,none,hf-leaderboard-v1,significance_level,236,1000,723,23.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,183,1000,648,18.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,352,1000,0,35.2,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,270,1000,648,27.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,338,1000,648,33.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,308,1000,648,30.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,local-paper-v1,perfect,372,1000,305,37.2,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,hf-leaderboard-v1,compilation_success,699,1000,30,69.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,hf-leaderboard-v1,partial_replication,556,1000,302,55.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,hf-leaderboard-v1,coefficient_direction,683,1000,302,68.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,hf-leaderboard-v1,significance_level,598,1000,304,59.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False

leaderboard.js

@@ -3,7 +3,7 @@ const $ = id => document.getElementById(id);
 const metricKeys = ['perfect', 'partial_replication', 'compilation_success', 'coefficient_direction', 'significance_level'];
 const en = Object.fromEntries([...document.querySelectorAll('[data-i18n]')].map(el => [el.dataset.i18n, el.innerHTML]));
 Object.assign(en, {
-  model: 'Model', rank: '#', score: 'Score', evidenceCol: 'Evidence', pp: 'pp',
+  model: 'Model / API alias', rank: '#', score: 'Score', evidenceCol: 'Evidence', pp: 'pp',
   passed: 'successes', metricUnknown: 'metric unknown', review: 'Evidence gaps',
   config: 'Config', recordsLink: 'Results', allSealed: 'Archived',
   taskUnknown: 'task status unknown', invalid: 'invalid evidence', interruptions: 'interruptions',
@@ -11,6 +11,8 @@ Object.assign(en, {
   groupLocal: 'Local paper profile', groupHF: 'HF-style profile · parity unverified',
   loading: 'Loading archived results…', searchPlaceholder: 'Search models…',
   matched: '{n} of {total} models', localProfile: 'local-paper-v1', hfProfile: 'hf-leaderboard-v1 · parity unverified',
+  matchedRuns: '{n} of {total} runs', protocol: 'Protocol', independentRun: 'Independent · alias unverified',
+  published: 'Published {date}',
   perfectDesc: 'Coefficient and standard-error relative errors ≤ 1%, and p-value absolute error ≤ 0.01, all at once.',
   partial_replicationDesc: 'Coefficient relative error ≤ 5%. No standard-error or p-value condition.',
   compilation_successDesc: 'Verified code execution completed successfully; a valid prediction JSON is not required.',
@@ -19,35 +21,41 @@ Object.assign(en, {
 });
 const zh = {
   dataset: '数据集 ↗', archive: '结果存档 ↗', edition: '研究榜单 · 2026 年 9 月',
-  title: '同样的模型，<br><span>两种执行方式。</span>',
-  intro: '在真实计量经济学复现任务上，对比单次 Agent 与加入 DeepAgents Harness 后的表现。',
-  models: '个模型', groups: '组实验', tasks: '题 / 组', updated: '发布于 2026 年 9 月 17 日',
-  single: '单次 Agent', harness: 'Agent + Harness',
+  title: '模型与 Harness，<br><span>复现真实研究。</span>',
+  intro: '查看单次 Agent、DeepAgents 和独立 DeepSeek Harness 实验在真实计量经济学复现任务上的表现。',
+  models: '种请求模型别名', groups: '组实验', tasks: '题 / 组', records: '条任务记录', published: '发布于 {date}',
+  single: '单次 Agent', harness: '全部 Harness 实验', deepagents: 'DeepAgents',
+  dsh: 'DeepSeek Harness · 独立实验',
+  dshDesc: '使用官方 DSH SDK，请求 v4-pro API 别名。按固定规则纳入技术补测，底层模型权重未经核实。',
+  dshNoticeTitle: 'DSH 作为独立实验展示。',
+  dshNotice: '请求使用 deepseek-v4-pro 别名，底层模型权重未经核实。固定的 1,000 题结果包含 87 次技术补测替换（其中 27 题调整了输入 schema 提示），仍有 30 题状态未知；不纳入 DeepAgents 配对提升计算。',
+  dshProtocol: '查看运行协议与证据 ↗',
   singleDesc: '一次模型调用生成代码，然后执行；不进行多轮 Agent 循环。',
   harnessDesc: 'DeepAgents 进行规划、工具调用与代码修订；最多 6 次模型调用，最后执行。',
   resultsLabel: '01 / 评测结果', rankings: 'Agent 与 Harness 对照榜单', research: '研究结果 · 阶段版',
   scope: '基于已存档结果的本地评分，尚未验证与官方评分器完全一致。两组预算及不同批次协议存在差异，提升值用于描述观察结果，不代表等成本对比。',
-  compare: '双组对照', download: '下载 CSV ↓', metric: '评分指标',
+  compare: 'Baseline / DeepAgents 配对', download: '下载 CSV ↓', metric: '评分指标',
   perfect: '完整复现率 · 本地口径', partial_replication: '部分复现率 · HF 口径',
   compilation_success: '执行成功率 · HF 口径', coefficient_direction: '系数方向正确率 · HF 口径',
   significance_level: '显著性水平正确率 · HF 口径',
-  sort: '排序方式', harnessScore: 'Harness 组成绩', singleScore: '单次组成绩', lift: '提升', search: '查找模型',
+  sort: '排序方式', harnessScore: 'DeepAgents 组成绩', singleScore: '单次组成绩', lift: '提升', search: '查找模型',
   error: '结果加载失败，请重试或打开结果存档。', retry: '重试', empty: '没有匹配的模型。',
   denominator: '所有指标固定以 1,000 题为分母，未知和无效记录不计为成功。',
   methodLabel: '02 / 如何解读', methodTitle: '每个成绩都有据可查。', historical: '查看历史榜单 ↗',
   fixedTasks: '固定的题目集合', fixedDesc: '所有组使用相同的 Selected_1000 题号和数据集版本。成绩 = 成功数 ÷ 1,000，失败和未知记录始终保留在分母中。',
   separateMetrics: '两套评分口径', profilesDesc: '完整复现率采用本地论文口径。其余四项 HF 指标由本地实现复现网页定义，尚未验证与官方评分器一致。',
-  budgetTitle: '结合预算看提升', budgetDesc: '单次组使用 1 次模型调用；DeepAgents 最多使用 6 次模型调用和 4 次试运行工具调用。历史恢复、时限和运行协议有差异，不能作为等成本的因果估计。',
+  budgetTitle: '结合预算看提升', budgetDesc: '单次组使用 1 次模型调用；DeepAgents 和 DSH 每次尝试最多 6 次模型调用、4 次试运行工具调用。DSH 额外进行了 87 次技术补测，完整环境和推理设定是否一致尚未核实；不计算 DSH 配对提升，也不能据此判断等成本的因果效果。',
   evidence: '证据状态与指标定义',
   unknownDesc: '“指标未知”表示该指标无法判断，可能源于执行失败或输出缺失；它与“任务最终状态未知”不同。两者均不剔除，也不计为成功。',
-  evidenceCaption: '各组已接收存档中的状态', model: '模型',
-  verification: '四项 HF 指标已与逐题存档标记核对。Sol 和 Opus 的完整复现率来自已接收汇总；其余四个模型还可逐题核对本地评分标记。本次网页更新没有重新运行或重新评分实验。',
+  evidenceCaption: '各组已接收存档中的状态', model: '模型 / API 别名', protocol: '执行方式',
+  verification: '四项 HF 指标已与逐题存档标记核对。Sol 和 Opus 的完整复现率来自已接收汇总；其余实验（含 DSH）均可逐题核对本地评分标记。本次网页更新没有重新运行或重新评分实验。',
   archiveNote: '各组原始存档及当时的发布元数据保持不变，本页是基于存档的新展示。本版不包含 GPT-5.5 和尚未存档的实验。',
   footer: '衡量计量经济学复现能力。', source: '来源快照 ↗', json: '数据 JSON ↗', project: '项目页 ↗',
   rank: '#', score: '成绩', evidenceCol: '证据', pp: '百分点', passed: '题成功', metricUnknown: '指标未知', review: '存在证据缺口',
   config: '配置', recordsLink: '逐题结果', allSealed: '已存档', taskUnknown: '任务状态未知', invalid: '无效证据', interruptions: '中断记录',
   historicalStatus: '历史存档，未导出任务未知总数', groupLocal: '本地论文口径', groupHF: 'HF 网页口径 · 一致性待验证',
   loading: '正在加载存档结果…', searchPlaceholder: '搜索模型…', matched: '显示 {n} / {total} 个模型',
+  matchedRuns: '显示 {n} / {total} 组实验', independentRun: '独立实验 · 别名权重未核实',
   localProfile: 'local-paper-v1 · 本地口径', hfProfile: 'hf-leaderboard-v1 · 一致性待验证',
   perfectDesc: '系数与标准误的相对误差均 ≤ 1%，同时 p 值绝对误差 ≤ 0.01。',
   partial_replicationDesc: '系数相对误差 ≤ 5%，不要求标准误或 p 值达标。',
@@ -58,7 +66,7 @@ const zh = {
 let language = 'en';
 try { language = localStorage.getItem('inferencenet-language') === 'zh' ? 'zh' : 'en'; } catch (_) {}
 let data = null;
-let view = 'compare';
+let view = 'harness';
 let sortBy = 'deepagents';
 const t = key => (language === 'zh' ? zh[key] : en[key]) ?? en[key] ?? key;
 const esc = value => String(value).replace(/[&<>"']/g, char => ({'&':'&amp;', '<':'&lt;', '>':'&gt;', '"':'&quot;', "'":'&#39;'}[char]));
@@ -79,23 +87,40 @@ function applyLanguage() {
 }
 
 function validate(candidate) {
-  if (candidate.schema_version !== 1 || candidate.models?.length !== 6 || candidate.groups !== 12 || candidate.records !== 12000 || candidate.tasks_per_group !== 1000 || candidate.official_parity_verified !== false || !/^[a-f0-9]{40}$/.test(candidate.source_revision)) throw new Error('Invalid leaderboard snapshot');
+  if (candidate.schema_version !== 1 || candidate.models?.length !== 6 || ![12, 13].includes(candidate.groups) || candidate.records !== candidate.groups * 1000 || candidate.tasks_per_group !== 1000 || candidate.official_parity_verified !== false || !/^[a-f0-9]{40}$/.test(candidate.source_revision) || !/^\d{4}-\d{2}-\d{2}$/.test(candidate.published_date)) throw new Error('Invalid leaderboard snapshot');
   const ids = new Set();
+  let groups = 0;
   for (const model of candidate.models) {
     if (!/^[a-z0-9.-]+$/.test(model.id) || ids.has(model.id) || typeof model.name !== 'string') throw new Error('Invalid model');
     ids.add(model.id);
-    for (const armId of ['baseline', 'deepagents']) {
+    if (!model.arms?.baseline || !model.arms?.deepagents) throw new Error('Missing paired arm');
+    for (const armId of Object.keys(model.arms)) {
+      if (!['baseline', 'deepagents'].includes(armId) && !(model.id === 'deepseek-v4-pro' && armId === 'dsh')) throw new Error('Unexpected arm');
       const arm = model.arms?.[armId];
       if (arm?.path !== `results/${model.id}/${armId}` || typeof arm.complete !== 'boolean') throw new Error('Invalid arm');
+      if (armId === 'dsh' && (arm.paired_comparison !== false || arm.comparison_group !== 'independent' || arm.underlying_model_weights_verified !== false || arm.model_alias !== 'deepseek-v4-pro' || typeof arm.display_name !== 'string' || arm.evidence_class !== 'technical_recovery_view' || arm.protocol_url !== arm.path + '/README.md')) throw new Error('Invalid independent result');
+      groups += 1;
       for (const key of metricKeys) {
         const metric = arm.metrics?.[key];
         if (!metric || metric.denominator !== 1000 || !Number.isInteger(metric.count) || !Number.isInteger(metric.unknown) || metric.count < 0 || metric.unknown < 0 || metric.count + metric.unknown > 1000 || !Number.isFinite(metric.score) || Math.abs(metric.score - metric.count / 10) > 1e-8 || metric.profile !== (key === 'perfect' ? 'local-paper-v1' : 'hf-leaderboard-v1')) throw new Error('Invalid metric');
       }
     }
   }
+  if (groups !== candidate.groups) throw new Error('Inconsistent group count');
   return candidate;
 }
 
+// Pairing is intentionally limited to the two historical arms. Independent
+// harness entries have their own rank and can never enter a paired lift.
+function rankedEntries(snapshot, selectedView, key, selectedSort, query) {
+  const entries = selectedView === 'harness'
+    ? snapshot.models.flatMap(model => Object.entries(model.arms).filter(([id]) => id !== 'baseline').map(([armId, arm]) => ({model, armId, arm, name: arm.display_name || model.name, value: arm.metrics[key].count})))
+    : snapshot.models.map(model => ({model, armId: selectedView === 'compare' ? null : 'baseline', arm: model.arms.baseline, name: model.name,
+      value: selectedView === 'baseline' ? model.arms.baseline.metrics[key].count : selectedSort === 'gain' ? model.arms.deepagents.metrics[key].count - model.arms.baseline.metrics[key].count : model.arms[selectedSort].metrics[key].count}));
+  const search = query.toLowerCase().trim();
+  return {total: entries.length, entries: entries.filter(entry => `${entry.name} ${entry.model.id} ${entry.armId || ''}`.toLowerCase().includes(search)).sort((a, b) => b.value - a.value || a.name.localeCompare(b.name))};
+}
+
 function scoreCell(arm, key, style) {
   const m = arm.metrics[key];
   return `<td class="score-cell ${style}"><span class="score">${m.score.toFixed(1)}<small>%</small></span><div class="track" aria-hidden="true"><div class="fill" style="width:${m.score}%"></div></div><span class="counts">${m.count} / 1,000 ${esc(t('passed'))}</span><span class="unknown">${esc(t('metricUnknown'))}: ${m.unknown}</span></td>`;
@@ -112,38 +137,43 @@ function render() {
   const key = $('metric').value;
   $('metric-description').innerHTML = `<span class="profile">${esc(t(key === 'perfect' ? 'localProfile' : 'hfProfile'))}</span>${esc(t(key + 'Desc'))}`;
   if (!data) return;
-  const search = $('search').value.toLowerCase().trim();
   const count = (m, arm) => m.arms[arm].metrics[key].count;
-  const value = m => view !== 'compare' ? count(m, view) : sortBy === 'gain' ? count(m, 'deepagents') - count(m, 'baseline') : count(m, sortBy);
-  const models = data.models.filter(m => `${m.name} ${m.id}`.toLowerCase().includes(search)).sort((a, b) => value(b) - value(a) || a.name.localeCompare(b.name));
+  const {total, entries} = rankedEntries(data, view, key, sortBy, $('search').value);
+  $('published-date').textContent = t('published').replace('{date}', data.published_date);
   $('sort-control').hidden = view !== 'compare';
-  $('table-wrap').hidden = models.length === 0;
-  $('empty').hidden = models.length !== 0;
-  $('result-count').textContent = t('matched').replace('{n}', models.length).replace('{total}', data.models.length);
+  $('table-wrap').hidden = entries.length === 0;
+  $('empty').hidden = entries.length !== 0;
+  $('result-count').textContent = t(view === 'compare' ? 'matched' : 'matchedRuns').replace('{n}', entries.length).replace('{total}', total);
   $('caption').textContent = `${t(view === 'compare' ? 'compare' : view === 'baseline' ? 'single' : 'harness')} — ${t(key)}`;
   const th = (text, cls = '') => `<th scope="col" class="${cls}">${esc(text)}</th>`;
-  $('table-head').innerHTML = '<tr>' + th(t('rank'), 'rank') + th(t('model')) + (view === 'compare' ? th(t('single'), 'baseline') + th(t('harness'), 'harness') + th(t('lift')) : th(t('score'), view === 'baseline' ? 'baseline' : 'harness')) + th(t('evidenceCol')) + '</tr>';
+  $('table-head').innerHTML = '<tr>' + th(t('rank'), 'rank') + th(t('model')) + (view === 'compare' ? th(t('single'), 'baseline') + th(t('deepagents'), 'harness') + th(t('lift')) : th(t('score'), view === 'baseline' ? 'baseline' : 'harness')) + th(t('evidenceCol')) + '</tr>';
   let lastValue = null, rank = 0;
-  $('table-body').innerHTML = models.map((model, index) => {
-    const currentValue = value(model);
+  $('table-body').innerHTML = entries.map((entry, index) => {
+    const {model, arm, armId, name, value: currentValue} = entry;
     if (currentValue !== lastValue) rank = index + 1;
     lastValue = currentValue;
     const arms = model.arms;
-    const gap = view === 'compare' ? !arms.baseline.complete || !arms.deepagents.complete : !arms[view].complete;
+    const gap = view === 'compare' ? !arms.baseline.complete || !arms.deepagents.complete : !arm.complete;
     const badge = gap ? `<span class="tag">${esc(t('review'))}</span>` : '';
-    const identity = `<td class="rank">${String(rank).padStart(2, '0')}</td><th scope="row" class="model-cell"><div class="model-name">${esc(model.name)}</div><div class="model-meta">1,000 ${language === 'zh' ? '题 / 组' : 'tasks / group'} ${badge}</div></th>`;
+    const independent = armId === 'dsh';
+    const protocol = view === 'harness' ? `<span class="run-protocol ${independent ? 'independent' : 'harness'}">${esc(independent ? t('independentRun') : 'DeepAgents')}</span>` : '';
+    const identity = `<td class="rank">${String(rank).padStart(2, '0')}</td><th scope="row" class="model-cell"><div class="model-name">${esc(name)}</div>${protocol}<div class="model-meta">1,000 ${language === 'zh' ? '题 / 组' : 'tasks / group'} ${badge}</div></th>`;
     let scores = '', sources = '';
     if (view === 'compare') {
       const gain = (count(model, 'deepagents') - count(model, 'baseline')) / 10;
       scores = scoreCell(arms.baseline, key, 'baseline') + scoreCell(arms.deepagents, key, 'harness') + `<td class="gain ${gain < 0 ? 'negative' : ''}">${gain > 0 ? '+' : ''}${gain.toFixed(1)}<small>${esc(t('pp'))}</small></td>`;
       sources = external(sourceURL(arms.baseline.path, true), 'A') + external(sourceURL(arms.deepagents.path, true), 'B');
     } else {
-      scores = scoreCell(arms[view], key, view === 'baseline' ? 'baseline' : 'harness');
-      sources = external(sourceURL(arms[view].path + '/results.jsonl'), t('recordsLink')) + external(sourceURL(arms[view].path + '/config.json'), t('config'));
+      scores = scoreCell(arm, key, view === 'baseline' ? 'baseline' : independent ? 'independent' : 'harness');
+      sources = external(sourceURL(arm.path + '/results.jsonl'), t('recordsLink')) + external(sourceURL(arm.path + '/config.json'), t('config'));
+      if (independent) sources += external(sourceURL(arm.protocol_url), t('protocol'));
     }
-    return `<tr data-model="${esc(model.id)}">${identity}${scores}<td class="source-cell">${sources}</td></tr>`;
+    return `<tr data-model="${esc(model.id)}" data-arm="${esc(armId || 'paired')}">${identity}${scores}<td class="source-cell">${sources}</td></tr>`;
   }).join('');
-  $('evidence-body').innerHTML = data.models.map(m => `<tr><th scope="row">${esc(m.name)}</th>${evidenceState(m.arms.baseline)}${evidenceState(m.arms.deepagents)}</tr>`).join('');
+  $('evidence-body').innerHTML = data.models.flatMap(m => Object.entries(m.arms).map(([id, arm]) => `<tr><th scope="row">${esc(arm.display_name || m.name)}</th><td>${esc(id === 'baseline' ? t('single') : id === 'dsh' ? t('dsh') : 'DeepAgents')}</td>${evidenceState(arm)}</tr>`)).join('');
+  const dsh = data.models.find(m => m.id === 'deepseek-v4-pro')?.arms.dsh;
+  $('independent-notice').hidden = !dsh;
+  if (dsh) $('dsh-protocol').href = sourceURL(dsh.protocol_url);
 }
 
 async function loadData() {
@@ -155,7 +185,8 @@ async function loadData() {
   $('empty').hidden = true;
   $('result-count').textContent = '';
   $('evidence-body').replaceChildren();
-  for (const id of ['model-count', 'group-count', 'task-count']) $(id).textContent = '—';
+  $('independent-notice').hidden = true;
+  for (const id of ['model-count', 'group-count', 'task-count', 'record-count']) $(id).textContent = '—';
   try {
     const response = await fetch('./leaderboard-data.json', {cache: 'no-cache'});
     if (!response.ok) throw new Error(`HTTP ${response.status}`);
@@ -163,6 +194,7 @@ async function loadData() {
     $('model-count').textContent = data.models.length;
     $('group-count').textContent = data.groups;
     $('task-count').textContent = data.tasks_per_group.toLocaleString('en-US');
+    $('record-count').textContent = data.records.toLocaleString('en-US');
     $('revision-link').href = sourceURL('results', true);
     render();
   } catch (error) {