title: InferenceNet Leaderboard
emoji: 📊
colorFrom: green
colorTo: blue
sdk: static
pinned: false
license: apache-2.0
short_description: Single-pass agents and DeepAgents on econometric replication
InferenceNet Agent & Harness Leaderboard
The September 2026 research leaderboard displays six models, twelve groups, and 12,000 archived task records. Switch between a paired comparison, a single-pass Agent ranking, and an Agent + Harness ranking. Every group uses the same 1,000 Selected_1000 task IDs.
- Single-pass Agent: one model call to generate code, followed by execution.
- Agent + Harness: DeepAgents with up to six model calls and four trial tool calls, followed by final execution.
Models: GPT-5.6 Sol, Claude Opus 4.8, Kimi K3, Gemini 3.1 Pro Preview, Qwen3.7-Max, and DeepSeek V4 Pro. GPT-5.5 is outside this edition.
Reading the results
The default metric is full replication, from local-paper-v1: coefficient
and standard-error relative errors ≤ 1%, and p-value absolute error ≤ 0.01.
Four separately labeled hf-leaderboard-v1 metrics are also available:
execution success, coefficient-only partial replication (≤ 5% relative error),
coefficient direction, and significance category.
These are locally scored research results. Official scorer parity has not been verified. All rates use 1,000 as the denominator; metric unknowns and invalid evidence are retained and earn no successes. Task-status unknown counts are reported separately from metric unknown counts.
The comparisons preserve historical recovery amendments. Call budgets, time limits, reasoning settings and source protocols differ. Differences are descriptive observations, not equal-cost causal estimates of a harness effect. See each run's configuration before making cross-model claims.
Kimi retains 5 baseline and 4 harness task-status unknowns. Qwen3.7-Max harness retains 6 task-status unknowns and 1 invalid-evidence record (task 453), which is not sealed. Historical Sol/Opus interruption counts are shown as originally exported. An archive of 1,000 task IDs does not imply 1,000 definitive outcomes.
Data and reproducibility
leaderboard-data.json: displayed data, metric profiles, counts, evidence status and source-file SHA-256 hashes.leaderboard.csv: 60 displayed metric rows; six models × two arms × five metrics. One metric profile is explicit on every row.results/: frozen per-task archives, configurations and summaries.legacy.html: original leaderboard with a historical banner. Its originalresults.csvis unchanged and is not mixed into the new comparison.
Source archive commit: 731111974cf4d1a3c7a4f848a899f0b0708dd12b.
Dataset: CamoAiLab/InferenceNet,
revision 59f9512a38e594528807744214a60ee00367434e, task list
Selected_1000/1000_new.csv.
The per-run archive files keep their original publication metadata. Their
historical included_in_displayed_leaderboard: false fields describe the archive
upload, not this later display release. The current display manifest is
leaderboard-data.json; no experiment evidence is edited to publish this page.
To rebuild the display exports from those archives:
python build_leaderboard.py --source . --output . \
--source-revision 731111974cf4d1a3c7a4f848a899f0b0708dd12b
The builder verifies all task populations and 56 metric counts against archived flags. The four Sol/Opus local full-replication summaries have no per-task local flags in the archive and are preserved from their accepted summaries. This is a display refresh, not a fresh inference run, native artifact audit, or rescore.