Buckets:

27.5 GB
404,819 files
Updated about 16 hours ago
Name
Size
claude-fable-5-2__high
claude-opus-5-5__high
claude-opus-5-5__medium
claude-sonnet-5-5__high
claude-sonnet-5-5__medium
deepseek-flash__high
glm-5.3__high
gpt-6-astra__high
gpt-6-luna__high
gpt-6-luna__medium
gpt-6.1-sol__high
hy4__high
kimi-k3.1-pro__high
mimo-v2.6-flash__high
mimo-v2.6-pro__high
qwen3.8-max__high
README.md1.72 kB
xet
index.json6.07 kB
xet
README.md

TwinLab-Bench paper logs

Run logs behind the 16 entries of the TwinLab-Bench leaderboard. Each entry has one folder, <model>__<effort>/, holding the 52 task runs that give its score. Environment and judge are deepseek-flash in all entries.

<model>__<effort>/
  runs/<task>/<run_id>/run_0/     one run per task (result.json, run_config.json, actor stream, environment and judge logs, ...)
  batches/<batch_id>/             batch_config.json and batch_summary.json of the batch that started those runs
  selected_runs.json              task -> run id, batch, judge file, derived score
  workspace_files_over_5MB_not_uploaded.tsv
index.json                        the 16 entries, scores recomputed from these logs next to the published ones

Score of an entry = mean over the 52 tasks of score.derived_100 (the newest rejudge-*.json when present, else result.json). Passed = tasks with a non-zero derived score.

Only the 52 main-pool tasks are included. Each task uses the first valid run of that model and effort.

Left out of each run_0/:

  • actor_config/ (agent login and hook files), env_bin/, env_tmp/, img_cache/, calc_state_before_request/, judge_tmp/, and the environment workspace env_ws/ except session_record.md and calculator_usage.txt.
  • In actor_workspace/, files byte-identical to the case input and files over 5 MB (listed in the tsv).

batch_config.json and batch_summary.json are cut down to the rows of the tasks above. One run of deepseek-flash__high (task 053) printed its agent account file; the e-mail address, display name and account ids in that run are replaced by REDACTED.

Total size
27.5 GB
Files
404,819
Last updated
Nov 20
Pre-warmed CDN
US EU US EU

Contributors