Buckets:
| # TwinLab-Bench paper logs | |
| Run logs behind the 16 entries of the [TwinLab-Bench leaderboard](https://huggingface.co/spaces/ZJUSCL/TwinLab-Bench_Leaderboard). | |
| Each entry has one folder, `<model>__<effort>/`, holding the 52 task runs that give its score. Environment and judge are `deepseek-flash` in all entries. | |
| ``` | |
| <model>__<effort>/ | |
| runs/<task>/<run_id>/run_0/ one run per task (result.json, run_config.json, actor stream, environment and judge logs, ...) | |
| batches/<batch_id>/ batch_config.json and batch_summary.json of the batch that started those runs | |
| selected_runs.json task -> run id, batch, judge file, derived score | |
| workspace_files_over_5MB_not_uploaded.tsv | |
| index.json the 16 entries, scores recomputed from these logs next to the published ones | |
| ``` | |
| Score of an entry = mean over the 52 tasks of `score.derived_100` (the newest `rejudge-*.json` when present, else `result.json`). Passed = tasks with a non-zero derived score. | |
| Only the 52 main-pool tasks are included. Each task uses the first valid run of that model and effort. | |
| Left out of each `run_0/`: | |
| - `actor_config/` (agent login and hook files), `env_bin/`, `env_tmp/`, `img_cache/`, `calc_state_before_request/`, `judge_tmp/`, and the environment workspace `env_ws/` except `session_record.md` and `calculator_usage.txt`. | |
| - In `actor_workspace/`, files byte-identical to the case input and files over 5 MB (listed in the tsv). | |
| `batch_config.json` and `batch_summary.json` are cut down to the rows of the tasks above. One run of `deepseek-flash__high` (task 053) printed its agent account file; the e-mail address, display name and account ids in that run are replaced by `REDACTED`. | |
Xet Storage Details
- Size:
- 1.72 kB
- Xet hash:
- 0a47f448eb9817a5edd99985178dfab3f5f26c455d3cf8f6ef4fe66f9c3c8220
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.