Spaces:
Running
Running
|
Download DATA_FORMAT.md from RLE-Bench/codex-benchmark: direct link, hf CLI and curl.
- Browser
- Download file 1.13 kB
-
https://huggingface.co/spaces/RLE-Bench/codex-benchmark/resolve/main/DATA_FORMAT.md
- Command line
-
hf download hf://spaces/RLE-Bench/codex-benchmark/DATA_FORMAT.md
-
curl -L -o DATA_FORMAT.md https://huggingface.co/spaces/RLE-Bench/codex-benchmark/resolve/main/DATA_FORMAT.md
1.13 kB
| # Data format | |
| `data.json` contains the task catalog, per-family summary and completed episode records. `episodes.json` and `episodes.csv` contain the same measured results. Pending tasks have no invented verdict or usage. `attempt-history.json` indexes preserved provider interruptions separately. | |
| Each `episodes/FAMILY/SLOT/seed-0/` directory contains the native verdict, protocol, owner journals, full Codex session, structured trajectory, provider usage records, transcript, observed images, saved resources, video and measured provenance. | |
| Codex native sessions use response_item records. `provider-usage.jsonl` contains native token_usage_record events, not reconstructed HTTP requests. Input includes cached input; output includes reasoning. Failed requests without reported usage add no tokens. Missing usage remains unknown. | |
| Public copies redact account identifiers and local host paths; raw evidence remains unchanged. Workspace archives exclude bytecode caches and use neutral ownership. The manifests describe public bytes. Videos are native spectator recordings edited to 4x speed; they are not simulator replays. | |