Spaces:
Running
Download DATA_FORMAT.md from RLE-Bench/codex-benchmark: direct link, hf CLI and curl.
- Browser
- Download file 1.13 kB
-
https://huggingface.co/spaces/RLE-Bench/codex-benchmark/resolve/main/DATA_FORMAT.md
- Command line
-
hf download hf://spaces/RLE-Bench/codex-benchmark/DATA_FORMAT.md
-
curl -L -o DATA_FORMAT.md https://huggingface.co/spaces/RLE-Bench/codex-benchmark/resolve/main/DATA_FORMAT.md
Data format
data.json contains the task catalog, per-family summary and completed episode records. episodes.json and episodes.csv contain the same measured results. Pending tasks have no invented verdict or usage. attempt-history.json indexes preserved provider interruptions separately.
Each episodes/FAMILY/SLOT/seed-0/ directory contains the native verdict, protocol, owner journals, full Codex session, structured trajectory, provider usage records, transcript, observed images, saved resources, video and measured provenance.
Codex native sessions use response_item records. provider-usage.jsonl contains native token_usage_record events, not reconstructed HTTP requests. Input includes cached input; output includes reasoning. Failed requests without reported usage add no tokens. Missing usage remains unknown.
Public copies redact account identifiers and local host paths; raw evidence remains unchanged. Workspace archives exclude bytecode caches and use neutral ownership. The manifests describe public bytes. Videos are native spectator recordings edited to 4x speed; they are not simulator replays.