Publish Codex Benchmark evaluation evidence

DATA_FORMAT.md

@@ -2,7 +2,7 @@
 
 `data.json` contains the task catalog, per-family summary and completed episode records. `episodes.json` and `episodes.csv` contain the same measured results. Pending tasks have no invented verdict or usage. `attempt-history.json` indexes preserved provider interruptions separately.
 
-Each `episodes/task04/SLOT/seed-0/` directory contains the native verdict, protocol, owner journals, full Codex session, structured trajectory, provider usage records, transcript, observed images, saved resources, video and measured provenance.
+Each `episodes/FAMILY/SLOT/seed-0/` directory contains the native verdict, protocol, owner journals, full Codex session, structured trajectory, provider usage records, transcript, observed images, saved resources, video and measured provenance.
 
 Codex native sessions use response_item records. `provider-usage.jsonl` contains native token_usage_record events, not reconstructed HTTP requests. Input includes cached input; output includes reasoning. Failed requests without reported usage add no tokens. Missing usage remains unknown.
 

README.md

@@ -10,12 +10,18 @@ pinned: false
 
 # Codex Benchmark
 
-42/42 ordinary RoboDojo tasks published: 31 successes, 11 valid native failures, 0 pending. Pending and interrupted attempts are excluded from the success-rate denominator.
+62 completed ordinary Codex tasks across RoboCasa, LIBERO Long, and RoboDojo. Results are reported separately by simulator family.
 
-Stock Codex CLI 0.160.0, GPT-6 Astra / high / default, seed 0, fresh independent workspace, 7,500 action frames, stepped simulation, 8-hour safety limit. Instructions match the final reviewed benchmark definitions.
+| Family | Native successes | Tasks | Success rate |
+| --- | ---: | ---: | ---: |
+| RoboCasa | 2 | 10 | 20.00% |
+| LIBERO Long | 9 | 10 | 90.00% |
+| RoboDojo | 31 | 42 | 73.81% |
 
-Videos are native spectator recordings edited to 4x speed at 20 FPS. Sessions, observed images, tools and memos, native verdicts, usage and source identities are linked per task. Detailed failure mechanisms remain unreviewed unless an explicit evidence-based review is provided.
+Stock Codex CLI 0.160.0, GPT-6 Astra / high / default, seed 0, fresh independent workspace. RoboCasa and LIBERO use 6,000 steps at 20 Hz; LIBERO uses initial state 0. RoboDojo retains 7,500 frames at 25 Hz. All use stepped simulation and an 8-hour safety limit. Each task has one formal episode.
 
-The HF Kinex baseline combines versions 0.10.0, 0.10.2 and 0.10.3; 24 tasks share this RoboEnv source/build and 18 use an older build. This is a comparison with published historical results, not a controlled harness-only ablation. See [comparison.json](comparison.json).
+RoboCasa and LIBERO use the latest reviewed catalog instructions and the same native simulator source, assets, and task settings as their Kinex references. LIBERO Kinex was fully rerun at caac19a; RoboCasa retains its current a1eaece results. The intervening Kinex commits affect session timing and footer display. See [comparison.json](comparison.json) for paired results, usage, timing, and evidence boundaries.
 
-[Results](episodes.csv) · [Attempt history](attempt-history.json) · [Protocol](protocol.json)
+Native spectator videos are presented at 4x speed and 20 FPS. Per-task pages retain full sessions, observed images, saved tools and memos, native verdicts, instructions, protocol, usage, and source identities. Native failures are valid results; detailed failure mechanisms are unreviewed unless supported by a separate review. The four historical RoboDojo provider interruptions remain in [attempt-history.json](attempt-history.json).
+
+[Results](episodes.csv) · [Protocol](protocol.json) · [Comparison](comparison.json)

REPORT.md

@@ -1,11 +1,17 @@
 # Codex Benchmark
 
-42/42 ordinary RoboDojo tasks published: 31 successes, 11 valid native failures, 0 pending. Pending and interrupted attempts are excluded from the success-rate denominator.
+62 completed ordinary Codex tasks across RoboCasa, LIBERO Long, and RoboDojo. Results are reported separately by simulator family.
 
-Stock Codex CLI 0.160.0, GPT-6 Astra / high / default, seed 0, fresh independent workspace, 7,500 action frames, stepped simulation, 8-hour safety limit. Instructions match the final reviewed benchmark definitions.
+| Family | Native successes | Tasks | Success rate |
+| --- | ---: | ---: | ---: |
+| RoboCasa | 2 | 10 | 20.00% |
+| LIBERO Long | 9 | 10 | 90.00% |
+| RoboDojo | 31 | 42 | 73.81% |
 
-Videos are native spectator recordings edited to 4x speed at 20 FPS. Sessions, observed images, tools and memos, native verdicts, usage and source identities are linked per task. Detailed failure mechanisms remain unreviewed unless an explicit evidence-based review is provided.
+Stock Codex CLI 0.160.0, GPT-6 Astra / high / default, seed 0, fresh independent workspace. RoboCasa and LIBERO use 6,000 steps at 20 Hz; LIBERO uses initial state 0. RoboDojo retains 7,500 frames at 25 Hz. All use stepped simulation and an 8-hour safety limit. Each task has one formal episode.
 
-The HF Kinex baseline combines versions 0.10.0, 0.10.2 and 0.10.3; 24 tasks share this RoboEnv source/build and 18 use an older build. This is a comparison with published historical results, not a controlled harness-only ablation. See [comparison.json](comparison.json).
+RoboCasa and LIBERO use the latest reviewed catalog instructions and the same native simulator source, assets, and task settings as their Kinex references. LIBERO Kinex was fully rerun at caac19a; RoboCasa retains its current a1eaece results. The intervening Kinex commits affect session timing and footer display. See [comparison.json](comparison.json) for paired results, usage, timing, and evidence boundaries.
 
-[Results](episodes.csv) · [Attempt history](attempt-history.json) · [Protocol](protocol.json)
+Native spectator videos are presented at 4x speed and 20 FPS. Per-task pages retain full sessions, observed images, saved tools and memos, native verdicts, instructions, protocol, usage, and source identities. Native failures are valid results; detailed failure mechanisms are unreviewed unless supported by a separate review. The four historical RoboDojo provider interruptions remain in [attempt-history.json](attempt-history.json).
+
+[Results](episodes.csv) · [Protocol](protocol.json) · [Comparison](comparison.json)

THIRD_PARTY_NOTICES.md

@@ -1,5 +1,16 @@
 # Third-party notices
 
-This edition presents RoboDojo evaluation recordings and agent evidence.
+This edition presents rendered episodes from RoboCasa, LIBERO Long and RoboDojo. No simulator asset packs are redistributed.
+
+- robocasa: pinned source `456174f62b89b8fca99eaaf33949c29fec9cfc2a`; see [license](licenses/robocasa-LICENSE.txt).
+- robosuite: pinned source `5ce6643f3092639d08f7b0f90ed1c6a84f50552c`; see [license](licenses/robosuite-LICENSE.txt).
+- LIBERO: pinned source `8f1084e3132a39270c3a13ebe37270a43ece2a01`; see [license](licenses/LIBERO-LICENSE.txt).
+- RoboDojo: pinned source `726e9aabfaa642203722eb126f5eaf0f37f3e1ad`; see [license](licenses/RoboDojo-LICENSE.txt).
+
+RoboCasa code is MIT-licensed. Its pinned README identifies assets and datasets as CC BY 4.0: https://github.com/robocasa/robocasa/tree/456174f62b89b8fca99eaaf33949c29fec9cfc2a#license . Video and observation images derive from that scene; the published video combines two native camera streams and accelerates simulation time by 4x. Attribution: the RoboCasa Team and the robosuite contributors, with component notices retained above. Underlying third-party assets retain their own terms. This publication makes no blanket license claim for all model outputs, recordings or bundled generated resources.
+
+LIBERO source: https://github.com/Lifelong-Robot-Learning/LIBERO/tree/8f1084e3132a39270c3a13ebe37270a43ece2a01 . Its README identifies code as MIT and datasets as CC BY 4.0. Attribution: the LIBERO authors and robosuite contributors; the original notices are preserved in licenses/.
+
+Instruction clarifications for three LIBERO Long tasks draw on [BenchMend](https://benchmend-gallery.static.hf.space/gallery/index.html). Attribution: the BenchMend contributors. Additional current catalog refinements are recorded in each task instruction and source provenance.
 
 RoboDojo source: https://github.com/RoboDojo-Benchmark/RoboDojo/tree/726e9aabfaa642203722eb126f5eaf0f37f3e1ad . The pinned source carries the RoboDojo Non-Commercial Research License; see licenses/RoboDojo-LICENSE.txt. Attribution: the RoboDojo authors and the Isaac Sim, robotics and scene-asset contributors. This release presents non-commercial evaluation results, rendered views and agent evidence. No simulator asset packs, saved-layout archives or Isaac Sim software are redistributed. Underlying components retain their own terms.

app.js

@@ -31,7 +31,7 @@ function failureCategories(review){
 }
 function renderFailureOverview(){
  const failures=data.episodes.filter(e=>!e.success);
- $('#failure-overview').innerHTML=`<div class="section-top"><h2>RoboDojo · 当前结果与失败分析</h2><a href="attempt-history.json" target="_blank" rel="noopener">Attempt history ↗</a></div><p class="section-note">${data.tasks.length} 个任务 · ${data.episodes.filter(e=>e.success).length} 个成功 · ${failures.length} 个原生失败 · ${data.tasks.length-data.episodes.length} 个待发布。原生判定与完整 session 已保留；具体失败机制尚未逐项定位。服务端中断及其补跑记录见尝试历史。</p>`;
+ $('#failure-overview').innerHTML=`<div class="section-top"><h2>当前结果与失败分析</h2><a href="attempt-history.json" target="_blank" rel="noopener">Attempt history ↗</a></div><p class="section-note">${data.tasks.length} 个任务 · ${data.episodes.filter(e=>e.success).length} 个成功 · ${failures.length} 个原生失败 · ${data.tasks.length-data.episodes.length} 个待发布。原生判定与完整 session 已保留；具体失败机制尚未逐项定位。服务端中断及其补跑记录见尝试历史。</p>`;
  $('#failure-filter').innerHTML='<option value="all">All results</option><option value="failed">All unsuccessful results</option>';
 }
 const successPercent=value=>value==null?'—':(value*100).toFixed(2)+'%';

comparison.json

@@ -1,354 +1,2301 @@
 {
-  "baseline_revision": "1a7f1c9f178ba7aeb659e5b527a77978e9136072",
-  "summary": {
-    "completed": 42,
-    "codex_successes": 31,
-    "codex_success_rate_completed_subset": 0.7380952380952381,
-    "kinex_successes_same_subset": 35,
-    "kinex_successes_all42": 35,
-    "all42_comparison_complete": true
+  "reference_repo": "RLE-Bench/kinex-v0.10.0-benchmark",
+  "reference_delivery_time": "2026-10-09T17:52:51.393664+00:00",
+  "reference_revisions": {
+    "dataset": "07dd924831cf10515ccda0a05d4a8ce169df7965",
+    "space": "83483dadd2a7140a4f23bd247ff91183444b1da5"
   },
-  "tasks": [
+  "families": [
     {
-      "task_key": "task04/01",
-      "native_id": "robodojo/make-toast",
-      "status": "completed",
-      "codex_success": false,
-      "kinex_success": false,
-      "kinex_version": "0.10.3"
+      "family": "task01",
+      "tasks": 10,
+      "codex_successes": 2,
+      "kinex_successes": 1,
+      "codex_usage": {
+        "input_tokens": 63285354,
+        "cached_input_tokens": 62436736,
+        "uncached_input_tokens": 848618,
+        "output_tokens": 204733,
+        "reasoning_output_tokens": 78505,
+        "response_count": 1112,
+        "cost_usd": null,
+        "attempts": 10,
+        "reported_attempts": {
+          "input_tokens": 10,
+          "cached_input_tokens": 10,
+          "uncached_input_tokens": 10,
+          "output_tokens": 10,
+          "reasoning_output_tokens": 10,
+          "response_count": 10,
+          "cost_usd": 0
+        },
+        "audit_complete_attempts": 10,
+        "cache_hit_rate": 0.9865906098905601,
+        "tokens_per_task": {
+          "input_tokens": 6328535.4,
+          "cached_input_tokens": 6243673.6,
+          "uncached_input_tokens": 84861.8,
+          "output_tokens": 20473.3,
+          "reasoning_output_tokens": 7850.5
+        }
+      },
+      "kinex_usage": {
+        "input_tokens": 120363624,
+        "cached_input_tokens": 118975104,
+        "uncached_input_tokens": 1388520,
+        "output_tokens": 247504,
+        "reasoning_output_tokens": 98713,
+        "response_count": 2003,
+        "cost_usd": null,
+        "attempts": 10,
+        "reported_attempts": {
+          "input_tokens": 10,
+          "cached_input_tokens": 10,
+          "uncached_input_tokens": 10,
+          "output_tokens": 10,
+          "reasoning_output_tokens": 10,
+          "response_count": 10,
+          "cost_usd": 0
+        },
+        "audit_complete_attempts": 10,
+        "cache_hit_rate": 0.9884639565189562,
+        "tokens_per_task": {
+          "input_tokens": 12036362.4,
+          "cached_input_tokens": 11897510.4,
+          "uncached_input_tokens": 138852.0,
+          "output_tokens": 24750.4,
+          "reasoning_output_tokens": 9871.3
+        }
+      },
+      "codex_wall_time": {
+        "reported_attempts": 10,
+        "sum_s": 11720.851507,
+        "mean_s": 1172.0851507,
+        "median_s": 1239.5881319999999
+      },
+      "kinex_wall_time": {
+        "reported_attempts": 10,
+        "sum_s": 21798.805297,
+        "mean_s": 2179.8805297,
+        "median_s": 2131.8586855000003
+      }
     },
     {
-      "task_key": "task04/02",
-      "native_id": "robodojo/classify-objects-by-language",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.0"
-    },
+      "family": "task02",
+      "tasks": 10,
+      "codex_successes": 9,
+      "kinex_successes": 10,
+      "codex_usage": {
+        "input_tokens": 12586499,
+        "cached_input_tokens": 12106752,
+        "uncached_input_tokens": 479747,
+        "output_tokens": 93969,
+        "reasoning_output_tokens": 34987,
+        "response_count": 344,
+        "cost_usd": null,
+        "attempts": 10,
+        "reported_attempts": {
+          "input_tokens": 10,
+          "cached_input_tokens": 10,
+          "uncached_input_tokens": 10,
+          "output_tokens": 10,
+          "reasoning_output_tokens": 10,
+          "response_count": 10,
+          "cost_usd": 0
+        },
+        "audit_complete_attempts": 10,
+        "cache_hit_rate": 0.9618839996729829,
+        "tokens_per_task": {
+          "input_tokens": 1258649.9,
+          "cached_input_tokens": 1210675.2,
+          "uncached_input_tokens": 47974.7,
+          "output_tokens": 9396.9,
+          "reasoning_output_tokens": 3498.7
+        }
+      },
+      "kinex_usage": {
+        "input_tokens": 17966387,
+        "cached_input_tokens": 17024128,
+        "uncached_input_tokens": 942259,
+        "output_tokens": 83439,
+        "reasoning_output_tokens": 26298,
+        "response_count": 599,
+        "cost_usd": null,
+        "attempts": 10,
+        "reported_attempts": {
+          "input_tokens": 10,
+          "cached_input_tokens": 10,
+          "uncached_input_tokens": 10,
+          "output_tokens": 10,
+          "reasoning_output_tokens": 10,
+          "response_count": 10,
+          "cost_usd": 0
+        },
+        "audit_complete_attempts": 10,
+        "cache_hit_rate": 0.947554341337521,
+        "tokens_per_task": {
+          "input_tokens": 1796638.7,
+          "cached_input_tokens": 1702412.8,
+          "uncached_input_tokens": 94225.9,
+          "output_tokens": 8343.9,
+          "reasoning_output_tokens": 2629.8
+        }
+      },
+      "codex_wall_time": {
+        "reported_attempts": 10,
+        "sum_s": 3681.418654,
+        "mean_s": 368.14186540000003,
+        "median_s": 266.88804849999997
+      },
+      "kinex_wall_time": {
+        "reported_attempts": 10,
+        "sum_s": 4811.317263,
+        "mean_s": 481.1317263,
+        "median_s": 409.560245
+      }
+    }
+  ],
+  "tasks": [
     {
-      "task_key": "task04/03",
-      "native_id": "robodojo/store-laptop-and-headphones",
-      "status": "completed",
+      "task_key": "task01/01",
+      "instruction": "Place the specified condiments from the counter on the top shelf of the fridge. Move any existing top-shelf items to other shelves. Release the items and move the gripper clear of the stored items.",
+      "instruction_policy": "modified",
       "codex_success": false,
       "kinex_success": false,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/04",
-      "native_id": "robodojo/cover-blocks",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.2"
+      "codex_steps": 5496,
+      "kinex_steps": 3704,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9875774826559943,
+        "cache_reported_input_tokens": 7077229,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 7077229,
+        "cached_input_tokens": 6989312,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 7077229,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 6989312,
+        "known_input_tokens": 7077229,
+        "known_output_tokens": 20496,
+        "known_reasoning_output_tokens": 5873,
+        "output_tokens": 20496,
+        "reasoning_output_tokens": 5873,
+        "reasoning_reported_output_tokens": 20496,
+        "reported_responses": {
+          "cache_reported_input_tokens": 140,
+          "cache_write_input_tokens": 140,
+          "cache_write_reported_input_tokens": 140,
+          "cached_input_tokens": 140,
+          "input_tokens": 140,
+          "output_tokens": 140,
+          "reasoning_output_tokens": 140,
+          "reasoning_reported_output_tokens": 140
+        },
+        "response_count": 140,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 87917,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 9503123,
+        "output_tokens": 23563,
+        "cached_input_tokens": 9397760,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 8032,
+        "cache_reported_input_tokens": 9503123,
+        "cache_write_reported_input_tokens": 9503123,
+        "reasoning_reported_output_tokens": 23563,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 191,
+        "reported_responses": {
+          "input_tokens": 191,
+          "output_tokens": 191,
+          "cached_input_tokens": 191,
+          "cache_write_input_tokens": 191,
+          "reasoning_output_tokens": 191,
+          "cache_reported_input_tokens": 191,
+          "cache_write_reported_input_tokens": 191,
+          "reasoning_reported_output_tokens": 191
+        },
+        "known_cached_input_tokens": 9397760,
+        "known_input_tokens": 9503123,
+        "known_output_tokens": 23563,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 8032,
+        "cache_hit_rate": 0.9889128026649766,
+        "uncached_input_tokens": 105363,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 192,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 1267.271401,
+      "kinex_wall_time_s": 1805.038681,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/05",
-      "native_id": "robodojo/play-xylophone",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.0"
-    },
-    {
-      "task_key": "task04/06",
-      "native_id": "robodojo/store-tools-in-toolbox",
-      "status": "completed",
+      "task_key": "task01/02",
+      "instruction": "Remove the specified fruit from the bowl and place it on the small plate. Then place the bowl with only the specified meat in the microwave, close the door, and press the start button to microwave the meat. Release the bowl and move the gripper clear of it.",
+      "instruction_policy": "modified",
       "codex_success": false,
       "kinex_success": false,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/07",
-      "native_id": "robodojo/insert-tubes",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/08",
-      "native_id": "robodojo/deposit-coin",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.2"
-    },
-    {
-      "task_key": "task04/09",
-      "native_id": "robodojo/fasten-screws",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/10",
-      "native_id": "robodojo/play-stacking-toy",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/11",
-      "native_id": "robodojo/align-blocks",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.0"
-    },
-    {
-      "task_key": "task04/12",
-      "native_id": "robodojo/arrange-largest-number",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
+      "codex_steps": 5903,
+      "kinex_steps": 5048,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9810887797632594,
+        "cache_reported_input_tokens": 4438106,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 4438106,
+        "cached_input_tokens": 4354176,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 4438106,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 4354176,
+        "known_input_tokens": 4438106,
+        "known_output_tokens": 22048,
+        "known_reasoning_output_tokens": 9003,
+        "output_tokens": 22048,
+        "reasoning_output_tokens": 9003,
+        "reasoning_reported_output_tokens": 22048,
+        "reported_responses": {
+          "cache_reported_input_tokens": 86,
+          "cache_write_input_tokens": 86,
+          "cache_write_reported_input_tokens": 86,
+          "cached_input_tokens": 86,
+          "input_tokens": 86,
+          "output_tokens": 86,
+          "reasoning_output_tokens": 86,
+          "reasoning_reported_output_tokens": 86
+        },
+        "response_count": 86,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 83930,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 13113774,
+        "output_tokens": 31080,
+        "cached_input_tokens": 12981632,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 13344,
+        "cache_reported_input_tokens": 13113774,
+        "cache_write_reported_input_tokens": 13113774,
+        "reasoning_reported_output_tokens": 31080,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 214,
+        "reported_responses": {
+          "input_tokens": 214,
+          "output_tokens": 214,
+          "cached_input_tokens": 214,
+          "cache_write_input_tokens": 214,
+          "reasoning_output_tokens": 214,
+          "cache_reported_input_tokens": 214,
+          "cache_write_reported_input_tokens": 214,
+          "reasoning_reported_output_tokens": 214
+        },
+        "known_cached_input_tokens": 12981632,
+        "known_input_tokens": 13113774,
+        "known_output_tokens": 31080,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 13344,
+        "cache_hit_rate": 0.9899234194519442,
+        "uncached_input_tokens": 132142,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 215,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 1211.904863,
+      "kinex_wall_time_s": 2458.67869,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/14",
-      "native_id": "robodojo/build-tower",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/15",
-      "native_id": "robodojo/classify-objects",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.2"
-    },
-    {
-      "task_key": "task04/16",
-      "native_id": "robodojo/fill-egg-holder",
-      "status": "completed",
-      "codex_success": true,
+      "task_key": "task01/03",
+      "instruction": "Place two dumplings into each of the tupperware containers and then place both containers on a shelf in the fridge. Release the dumplings and containers and move the gripper clear of them.",
+      "instruction_policy": "modified",
+      "codex_success": false,
       "kinex_success": false,
-      "kinex_version": "0.10.3"
+      "codex_steps": 5053,
+      "kinex_steps": 2809,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9899699443798293,
+        "cache_reported_input_tokens": 11115392,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 11115392,
+        "cached_input_tokens": 11003904,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 11115392,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 11003904,
+        "known_input_tokens": 11115392,
+        "known_output_tokens": 29598,
+        "known_reasoning_output_tokens": 14166,
+        "output_tokens": 29598,
+        "reasoning_output_tokens": 14166,
+        "reasoning_reported_output_tokens": 29598,
+        "reported_responses": {
+          "cache_reported_input_tokens": 171,
+          "cache_write_input_tokens": 171,
+          "cache_write_reported_input_tokens": 171,
+          "cached_input_tokens": 171,
+          "input_tokens": 171,
+          "output_tokens": 171,
+          "reasoning_output_tokens": 171,
+          "reasoning_reported_output_tokens": 171
+        },
+        "response_count": 171,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 111488,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 6179963,
+        "output_tokens": 17710,
+        "cached_input_tokens": 6033792,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 5948,
+        "cache_reported_input_tokens": 6179963,
+        "cache_write_reported_input_tokens": 6179963,
+        "reasoning_reported_output_tokens": 17710,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 135,
+        "reported_responses": {
+          "input_tokens": 135,
+          "output_tokens": 135,
+          "cached_input_tokens": 135,
+          "cache_write_input_tokens": 135,
+          "reasoning_output_tokens": 135,
+          "cache_reported_input_tokens": 135,
+          "cache_write_reported_input_tokens": 135,
+          "reasoning_reported_output_tokens": 135
+        },
+        "known_cached_input_tokens": 6033792,
+        "known_input_tokens": 6179963,
+        "known_output_tokens": 17710,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 5948,
+        "cache_hit_rate": 0.9763475930195699,
+        "uncached_input_tokens": 146171,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 135,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 1749.701956,
+      "kinex_wall_time_s": 1444.988461,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/17",
-      "native_id": "robodojo/fill-pen-holder",
-      "status": "completed",
+      "task_key": "task01/04",
+      "instruction": "Gather the specified vegetables from the fridge and place them on a tray on the dining counter. Then gather the specified meats from the fridge and place them on the other tray. Release the food and move the gripper clear of the food and both trays.",
+      "instruction_policy": "modified",
       "codex_success": false,
       "kinex_success": false,
-      "kinex_version": "0.10.3"
+      "codex_steps": 5435,
+      "kinex_steps": 5449,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9899239913597869,
+        "cache_reported_input_tokens": 11714063,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 11714063,
+        "cached_input_tokens": 11596032,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 11714063,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 11596032,
+        "known_input_tokens": 11714063,
+        "known_output_tokens": 32111,
+        "known_reasoning_output_tokens": 11814,
+        "output_tokens": 32111,
+        "reasoning_output_tokens": 11814,
+        "reasoning_reported_output_tokens": 32111,
+        "reported_responses": {
+          "cache_reported_input_tokens": 190,
+          "cache_write_input_tokens": 190,
+          "cache_write_reported_input_tokens": 190,
+          "cached_input_tokens": 190,
+          "input_tokens": 190,
+          "output_tokens": 190,
+          "reasoning_output_tokens": 190,
+          "reasoning_reported_output_tokens": 190
+        },
+        "response_count": 190,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 118031,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 18456926,
+        "output_tokens": 44833,
+        "cached_input_tokens": 18169984,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 22099,
+        "cache_reported_input_tokens": 18456926,
+        "cache_write_reported_input_tokens": 18456926,
+        "reasoning_reported_output_tokens": 44833,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 236,
+        "reported_responses": {
+          "input_tokens": 236,
+          "output_tokens": 236,
+          "cached_input_tokens": 236,
+          "cache_write_input_tokens": 236,
+          "reasoning_output_tokens": 236,
+          "cache_reported_input_tokens": 236,
+          "cache_write_reported_input_tokens": 236,
+          "reasoning_reported_output_tokens": 236
+        },
+        "known_cached_input_tokens": 18169984,
+        "known_input_tokens": 18456926,
+        "known_output_tokens": 44833,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 22099,
+        "cache_hit_rate": 0.9844534241509122,
+        "uncached_input_tokens": 286942,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 237,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 1973.915146,
+      "kinex_wall_time_s": 3231.211091,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/18",
-      "native_id": "robodojo/fold-clothes",
-      "status": "completed",
+      "task_key": "task01/05",
+      "instruction": "Add the butter stick, sugar cube, and cream cheese stick to the stand mixer bowl, lower the mixer head fully, and turn the speed knob to begin making cheesecake filling.",
+      "instruction_policy": "modified",
       "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/20",
-      "native_id": "robodojo/general-pickup",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.0"
-    },
-    {
-      "task_key": "task04/21",
-      "native_id": "robodojo/hang-mugs",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/23",
-      "native_id": "robodojo/imitate-sorting-sequence",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.2"
-    },
-    {
-      "task_key": "task04/24",
-      "native_id": "robodojo/insert-key",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.2"
+      "kinex_success": false,
+      "codex_steps": 5224,
+      "kinex_steps": 5930,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9885663091343345,
+        "cache_reported_input_tokens": 12355153,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 12355153,
+        "cached_input_tokens": 12213888,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 12355153,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 12213888,
+        "known_input_tokens": 12355153,
+        "known_output_tokens": 35229,
+        "known_reasoning_output_tokens": 13787,
+        "output_tokens": 35229,
+        "reasoning_output_tokens": 13787,
+        "reasoning_reported_output_tokens": 35229,
+        "reported_responses": {
+          "cache_reported_input_tokens": 202,
+          "cache_write_input_tokens": 202,
+          "cache_write_reported_input_tokens": 202,
+          "cached_input_tokens": 202,
+          "input_tokens": 202,
+          "output_tokens": 202,
+          "reasoning_output_tokens": 202,
+          "reasoning_reported_output_tokens": 202
+        },
+        "response_count": 202,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 141265,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 10985080,
+        "output_tokens": 19054,
+        "cached_input_tokens": 10880128,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 6692,
+        "cache_reported_input_tokens": 10985080,
+        "cache_write_reported_input_tokens": 10985080,
+        "reasoning_reported_output_tokens": 19054,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 217,
+        "reported_responses": {
+          "input_tokens": 217,
+          "output_tokens": 217,
+          "cached_input_tokens": 217,
+          "cache_write_input_tokens": 217,
+          "reasoning_output_tokens": 217,
+          "cache_reported_input_tokens": 217,
+          "cache_write_reported_input_tokens": 217,
+          "reasoning_reported_output_tokens": 217
+        },
+        "known_cached_input_tokens": 10880128,
+        "known_input_tokens": 10985080,
+        "known_output_tokens": 19054,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 6692,
+        "cache_hit_rate": 0.9904459503253504,
+        "uncached_input_tokens": 104952,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 217,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 1964.925029,
+      "kinex_wall_time_s": 2570.336196,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/25",
-      "native_id": "robodojo/make-kong",
-      "status": "completed",
+      "task_key": "task01/06",
+      "instruction": "Turn on the sink faucet. Move the specified vegetable from the counter into the sink while the water is running. Turn off the faucet, move the vegetable into the pot next to the stove, and move the pot to the specified burner.",
+      "instruction_policy": "modified",
       "codex_success": false,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/27",
-      "native_id": "robodojo/match-and-pick-from-conveyor",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
+      "kinex_success": false,
+      "codex_steps": 5416,
+      "kinex_steps": 5856,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9880091550479709,
+        "cache_reported_input_tokens": 9630097,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 9630097,
+        "cached_input_tokens": 9514624,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 9630097,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 9514624,
+        "known_input_tokens": 9630097,
+        "known_output_tokens": 24897,
+        "known_reasoning_output_tokens": 10825,
+        "output_tokens": 24897,
+        "reasoning_output_tokens": 10825,
+        "reasoning_reported_output_tokens": 24897,
+        "reported_responses": {
+          "cache_reported_input_tokens": 145,
+          "cache_write_input_tokens": 145,
+          "cache_write_reported_input_tokens": 145,
+          "cached_input_tokens": 145,
+          "input_tokens": 145,
+          "output_tokens": 145,
+          "reasoning_output_tokens": 145,
+          "reasoning_reported_output_tokens": 145
+        },
+        "response_count": 145,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 115473,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 26424116,
+        "output_tokens": 28463,
+        "cached_input_tokens": 26255616,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 11086,
+        "cache_reported_input_tokens": 26424116,
+        "cache_write_reported_input_tokens": 26424116,
+        "reasoning_reported_output_tokens": 28463,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 347,
+        "reported_responses": {
+          "input_tokens": 347,
+          "output_tokens": 347,
+          "cached_input_tokens": 347,
+          "cache_write_input_tokens": 347,
+          "reasoning_output_tokens": 347,
+          "cache_reported_input_tokens": 347,
+          "cache_write_reported_input_tokens": 347,
+          "reasoning_reported_output_tokens": 347
+        },
+        "known_cached_input_tokens": 26255616,
+        "known_input_tokens": 26424116,
+        "known_output_tokens": 28463,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 11086,
+        "cache_hit_rate": 0.9936232493075643,
+        "uncached_input_tokens": 168500,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 347,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 1456.598048,
+      "kinex_wall_time_s": 3438.331724,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/28",
-      "native_id": "robodojo/organize-table",
-      "status": "completed",
+      "task_key": "task01/07",
+      "instruction": "Take the specified meat from the fridge and place it on the digital scale on the counter by the fridge. Release it and move the gripper clear while waiting a few seconds for a reading. Then move it to the plate on the dining counter, release it, and move the gripper clear.",
+      "instruction_policy": "modified",
       "codex_success": false,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
+      "kinex_success": false,
+      "codex_steps": 1426,
+      "kinex_steps": 2427,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9663948735854094,
+        "cache_reported_input_tokens": 1217225,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 1217225,
+        "cached_input_tokens": 1176320,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 1217225,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 1176320,
+        "known_input_tokens": 1217225,
+        "known_output_tokens": 8600,
+        "known_reasoning_output_tokens": 2758,
+        "output_tokens": 8600,
+        "reasoning_output_tokens": 2758,
+        "reasoning_reported_output_tokens": 8600,
+        "reported_responses": {
+          "cache_reported_input_tokens": 36,
+          "cache_write_input_tokens": 36,
+          "cache_write_reported_input_tokens": 36,
+          "cached_input_tokens": 36,
+          "input_tokens": 36,
+          "output_tokens": 36,
+          "reasoning_output_tokens": 36,
+          "reasoning_reported_output_tokens": 36
+        },
+        "response_count": 36,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 40905,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 7005688,
+        "output_tokens": 18192,
+        "cached_input_tokens": 6921216,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 6177,
+        "cache_reported_input_tokens": 7005688,
+        "cache_write_reported_input_tokens": 7005688,
+        "reasoning_reported_output_tokens": 18192,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 153,
+        "reported_responses": {
+          "input_tokens": 153,
+          "output_tokens": 153,
+          "cached_input_tokens": 153,
+          "cache_write_input_tokens": 153,
+          "reasoning_output_tokens": 153,
+          "cache_reported_input_tokens": 153,
+          "cache_write_reported_input_tokens": 153,
+          "reasoning_reported_output_tokens": 153
+        },
+        "known_cached_input_tokens": 6921216,
+        "known_input_tokens": 7005688,
+        "known_output_tokens": 18192,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 6177,
+        "cache_hit_rate": 0.987942369114925,
+        "uncached_input_tokens": 84472,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 155,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 394.373333,
+      "kinex_wall_time_s": 1396.812595,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/29",
-      "native_id": "robodojo/pack-objects-into-box",
-      "status": "completed",
-      "codex_success": true,
+      "task_key": "task01/08",
+      "instruction": "Pick up the sponge from the counter and scrub across a broad area of the cutting board, keeping the sponge grasped and in contact with the board throughout the scrubbing motion. Once finished, release the sponge and retract the gripper well away from it.",
+      "instruction_policy": "modified",
+      "codex_success": false,
       "kinex_success": true,
-      "kinex_version": "0.10.2"
+      "codex_steps": 1212,
+      "kinex_steps": 1007,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9606465096549688,
+        "cache_reported_input_tokens": 775611,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 775611,
+        "cached_input_tokens": 745088,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 775611,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 745088,
+        "known_input_tokens": 775611,
+        "known_output_tokens": 6607,
+        "known_reasoning_output_tokens": 1428,
+        "output_tokens": 6607,
+        "reasoning_output_tokens": 1428,
+        "reasoning_reported_output_tokens": 6607,
+        "reported_responses": {
+          "cache_reported_input_tokens": 26,
+          "cache_write_input_tokens": 26,
+          "cache_write_reported_input_tokens": 26,
+          "cached_input_tokens": 26,
+          "input_tokens": 26,
+          "output_tokens": 26,
+          "reasoning_output_tokens": 26,
+          "reasoning_reported_output_tokens": 26
+        },
+        "response_count": 26,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 30523,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 1225659,
+        "output_tokens": 6356,
+        "cached_input_tokens": 1182720,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 1012,
+        "cache_reported_input_tokens": 1225659,
+        "cache_write_reported_input_tokens": 1225659,
+        "reasoning_reported_output_tokens": 6356,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 42,
+        "reported_responses": {
+          "input_tokens": 42,
+          "output_tokens": 42,
+          "cached_input_tokens": 42,
+          "cache_write_input_tokens": 42,
+          "reasoning_output_tokens": 42,
+          "cache_reported_input_tokens": 42,
+          "cache_write_reported_input_tokens": 42,
+          "reasoning_reported_output_tokens": 42
+        },
+        "known_cached_input_tokens": 1182720,
+        "known_input_tokens": 1225659,
+        "known_output_tokens": 6356,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 1012,
+        "cache_hit_rate": 0.9649666016404237,
+        "uncached_input_tokens": 42939,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 42,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 276.421696,
+      "kinex_wall_time_s": 418.12241,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/31",
-      "native_id": "robodojo/pick-from-conveyor-by-image",
-      "status": "completed",
+      "task_key": "task01/09",
+      "instruction": "Pick the specified vegetable and the cream cheese from the fridge, place both fully inside the blender, and turn it on.",
+      "instruction_policy": "modified",
       "codex_success": false,
       "kinex_success": false,
-      "kinex_version": "0.10.3"
+      "codex_steps": 2216,
+      "kinex_steps": 4628,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9733556695336862,
+        "cache_reported_input_tokens": 1973478,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 1973478,
+        "cached_input_tokens": 1920896,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 1973478,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 1920896,
+        "known_input_tokens": 1973478,
+        "known_output_tokens": 11695,
+        "known_reasoning_output_tokens": 4578,
+        "output_tokens": 11695,
+        "reasoning_output_tokens": 4578,
+        "reasoning_reported_output_tokens": 11695,
+        "reported_responses": {
+          "cache_reported_input_tokens": 48,
+          "cache_write_input_tokens": 48,
+          "cache_write_reported_input_tokens": 48,
+          "cached_input_tokens": 48,
+          "input_tokens": 48,
+          "output_tokens": 48,
+          "reasoning_output_tokens": 48,
+          "reasoning_reported_output_tokens": 48
+        },
+        "response_count": 48,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 52582,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 19377448,
+        "output_tokens": 36860,
+        "cached_input_tokens": 19235968,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 16258,
+        "cache_reported_input_tokens": 19377448,
+        "cache_write_reported_input_tokens": 19377448,
+        "reasoning_reported_output_tokens": 36860,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 308,
+        "reported_responses": {
+          "input_tokens": 308,
+          "output_tokens": 308,
+          "cached_input_tokens": 308,
+          "cache_write_input_tokens": 308,
+          "reasoning_output_tokens": 308,
+          "cache_reported_input_tokens": 308,
+          "cache_write_reported_input_tokens": 308,
+          "reasoning_reported_output_tokens": 308
+        },
+        "known_cached_input_tokens": 19235968,
+        "known_input_tokens": 19377448,
+        "known_output_tokens": 36860,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 16258,
+        "cache_hit_rate": 0.9926987289554331,
+        "uncached_input_tokens": 141480,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 308,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 625.032726,
+      "kinex_wall_time_s": 3327.021406,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/32",
-      "native_id": "robodojo/play-tic-tac-toe",
-      "status": "completed",
+      "task_key": "task01/10",
+      "instruction": "Pick the specified vegetable from the fridge and hold it under running water from the sink faucet to wash it. Then place it on the tray next to the sink to prepare for roasting. Release it and move the gripper clear.",
+      "instruction_policy": "modified",
       "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
+      "kinex_success": false,
+      "codex_steps": 3969,
+      "kinex_steps": 4505,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9777504182000669,
+        "cache_reported_input_tokens": 2989000,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 2989000,
+        "cached_input_tokens": 2922496,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 2989000,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 2922496,
+        "known_input_tokens": 2989000,
+        "known_output_tokens": 13452,
+        "known_reasoning_output_tokens": 4273,
+        "output_tokens": 13452,
+        "reasoning_output_tokens": 4273,
+        "reasoning_reported_output_tokens": 13452,
+        "reported_responses": {
+          "cache_reported_input_tokens": 68,
+          "cache_write_input_tokens": 68,
+          "cache_write_reported_input_tokens": 68,
+          "cached_input_tokens": 68,
+          "input_tokens": 68,
+          "output_tokens": 68,
+          "reasoning_output_tokens": 68,
+          "reasoning_reported_output_tokens": 68
+        },
+        "response_count": 68,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 66504,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 8091847,
+        "output_tokens": 21393,
+        "cached_input_tokens": 7916288,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 8065,
+        "cache_reported_input_tokens": 8091847,
+        "cache_write_reported_input_tokens": 8091847,
+        "reasoning_reported_output_tokens": 21393,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 160,
+        "reported_responses": {
+          "input_tokens": 160,
+          "output_tokens": 160,
+          "cached_input_tokens": 160,
+          "cache_write_input_tokens": 160,
+          "reasoning_output_tokens": 160,
+          "cache_reported_input_tokens": 160,
+          "cache_write_reported_input_tokens": 160,
+          "reasoning_reported_output_tokens": 160
+        },
+        "known_cached_input_tokens": 7916288,
+        "known_input_tokens": 8091847,
+        "known_output_tokens": 21393,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 8065,
+        "cache_hit_rate": 0.9783042116342536,
+        "uncached_input_tokens": 175559,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "usage_source": "provider-raw",
+        "request_attempts": 160,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true,
+        "accounting": "reported-responses"
+      },
+      "codex_wall_time_s": 800.707309,
+      "kinex_wall_time_s": 1708.264043,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "a1eaece0bd01a25177a43c2091c10b213b416aa5",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/33",
-      "native_id": "robodojo/plug-in-charger",
-      "status": "completed",
+      "task_key": "task02/03",
+      "instruction": "turn on the stove and put the moka pot on it",
+      "instruction_policy": "original_native",
       "codex_success": true,
       "kinex_success": true,
-      "kinex_version": "0.10.3"
+      "codex_steps": 2013,
+      "kinex_steps": 706,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.967178914688548,
+        "cache_reported_input_tokens": 2225094,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 2225094,
+        "cached_input_tokens": 2152064,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 2225094,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 2152064,
+        "known_input_tokens": 2225094,
+        "known_output_tokens": 13673,
+        "known_reasoning_output_tokens": 5025,
+        "output_tokens": 13673,
+        "reasoning_output_tokens": 5025,
+        "reasoning_reported_output_tokens": 13673,
+        "reported_responses": {
+          "cache_reported_input_tokens": 50,
+          "cache_write_input_tokens": 50,
+          "cache_write_reported_input_tokens": 50,
+          "cached_input_tokens": 50,
+          "input_tokens": 50,
+          "output_tokens": 50,
+          "reasoning_output_tokens": 50,
+          "reasoning_reported_output_tokens": 50
+        },
+        "response_count": 50,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 73030,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 1221677,
+        "output_tokens": 6604,
+        "cached_input_tokens": 1146496,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 1955,
+        "cache_reported_input_tokens": 1221677,
+        "cache_write_reported_input_tokens": 1221677,
+        "reasoning_reported_output_tokens": 6604,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 47,
+        "reported_responses": {
+          "input_tokens": 47,
+          "output_tokens": 47,
+          "cached_input_tokens": 47,
+          "cache_write_input_tokens": 47,
+          "reasoning_output_tokens": 47,
+          "cache_reported_input_tokens": 47,
+          "cache_write_reported_input_tokens": 47,
+          "reasoning_reported_output_tokens": 47
+        },
+        "known_cached_input_tokens": 1146496,
+        "known_input_tokens": 1221677,
+        "known_output_tokens": 6604,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 1955,
+        "cache_hit_rate": 0.9384608206588158,
+        "uncached_input_tokens": 75181,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 47,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 531.974225,
+      "kinex_wall_time_s": 371.824558,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/34",
-      "native_id": "robodojo/pour-balls-into-vase",
-      "status": "completed",
+      "task_key": "task02/01",
+      "instruction": "put both the alphabet soup and the tomato sauce in the basket",
+      "instruction_policy": "original_native",
       "codex_success": true,
       "kinex_success": true,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/35",
-      "native_id": "robodojo/pour-by-language",
-      "status": "completed",
-      "codex_success": false,
-      "kinex_success": true,
-      "kinex_version": "0.10.0"
+      "codex_steps": 969,
+      "kinex_steps": 1094,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9509386255123954,
+        "cache_reported_input_tokens": 832121,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 832121,
+        "cached_input_tokens": 791296,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 832121,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 791296,
+        "known_input_tokens": 832121,
+        "known_output_tokens": 6802,
+        "known_reasoning_output_tokens": 1635,
+        "output_tokens": 6802,
+        "reasoning_output_tokens": 1635,
+        "reasoning_reported_output_tokens": 6802,
+        "reported_responses": {
+          "cache_reported_input_tokens": 27,
+          "cache_write_input_tokens": 27,
+          "cache_write_reported_input_tokens": 27,
+          "cached_input_tokens": 27,
+          "input_tokens": 27,
+          "output_tokens": 27,
+          "reasoning_output_tokens": 27,
+          "reasoning_reported_output_tokens": 27
+        },
+        "response_count": 27,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 40825,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 1546168,
+        "output_tokens": 7604,
+        "cached_input_tokens": 1461888,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 2069,
+        "cache_reported_input_tokens": 1546168,
+        "cache_write_reported_input_tokens": 1546168,
+        "reasoning_reported_output_tokens": 7604,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 55,
+        "reported_responses": {
+          "input_tokens": 55,
+          "output_tokens": 55,
+          "cached_input_tokens": 55,
+          "cache_write_input_tokens": 55,
+          "reasoning_output_tokens": 55,
+          "cache_reported_input_tokens": 55,
+          "cache_write_reported_input_tokens": 55,
+          "reasoning_reported_output_tokens": 55
+        },
+        "known_cached_input_tokens": 1461888,
+        "known_input_tokens": 1546168,
+        "known_output_tokens": 7604,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 2069,
+        "cache_hit_rate": 0.9454910462511189,
+        "uncached_input_tokens": 84280,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 55,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 272.779105,
+      "kinex_wall_time_s": 406.249632,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/36",
-      "native_id": "robodojo/pour-liquid-into-cup",
-      "status": "completed",
+      "task_key": "task02/02",
+      "instruction": "put both the cream cheese box and the butter in the basket",
+      "instruction_policy": "original_native",
       "codex_success": true,
       "kinex_success": true,
-      "kinex_version": "0.10.3"
+      "codex_steps": 722,
+      "kinex_steps": 1011,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9515345728205089,
+        "cache_reported_input_tokens": 784518,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 784518,
+        "cached_input_tokens": 746496,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 784518,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 746496,
+        "known_input_tokens": 784518,
+        "known_output_tokens": 6225,
+        "known_reasoning_output_tokens": 1371,
+        "output_tokens": 6225,
+        "reasoning_output_tokens": 1371,
+        "reasoning_reported_output_tokens": 6225,
+        "reported_responses": {
+          "cache_reported_input_tokens": 28,
+          "cache_write_input_tokens": 28,
+          "cache_write_reported_input_tokens": 28,
+          "cached_input_tokens": 28,
+          "input_tokens": 28,
+          "output_tokens": 28,
+          "reasoning_output_tokens": 28,
+          "reasoning_reported_output_tokens": 28
+        },
+        "response_count": 28,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 38022,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 2453051,
+        "output_tokens": 9069,
+        "cached_input_tokens": 2363776,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 2858,
+        "cache_reported_input_tokens": 2453051,
+        "cache_write_reported_input_tokens": 2453051,
+        "reasoning_reported_output_tokens": 9069,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 84,
+        "reported_responses": {
+          "input_tokens": 84,
+          "output_tokens": 84,
+          "cached_input_tokens": 84,
+          "cache_write_input_tokens": 84,
+          "reasoning_output_tokens": 84,
+          "cache_reported_input_tokens": 84,
+          "cache_write_reported_input_tokens": 84,
+          "reasoning_reported_output_tokens": 84
+        },
+        "known_cached_input_tokens": 2363776,
+        "known_input_tokens": 2453051,
+        "known_output_tokens": 9069,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 2858,
+        "cache_hit_rate": 0.9636065454815248,
+        "uncached_input_tokens": 89275,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 84,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 231.024585,
+      "kinex_wall_time_s": 611.087641,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/38",
-      "native_id": "robodojo/press-by-number",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/39",
-      "native_id": "robodojo/push-t",
-      "status": "completed",
+      "task_key": "task02/04",
+      "instruction": "put the black bowl in the bottom drawer of the cabinet and close it",
+      "instruction_policy": "original_native",
       "codex_success": false,
       "kinex_success": true,
-      "kinex_version": "0.10.3"
+      "codex_steps": 3687,
+      "kinex_steps": 1483,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9745746678635921,
+        "cache_reported_input_tokens": 3356377,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 3356377,
+        "cached_input_tokens": 3271040,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 3356377,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 3271040,
+        "known_input_tokens": 3356377,
+        "known_output_tokens": 23446,
+        "known_reasoning_output_tokens": 11600,
+        "output_tokens": 23446,
+        "reasoning_output_tokens": 11600,
+        "reasoning_reported_output_tokens": 23446,
+        "reported_responses": {
+          "cache_reported_input_tokens": 67,
+          "cache_write_input_tokens": 67,
+          "cache_write_reported_input_tokens": 67,
+          "cached_input_tokens": 67,
+          "input_tokens": 67,
+          "output_tokens": 67,
+          "reasoning_output_tokens": 67,
+          "reasoning_reported_output_tokens": 67
+        },
+        "response_count": 67,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 85337,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 1990960,
+        "output_tokens": 12374,
+        "cached_input_tokens": 1883904,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 5081,
+        "cache_reported_input_tokens": 1990960,
+        "cache_write_reported_input_tokens": 1990960,
+        "reasoning_reported_output_tokens": 12374,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 59,
+        "reported_responses": {
+          "input_tokens": 59,
+          "output_tokens": 59,
+          "cached_input_tokens": 59,
+          "cache_write_input_tokens": 59,
+          "reasoning_output_tokens": 59,
+          "cache_reported_input_tokens": 59,
+          "cache_write_reported_input_tokens": 59,
+          "reasoning_reported_output_tokens": 59
+        },
+        "known_cached_input_tokens": 1883904,
+        "known_input_tokens": 1990960,
+        "known_output_tokens": 12374,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 5081,
+        "cache_hit_rate": 0.9462289548760398,
+        "uncached_input_tokens": 107056,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 59,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 919.204412,
+      "kinex_wall_time_s": 624.888736,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/41",
-      "native_id": "robodojo/put-bottles-into-dustbin",
-      "status": "completed",
+      "task_key": "task02/05",
+      "instruction": "put the white mug in the center of the left plate and put the yellow and white mug in the center of the right plate, with each mug resting on its plate",
+      "instruction_policy": "modified",
       "codex_success": true,
       "kinex_success": true,
-      "kinex_version": "0.10.2"
+      "codex_steps": 634,
+      "kinex_steps": 765,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9493350495033964,
+        "cache_reported_input_tokens": 509662,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 509662,
+        "cached_input_tokens": 483840,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 509662,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 483840,
+        "known_input_tokens": 509662,
+        "known_output_tokens": 5212,
+        "known_reasoning_output_tokens": 1183,
+        "output_tokens": 5212,
+        "reasoning_output_tokens": 1183,
+        "reasoning_reported_output_tokens": 5212,
+        "reported_responses": {
+          "cache_reported_input_tokens": 20,
+          "cache_write_input_tokens": 20,
+          "cache_write_reported_input_tokens": 20,
+          "cached_input_tokens": 20,
+          "input_tokens": 20,
+          "output_tokens": 20,
+          "reasoning_output_tokens": 20,
+          "reasoning_reported_output_tokens": 20
+        },
+        "response_count": 20,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 25822,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 2532606,
+        "output_tokens": 7281,
+        "cached_input_tokens": 2444032,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 1949,
+        "cache_reported_input_tokens": 2532606,
+        "cache_write_reported_input_tokens": 2532606,
+        "reasoning_reported_output_tokens": 7281,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 88,
+        "reported_responses": {
+          "input_tokens": 88,
+          "output_tokens": 88,
+          "cached_input_tokens": 88,
+          "cache_write_input_tokens": 88,
+          "reasoning_output_tokens": 88,
+          "cache_reported_input_tokens": 88,
+          "cache_write_reported_input_tokens": 88,
+          "reasoning_reported_output_tokens": 88
+        },
+        "known_cached_input_tokens": 2444032,
+        "known_input_tokens": 2532606,
+        "known_output_tokens": 7281,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 1949,
+        "cache_hit_rate": 0.9650265378823236,
+        "uncached_input_tokens": 88574,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 88,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 195.147246,
+      "kinex_wall_time_s": 569.224064,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/42",
-      "native_id": "robodojo/solve-equation",
-      "status": "completed",
+      "task_key": "task02/06",
+      "instruction": "pick up the book and place it in the back compartment of the caddy, between the two large side compartments",
+      "instruction_policy": "modified",
       "codex_success": true,
       "kinex_success": true,
-      "kinex_version": "0.10.0"
+      "codex_steps": 325,
+      "kinex_steps": 441,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.956185574299006,
+        "cache_reported_input_tokens": 707210,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 707210,
+        "cached_input_tokens": 676224,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 707210,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 676224,
+        "known_input_tokens": 707210,
+        "known_output_tokens": 6550,
+        "known_reasoning_output_tokens": 2424,
+        "output_tokens": 6550,
+        "reasoning_output_tokens": 2424,
+        "reasoning_reported_output_tokens": 6550,
+        "reported_responses": {
+          "cache_reported_input_tokens": 25,
+          "cache_write_input_tokens": 25,
+          "cache_write_reported_input_tokens": 25,
+          "cached_input_tokens": 25,
+          "input_tokens": 25,
+          "output_tokens": 25,
+          "reasoning_output_tokens": 25,
+          "reasoning_reported_output_tokens": 25
+        },
+        "response_count": 25,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 30986,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 1345973,
+        "output_tokens": 6571,
+        "cached_input_tokens": 1272960,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 2281,
+        "cache_reported_input_tokens": 1345973,
+        "cache_write_reported_input_tokens": 1345973,
+        "reasoning_reported_output_tokens": 6571,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 54,
+        "reported_responses": {
+          "input_tokens": 54,
+          "output_tokens": 54,
+          "cached_input_tokens": 54,
+          "cache_write_input_tokens": 54,
+          "reasoning_output_tokens": 54,
+          "cache_reported_input_tokens": 54,
+          "cache_write_reported_input_tokens": 54,
+          "reasoning_reported_output_tokens": 54
+        },
+        "known_cached_input_tokens": 1272960,
+        "known_input_tokens": 1345973,
+        "known_output_tokens": 6571,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 2281,
+        "cache_hit_rate": 0.9457544839309555,
+        "uncached_input_tokens": 73013,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 54,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 241.745547,
+      "kinex_wall_time_s": 358.977488,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/43",
-      "native_id": "robodojo/sort-nesting-dolls-by-size",
-      "status": "completed",
+      "task_key": "task02/07",
+      "instruction": "put the white mug in the center of the plate and put the chocolate pudding immediately to the right of the plate",
+      "instruction_policy": "modified",
       "codex_success": true,
       "kinex_success": true,
-      "kinex_version": "0.10.3"
+      "codex_steps": 1016,
+      "kinex_steps": 1153,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9487277032832223,
+        "cache_reported_input_tokens": 701841,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 701841,
+        "cached_input_tokens": 665856,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 701841,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 665856,
+        "known_input_tokens": 701841,
+        "known_output_tokens": 6754,
+        "known_reasoning_output_tokens": 1904,
+        "output_tokens": 6754,
+        "reasoning_output_tokens": 1904,
+        "reasoning_reported_output_tokens": 6754,
+        "reported_responses": {
+          "cache_reported_input_tokens": 24,
+          "cache_write_input_tokens": 24,
+          "cache_write_reported_input_tokens": 24,
+          "cached_input_tokens": 24,
+          "input_tokens": 24,
+          "output_tokens": 24,
+          "reasoning_output_tokens": 24,
+          "reasoning_reported_output_tokens": 24
+        },
+        "response_count": 24,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 35985,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 1086944,
+        "output_tokens": 7094,
+        "cached_input_tokens": 980736,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 2071,
+        "cache_reported_input_tokens": 1086944,
+        "cache_write_reported_input_tokens": 1086944,
+        "reasoning_reported_output_tokens": 7094,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 40,
+        "reported_responses": {
+          "input_tokens": 40,
+          "output_tokens": 40,
+          "cached_input_tokens": 40,
+          "cache_write_input_tokens": 40,
+          "reasoning_output_tokens": 40,
+          "cache_reported_input_tokens": 40,
+          "cache_write_reported_input_tokens": 40,
+          "reasoning_reported_output_tokens": 40
+        },
+        "known_cached_input_tokens": 980736,
+        "known_input_tokens": 1086944,
+        "known_output_tokens": 7094,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 2071,
+        "cache_hit_rate": 0.9022875143521654,
+        "uncached_input_tokens": 106208,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 40,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 260.996992,
+      "kinex_wall_time_s": 349.052826,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/45",
-      "native_id": "robodojo/stack-blocks",
-      "status": "completed",
+      "task_key": "task02/08",
+      "instruction": "put both the alphabet soup and the cream cheese box fully inside the basket",
+      "instruction_policy": "modified",
       "codex_success": true,
       "kinex_success": true,
-      "kinex_version": "0.10.2"
+      "codex_steps": 1115,
+      "kinex_steps": 926,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9604073765886478,
+        "cache_reported_input_tokens": 1406070,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 1406070,
+        "cached_input_tokens": 1350400,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 1406070,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 1350400,
+        "known_input_tokens": 1406070,
+        "known_output_tokens": 9396,
+        "known_reasoning_output_tokens": 4471,
+        "output_tokens": 9396,
+        "reasoning_output_tokens": 4471,
+        "reasoning_reported_output_tokens": 9396,
+        "reported_responses": {
+          "cache_reported_input_tokens": 40,
+          "cache_write_input_tokens": 40,
+          "cache_write_reported_input_tokens": 40,
+          "cached_input_tokens": 40,
+          "input_tokens": 40,
+          "output_tokens": 40,
+          "reasoning_output_tokens": 40,
+          "reasoning_reported_output_tokens": 40
+        },
+        "response_count": 40,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 55670,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 1201086,
+        "output_tokens": 6987,
+        "cached_input_tokens": 1119616,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 2114,
+        "cache_reported_input_tokens": 1201086,
+        "cache_write_reported_input_tokens": 1201086,
+        "reasoning_reported_output_tokens": 6987,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 43,
+        "reported_responses": {
+          "input_tokens": 43,
+          "output_tokens": 43,
+          "cached_input_tokens": 43,
+          "cache_write_input_tokens": 43,
+          "reasoning_output_tokens": 43,
+          "cache_reported_input_tokens": 43,
+          "cache_write_reported_input_tokens": 43,
+          "reasoning_reported_output_tokens": 43
+        },
+        "known_cached_input_tokens": 1119616,
+        "known_input_tokens": 1201086,
+        "known_output_tokens": 6987,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 2114,
+        "cache_hit_rate": 0.9321697197369714,
+        "uncached_input_tokens": 81470,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 44,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 412.342365,
+      "kinex_wall_time_s": 389.816733,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/46",
-      "native_id": "robodojo/stack-blocks-by-language",
-      "status": "completed",
+      "task_key": "task02/09",
+      "instruction": "put both moka pots on the stove and turn the stove on",
+      "instruction_policy": "modified",
       "codex_success": true,
       "kinex_success": true,
-      "kinex_version": "0.10.0"
+      "codex_steps": 560,
+      "kinex_steps": 890,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9551717429852196,
+        "cache_reported_input_tokens": 714527,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 714527,
+        "cached_input_tokens": 682496,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 714527,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 682496,
+        "known_input_tokens": 714527,
+        "known_output_tokens": 5355,
+        "known_reasoning_output_tokens": 1306,
+        "output_tokens": 5355,
+        "reasoning_output_tokens": 1306,
+        "reasoning_reported_output_tokens": 5355,
+        "reported_responses": {
+          "cache_reported_input_tokens": 24,
+          "cache_write_input_tokens": 24,
+          "cache_write_reported_input_tokens": 24,
+          "cached_input_tokens": 24,
+          "input_tokens": 24,
+          "output_tokens": 24,
+          "reasoning_output_tokens": 24,
+          "reasoning_reported_output_tokens": 24
+        },
+        "response_count": 24,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 32031,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 1251289,
+        "output_tokens": 7233,
+        "cached_input_tokens": 1150848,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 2182,
+        "cache_reported_input_tokens": 1251289,
+        "cache_write_reported_input_tokens": 1251289,
+        "reasoning_reported_output_tokens": 7233,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 47,
+        "reported_responses": {
+          "input_tokens": 47,
+          "output_tokens": 47,
+          "cached_input_tokens": 47,
+          "cache_write_input_tokens": 47,
+          "reasoning_output_tokens": 47,
+          "cache_reported_input_tokens": 47,
+          "cache_write_reported_input_tokens": 47,
+          "reasoning_reported_output_tokens": 47
+        },
+        "known_cached_input_tokens": 1150848,
+        "known_input_tokens": 1251289,
+        "known_output_tokens": 7233,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 2182,
+        "cache_hit_rate": 0.9197299744503468,
+        "uncached_input_tokens": 100441,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 47,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 216.244925,
+      "kinex_wall_time_s": 412.870858,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     },
     {
-      "task_key": "task04/48",
-      "native_id": "robodojo/stack-bowls",
-      "status": "completed",
+      "task_key": "task02/10",
+      "instruction": "put the yellow and white mug in the microwave and close it",
+      "instruction_policy": "original_native",
       "codex_success": true,
       "kinex_success": true,
-      "kinex_version": "0.10.2"
-    },
-    {
-      "task_key": "task04/51",
-      "native_id": "robodojo/swap-t",
-      "status": "completed",
-      "codex_success": true,
-      "kinex_success": true,
-      "kinex_version": "0.10.2"
-    },
-    {
-      "task_key": "task04/52",
-      "native_id": "robodojo/swap-blocks",
-      "status": "completed",
-      "codex_success": false,
-      "kinex_success": false,
-      "kinex_version": "0.10.3"
-    },
-    {
-      "task_key": "task04/53",
-      "native_id": "robodojo/sweep-blocks",
-      "status": "completed",
-      "codex_success": false,
-      "kinex_success": true,
-      "kinex_version": "0.10.2"
+      "codex_steps": 1210,
+      "kinex_steps": 1950,
+      "codex_usage": {
+        "accounting": "reported-responses",
+        "audit_complete": true,
+        "cache_hit_rate": 0.9540138123860797,
+        "cache_reported_input_tokens": 1349079,
+        "cache_write_input_tokens": 0,
+        "cache_write_reported_input_tokens": 1349079,
+        "cached_input_tokens": 1287040,
+        "completed_turns": 1,
+        "cost_usd": null,
+        "failed_turns": 0,
+        "input_tokens": 1349079,
+        "known_cache_write_input_tokens": 0,
+        "known_cached_input_tokens": 1287040,
+        "known_input_tokens": 1349079,
+        "known_output_tokens": 10556,
+        "known_reasoning_output_tokens": 4068,
+        "output_tokens": 10556,
+        "reasoning_output_tokens": 4068,
+        "reasoning_reported_output_tokens": 10556,
+        "reported_responses": {
+          "cache_reported_input_tokens": 39,
+          "cache_write_input_tokens": 39,
+          "cache_write_reported_input_tokens": 39,
+          "cached_input_tokens": 39,
+          "input_tokens": 39,
+          "output_tokens": 39,
+          "reasoning_output_tokens": 39,
+          "reasoning_reported_output_tokens": 39
+        },
+        "response_count": 39,
+        "response_ids_complete": true,
+        "schema": "rlebench/token-usage/1",
+        "source": "Codex token_usage_record per response",
+        "uncached_input_tokens": 62039,
+        "unidentified_usage_records": 0
+      },
+      "kinex_usage": {
+        "input_tokens": 3336633,
+        "output_tokens": 12622,
+        "cached_input_tokens": 3199872,
+        "cache_write_input_tokens": 0,
+        "reasoning_output_tokens": 3738,
+        "cache_reported_input_tokens": 3336633,
+        "cache_write_reported_input_tokens": 3336633,
+        "reasoning_reported_output_tokens": 12622,
+        "schema": "rlebench/token-usage/1",
+        "source": "Provider terminal response usage",
+        "response_count": 82,
+        "reported_responses": {
+          "input_tokens": 82,
+          "output_tokens": 82,
+          "cached_input_tokens": 82,
+          "cache_write_input_tokens": 82,
+          "reasoning_output_tokens": 82,
+          "cache_reported_input_tokens": 82,
+          "cache_write_reported_input_tokens": 82,
+          "reasoning_reported_output_tokens": 82
+        },
+        "known_cached_input_tokens": 3199872,
+        "known_input_tokens": 3336633,
+        "known_output_tokens": 12622,
+        "known_cache_write_input_tokens": 0,
+        "known_reasoning_output_tokens": 3738,
+        "cache_hit_rate": 0.9590122737502147,
+        "uncached_input_tokens": 136761,
+        "cost_usd": null,
+        "response_ids_complete": true,
+        "accounting": "reported-responses",
+        "usage_source": "provider-raw",
+        "request_attempts": 82,
+        "rejected_attempts": 0,
+        "unresolved_attempts": [],
+        "audit_complete": true
+      },
+      "codex_wall_time_s": 399.959252,
+      "kinex_wall_time_s": 717.324727,
+      "kinex_version": "0.10.3",
+      "kinex_revision": "caac19a8a36272972f762e0f74cfe381e8500048",
+      "same_roboenv_source": true
     }
   ],
+  "preserved_robodojo_comparison": {
+    "baseline_revision": "1a7f1c9f178ba7aeb659e5b527a77978e9136072",
+    "summary": {
+      "completed": 42,
+      "codex_successes": 31,
+      "codex_success_rate_completed_subset": 0.7380952380952381,
+      "kinex_successes_same_subset": 35,
+      "kinex_successes_all42": 35,
+      "all42_comparison_complete": true
+    },
+    "tasks": [
+      {
+        "task_key": "task04/01",
+        "native_id": "robodojo/make-toast",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": false,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/02",
+        "native_id": "robodojo/classify-objects-by-language",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.0"
+      },
+      {
+        "task_key": "task04/03",
+        "native_id": "robodojo/store-laptop-and-headphones",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": false,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/04",
+        "native_id": "robodojo/cover-blocks",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/05",
+        "native_id": "robodojo/play-xylophone",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.0"
+      },
+      {
+        "task_key": "task04/06",
+        "native_id": "robodojo/store-tools-in-toolbox",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": false,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/07",
+        "native_id": "robodojo/insert-tubes",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/08",
+        "native_id": "robodojo/deposit-coin",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/09",
+        "native_id": "robodojo/fasten-screws",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/10",
+        "native_id": "robodojo/play-stacking-toy",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/11",
+        "native_id": "robodojo/align-blocks",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.0"
+      },
+      {
+        "task_key": "task04/12",
+        "native_id": "robodojo/arrange-largest-number",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/14",
+        "native_id": "robodojo/build-tower",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/15",
+        "native_id": "robodojo/classify-objects",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/16",
+        "native_id": "robodojo/fill-egg-holder",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": false,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/17",
+        "native_id": "robodojo/fill-pen-holder",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": false,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/18",
+        "native_id": "robodojo/fold-clothes",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/20",
+        "native_id": "robodojo/general-pickup",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.0"
+      },
+      {
+        "task_key": "task04/21",
+        "native_id": "robodojo/hang-mugs",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/23",
+        "native_id": "robodojo/imitate-sorting-sequence",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/24",
+        "native_id": "robodojo/insert-key",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/25",
+        "native_id": "robodojo/make-kong",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/27",
+        "native_id": "robodojo/match-and-pick-from-conveyor",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/28",
+        "native_id": "robodojo/organize-table",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/29",
+        "native_id": "robodojo/pack-objects-into-box",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/31",
+        "native_id": "robodojo/pick-from-conveyor-by-image",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": false,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/32",
+        "native_id": "robodojo/play-tic-tac-toe",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/33",
+        "native_id": "robodojo/plug-in-charger",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/34",
+        "native_id": "robodojo/pour-balls-into-vase",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/35",
+        "native_id": "robodojo/pour-by-language",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": true,
+        "kinex_version": "0.10.0"
+      },
+      {
+        "task_key": "task04/36",
+        "native_id": "robodojo/pour-liquid-into-cup",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/38",
+        "native_id": "robodojo/press-by-number",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/39",
+        "native_id": "robodojo/push-t",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/41",
+        "native_id": "robodojo/put-bottles-into-dustbin",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/42",
+        "native_id": "robodojo/solve-equation",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.0"
+      },
+      {
+        "task_key": "task04/43",
+        "native_id": "robodojo/sort-nesting-dolls-by-size",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/45",
+        "native_id": "robodojo/stack-blocks",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/46",
+        "native_id": "robodojo/stack-blocks-by-language",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.0"
+      },
+      {
+        "task_key": "task04/48",
+        "native_id": "robodojo/stack-bowls",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/51",
+        "native_id": "robodojo/swap-t",
+        "status": "completed",
+        "codex_success": true,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      },
+      {
+        "task_key": "task04/52",
+        "native_id": "robodojo/swap-blocks",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": false,
+        "kinex_version": "0.10.3"
+      },
+      {
+        "task_key": "task04/53",
+        "native_id": "robodojo/sweep-blocks",
+        "status": "completed",
+        "codex_success": false,
+        "kinex_success": true,
+        "kinex_version": "0.10.2"
+      }
+    ],
+    "limitations": [
+      "Historical Kinex baseline mixes three runtime versions.",
+      "24 tasks share RoboEnv source/build; 18 use an older build.",
+      "The first six completed Codex tasks used original request defaults; later attempts use 50 retries."
+    ]
+  },
   "limitations": [
-    "Historical Kinex baseline mixes three runtime versions.",
-    "24 tasks share RoboEnv source/build; 18 use an older build.",
-    "The first six completed Codex tasks used original request defaults; later attempts use 50 retries."
+    "Single seed and one native episode per task; these are observed success rates, not population estimates.",
+    "Wall time is operational latency under different concurrency, not a controlled speed benchmark; Kinex used up to five concurrent tasks and the new Codex dispatcher admits up to twenty.",
+    "LIBERO Kinex uses caac19a. The retained RoboCasa Kinex reference uses a1eaece; the two intervening Kinex commits change session timing and footer display.",
+    "The ordinary Codex baseline uses its native exec tools and the public RoboEnv SDK/CLI. Kinex uses its robotics tools.",
+    "RoboDojo retains its previously published protocol and historical source differences."
   ]
 }

episodes.csv

@@ -1,4 +1,24 @@
 task_key,native_id,status,run_status,success,steps,input_tokens,cached_input_tokens,output_tokens
+task01/01,robocasa/load-condiments-in-fridge,completed,finished,False,5496,7077229,6989312,20496
+task01/02,robocasa/filter-microwavable-item,completed,finished,False,5903,4438106,4354176,22048
+task01/03,robocasa/store-dumplings,completed,finished,False,5053,11115392,11003904,29598
+task01/04,robocasa/divide-buffet-trays,completed,finished,False,5435,11714063,11596032,32111
+task01/05,robocasa/make-cheesecake-filling,completed,finished,True,5224,12355153,12213888,35229
+task01/06,robocasa/multistep-steaming,completed,finished,False,5416,9630097,9514624,24897
+task01/07,robocasa/scale-portioning,completed,finished,False,1426,1217225,1176320,8600
+task01/08,robocasa/scrub-cutting-board,completed,finished,False,1212,775611,745088,6607
+task01/09,robocasa/prepare-veggie-dip,completed,finished,False,2216,1973478,1920896,11695
+task01/10,robocasa/prepare-vegetable-roasting,completed,finished,True,3969,2989000,2922496,13452
+task02/01,libero/10-0,completed,finished,True,969,832121,791296,6802
+task02/02,libero/10-1,completed,finished,True,722,784518,746496,6225
+task02/03,libero/10-2,completed,finished,True,2013,2225094,2152064,13673
+task02/04,libero/bowl-into-bottom-drawer,completed,finished,False,3687,3356377,3271040,23446
+task02/05,libero/10-4,completed,finished,True,634,509662,483840,5212
+task02/06,libero/10-5,completed,finished,True,325,707210,676224,6550
+task02/07,libero/10-6,completed,finished,True,1016,701841,665856,6754
+task02/08,libero/10-7,completed,finished,True,1115,1406070,1350400,9396
+task02/09,libero/10-8,completed,finished,True,560,714527,682496,5355
+task02/10,libero/mug-into-microwave,completed,finished,True,1210,1349079,1287040,10556
 task04/01,robodojo/make-toast,completed,finished,False,6884,13262282,13103360,42295
 task04/02,robodojo/classify-objects-by-language,completed,finished,True,1642,1667619,1598848,9371
 task04/03,robodojo/store-laptop-and-headphones,completed,finished,False,5696,18189657,17666048,45612

index.html

@@ -1,5 +1,5 @@
 <!doctype html>
-<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><meta name="description" content="Codex robotics benchmark: original and revised instructions, seed 0, GPT-6 Astra high. Explore task videos, sessions and evidence on RoboDojo."><title>Codex Benchmark · RLE-Bench</title><link rel="icon" href="favicon.svg"><link rel="stylesheet" href="style.css"><script defer src="app.js"></script></head>
-<body><header class="hero"><div class="wrap"><div class="topbar"><a class="brand" href="#"><span class="brand-icon" aria-hidden="true">◇</span> RLE-Bench</a><nav aria-label="Primary"><a href="#tasks">Tasks</a><a href="https://huggingface.co/datasets/RLE-Bench/codex-benchmark" target="_blank" rel="noopener">Dataset ↗</a><a href="REPORT.md" target="_blank" rel="noopener">Report ↗</a></nav></div><div class="hero-content"><div class="eyebrow">ROBOTICS EVALUATION / STOCK CODEX</div><h1>Codex Benchmark</h1><p>GPT-6 Astra · high · seed 0 · original &amp; modified instructions</p><div class="hero-bottom"><span>42 RoboDojo tasks. Native success criteria.</span><a class="hero-button" href="#task04">Explore the results <span>↗</span></a></div></div></div></header>
-<main class="wrap"><div id="loading" role="status">Loading benchmark…</div><div id="browse" hidden><div class="edition"><span id="edition-count">Loading results…</span><span class="chip" id="edition-label">Benchmark results</span></div><section aria-labelledby="overview-heading"><div class="section-top"><h2 id="overview-heading">Task overview</h2><a href="summary.json" target="_blank" rel="noopener">Summary JSON ↗</a></div><div class="table-wrap"><table><thead><tr><th>Environment</th><th>Coverage</th><th>Native success</th><th>Input tokens</th><th>Output tokens</th><th>Usage coverage</th></tr></thead><tbody id="overview"></tbody></table></div><p class="footnote">Current recorded result per task · Seed 0 · Native success criteria. Each task shows its latest result and the instruction used for that recording. Rates include valid completed results only; pending tasks are excluded. No pooled score across environments. Input includes cached tokens; output includes reasoning tokens.</p></section><section id="failure-overview" lang="zh-CN" aria-label="失败原因"></section><p id="readiness-note" class="readiness-note" hidden></p><div id="tasks" class="task-controls"><nav id="family-nav" aria-label="Task families"></nav><div class="filters"><label class="search-label"><span class="sr-only">Search tasks</span><input id="search" type="search" placeholder="Search 42 tasks…"></label><label><span class="sr-only">Filter by status</span><select id="status-filter"><option value="all">All tasks</option><option value="completed">Results available</option><option value="pending">Pending</option></select></label><label><span class="sr-only">Filter by failure category</span><select id="failure-filter" aria-label="Filter by failure category"><option value="all">All failure categories</option></select></label><span id="filter-count" aria-live="polite"></span></div></div><div id="families"></div><p id="empty" hidden>No tasks match this search.</p><section class="protocol-box"><h2>The evaluation protocol</h2><div class="protocol-grid"><div><b>Instruction versions</b><p>Modified instructions are labeled, with changes highlighted. Each recording retains the instruction it used; native success predicates are unchanged.</p></div><div><b>Single seed</b><p>One independent episode, seed 0. GPT-6 Astra with high reasoning effort. Stock Codex CLI 0.160.0; each task starts with a fresh workspace and no inherited trajectory.</p></div><div><b>Native control budgets</b><p>RoboDojo: 7,500 native action frames at 25 Hz. Stepped simulation, with an eight-hour wall-clock limit.</p></div></div></section></div><article id="detail" hidden></article></main>
+<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><meta name="description" content="Codex robotics benchmark: original and revised instructions, seed 0, GPT-6 Astra high. Explore task videos, sessions and evidence across RoboCasa, LIBERO and RoboDojo."><title>Codex Benchmark · RLE-Bench</title><link rel="icon" href="favicon.svg"><link rel="stylesheet" href="style.css"><script defer src="app.js"></script></head>
+<body><header class="hero"><div class="wrap"><div class="topbar"><a class="brand" href="#"><span class="brand-icon" aria-hidden="true">◇</span> RLE-Bench</a><nav aria-label="Primary"><a href="#tasks">Tasks</a><a href="https://huggingface.co/datasets/RLE-Bench/codex-benchmark" target="_blank" rel="noopener">Dataset ↗</a><a href="REPORT.md" target="_blank" rel="noopener">Report ↗</a></nav></div><div class="hero-content"><div class="eyebrow">ROBOTICS EVALUATION / STOCK CODEX</div><h1>Codex Benchmark</h1><p>GPT-6 Astra · high · seed 0 · original &amp; modified instructions</p><div class="hero-bottom"><span>62 tasks across three environments. Native success criteria.</span><a class="hero-button" href="#task04">Explore the results <span>↗</span></a></div></div></div></header>
+<main class="wrap"><div id="loading" role="status">Loading benchmark…</div><div id="browse" hidden><div class="edition"><span id="edition-count">Loading results…</span><span class="chip" id="edition-label">Benchmark results</span></div><section aria-labelledby="overview-heading"><div class="section-top"><h2 id="overview-heading">Task overview</h2><a href="summary.json" target="_blank" rel="noopener">Summary JSON ↗</a></div><div class="table-wrap"><table><thead><tr><th>Environment</th><th>Coverage</th><th>Native success</th><th>Input tokens</th><th>Output tokens</th><th>Usage coverage</th></tr></thead><tbody id="overview"></tbody></table></div><p class="footnote">Current recorded result per task · Seed 0 · Native success criteria. Each task shows its latest result and the instruction used for that recording. Rates include valid completed results only; pending tasks are excluded. No pooled score across environments. Input includes cached tokens; output includes reasoning tokens.</p></section><section id="failure-overview" lang="zh-CN" aria-label="失败原因"></section><p id="readiness-note" class="readiness-note" hidden></p><div id="tasks" class="task-controls"><nav id="family-nav" aria-label="Task families"></nav><div class="filters"><label class="search-label"><span class="sr-only">Search tasks</span><input id="search" type="search" placeholder="Search 62 tasks…"></label><label><span class="sr-only">Filter by status</span><select id="status-filter"><option value="all">All tasks</option><option value="completed">Results available</option><option value="pending">Pending</option></select></label><label><span class="sr-only">Filter by failure category</span><select id="failure-filter" aria-label="Filter by failure category"><option value="all">All failure categories</option></select></label><span id="filter-count" aria-live="polite"></span></div></div><div id="families"></div><p id="empty" hidden>No tasks match this search.</p><section class="protocol-box"><h2>The evaluation protocol</h2><div class="protocol-grid"><div><b>Instruction versions</b><p>Modified instructions are labeled, with changes highlighted. Each recording retains the instruction it used; native success predicates are unchanged.</p></div><div><b>Single seed</b><p>One independent episode, seed 0. GPT-6 Astra with high reasoning effort. Stock Codex CLI 0.160.0; each task starts with a fresh workspace and no inherited trajectory.</p></div><div><b>Native control budgets</b><p>RoboCasa and LIBERO: 6,000 steps at 20 Hz. RoboDojo: 7,500 frames at 25 Hz. Stepped simulation, with an eight-hour wall-clock limit.</p></div></div></section></div><article id="detail" hidden></article></main>
 <footer class="wrap"><span>RLE-Bench · Codex Benchmark</span><nav aria-label="Downloads"><a href="data.json">Data JSON</a><a href="episodes.csv">CSV</a><a href="DATA_FORMAT.md">Data format</a><a href="THIRD_PARTY_NOTICES.md">Attribution</a><a href="https://huggingface.co/datasets/RLE-Bench/codex-benchmark">Dataset ↗</a></nav></footer></body></html>

licenses/LIBERO-LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Lifelong Robot Learning
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

licenses/robocasa-LICENSE.txt

@@ -0,0 +1,28 @@
+MIT License
+
+Copyright (c) 2026 the RoboCasa Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+This software includes the partial implementation of Deepmind Mujoco https://github.com/deepmind/mujoco. 
+Deepmind Mujoco is licensed under the Apache License, Version 2.0 (the "License");
+you may not use the files except in compliance with the License. 
+
+You may obtain a copy of the License at
+    http://www.apache.org/licenses/LICENSE-2.0

licenses/robosuite-LICENSE.txt

@@ -0,0 +1,28 @@
+MIT License
+
+Copyright (c) 2022 Stanford Vision and Learning Lab and UT Robot Perception and Learning Lab
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+This software includes the partial implementation of Deepmind Mujoco https://github.com/deepmind/mujoco. 
+Deepmind Mujoco is licensed under the Apache License, Version 2.0 (the "License");
+you may not use the files except in compliance with the License. 
+
+You may obtain a copy of the License at
+    http://www.apache.org/licenses/LICENSE-2.0

manifest.json

@@ -1,21 +1,24 @@
 {
   ".gitattributes": "51dec59a208c8656ce7d6614b7b547022f7a4720b1cdad6a20aeeda717ec090b",
-  "DATA_FORMAT.md": "90cde3a7afcec6312f6705d47a959bd2135f6ed599b1ef70944eeca209b96778",
-  "README.md": "a89f667f1d8dab1804854081732761d76aaef1c086a6ca68e406e7ab32dcdef7",
-  "REPORT.md": "680246aca2267b3532a100e122cfa08dfb5c74cd6a7786818669e27820b62772",
-  "THIRD_PARTY_NOTICES.md": "b90a0d2dc93b1686caed55e3c68f85a1c761cb95a4935c692a223ab7e19c4c28",
-  "app.js": "3ef085aa517e167f8020eaef3ef26bbf4ca71f759e332c2358ddf3411c25c771",
+  "DATA_FORMAT.md": "e2fd4f6b5c8be9d9e008db7d0c1f5d52b419d68328a1c86625c9f67c49b03403",
+  "README.md": "43d89b6c59fdee0a1f5bddf3bd21cf0d809fee832f115771da95389f40f421b8",
+  "REPORT.md": "696454f04ac34467d2313e28b1e58b7095b02dc09e2b33b4b597594e10e3b59d",
+  "THIRD_PARTY_NOTICES.md": "8f38aa1ab2e03203b438e095a6325b3914541efe434290a9a2f786087d8f657d",
+  "app.js": "eb013cc9d24382410953a1a8c81e446a6592a81b32887749d17bfc863936556b",
   "attempt-history.json": "8555f458c1dd362f5a07447342994fddf265ff23bbc40e9bd6f9984a892ddb4e",
-  "comparison.json": "2d2ab0c2400ea6e4f14ffc3ec0acb536b1ddb8f32cc21394816d1e098e8b518c",
-  "data.json": "3411dc8b46b40e7a1336a197ba403402dd16fff11fb1b148add25ca55f6ea83f",
-  "episodes.csv": "cd6c36092565b08dd43863edea1a0c1c7af8b30c85a27184ae2e9e91aa8ba13f",
-  "episodes.json": "5f24633b68ad55179be36cd32fbb0317d534796f41e11f2a3262e58f7f108bfb",
+  "comparison.json": "012c7a64ecfde3352d258e332eae3b52de3d0c253663841e1a7dc9af3b3dfab7",
+  "data.json": "ca7388d97a0ed696d3b173eef5f8264bc76451b864baf2678df3144943bfe53e",
+  "episodes.csv": "ef6dc2beac16fbf02f603a770de4f3045eede1696d04c25cb0de789745fd4c69",
+  "episodes.json": "4dfa2559eee240954b0c2bf1dc8e15085f03d34dde749a21497c805cb7f57129",
   "failure-reviews.json": "c66a997c9ecd6f7f43f130a8a33cd64c7a5b2049033e9ccde653ee30e4796f23",
   "favicon.svg": "46063791218655f70c828ef71a1232a34fd38181da938338badec929ad226819",
-  "index.html": "da4a91fd226b8792395edbd3d2badda0e1fd69913d9c27480c12bcbdbbc73c37",
+  "index.html": "80694c5b55656fd3392b354a0d122586bb86a2a1466ca23d2031092299d72202",
+  "licenses/LIBERO-LICENSE.txt": "e2885fd30a08381b799c4a33385522b23d637b4051b8f9a7f9f2519944b68ff6",
   "licenses/RoboDojo-LICENSE.txt": "7794bb06af8fe5485ca912454ad1665ccd7846c45f9c2b1258658e480948cdbe",
-  "protocol.json": "b6a7aa409498f040e3edc1295594e68e6b3ec4b1d32a18499c3adf35a8a6ebd9",
+  "licenses/robocasa-LICENSE.txt": "5da18670b3f00c59847b1ded9c28dee59940d963b1e03b528b0108d9c5a09885",
+  "licenses/robosuite-LICENSE.txt": "177978cbece0a4c454c2aaec5b3f145b39270814874c43109da9e829c39d9cba",
+  "protocol.json": "8368960a967825a8d721cf982e61d028c5d72091530b6d54316e7626955e6dbd",
   "style.css": "dd31b30951c220ebb82bc96224b5e640913945679e42b2e0a42e2c373a329e6c",
-  "summary.json": "cdc48cc40a7726b4101c123c8ca931af17721aab78e649c07cfa32ca095fc76c",
-  "task-index.json": "75fd794d01e610337e8800d431a1d79b3dcc59f93dfeedf33f91b6232791d833"
+  "summary.json": "752f4bccc2f906bc94f05113a9fd906d8bc1ad512d3a4057ae885479b1b94611",
+  "task-index.json": "c5bee5272347efefba25529144c9e752bf3ca2d336942c6b57a34036d1774517"
 }

protocol.json

@@ -4,14 +4,52 @@
   "effort": "high",
   "seed": 0,
   "episodes_per_task": 1,
-  "max_steps": 7500,
-  "control_frequency_hz": 25,
   "mode": "stepped",
   "safety_timeout_sec": 28800,
   "fresh_workspace": true,
   "resume_trajectory": false,
   "source_jobs": [],
   "families": [
+    {
+      "id": "task01",
+      "name": "RoboCasa",
+      "total": 10,
+      "completed": 10,
+      "pending": 0,
+      "successes": 2,
+      "valid_results": 10,
+      "success_rate": 0.2,
+      "input_tokens": 63285354,
+      "cached_input_tokens": 62436736,
+      "output_tokens": 204733,
+      "usage_complete": 10,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "preflight_results": 0,
+      "formal_results": 10,
+      "modified_results": 10,
+      "original_results": 0
+    },
+    {
+      "id": "task02",
+      "name": "LIBERO Long",
+      "total": 10,
+      "completed": 10,
+      "pending": 0,
+      "successes": 9,
+      "valid_results": 10,
+      "success_rate": 0.9,
+      "input_tokens": 12586499,
+      "cached_input_tokens": 12106752,
+      "output_tokens": 93969,
+      "usage_complete": 10,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "preflight_results": 0,
+      "formal_results": 10,
+      "modified_results": 5,
+      "original_results": 5
+    },
     {
       "id": "task04",
       "name": "RoboDojo",
@@ -34,5 +72,26 @@
     }
   ],
   "usage_accounting": "reported-responses",
-  "retry_policy_note": "The first six completed results use original Codex request defaults. Remaining tasks and two authorized fresh replacements use a 50-retry request policy."
+  "retry_policy_note": "The first six completed results use original Codex request defaults. Remaining tasks and two authorized fresh replacements use a 50-retry request policy. All new RoboCasa and LIBERO runs use explicit 50-retry SSE request policy; no automatic episode retry.",
+  "family_protocols": {
+    "task01": {
+      "episodes": 1,
+      "seed": 0,
+      "max_steps": 6000,
+      "control_frequency_hz": 20
+    },
+    "task02": {
+      "episodes": 1,
+      "seed": 0,
+      "init_state_index": 0,
+      "max_steps": 6000,
+      "control_frequency_hz": 20
+    },
+    "task04": {
+      "episodes": 1,
+      "seed": 0,
+      "max_steps": 7500,
+      "control_frequency_hz": 25
+    }
+  }
 }

publication-manifest.json

@@ -6,44 +6,44 @@
       "bytes": 215
     },
     "DATA_FORMAT.md": {
-      "sha256": "90cde3a7afcec6312f6705d47a959bd2135f6ed599b1ef70944eeca209b96778",
+      "sha256": "e2fd4f6b5c8be9d9e008db7d0c1f5d52b419d68328a1c86625c9f67c49b03403",
       "bytes": 1129
     },
     "README.md": {
-      "sha256": "a89f667f1d8dab1804854081732761d76aaef1c086a6ca68e406e7ab32dcdef7",
-      "bytes": 1180
+      "sha256": "43d89b6c59fdee0a1f5bddf3bd21cf0d809fee832f115771da95389f40f421b8",
+      "bytes": 1688
     },
     "REPORT.md": {
-      "sha256": "680246aca2267b3532a100e122cfa08dfb5c74cd6a7786818669e27820b62772",
-      "bytes": 1057
+      "sha256": "696454f04ac34467d2313e28b1e58b7095b02dc09e2b33b4b597594e10e3b59d",
+      "bytes": 1565
     },
     "THIRD_PARTY_NOTICES.md": {
-      "sha256": "b90a0d2dc93b1686caed55e3c68f85a1c761cb95a4935c692a223ab7e19c4c28",
-      "bytes": 635
+      "sha256": "8f38aa1ab2e03203b438e095a6325b3914541efe434290a9a2f786087d8f657d",
+      "bytes": 2319
     },
     "app.js": {
-      "sha256": "3ef085aa517e167f8020eaef3ef26bbf4ca71f759e332c2358ddf3411c25c771",
-      "bytes": 24230
+      "sha256": "eb013cc9d24382410953a1a8c81e446a6592a81b32887749d17bfc863936556b",
+      "bytes": 24218
     },
     "attempt-history.json": {
       "sha256": "8555f458c1dd362f5a07447342994fddf265ff23bbc40e9bd6f9984a892ddb4e",
       "bytes": 36661
     },
     "comparison.json": {
-      "sha256": "2d2ab0c2400ea6e4f14ffc3ec0acb536b1ddb8f32cc21394816d1e098e8b518c",
-      "bytes": 9469
+      "sha256": "012c7a64ecfde3352d258e332eae3b52de3d0c253663841e1a7dc9af3b3dfab7",
+      "bytes": 84718
     },
     "data.json": {
-      "sha256": "3411dc8b46b40e7a1336a197ba403402dd16fff11fb1b148add25ca55f6ea83f",
-      "bytes": 611552
+      "sha256": "ca7388d97a0ed696d3b173eef5f8264bc76451b864baf2678df3144943bfe53e",
+      "bytes": 907017
     },
     "episodes.csv": {
-      "sha256": "cd6c36092565b08dd43863edea1a0c1c7af8b30c85a27184ae2e9e91aa8ba13f",
-      "bytes": 3796
+      "sha256": "ef6dc2beac16fbf02f603a770de4f3045eede1696d04c25cb0de789745fd4c69",
+      "bytes": 5479
     },
     "episodes.json": {
-      "sha256": "5f24633b68ad55179be36cd32fbb0317d534796f41e11f2a3262e58f7f108bfb",
-      "bytes": 440615
+      "sha256": "4dfa2559eee240954b0c2bf1dc8e15085f03d34dde749a21497c805cb7f57129",
+      "bytes": 667599
     },
     "failure-reviews.json": {
       "sha256": "c66a997c9ecd6f7f43f130a8a33cd64c7a5b2049033e9ccde653ee30e4796f23",
@@ -54,32 +54,44 @@
       "bytes": 227
     },
     "index.html": {
-      "sha256": "da4a91fd226b8792395edbd3d2badda0e1fd69913d9c27480c12bcbdbbc73c37",
-      "bytes": 4269
+      "sha256": "80694c5b55656fd3392b354a0d122586bb86a2a1466ca23d2031092299d72202",
+      "bytes": 4340
+    },
+    "licenses/LIBERO-LICENSE.txt": {
+      "sha256": "e2885fd30a08381b799c4a33385522b23d637b4051b8f9a7f9f2519944b68ff6",
+      "bytes": 1080
     },
     "licenses/RoboDojo-LICENSE.txt": {
       "sha256": "7794bb06af8fe5485ca912454ad1665ccd7846c45f9c2b1258658e480948cdbe",
       "bytes": 1091
     },
+    "licenses/robocasa-LICENSE.txt": {
+      "sha256": "5da18670b3f00c59847b1ded9c28dee59940d963b1e03b528b0108d9c5a09885",
+      "bytes": 1418
+    },
+    "licenses/robosuite-LICENSE.txt": {
+      "sha256": "177978cbece0a4c454c2aaec5b3f145b39270814874c43109da9e829c39d9cba",
+      "bytes": 1474
+    },
     "protocol.json": {
-      "sha256": "b6a7aa409498f040e3edc1295594e68e6b3ec4b1d32a18499c3adf35a8a6ebd9",
-      "bytes": 1083
+      "sha256": "8368960a967825a8d721cf982e61d028c5d72091530b6d54316e7626955e6dbd",
+      "bytes": 2574
     },
     "style.css": {
       "sha256": "dd31b30951c220ebb82bc96224b5e640913945679e42b2e0a42e2c373a329e6c",
       "bytes": 16560
     },
     "summary.json": {
-      "sha256": "cdc48cc40a7726b4101c123c8ca931af17721aab78e649c07cfa32ca095fc76c",
-      "bytes": 897
+      "sha256": "752f4bccc2f906bc94f05113a9fd906d8bc1ad512d3a4057ae885479b1b94611",
+      "bytes": 1922
     },
     "task-index.json": {
-      "sha256": "75fd794d01e610337e8800d431a1d79b3dcc59f93dfeedf33f91b6232791d833",
-      "bytes": 140012
+      "sha256": "c5bee5272347efefba25529144c9e752bf3ca2d336942c6b57a34036d1774517",
+      "bytes": 192726
     },
     "manifest.json": {
-      "sha256": "e96c5856c4c6770ab02a4623c1af0acfaf66c63a80a4803407e3abd6d29f1343",
-      "bytes": 1671
+      "sha256": "65df6bd8f0503fc521d5101cd2d41f2c4d1bd408d3f8ea672fd8363211d68e14",
+      "bytes": 1979
     }
   }
 }

summary.json

@@ -1,8 +1,48 @@
 {
-  "planned_tasks": 42,
-  "published_results": 42,
+  "planned_tasks": 62,
+  "published_results": 62,
   "pending_tasks": 0,
   "families": [
+    {
+      "id": "task01",
+      "name": "RoboCasa",
+      "total": 10,
+      "completed": 10,
+      "pending": 0,
+      "successes": 2,
+      "valid_results": 10,
+      "success_rate": 0.2,
+      "input_tokens": 63285354,
+      "cached_input_tokens": 62436736,
+      "output_tokens": 204733,
+      "usage_complete": 10,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "preflight_results": 0,
+      "formal_results": 10,
+      "modified_results": 10,
+      "original_results": 0
+    },
+    {
+      "id": "task02",
+      "name": "LIBERO Long",
+      "total": 10,
+      "completed": 10,
+      "pending": 0,
+      "successes": 9,
+      "valid_results": 10,
+      "success_rate": 0.9,
+      "input_tokens": 12586499,
+      "cached_input_tokens": 12106752,
+      "output_tokens": 93969,
+      "usage_complete": 10,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "preflight_results": 0,
+      "formal_results": 10,
+      "modified_results": 5,
+      "original_results": 5
+    },
     {
       "id": "task04",
       "name": "RoboDojo",
@@ -25,13 +65,13 @@
     }
   ],
   "progress": {
-    "finished": 42,
-    "interrupted": 0,
-    "native_failures": 11,
-    "native_successes": 31,
-    "needs_review": 0,
+    "finished": 62,
+    "running": 0,
     "queued": 0,
-    "running": 0
+    "needs_review": 0,
+    "interrupted": 0,
+    "native_successes": 42,
+    "native_failures": 20
   },
   "interrupted_attempts": 4,
   "usage_incomplete_tasks": [],

task-index.json

@@ -1,4 +1,1467 @@
 [
+  {
+    "key": "task01/01",
+    "family": "task01",
+    "slot": "01",
+    "native_id": "robocasa/load-condiments-in-fridge",
+    "title": "Load condiments in fridge",
+    "catalog_instruction": "Place the specified condiments from the counter on the top shelf of the fridge. Move any existing top-shelf items to other shelves. Release the items and move the gripper clear of the stored items.",
+    "native_instruction": "Place the {condiment1} and {condiment2} from the counter to the top shelf of the fridge. If the existing items in the fridge are on the top shelf, move them to other shelves.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-01-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/load-condiments-in-fridge",
+    "native_identity": {
+      "task_name": "LoadCondimentsInFridge",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Place the {condiment1} and {condiment2} from the counter to the top shelf of the fridge. If the existing items in the fridge are on the top shelf, move them to other shelves.",
+      "instruction": "Place the specified condiments from the counter on the top shelf of the fridge. Move any existing top-shelf items to other shelves. Release the items and move the gripper clear of the stored items.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Place the ",
+          "modified": "Place the "
+        },
+        {
+          "op": "replace",
+          "original": "{condiment1}",
+          "modified": "specified"
+        },
+        {
+          "op": "equal",
+          "original": " ",
+          "modified": " "
+        },
+        {
+          "op": "replace",
+          "original": "and {condiment2}",
+          "modified": "condiments"
+        },
+        {
+          "op": "equal",
+          "original": " from the counter ",
+          "modified": " from the counter "
+        },
+        {
+          "op": "replace",
+          "original": "to",
+          "modified": "on"
+        },
+        {
+          "op": "equal",
+          "original": " the top shelf of the fridge. ",
+          "modified": " the top shelf of the fridge. "
+        },
+        {
+          "op": "replace",
+          "original": "If",
+          "modified": "Move"
+        },
+        {
+          "op": "equal",
+          "original": " ",
+          "modified": " "
+        },
+        {
+          "op": "replace",
+          "original": "the",
+          "modified": "any"
+        },
+        {
+          "op": "equal",
+          "original": " existing ",
+          "modified": " existing "
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": "top-shelf "
+        },
+        {
+          "op": "equal",
+          "original": "items",
+          "modified": "items"
+        },
+        {
+          "op": "delete",
+          "original": " in the fridge are on the top shelf, move them",
+          "modified": ""
+        },
+        {
+          "op": "equal",
+          "original": " to other shelves.",
+          "modified": " to other shelves."
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": " Release the items and move the gripper clear of the stored items."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/01-load-condiments-in-fridge/task.yaml"
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task01/02",
+    "family": "task01",
+    "slot": "02",
+    "native_id": "robocasa/filter-microwavable-item",
+    "title": "Filter microwavable item",
+    "catalog_instruction": "Remove the specified fruit from the bowl and place it on the small plate. Then place the bowl with only the specified meat in the microwave, close the door, and press the start button to microwave the meat. Release the bowl and move the gripper clear of it.",
+    "native_instruction": "Remove the {fruit} from the bowl and place it on the small plate. Then place the bowl with only the {meat} in the microwave, close the door, and press the start button to microwave the {meat}.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-02-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/filter-microwavable-item",
+    "native_identity": {
+      "task_name": "FilterMicrowavableItem",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Remove the {fruit} from the bowl and place it on the small plate. Then place the bowl with only the {meat} in the microwave, close the door, and press the start button to microwave the {meat}.",
+      "instruction": "Remove the specified fruit from the bowl and place it on the small plate. Then place the bowl with only the specified meat in the microwave, close the door, and press the start button to microwave the meat. Release the bowl and move the gripper clear of it.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Remove the ",
+          "modified": "Remove the "
+        },
+        {
+          "op": "replace",
+          "original": "{fruit}",
+          "modified": "specified fruit"
+        },
+        {
+          "op": "equal",
+          "original": " from the bowl and place it on the small plate. Then place the bowl with only the ",
+          "modified": " from the bowl and place it on the small plate. Then place the bowl with only the "
+        },
+        {
+          "op": "replace",
+          "original": "{meat}",
+          "modified": "specified meat"
+        },
+        {
+          "op": "equal",
+          "original": " in the microwave, close the door, and press the start button to microwave the ",
+          "modified": " in the microwave, close the door, and press the start button to microwave the "
+        },
+        {
+          "op": "replace",
+          "original": "{meat}.",
+          "modified": "meat. Release the bowl and move the gripper clear of it."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/02-filter-microwavable-item/task.yaml"
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task01/03",
+    "family": "task01",
+    "slot": "03",
+    "native_id": "robocasa/store-dumplings",
+    "title": "Store dumplings",
+    "catalog_instruction": "Place two dumplings into each of the tupperware containers and then place both containers on a shelf in the fridge. Release the dumplings and containers and move the gripper clear of them.",
+    "native_instruction": "Place two dumplings into each of the tupperware containers and then place the containers in the fridge.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-03-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/store-dumplings",
+    "native_identity": {
+      "task_name": "StoreDumplings",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Place two dumplings into each of the tupperware containers and then place the containers in the fridge.",
+      "instruction": "Place two dumplings into each of the tupperware containers and then place both containers on a shelf in the fridge. Release the dumplings and containers and move the gripper clear of them.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Place two dumplings into each of the tupperware containers and then place ",
+          "modified": "Place two dumplings into each of the tupperware containers and then place "
+        },
+        {
+          "op": "replace",
+          "original": "the",
+          "modified": "both"
+        },
+        {
+          "op": "equal",
+          "original": " containers",
+          "modified": " containers"
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": " on a shelf"
+        },
+        {
+          "op": "equal",
+          "original": " in the fridge.",
+          "modified": " in the fridge."
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": " Release the dumplings and containers and move the gripper clear of them."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/03-store-dumplings/task.yaml"
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task01/04",
+    "family": "task01",
+    "slot": "04",
+    "native_id": "robocasa/divide-buffet-trays",
+    "title": "Divide buffet trays",
+    "catalog_instruction": "Gather the specified vegetables from the fridge and place them on a tray on the dining counter. Then gather the specified meats from the fridge and place them on the other tray. Release the food and move the gripper clear of the food and both trays.",
+    "native_instruction": "Gather the {vegetables} from the fridge and place them on a tray on the dining counter. Then gather the {meats} from the fridge and place them on the other tray.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-04-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/divide-buffet-trays",
+    "native_identity": {
+      "task_name": "DivideBuffetTrays",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Gather the {vegetables} from the fridge and place them on a tray on the dining counter. Then gather the {meats} from the fridge and place them on the other tray.",
+      "instruction": "Gather the specified vegetables from the fridge and place them on a tray on the dining counter. Then gather the specified meats from the fridge and place them on the other tray. Release the food and move the gripper clear of the food and both trays.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Gather the ",
+          "modified": "Gather the "
+        },
+        {
+          "op": "replace",
+          "original": "{vegetables}",
+          "modified": "specified vegetables"
+        },
+        {
+          "op": "equal",
+          "original": " from the fridge and place them on a tray on the dining counter. Then gather the ",
+          "modified": " from the fridge and place them on a tray on the dining counter. Then gather the "
+        },
+        {
+          "op": "replace",
+          "original": "{meats}",
+          "modified": "specified meats"
+        },
+        {
+          "op": "equal",
+          "original": " from the fridge and place them on the other tray.",
+          "modified": " from the fridge and place them on the other tray."
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": " Release the food and move the gripper clear of the food and both trays."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/04-divide-buffet-trays/task.yaml"
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task01/05",
+    "family": "task01",
+    "slot": "05",
+    "native_id": "robocasa/make-cheesecake-filling",
+    "title": "Make cheesecake filling",
+    "catalog_instruction": "Add the butter stick, sugar cube, and cream cheese stick to the stand mixer bowl, lower the mixer head fully, and turn the speed knob to begin making cheesecake filling.",
+    "native_instruction": "Add the butter stick, sugar cube, and cream cheese stick to the stand mixer bowl and then turn the speed knob to begin making cheesecake filling.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-05-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/make-cheesecake-filling",
+    "native_identity": {
+      "task_name": "MakeCheesecakeFilling",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Add the butter stick, sugar cube, and cream cheese stick to the stand mixer bowl and then turn the speed knob to begin making cheesecake filling.",
+      "instruction": "Add the butter stick, sugar cube, and cream cheese stick to the stand mixer bowl, lower the mixer head fully, and turn the speed knob to begin making cheesecake filling.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Add the butter stick, sugar cube, and cream cheese stick to the stand mixer ",
+          "modified": "Add the butter stick, sugar cube, and cream cheese stick to the stand mixer "
+        },
+        {
+          "op": "replace",
+          "original": "bowl",
+          "modified": "bowl, lower the mixer head fully,"
+        },
+        {
+          "op": "equal",
+          "original": " and",
+          "modified": " and"
+        },
+        {
+          "op": "delete",
+          "original": " then",
+          "modified": ""
+        },
+        {
+          "op": "equal",
+          "original": " turn the speed knob to begin making cheesecake filling.",
+          "modified": " turn the speed knob to begin making cheesecake filling."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/05-make-cheesecake-filling/task.yaml"
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task01/06",
+    "family": "task01",
+    "slot": "06",
+    "native_id": "robocasa/multistep-steaming",
+    "title": "Multistep steaming",
+    "catalog_instruction": "Turn on the sink faucet. Move the specified vegetable from the counter into the sink while the water is running. Turn off the faucet, move the vegetable into the pot next to the stove, and move the pot to the specified burner.",
+    "native_instruction": "Turn on the sink faucet. Then move the {vegetable} from the counter to the sink. Turn off the sink. Move the vegetable from the sink to the pot next to the stove. Finally move the pot to the {burner} burner.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-06-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/multistep-steaming",
+    "native_identity": {
+      "task_name": "MultistepSteaming",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Turn on the sink faucet. Then move the {vegetable} from the counter to the sink. Turn off the sink. Move the vegetable from the sink to the pot next to the stove. Finally move the pot to the {burner} burner.",
+      "instruction": "Turn on the sink faucet. Move the specified vegetable from the counter into the sink while the water is running. Turn off the faucet, move the vegetable into the pot next to the stove, and move the pot to the specified burner.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Turn on the sink faucet. ",
+          "modified": "Turn on the sink faucet. "
+        },
+        {
+          "op": "replace",
+          "original": "Then move",
+          "modified": "Move"
+        },
+        {
+          "op": "equal",
+          "original": " the ",
+          "modified": " the "
+        },
+        {
+          "op": "replace",
+          "original": "{vegetable}",
+          "modified": "specified vegetable"
+        },
+        {
+          "op": "equal",
+          "original": " from the counter ",
+          "modified": " from the counter "
+        },
+        {
+          "op": "replace",
+          "original": "to",
+          "modified": "into"
+        },
+        {
+          "op": "equal",
+          "original": " the ",
+          "modified": " the "
+        },
+        {
+          "op": "replace",
+          "original": "sink.",
+          "modified": "sink while the water is running."
+        },
+        {
+          "op": "equal",
+          "original": " Turn off the ",
+          "modified": " Turn off the "
+        },
+        {
+          "op": "replace",
+          "original": "sink.",
+          "modified": "faucet,"
+        },
+        {
+          "op": "equal",
+          "original": " ",
+          "modified": " "
+        },
+        {
+          "op": "replace",
+          "original": "Move",
+          "modified": "move"
+        },
+        {
+          "op": "equal",
+          "original": " the vegetable ",
+          "modified": " the vegetable "
+        },
+        {
+          "op": "replace",
+          "original": "from the sink to",
+          "modified": "into"
+        },
+        {
+          "op": "equal",
+          "original": " the pot next to the ",
+          "modified": " the pot next to the "
+        },
+        {
+          "op": "replace",
+          "original": "stove.",
+          "modified": "stove,"
+        },
+        {
+          "op": "equal",
+          "original": " ",
+          "modified": " "
+        },
+        {
+          "op": "replace",
+          "original": "Finally",
+          "modified": "and"
+        },
+        {
+          "op": "equal",
+          "original": " move the pot to the ",
+          "modified": " move the pot to the "
+        },
+        {
+          "op": "replace",
+          "original": "{burner}",
+          "modified": "specified"
+        },
+        {
+          "op": "equal",
+          "original": " burner.",
+          "modified": " burner."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/06-multistep-steaming/task.yaml"
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task01/07",
+    "family": "task01",
+    "slot": "07",
+    "native_id": "robocasa/scale-portioning",
+    "title": "Scale portioning",
+    "catalog_instruction": "Take the specified meat from the fridge and place it on the digital scale on the counter by the fridge. Release it and move the gripper clear while waiting a few seconds for a reading. Then move it to the plate on the dining counter, release it, and move the gripper clear.",
+    "native_instruction": "Take the {meat} from the fridge and place it on the digital scale on the counter by the fridge. Wait a few seconds for a reading, then move it to the plate on the dining counter.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-07-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/scale-portioning",
+    "native_identity": {
+      "task_name": "ScalePortioning",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Take the {meat} from the fridge and place it on the digital scale on the counter by the fridge. Wait a few seconds for a reading, then move it to the plate on the dining counter.",
+      "instruction": "Take the specified meat from the fridge and place it on the digital scale on the counter by the fridge. Release it and move the gripper clear while waiting a few seconds for a reading. Then move it to the plate on the dining counter, release it, and move the gripper clear.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Take the ",
+          "modified": "Take the "
+        },
+        {
+          "op": "replace",
+          "original": "{meat}",
+          "modified": "specified meat"
+        },
+        {
+          "op": "equal",
+          "original": " from the fridge and place it on the digital scale on the counter by the fridge. ",
+          "modified": " from the fridge and place it on the digital scale on the counter by the fridge. "
+        },
+        {
+          "op": "replace",
+          "original": "Wait",
+          "modified": "Release it and move the gripper clear while waiting"
+        },
+        {
+          "op": "equal",
+          "original": " a few seconds for a ",
+          "modified": " a few seconds for a "
+        },
+        {
+          "op": "replace",
+          "original": "reading,",
+          "modified": "reading."
+        },
+        {
+          "op": "equal",
+          "original": " ",
+          "modified": " "
+        },
+        {
+          "op": "replace",
+          "original": "then",
+          "modified": "Then"
+        },
+        {
+          "op": "equal",
+          "original": " move it to the plate on the dining ",
+          "modified": " move it to the plate on the dining "
+        },
+        {
+          "op": "replace",
+          "original": "counter.",
+          "modified": "counter, release it, and move the gripper clear."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/07-scale-portioning/task.yaml"
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task01/08",
+    "family": "task01",
+    "slot": "08",
+    "native_id": "robocasa/scrub-cutting-board",
+    "title": "Scrub cutting board",
+    "catalog_instruction": "Pick up the sponge from the counter and scrub across a broad area of the cutting board, keeping the sponge grasped and in contact with the board throughout the scrubbing motion. Once finished, release the sponge and retract the gripper well away from it.",
+    "native_instruction": "Pick up the sponge from the counter and clean the cutting board by briefly scrubbing or pressing down on the cutting board. Once finished, release the sponge.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-08-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/scrub-cutting-board",
+    "native_identity": {
+      "task_name": "ScrubCuttingBoard",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Pick up the sponge from the counter and clean the cutting board by briefly scrubbing or pressing down on the cutting board. Once finished, release the sponge.",
+      "instruction": "Pick up the sponge from the counter and scrub across a broad area of the cutting board, keeping the sponge grasped and in contact with the board throughout the scrubbing motion. Once finished, release the sponge and retract the gripper well away from it.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Pick up the sponge from the counter and ",
+          "modified": "Pick up the sponge from the counter and "
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": "s"
+        },
+        {
+          "op": "equal",
+          "original": "c",
+          "modified": "c"
+        },
+        {
+          "op": "replace",
+          "original": "l",
+          "modified": "rub across a broad ar"
+        },
+        {
+          "op": "equal",
+          "original": "ea",
+          "modified": "ea"
+        },
+        {
+          "op": "replace",
+          "original": "n",
+          "modified": " of"
+        },
+        {
+          "op": "equal",
+          "original": " the cutting board",
+          "modified": " the cutting board"
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": ", keeping the sponge grasped and in contact with the"
+        },
+        {
+          "op": "equal",
+          "original": " b",
+          "modified": " b"
+        },
+        {
+          "op": "replace",
+          "original": "y",
+          "modified": "oard"
+        },
+        {
+          "op": "equal",
+          "original": " ",
+          "modified": " "
+        },
+        {
+          "op": "replace",
+          "original": "b",
+          "modified": "th"
+        },
+        {
+          "op": "equal",
+          "original": "r",
+          "modified": "r"
+        },
+        {
+          "op": "replace",
+          "original": "i",
+          "modified": "oughout th"
+        },
+        {
+          "op": "equal",
+          "original": "e",
+          "modified": "e"
+        },
+        {
+          "op": "delete",
+          "original": "fly",
+          "modified": ""
+        },
+        {
+          "op": "equal",
+          "original": " scrubbing ",
+          "modified": " scrubbing "
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": "m"
+        },
+        {
+          "op": "equal",
+          "original": "o",
+          "modified": "o"
+        },
+        {
+          "op": "replace",
+          "original": "r press",
+          "modified": "t"
+        },
+        {
+          "op": "equal",
+          "original": "i",
+          "modified": "i"
+        },
+        {
+          "op": "delete",
+          "original": "ng down ",
+          "modified": ""
+        },
+        {
+          "op": "equal",
+          "original": "on",
+          "modified": "on"
+        },
+        {
+          "op": "delete",
+          "original": " the cutting board",
+          "modified": ""
+        },
+        {
+          "op": "equal",
+          "original": ". Once finished, release the sponge",
+          "modified": ". Once finished, release the sponge"
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": " and retract the gripper well away from it"
+        },
+        {
+          "op": "equal",
+          "original": ".",
+          "modified": "."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/08-scrub-cutting-board/task.yaml",
+      "native_predicates_unchanged": true,
+      "evaluation_status": "Evaluated with this modified instruction.",
+      "review_reason": "Clarify broad board coverage, maintained grasp/contact, and final retraction without numeric thresholds."
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task01/09",
+    "family": "task01",
+    "slot": "09",
+    "native_id": "robocasa/prepare-veggie-dip",
+    "title": "Prepare veggie dip",
+    "catalog_instruction": "Pick the specified vegetable and the cream cheese from the fridge, place both fully inside the blender, and turn it on.",
+    "native_instruction": "Pick the {vegetable} and the cream cheese from the fridge, place them in the blender, and turn it on.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-09-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/prepare-veggie-dip",
+    "native_identity": {
+      "task_name": "PrepareVeggieDip",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Pick the {vegetable} and the cream cheese from the fridge, place them in the blender, and turn it on.",
+      "instruction": "Pick the specified vegetable and the cream cheese from the fridge, place both fully inside the blender, and turn it on.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Pick the ",
+          "modified": "Pick the "
+        },
+        {
+          "op": "replace",
+          "original": "{vegetable}",
+          "modified": "specified vegetable"
+        },
+        {
+          "op": "equal",
+          "original": " and the cream cheese from the fridge, place ",
+          "modified": " and the cream cheese from the fridge, place "
+        },
+        {
+          "op": "replace",
+          "original": "them",
+          "modified": "both"
+        },
+        {
+          "op": "equal",
+          "original": " ",
+          "modified": " "
+        },
+        {
+          "op": "replace",
+          "original": "in",
+          "modified": "fully inside"
+        },
+        {
+          "op": "equal",
+          "original": " the blender, and turn it on.",
+          "modified": " the blender, and turn it on."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/09-prepare-veggie-dip/task.yaml"
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task01/10",
+    "family": "task01",
+    "slot": "10",
+    "native_id": "robocasa/prepare-vegetable-roasting",
+    "title": "Prepare vegetable roasting",
+    "catalog_instruction": "Pick the specified vegetable from the fridge and hold it under running water from the sink faucet to wash it. Then place it on the tray next to the sink to prepare for roasting. Release it and move the gripper clear.",
+    "native_instruction": "Pick the {vegetable} from the fridge and hold it under the sink faucet to wash it. Then place it on the tray next to the sink to prepare for roasting.",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task01-10-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "robocasa/prepare-vegetable-roasting",
+    "native_identity": {
+      "task_name": "PrepareVegetableRoasting",
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "Pick the {vegetable} from the fridge and hold it under the sink faucet to wash it. Then place it on the tray next to the sink to prepare for roasting.",
+      "instruction": "Pick the specified vegetable from the fridge and hold it under running water from the sink faucet to wash it. Then place it on the tray next to the sink to prepare for roasting. Release it and move the gripper clear.",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "Pick the ",
+          "modified": "Pick the "
+        },
+        {
+          "op": "replace",
+          "original": "{vegetable}",
+          "modified": "specified vegetable"
+        },
+        {
+          "op": "equal",
+          "original": " from the fridge and hold it under",
+          "modified": " from the fridge and hold it under"
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": " running water from"
+        },
+        {
+          "op": "equal",
+          "original": " the sink faucet to wash it. Then place it on the tray next to the sink to prepare for roasting.",
+          "modified": " the sink faucet to wash it. Then place it on the tray next to the sink to prepare for roasting."
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": " Release it and move the gripper clear."
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task01/10-prepare-vegetable-roasting/task.yaml"
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/01",
+    "family": "task02",
+    "slot": "01",
+    "native_id": "libero/10-0",
+    "title": "LIBERO-10-01",
+    "catalog_instruction": "put both the alphabet soup and the tomato sauce in the basket",
+    "native_instruction": "put both the alphabet soup and the tomato sauce in the basket",
+    "instruction_source": "runtime native task.instruction",
+    "status": "completed",
+    "episode_id": "task02-01-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "original_native"
+    },
+    "catalog_id": "libero/libero-10-01",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 0,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/02",
+    "family": "task02",
+    "slot": "02",
+    "native_id": "libero/10-1",
+    "title": "LIBERO-10-02",
+    "catalog_instruction": "put both the cream cheese box and the butter in the basket",
+    "native_instruction": "put both the cream cheese box and the butter in the basket",
+    "instruction_source": "runtime native task.instruction",
+    "status": "completed",
+    "episode_id": "task02-02-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "original_native"
+    },
+    "catalog_id": "libero/libero-10-02",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 1,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/03",
+    "family": "task02",
+    "slot": "03",
+    "native_id": "libero/10-2",
+    "title": "LIBERO-10-03",
+    "catalog_instruction": "turn on the stove and put the moka pot on it",
+    "native_instruction": "turn on the stove and put the moka pot on it",
+    "instruction_source": "runtime native task.instruction",
+    "status": "completed",
+    "episode_id": "task02-03-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "original_native"
+    },
+    "catalog_id": "libero/libero-10-03",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 2,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/04",
+    "family": "task02",
+    "slot": "04",
+    "native_id": "libero/bowl-into-bottom-drawer",
+    "title": "LIBERO-10-04",
+    "catalog_instruction": "put the black bowl in the bottom drawer of the cabinet and close it",
+    "native_instruction": "put the black bowl in the bottom drawer of the cabinet and close it",
+    "instruction_source": "runtime native task.instruction",
+    "status": "completed",
+    "episode_id": "task02-04-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "original_native"
+    },
+    "catalog_id": "libero/libero-10-04",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 3,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/05",
+    "family": "task02",
+    "slot": "05",
+    "native_id": "libero/10-4",
+    "title": "LIBERO-10-05",
+    "catalog_instruction": "put the white mug in the center of the left plate and put the yellow and white mug in the center of the right plate, with each mug resting on its plate",
+    "native_instruction": "put the white mug on the left plate and put the yellow and white mug on the right plate",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task02-05-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "libero/libero-10-05",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 4,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "put the white mug on the left plate and put the yellow and white mug on the right plate",
+      "instruction": "put the white mug in the center of the left plate and put the yellow and white mug in the center of the right plate, with each mug resting on its plate",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "put the white mug ",
+          "modified": "put the white mug "
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": "in the center "
+        },
+        {
+          "op": "equal",
+          "original": "o",
+          "modified": "o"
+        },
+        {
+          "op": "replace",
+          "original": "n",
+          "modified": "f"
+        },
+        {
+          "op": "equal",
+          "original": " the left plate and put the yellow and white mug ",
+          "modified": " the left plate and put the yellow and white mug "
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": "in the center "
+        },
+        {
+          "op": "equal",
+          "original": "o",
+          "modified": "o"
+        },
+        {
+          "op": "replace",
+          "original": "n",
+          "modified": "f"
+        },
+        {
+          "op": "equal",
+          "original": " the right plate",
+          "modified": " the right plate"
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": ", with each mug resting on its plate"
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task02/05-libero-10-05/task.yaml",
+      "native_predicates_unchanged": true,
+      "evaluation_status": "Evaluated with this modified instruction.",
+      "review_reason": "Authorized full current-source LIBERO comparison refresh."
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/06",
+    "family": "task02",
+    "slot": "06",
+    "native_id": "libero/10-5",
+    "title": "LIBERO-10-06",
+    "catalog_instruction": "pick up the book and place it in the back compartment of the caddy, between the two large side compartments",
+    "native_instruction": "pick up the book and place it in the back compartment of the caddy",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task02-06-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "libero/libero-10-06",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 5,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "pick up the book and place it in the back compartment of the caddy",
+      "instruction": "pick up the book and place it in the back compartment of the caddy, between the two large side compartments",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "pick up the book and place it in the back compartment of the caddy",
+          "modified": "pick up the book and place it in the back compartment of the caddy"
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": ", between the two large side compartments"
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task02/06-libero-10-06/task.yaml",
+      "native_predicates_unchanged": true,
+      "evaluation_status": "Evaluated with this modified instruction.",
+      "review_reason": "Authorized full current-source LIBERO comparison refresh."
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/07",
+    "family": "task02",
+    "slot": "07",
+    "native_id": "libero/10-6",
+    "title": "LIBERO-10-07",
+    "catalog_instruction": "put the white mug in the center of the plate and put the chocolate pudding immediately to the right of the plate",
+    "native_instruction": "put the white mug on the plate and put the chocolate pudding to the right of the plate",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task02-07-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "libero/libero-10-07",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 6,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "put the white mug on the plate and put the chocolate pudding to the right of the plate",
+      "instruction": "put the white mug in the center of the plate and put the chocolate pudding immediately to the right of the plate",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "put the white mug ",
+          "modified": "put the white mug "
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": "in the center "
+        },
+        {
+          "op": "equal",
+          "original": "o",
+          "modified": "o"
+        },
+        {
+          "op": "replace",
+          "original": "n",
+          "modified": "f"
+        },
+        {
+          "op": "equal",
+          "original": " the plate and put the chocolate pudding ",
+          "modified": " the plate and put the chocolate pudding "
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": "immediately "
+        },
+        {
+          "op": "equal",
+          "original": "to the right of the plate",
+          "modified": "to the right of the plate"
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task02/07-libero-10-07/task.yaml",
+      "native_predicates_unchanged": true,
+      "evaluation_status": "Evaluated with this modified instruction.",
+      "review_reason": "Authorized full current-source LIBERO comparison refresh."
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/08",
+    "family": "task02",
+    "slot": "08",
+    "native_id": "libero/10-7",
+    "title": "LIBERO-10-08",
+    "catalog_instruction": "put both the alphabet soup and the cream cheese box fully inside the basket",
+    "native_instruction": "put both the alphabet soup and the cream cheese box in the basket",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task02-08-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "libero/libero-10-08",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 7,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "put both the alphabet soup and the cream cheese box in the basket",
+      "instruction": "put both the alphabet soup and the cream cheese box fully inside the basket",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "put both the alphabet soup and the cream cheese box ",
+          "modified": "put both the alphabet soup and the cream cheese box "
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": "fully "
+        },
+        {
+          "op": "equal",
+          "original": "in",
+          "modified": "in"
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": "side"
+        },
+        {
+          "op": "equal",
+          "original": " the basket",
+          "modified": " the basket"
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task02/08-libero-10-08/task.yaml",
+      "native_predicates_unchanged": true,
+      "evaluation_status": "Evaluated with this modified instruction.",
+      "review_reason": "Authorized full current-source LIBERO comparison refresh."
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/09",
+    "family": "task02",
+    "slot": "09",
+    "native_id": "libero/10-8",
+    "title": "LIBERO-10-09",
+    "catalog_instruction": "put both moka pots on the stove and turn the stove on",
+    "native_instruction": "put both moka pots on the stove",
+    "instruction_source": "Current reviewed benchmark instruction; each recording retains its measured instruction.",
+    "status": "completed",
+    "episode_id": "task02-09-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "modified"
+    },
+    "catalog_id": "libero/libero-10-09",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 8,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "instruction_revision": {
+      "native_instruction": "put both moka pots on the stove",
+      "instruction": "put both moka pots on the stove and turn the stove on",
+      "native_instruction_kind": "source",
+      "diff": [
+        {
+          "op": "equal",
+          "original": "put both moka pots on the stove",
+          "modified": "put both moka pots on the stove"
+        },
+        {
+          "op": "insert",
+          "original": "",
+          "modified": " and turn the stove on"
+        }
+      ],
+      "changed": true,
+      "preserve_whitespace": true,
+      "label": "Modified instruction",
+      "source_url": "https://huggingface.co/datasets/RLE-Bench/codex-benchmark/resolve/main/catalog/task02/09-libero-10-09/task.yaml",
+      "native_predicates_unchanged": true,
+      "evaluation_status": "Evaluated with this modified instruction.",
+      "review_reason": "Authorized full current-source LIBERO comparison refresh."
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
+  {
+    "key": "task02/10",
+    "family": "task02",
+    "slot": "10",
+    "native_id": "libero/mug-into-microwave",
+    "title": "LIBERO-10-10",
+    "catalog_instruction": "put the yellow and white mug in the microwave and close it",
+    "native_instruction": "put the yellow and white mug in the microwave and close it",
+    "instruction_source": "runtime native task.instruction",
+    "status": "completed",
+    "episode_id": "task02-10-seed0-formal",
+    "planned_protocol": {
+      "episodes": 1,
+      "seed": 0,
+      "control_frequency_hz": 20,
+      "max_control_steps": 6000,
+      "timeout_s": 28800,
+      "mode": "stepped",
+      "instruction_policy": "original_native"
+    },
+    "catalog_id": "libero/libero-10-10",
+    "native_identity": {
+      "suite_name": "libero_10",
+      "task_id": 9,
+      "init_state_index": 0,
+      "max_steps": 6000
+    },
+    "run_status": "finished",
+    "attempt_history": [],
+    "status_note": "Evaluation and native evidence complete."
+  },
   {
     "key": "task04/01",
     "family": "task04",