Publish image-goal diagnostics with explicit provider status and complete evidence

README.md

@@ -482,3 +482,10 @@ The Hugging Face workstation can load its three compressed simulator resources f
 [Inspect the image-goal benchmark](image-goals/v01/index.html). Explore 32 frozen PiPER image-goal cases, including absent targets, wrong arrangements and visually identical moving scenes. Native fixture validation is published. Jev accuracy and service latency are not yet measured. Synthetic diagnostic fixtures; passive evaluation; no robot controller or hardware qualification.
 
 [Pinned source](https://github.com/Dvidia-Inference/dvidia-training/tree/19acf0734d63ecebc07140c8d84302a54baee4c4) · [Research](https://research.dvidia.org/papers/image-goal-jev-v01/)
+
+
+## Image-goal measured follow-up (publication v0.2)
+
+[Inspect the image-goal benchmark](image-goals/v02/index.html). Explore 32 frozen PiPER image-goal cases, including absent targets, wrong arrangements and visually identical moving scenes. Inspect all paired Jev judgments, failures, abstentions and measured service latency. Synthetic diagnostic fixtures; passive evaluation; no robot controller or hardware qualification.
+
+[Pinned source](https://github.com/Dvidia-Inference/dvidia-training/tree/19acf0734d63ecebc07140c8d84302a54baee4c4) · [Research](https://research.dvidia.org/papers/image-goal-jev-v02/)

image-goals/v02/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 DVIDIA contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

image-goals/v02/README.md

@@ -0,0 +1,60 @@
+# Jev image-goal verification: measured follow-up
+
+DVIDIA robotics research · 19 October 2026.
+
+All 64 frozen Jev requests returned valid structured responses, with no retries or omitted trials. There were zero false-completion declarations among 24 physically incomplete cases per input condition. That result must be read beside poor positive recognition: images alone recognized 0/8 physically complete cases, and images plus privileged simulator telemetry recognized 1/8. All four occluded complete fixtures were called incomplete in both conditions.
+
+Observability-aware verdict agreement was 24/32 for images and 25/32 for images plus telemetry; the conditions have different expected labels, so this difference is not a broad accuracy improvement estimate. The RGB evaluator correctly abstained on 8/16 ambiguous cases. Overall online latency was 1.776 s median, 3.287 s p95 and 4.091 s maximum. These are hosted evaluation times, not robot reaction times.
+
+This publication v0.2 is a measured follow-up to the unchanged v0.1 protocol. The original unrun release remains at `image-goals/v01` on Hugging Face. Frozen native fixtures, prompts, conditions, thresholds and scoring are preserved byte-for-byte. There is no learned policy, trained model-weight release or hardware validation.
+
+Open `index.html` to inspect the current/goal images, positive cases, missing targets and every recorded judgment. `results.json` retains all typed outputs, probabilities, semantic request hashes, usage, timing and independently recomputed scores; raw provider response bodies are omitted. `report.json` is a smaller explicit allowlist for the offline viewer. No credential is included. `evaluation-plan.json` and `cases.json` preserve the original protocol and physical/observability references.
+
+`integrity.json` remains the original native fixture inventory. `report-integrity.json` binds the measured viewer. `publication-integrity.json` binds this full public package. No inventory replaces an earlier seal.
+
+The [independent run review](independent-run-review.json) verifies the frozen requests, append-only attempt ledger, stored provider records and metric arithmetic without rerunning the model.
+
+## Reproduce
+
+Use publication source commit `a3e9afb59f3a9cd7da617cf61416e24fad0a0a5c` (also recorded in `provenance.json`) and install Python 3.12+ with the repository's `.[arm]` extra. The native generator, client and evaluator remain frozen at core commit `19acf0734d63ecebc07140c8d84302a54baee4c4`. PiPER assets are fetched once and verified against the asset lock. Native fixture generation and the static viewer run locally; hosted Jev evaluation requires a TypeSafe account, network access and a securely provided credential.
+
+The frozen evaluation command is `python -m dvidia_training.image_goal_benchmark evaluate --fixtures PATH --output NEW_DIRECTORY`. A new run has new latency and possibly different stochastic outputs; it does not replace this evidence. See `docs/image-goal-benchmark.md` in the source for complete setup and limits. Never paste tokens into a report or source file.
+
+## Independently check the native evidence
+
+`native-validation.json` records the completed independent audit of all 32 physical labels and full-window telemetry extrema. `verify-image-goal-fixtures.py` recomputes those values from the saved native records; it does not call Jev, render images, or run a new physical simulation. Its source hash is recorded in both the audit and publication provenance.
+
+After installing the pinned source above, extract only the original sealed fixture files to a fresh directory. From this downloaded package directory:
+
+```sh
+python - <<'PYTHON'
+import hashlib, json
+from pathlib import Path, PurePosixPath
+source = Path.cwd()
+destination = source.parent / "image-goal-native-fixtures"
+destination.mkdir(exist_ok=False)
+inventory = json.loads((source / "integrity.json").read_text())
+for item in inventory["files"]:
+    relative = PurePosixPath(item["path"])
+    assert not relative.is_absolute() and ".." not in relative.parts
+    data = (source / relative).read_bytes()
+    assert len(data) == item["bytes"]
+    assert hashlib.sha256(data).hexdigest() == item["sha256"]
+    target = destination / relative
+    target.parent.mkdir(parents=True, exist_ok=True)
+    target.write_bytes(data)
+for name in ("integrity.json", "evaluation-plan.json"):
+    (destination / name).write_bytes((source / name).read_bytes())
+PYTHON
+python verify-image-goal-fixtures.py --fixtures ../image-goal-native-fixtures --output ../native-validation-reproduced.json
+```
+
+The verifier intentionally rejects extra files inside the native fixture directory. Keeping the report and its publication metadata outside that directory preserves the original manifest boundary. Expected result: 32 checked cases, 8 physically complete, 24 incomplete, 67 original files verified. Host metadata in a reproduction can differ.
+
+## Licenses and scope
+
+Original DVIDIA code and authored synthetic fixture records: MIT (`LICENSE`). The PiPER simulation asset is from MuJoCo Menagerie commit `0059d4335f8156206f63a35662313385f7ad6d74`; its MIT notice is preserved in `licenses/agilex_piper/LICENSE`. No MuJoCo runtime or upstream mesh files are redistributed in this viewer. DVIDIA marks are brand identifiers; the code license grants no trademark rights.
+
+Jev is TypeSafe's hosted model and retains its own service terms. Its output is evaluated, not used to train or distill a DVIDIA policy. The two model-input conditions are RGB images alone and RGB plus privileged simulator measurements. The latter supplies object identity/presence, pose errors and complete native-window extrema; it is not a demonstrated real-camera or contact estimator. There is no direct motor-control path.
+
+[Research](https://research.dvidia.org/papers/image-goal-jev-v02/) · [Source](https://github.com/Dvidia-Inference/dvidia-training/tree/a3e9afb59f3a9cd7da617cf61416e24fad0a0a5c) · [Hugging Face](https://huggingface.co/spaces/Dvidia/robot-simulation-lab)

image-goals/v02/evaluation-plan.json

@@ -0,0 +1,643 @@
+{
+  "automatic_retries": 0,
+  "cohort": "Authored diagnostic fixtures; four scene layouts are correlated clusters, not 32 independent real-world tasks.",
+  "completion_decision": "Unmodified Jev Choice verdict; unknown is abstention.",
+  "conditions": [
+    "rgb",
+    "rgb_telemetry"
+  ],
+  "cost_scope": "Planning allowance assumes documented roughly 64k default context. Not a billing guarantee; receipts retain known usage and unknown-cost attempts.",
+  "documented_input_price_usd_per_million_tokens": 0.042,
+  "files": {
+    "cases.json": {
+      "bytes": 103511,
+      "sha256": "f8b1b205b16e9833df7bd7063e0572355794dcede49a024f5d73ae6f65bbb1aa"
+    },
+    "images/layout-01-complete.png": {
+      "bytes": 79345,
+      "sha256": "550516b315cb198407b0ef3be997af97805c11fcfe6b587b1091affd8060e8fb"
+    },
+    "images/layout-01-missing_target.png": {
+      "bytes": 72825,
+      "sha256": "b956ae597b88d4b82b67b00d6ad6e41bb75eb184fd320d94e2fa76115857d43d"
+    },
+    "images/layout-01-moving_alias.png": {
+      "bytes": 79345,
+      "sha256": "550516b315cb198407b0ef3be997af97805c11fcfe6b587b1091affd8060e8fb"
+    },
+    "images/layout-01-occluded.png": {
+      "bytes": 68346,
+      "sha256": "747b50dcdcb37d89c0a4a882fff12057e9062bd2c171154d1a6d408ef3e8af0d"
+    },
+    "images/layout-01-unreleased.png": {
+      "bytes": 84090,
+      "sha256": "3816b70bacff446c8901713b140fe6d615c880605cdd3ea89089495c641b77d2"
+    },
+    "images/layout-01-wrong_object.png": {
+      "bytes": 80061,
+      "sha256": "83af097bb71ac714e02954029a91edc4fb8f51948e01ed70d31d705dcf593452"
+    },
+    "images/layout-01-wrong_orientation.png": {
+      "bytes": 79111,
+      "sha256": "56253438617f3ff5caf1700673122b6e7260601fad6700f519a8442f3e57bd37"
+    },
+    "images/layout-01-wrong_position.png": {
+      "bytes": 83583,
+      "sha256": "37862bb8c714993723236aa25d806a62229856edb4d17c489a02ea1fd777326a"
+    },
+    "images/layout-02-complete.png": {
+      "bytes": 79868,
+      "sha256": "a3221e451c24b164f0348f2cb6517d2bd2589a444c3d8d71a9942a98dee0bdf3"
+    },
+    "images/layout-02-missing_target.png": {
+      "bytes": 71367,
+      "sha256": "dd863d7a52be1c7a4675c56c7154e2b572a5960be421b9f2a7ca27a7f2836348"
+    },
+    "images/layout-02-moving_alias.png": {
+      "bytes": 79868,
+      "sha256": "a3221e451c24b164f0348f2cb6517d2bd2589a444c3d8d71a9942a98dee0bdf3"
+    },
+    "images/layout-02-occluded.png": {
+      "bytes": 64921,
+      "sha256": "3397103762d94870c515de9294b6999ad2c2b1cfc570daa1320b5724bf556fa8"
+    },
+    "images/layout-02-unreleased.png": {
+      "bytes": 85815,
+      "sha256": "9357687d5747c98916e705103aebc6f9b6900f7aef5ed675ca6dd942a19a66e7"
+    },
+    "images/layout-02-wrong_object.png": {
+      "bytes": 80112,
+      "sha256": "441dadb181f982e7393eb40642a2a035037f88a13c170b9b227cd6e46f3f4ebc"
+    },
+    "images/layout-02-wrong_orientation.png": {
+      "bytes": 78325,
+      "sha256": "66935db963dd00862c83e0fc8af06939d0f017d7801a328870f0feb49e64a144"
+    },
+    "images/layout-02-wrong_position.png": {
+      "bytes": 81061,
+      "sha256": "f31324821c1b69ed64bc7776437f292b8d20e87e2aa237135cb2b084fd6f0e76"
+    },
+    "images/layout-03-complete.png": {
+      "bytes": 78462,
+      "sha256": "9e077553701f0b54929386b69abc0d4a531f4dd8d0ef8c164dfbd4e11c2eec22"
+    },
+    "images/layout-03-missing_target.png": {
+      "bytes": 72934,
+      "sha256": "d091c877c2762f156c115dc2a2d960d2619c4a47279dbf5bee136aaa8ace6d1d"
+    },
+    "images/layout-03-moving_alias.png": {
+      "bytes": 78462,
+      "sha256": "9e077553701f0b54929386b69abc0d4a531f4dd8d0ef8c164dfbd4e11c2eec22"
+    },
+    "images/layout-03-occluded.png": {
+      "bytes": 69903,
+      "sha256": "0fd28594d7bee6d76cb13dd30e3bfe520918d470fa2bec53ecf84a493aed0556"
+    },
+    "images/layout-03-unreleased.png": {
+      "bytes": 87865,
+      "sha256": "2679f2897c559ff07190c9bee25b0fbdb9f6cda29847c287cbadf74d342b4c14"
+    },
+    "images/layout-03-wrong_object.png": {
+      "bytes": 79100,
+      "sha256": "78733a16c745bbd647514a3aa15b486f2b799cd413027728fd933b4a72954ab8"
+    },
+    "images/layout-03-wrong_orientation.png": {
+      "bytes": 79569,
+      "sha256": "66c3cc6f1378b262837d1b1d81c60d60bd9e3c1d25e59e28626810526d78b494"
+    },
+    "images/layout-03-wrong_position.png": {
+      "bytes": 81396,
+      "sha256": "266dcb71e61f8151fa2e56e7e3db4f561ff42830e44d0c7ff25cd7ea59843957"
+    },
+    "images/layout-04-complete.png": {
+      "bytes": 77762,
+      "sha256": "76cba333e8bdf56f1b9e6011247095507ede97697ef99ada4b47ea7d1c71d3fa"
+    },
+    "images/layout-04-missing_target.png": {
+      "bytes": 70111,
+      "sha256": "9d747de133583f8a3977332732a609f68eed97879615fdcee73ec0b12d091ecf"
+    },
+    "images/layout-04-moving_alias.png": {
+      "bytes": 77762,
+      "sha256": "76cba333e8bdf56f1b9e6011247095507ede97697ef99ada4b47ea7d1c71d3fa"
+    },
+    "images/layout-04-occluded.png": {
+      "bytes": 62348,
+      "sha256": "29772e10911231b9260ebe8b91a7c7f8f482324bd5002900affc445a0eeb8106"
+    },
+    "images/layout-04-unreleased.png": {
+      "bytes": 86362,
+      "sha256": "8d0290813f4288bddf5287eaf2d7e5b12e3710a284a5028fe229d57da3c4c736"
+    },
+    "images/layout-04-wrong_object.png": {
+      "bytes": 78104,
+      "sha256": "3e75f36cfbb95243a844e72ef822049b103cdf547c446d6e8e5652abf499a236"
+    },
+    "images/layout-04-wrong_orientation.png": {
+      "bytes": 76269,
+      "sha256": "a827a2f466b93f84ffc40ae11374a03bc7cbb222d194387bca4e1ee48d15887a"
+    },
+    "images/layout-04-wrong_position.png": {
+      "bytes": 80849,
+      "sha256": "a000a38d4e067649cdcac56ec77942730e0a67093338880d974e90e912b7df48"
+    },
+    "integrity.json": {
+      "bytes": 11372,
+      "sha256": "e8b6bbe4f2928e85f6e88d16af7d719dc32b1a3b588ce9633805061528cbe682"
+    },
+    "protocol.json": {
+      "bytes": 4867,
+      "sha256": "d48e363d24cc3c9f3be67ffb87390aa3f9b37272bd28d4b9b22424ccf336fba4"
+    },
+    "protocol.sha256": {
+      "bytes": 65,
+      "sha256": "74830b7ee22d52e31a784d13fe3f3220b9b6a013420a0d11042143b0d7f556ac"
+    },
+    "states/layout-01-complete.json": {
+      "bytes": 102367,
+      "sha256": "7b6031a83dadecba56c42af3f307c10866bc6d51ec9a178a0babcb611fb1f570"
+    },
+    "states/layout-01-missing_target.json": {
+      "bytes": 35731,
+      "sha256": "b908b17f09583053fa74aa01d00b7ce0be7d004b6a3cd0f11fd262f694167d36"
+    },
+    "states/layout-01-moving_alias.json": {
+      "bytes": 2373,
+      "sha256": "9f2179f911033c3a7737da3ec6e417af6efbdaaeeffa3e97b9be566df8643ea8"
+    },
+    "states/layout-01-occluded.json": {
+      "bytes": 102367,
+      "sha256": "7b6031a83dadecba56c42af3f307c10866bc6d51ec9a178a0babcb611fb1f570"
+    },
+    "states/layout-01-unreleased.json": {
+      "bytes": 123199,
+      "sha256": "2ce46f88bcb592de3dfbf8b3e277f111d256c65d876aae438a3508794ac23864"
+    },
+    "states/layout-01-wrong_object.json": {
+      "bytes": 36081,
+      "sha256": "af38dc0499c218d2dcf4a7696dce521c725e67873d5a6c2bbdba391438cae1c2"
+    },
+    "states/layout-01-wrong_orientation.json": {
+      "bytes": 102081,
+      "sha256": "a70e473947993f6d3bb3326e6007ec22c22d5633a5f123cf0baa34ca21575e04"
+    },
+    "states/layout-01-wrong_position.json": {
+      "bytes": 102056,
+      "sha256": "a63d248d85307cdce95a7d0ab9ea6f6d398b3d2b2a0727fa0c0a29326c738c67"
+    },
+    "states/layout-02-complete.json": {
+      "bytes": 102327,
+      "sha256": "a103b2a6aef0d8b1c025e6d58d1e5c4485762774055eb68c6c002d1ba0c3e899"
+    },
+    "states/layout-02-missing_target.json": {
+      "bytes": 35731,
+      "sha256": "b908b17f09583053fa74aa01d00b7ce0be7d004b6a3cd0f11fd262f694167d36"
+    },
+    "states/layout-02-moving_alias.json": {
+      "bytes": 2372,
+      "sha256": "3ae38f625857a3299886d47d8f20b62dd561c7af2f5377e7fa35df65a478f099"
+    },
+    "states/layout-02-occluded.json": {
+      "bytes": 102327,
+      "sha256": "a103b2a6aef0d8b1c025e6d58d1e5c4485762774055eb68c6c002d1ba0c3e899"
+    },
+    "states/layout-02-unreleased.json": {
+      "bytes": 123610,
+      "sha256": "8d10fcd6cea4fa17083334c7b08c8273e36ce7457652cf6bf6f35645c5f66a89"
+    },
+    "states/layout-02-wrong_object.json": {
+      "bytes": 36078,
+      "sha256": "7a0ed67229c85fd9e261b73d793eee86507e22e41af0ce4d33f5d9bc9899148e"
+    },
+    "states/layout-02-wrong_orientation.json": {
+      "bytes": 102053,
+      "sha256": "d7b852a42b846ce75d8c1410698435581d4cd79f1b6e0c41d7ab2f83a1e59c40"
+    },
+    "states/layout-02-wrong_position.json": {
+      "bytes": 102069,
+      "sha256": "528b489a5046025fced8eff4957848115b5fd693e8f4f09b2814acf99887b9f4"
+    },
+    "states/layout-03-complete.json": {
+      "bytes": 102349,
+      "sha256": "0526bbbf5cdb7723c7ab64344ea5b2a233e456d6e59a886040462535c4dcde2b"
+    },
+    "states/layout-03-missing_target.json": {
+      "bytes": 35731,
+      "sha256": "b908b17f09583053fa74aa01d00b7ce0be7d004b6a3cd0f11fd262f694167d36"
+    },
+    "states/layout-03-moving_alias.json": {
+      "bytes": 2365,
+      "sha256": "3aaab8cba27d7f854bff90077682749c92dd5be94d8605fb72f7ee38e0c54230"
+    },
+    "states/layout-03-occluded.json": {
+      "bytes": 102349,
+      "sha256": "0526bbbf5cdb7723c7ab64344ea5b2a233e456d6e59a886040462535c4dcde2b"
+    },
+    "states/layout-03-unreleased.json": {
+      "bytes": 123520,
+      "sha256": "a857eeed1136e6c8b72a0c8a998569f294f91a27363a3dbad76b2d3de1ec05dc"
+    },
+    "states/layout-03-wrong_object.json": {
+      "bytes": 36080,
+      "sha256": "5475197b44eeb73d74fa116634d9eacd6eb52135ad7c16fafbca4bacecc955b5"
+    },
+    "states/layout-03-wrong_orientation.json": {
+      "bytes": 102072,
+      "sha256": "c637ca772d171828be8ad1c7015209ee628241326a5a854b65f17af0aa283910"
+    },
+    "states/layout-03-wrong_position.json": {
+      "bytes": 102100,
+      "sha256": "47963ad58529f64243fa41ab426546a8324833a619661ddd997e96dccc79acf3"
+    },
+    "states/layout-04-complete.json": {
+      "bytes": 102327,
+      "sha256": "45062ce2bb746918929611edf611f4ca2b57944b8400636e474bec29af906eab"
+    },
+    "states/layout-04-missing_target.json": {
+      "bytes": 35731,
+      "sha256": "b908b17f09583053fa74aa01d00b7ce0be7d004b6a3cd0f11fd262f694167d36"
+    },
+    "states/layout-04-moving_alias.json": {
+      "bytes": 2371,
+      "sha256": "6e92f4341e7f378a538bcec8884053d9792f3e9c7535fca58f7e2460ff30b14d"
+    },
+    "states/layout-04-occluded.json": {
+      "bytes": 102327,
+      "sha256": "45062ce2bb746918929611edf611f4ca2b57944b8400636e474bec29af906eab"
+    },
+    "states/layout-04-unreleased.json": {
+      "bytes": 123342,
+      "sha256": "b1912487c8c421d3d0e6e5cc97a5f3b63f7a8625f441d5b26857955196f00e67"
+    },
+    "states/layout-04-wrong_object.json": {
+      "bytes": 36077,
+      "sha256": "59a4c4c52f8ec8db8c9698d62dfedfc08c7cf106bda2446da852699f21e35cb0"
+    },
+    "states/layout-04-wrong_orientation.json": {
+      "bytes": 102035,
+      "sha256": "78709db4cd96f5a1106cbd80a607b4a6ba91bfe587cdfa30becd5eafe5675ead"
+    },
+    "states/layout-04-wrong_position.json": {
+      "bytes": 102101,
+      "sha256": "a4946f69e2ca43bafb326fd9d5df6255df23506902348af9df6b5785669153e9"
+    }
+  },
+  "frozen_at": "2026-10-19T03:32:07.942824+00:00",
+  "latency_scope": "Preloaded image bytes + request construction + image upload + hosted response + parsing; excludes fixture file loading, prerecorded rendering, live camera, actuation and physical task duration.",
+  "max_requests": 64,
+  "model": "jev-1.14.0",
+  "planning_allowance_per_request_usd": 0.003,
+  "presence_thresholds": [
+    0.2,
+    0.8
+  ],
+  "price_checked_date": "2026-10-18",
+  "rules": [
+    "Keep every scheduled trial, including errors and unrun trials.",
+    "No retries, prompt tuning, threshold tuning or case selection after evaluation begins.",
+    "No direct actuator commands and no model training or distillation.",
+    "Report physical false completion separately from observability-aware accuracy.",
+    "No remote calls when credentials are absent; missing results are never scored as accurate."
+  ],
+  "schema": "dvidia.image-goal-evaluation-plan/v1",
+  "source_commit": "19acf0734d63ecebc07140c8d84302a54baee4c4",
+  "source_files": {
+    "image_goal_benchmark.py": {
+      "bytes": 19367,
+      "sha256": "38fa1de5da6ee9f8e0ab60bc77d2561e82de0dad17dcc1d067b5816d8633996a"
+    },
+    "image_goal_scenes.py": {
+      "bytes": 21128,
+      "sha256": "1fe41c3fe4d372b068681caf3f63766285b9238976043a9e15fe4dbadb40b79a"
+    },
+    "jev_client.py": {
+      "bytes": 18651,
+      "sha256": "f5dad86c692144a1cf85bfeecf68b99318ea8d9df08e0fe5d47d144f80a380ee"
+    }
+  },
+  "task": "Place the green shoe inside the outlined goal zone, matching the goal image's position and orientation. Completion requires the target shoe to have table support force greater than 0.01 N, robot contact force at most 0.01 N, XY position error at most 25 mm, yaw error at most 15 degrees, linear speed at most 0.01 m/s and angular speed at most 0.05 rad/s, all continuously for at least 200 ms. The goal picture is a visual reference, not proof that these physical conditions hold in the current picture. Answer unknown when the inputs cannot establish the required physical conditions. An orange shoe is a different object. The opaque inspection cover can hide the target.",
+  "timeout_s": 20.0,
+  "timeout_scope": "Per socket operation; not a hard wall-clock deadline.",
+  "trials": [
+    {
+      "case_id": "layout-02-wrong_position",
+      "condition": "rgb_telemetry",
+      "request_sha256": "4b2d21390d91a0e27ed7498506c325fb33f997108f0cd688646df0c8767b06b5"
+    },
+    {
+      "case_id": "layout-02-wrong_orientation",
+      "condition": "rgb",
+      "request_sha256": "cf2c595e98a0953986643451655b82b56ec9b6797f0385f172992a19a09e2902"
+    },
+    {
+      "case_id": "layout-01-moving_alias",
+      "condition": "rgb",
+      "request_sha256": "571f6b7928dbc9d2a6f2c0846975669e2f0f928efad97c69aafaa7cb56b66e15"
+    },
+    {
+      "case_id": "layout-03-occluded",
+      "condition": "rgb",
+      "request_sha256": "6dc62c2ce15e117efcf3c31ebf6559225eec602a00771ca335893539e0ce7b17"
+    },
+    {
+      "case_id": "layout-03-moving_alias",
+      "condition": "rgb_telemetry",
+      "request_sha256": "562192137127c05e4d4f824bf7f12fd6f1885900aad2a486b8b2026602dc2e0b"
+    },
+    {
+      "case_id": "layout-03-complete",
+      "condition": "rgb_telemetry",
+      "request_sha256": "9cdeacbb6a7bfb89a52b1440577dc93a86581c29ac4a50445c64fbb9436a38cc"
+    },
+    {
+      "case_id": "layout-03-missing_target",
+      "condition": "rgb_telemetry",
+      "request_sha256": "a45d75d5ee6d6d2f3f98afba864906ff46b15fdececc6dfee1f03dffe06c816c"
+    },
+    {
+      "case_id": "layout-02-unreleased",
+      "condition": "rgb_telemetry",
+      "request_sha256": "ba1ab4131973a5d3833b50c1af81ec168dded4ea50209a2b9e88b6d64c07849a"
+    },
+    {
+      "case_id": "layout-04-complete",
+      "condition": "rgb",
+      "request_sha256": "63367143b05302c30c2cba9bf7c4bff0dafa2b2cff9280faceb1ceb80f2ef855"
+    },
+    {
+      "case_id": "layout-03-wrong_orientation",
+      "condition": "rgb_telemetry",
+      "request_sha256": "2ea701cfba448a105267bbb27f8303456ca072d2d07a4a930d21034853fb262d"
+    },
+    {
+      "case_id": "layout-04-unreleased",
+      "condition": "rgb_telemetry",
+      "request_sha256": "6517dc9913124d034b3a54349b77f81feb05e049b0660a75c9b93d1443d0c425"
+    },
+    {
+      "case_id": "layout-03-occluded",
+      "condition": "rgb_telemetry",
+      "request_sha256": "0b9a08881ff8452447d301dc7e4379139afd0cda5b98a0ad7699e03a6e433c53"
+    },
+    {
+      "case_id": "layout-02-occluded",
+      "condition": "rgb",
+      "request_sha256": "1d6c918bd75bbb3d3544876daee9b4f4439c27f76d4a97126c1734d831add576"
+    },
+    {
+      "case_id": "layout-03-unreleased",
+      "condition": "rgb",
+      "request_sha256": "803f461156bfe7fc05d5fbbdab899f45c52b561e1700f4c8ba68873e3635d6dd"
+    },
+    {
+      "case_id": "layout-02-missing_target",
+      "condition": "rgb_telemetry",
+      "request_sha256": "6e3c7bda63e2584be56d9f5ffbcdf6228ba3f44c1d45e4d53f708f7e873d12da"
+    },
+    {
+      "case_id": "layout-01-wrong_orientation",
+      "condition": "rgb_telemetry",
+      "request_sha256": "5427dbcbe70b437b4949c2ef974905d4710ea0c994e8d92fd207d1017d494d6d"
+    },
+    {
+      "case_id": "layout-04-unreleased",
+      "condition": "rgb",
+      "request_sha256": "053c7b9d62bc72c6fccfee44a071cc0cb5c9fc4a09cef15f568ae958532d7e8e"
+    },
+    {
+      "case_id": "layout-01-unreleased",
+      "condition": "rgb",
+      "request_sha256": "29650676bc836d7065cc13f79ef3e1dbdadc7a139bfa45564dfd337c24f4918c"
+    },
+    {
+      "case_id": "layout-04-occluded",
+      "condition": "rgb_telemetry",
+      "request_sha256": "ec0eb371382c7fd7b35408483f2c9978fce56c17d2f6355bb4027ed68d4977af"
+    },
+    {
+      "case_id": "layout-04-occluded",
+      "condition": "rgb",
+      "request_sha256": "644de408fef8e6cb532ef8b5144f8d87742837887e4f05984b55341187a23a5b"
+    },
+    {
+      "case_id": "layout-01-complete",
+      "condition": "rgb_telemetry",
+      "request_sha256": "22c765474ea8183e3002b7f92eb27d13136b75d5e1515a007133afc05d459ec9"
+    },
+    {
+      "case_id": "layout-02-missing_target",
+      "condition": "rgb",
+      "request_sha256": "f73d1ad36b4b6537aec4ac4c603a04db290f1a02263ed7b48f65de7f8641edc9"
+    },
+    {
+      "case_id": "layout-02-wrong_object",
+      "condition": "rgb",
+      "request_sha256": "14d0e74adc997a0af6a29feb0bb8b5c7180fe519679936fa90265f896a87a350"
+    },
+    {
+      "case_id": "layout-02-moving_alias",
+      "condition": "rgb",
+      "request_sha256": "aae2ccbe47bbe8cb832b48b231d1b1c152c92a5e96417227a5dd2b981a394269"
+    },
+    {
+      "case_id": "layout-01-wrong_object",
+      "condition": "rgb_telemetry",
+      "request_sha256": "b9b3db8ea54fbadede8971e74c99fd7fecb513696e6a57c86f43b8a392b15cee"
+    },
+    {
+      "case_id": "layout-02-wrong_position",
+      "condition": "rgb",
+      "request_sha256": "0caffd3a26a3698840489fb4f7d7628a99d28dd969ab7e968b681dbf57380666"
+    },
+    {
+      "case_id": "layout-03-unreleased",
+      "condition": "rgb_telemetry",
+      "request_sha256": "8e094213a08d6ed860d1f868517b5a22417bb89f93029779f41f2d5974f02ad2"
+    },
+    {
+      "case_id": "layout-04-moving_alias",
+      "condition": "rgb_telemetry",
+      "request_sha256": "626df301847db14cad7060405b1875996cef9c3f6802e820440fc5cc3f47ac22"
+    },
+    {
+      "case_id": "layout-01-missing_target",
+      "condition": "rgb",
+      "request_sha256": "6680c0e5be92f50394933cd91f7fb856cedfc428d98983f245ba5719c920bb37"
+    },
+    {
+      "case_id": "layout-02-unreleased",
+      "condition": "rgb",
+      "request_sha256": "f6c0b2aa823c13f89f27aabcbb83816816e0f68cf026923a8d8916c170287a83"
+    },
+    {
+      "case_id": "layout-01-complete",
+      "condition": "rgb",
+      "request_sha256": "571f6b7928dbc9d2a6f2c0846975669e2f0f928efad97c69aafaa7cb56b66e15"
+    },
+    {
+      "case_id": "layout-02-occluded",
+      "condition": "rgb_telemetry",
+      "request_sha256": "f58a5417aaa6d0d5528b2c0c672e2fd6dfd9fb13080c382c080475529b938ca9"
+    },
+    {
+      "case_id": "layout-01-wrong_position",
+      "condition": "rgb",
+      "request_sha256": "fbdf1c3a54e1d8f3a51b5542528e4f621b5dd94e93db2228bc21828fbf8e4d85"
+    },
+    {
+      "case_id": "layout-03-wrong_position",
+      "condition": "rgb",
+      "request_sha256": "3908d00ae374e96f852d1e026bfd79ba6c1b2457611d9ae362168ae2ab1fd5b1"
+    },
+    {
+      "case_id": "layout-03-moving_alias",
+      "condition": "rgb",
+      "request_sha256": "727e854ff9361e8e1948877d2d380e35c57ed6860a950f0e281a7f884a68725c"
+    },
+    {
+      "case_id": "layout-01-moving_alias",
+      "condition": "rgb_telemetry",
+      "request_sha256": "0b5063721c28b72b90cd69cf3ff5eccd3d6a9b4e1f4b6e04d4acdeba9c5813e1"
+    },
+    {
+      "case_id": "layout-04-wrong_object",
+      "condition": "rgb",
+      "request_sha256": "e0143d52be87f1d72e4d955ea76e61e6fcdda047f9725766666cc6be808135ed"
+    },
+    {
+      "case_id": "layout-03-wrong_object",
+      "condition": "rgb_telemetry",
+      "request_sha256": "c48a75ef61b1c37da9f78fced2ad8c8fb64bb824764df2bafb810b5272ddb9b0"
+    },
+    {
+      "case_id": "layout-04-moving_alias",
+      "condition": "rgb",
+      "request_sha256": "63367143b05302c30c2cba9bf7c4bff0dafa2b2cff9280faceb1ceb80f2ef855"
+    },
+    {
+      "case_id": "layout-04-wrong_position",
+      "condition": "rgb",
+      "request_sha256": "691dbe273c38c698069ed53d87cc9b4bce9c714f66fd76e11d89569a7be6de69"
+    },
+    {
+      "case_id": "layout-03-wrong_orientation",
+      "condition": "rgb",
+      "request_sha256": "fd64d5be5c9fefcc4cbb3c3b42e73bafb28bf0fe169af7cb0dc2b64ec7b0bc32"
+    },
+    {
+      "case_id": "layout-01-wrong_orientation",
+      "condition": "rgb",
+      "request_sha256": "9aad7d221c1a2815360c5a7c7fc5d71ed186731fe9d727724868bedb5eb2042c"
+    },
+    {
+      "case_id": "layout-01-occluded",
+      "condition": "rgb_telemetry",
+      "request_sha256": "207e6f5519077f996117540549c91f9b242b6811633219a3d98f3141966939c6"
+    },
+    {
+      "case_id": "layout-04-missing_target",
+      "condition": "rgb_telemetry",
+      "request_sha256": "38c739b8b55cb6da3677e531b81e3868e16ced493b42c708c8e027fb8a653b71"
+    },
+    {
+      "case_id": "layout-04-wrong_orientation",
+      "condition": "rgb_telemetry",
+      "request_sha256": "cc06c03336fd573f3f93300c40f09356b1e6e25ce2e0e7f4c6f6c2b7e5b9e0eb"
+    },
+    {
+      "case_id": "layout-04-wrong_orientation",
+      "condition": "rgb",
+      "request_sha256": "c7abe212b11813b00ca8d4dcea622a21d1aff59d03c4a29bb21feb10939d813b"
+    },
+    {
+      "case_id": "layout-01-unreleased",
+      "condition": "rgb_telemetry",
+      "request_sha256": "27fd66094c3efe56842da714acc7be74fc97e09dea4ad67d7a4fe29ffa17d7a7"
+    },
+    {
+      "case_id": "layout-03-wrong_object",
+      "condition": "rgb",
+      "request_sha256": "3650b3cd6be734b696e8157da729427b1856d68eb905bce765cff3556c099814"
+    },
+    {
+      "case_id": "layout-02-complete",
+      "condition": "rgb",
+      "request_sha256": "aae2ccbe47bbe8cb832b48b231d1b1c152c92a5e96417227a5dd2b981a394269"
+    },
+    {
+      "case_id": "layout-02-complete",
+      "condition": "rgb_telemetry",
+      "request_sha256": "faa58fbc2678ab387f1f7f8b7c24aa781e0259cbf19a89a7459932ecb2a5d768"
+    },
+    {
+      "case_id": "layout-03-wrong_position",
+      "condition": "rgb_telemetry",
+      "request_sha256": "8a5b29305dec3510c2c7960557a0ef043eafb4237293f6b1507f3197242607fa"
+    },
+    {
+      "case_id": "layout-01-missing_target",
+      "condition": "rgb_telemetry",
+      "request_sha256": "150e2666d6e63b662012e973a5ab1e6e614d3492b7f324cdf3e9de9a3f0f4c98"
+    },
+    {
+      "case_id": "layout-01-occluded",
+      "condition": "rgb",
+      "request_sha256": "f653413e688093b5eec5d7b0597618d5946991f0921fb372650c7f562decfe95"
+    },
+    {
+      "case_id": "layout-04-wrong_object",
+      "condition": "rgb_telemetry",
+      "request_sha256": "b364766ccfff769c100afd83646101a3b4660191294982804942390f264d1cc3"
+    },
+    {
+      "case_id": "layout-01-wrong_position",
+      "condition": "rgb_telemetry",
+      "request_sha256": "a0ed0c8c2f72b936e34711c88f874179a2af286a17e8eaf9068dfe41bdf8e778"
+    },
+    {
+      "case_id": "layout-04-missing_target",
+      "condition": "rgb",
+      "request_sha256": "517d5035ec97f26d3d2c5ba307c827dac3f5e5a2cc831b82d648c94cdb6f2ef8"
+    },
+    {
+      "case_id": "layout-03-missing_target",
+      "condition": "rgb",
+      "request_sha256": "4a5cfb98445b511272f4eece16db8e09eedb075da0ce456fedc68d1280dda3f6"
+    },
+    {
+      "case_id": "layout-03-complete",
+      "condition": "rgb",
+      "request_sha256": "727e854ff9361e8e1948877d2d380e35c57ed6860a950f0e281a7f884a68725c"
+    },
+    {
+      "case_id": "layout-02-moving_alias",
+      "condition": "rgb_telemetry",
+      "request_sha256": "2b2cf534f89561d955526b1bb1db41baba75a591546dcd80349293a345ae7877"
+    },
+    {
+      "case_id": "layout-04-wrong_position",
+      "condition": "rgb_telemetry",
+      "request_sha256": "9a86c3e586b664e74763f17964bb1f41aae3dc2d3e938a2af44de193e52754d8"
+    },
+    {
+      "case_id": "layout-01-wrong_object",
+      "condition": "rgb",
+      "request_sha256": "57238b47f9786cb04bcc447130e5b4be6db4fe108092005392ea161d36e7b8f9"
+    },
+    {
+      "case_id": "layout-04-complete",
+      "condition": "rgb_telemetry",
+      "request_sha256": "63d8307d311a3bfd90ef515017c251ee82d3e4b714f7fd7bd716bf53b4d5974d"
+    },
+    {
+      "case_id": "layout-02-wrong_object",
+      "condition": "rgb_telemetry",
+      "request_sha256": "01b8450027128d7fe455f7d3b39c54fcac74a8fe25ff196aa0e15237b9f4f35f"
+    },
+    {
+      "case_id": "layout-02-wrong_orientation",
+      "condition": "rgb_telemetry",
+      "request_sha256": "b525b55a98301c6e1360cf7f82ec7a4215658fd7ba841fa75de19aa3ceea649e"
+    }
+  ]
+}