Read the scene from the host in the G1 saved configuration

The saved configuration's scene_boxes control publishes an empty scene of fixed capacity (one object, two boxes) in the simulator, and the two scene observations compute the box corners inside the policy from the joint readings and the torso anchor pose with the goal's FK kernel, replacing the two dummy observations. The generic export takes eight scene_boxes_* inputs beside the sensor readings (37 in all). Weights, launch config, provenance and browser graphs unchanged; the model card's license sentence regains its subject. Verified against the pinned Oct-25 graph on 64 MuJoCo playback states and four scene cases: raw action within 1.1e-5, gains exact.

G1/MODEL_CARD.md

@@ -14,8 +14,8 @@ See [NOTICE.txt](NOTICE.txt) for attribution.
 - Release: Metatron v1; native package hosted in `nvidia/Metatron-v1/G1`.
   Use the pinned revision selected by the ProtoMotions download helper. No public
   ProtoMotions release tag is asserted by this model card.
- Include both files when
-redistributing these artifacts.
+- License and notice: `LICENSE.pdf` and `NOTICE.txt` ship beside the weights.
+  Include both files when redistributing these artifacts.
 
 ProtoMotions source code remains Apache-2.0. Training datasets and robot assets
 have separate terms; this model license does not grant rights to those materials.
@@ -110,15 +110,22 @@ frame) and `scene_box_mask` (`true` means visible). Our separately pre-exported
 39-input ONNX includes the original scene preprocessing and also retains this
 capability. A scene-capable host must supply the matching object measurements.
 
-The **native preset and generic re-export disable scenes**: the bundled
-observation functions always produce zero corner tokens and false visibility
-masks. Running `deployment/export_to_onnx.py` on this checkpoint preserves the
-weights but builds that scene-disabled observation path. It does not reproduce
-the scene-capable 39-input ONNX. The preset rejects a furniture scene file
-rather than silently ignoring it.
-
-This distinction is intentional: these weights support scenes, but this
-package does not add a scene provider or a SAMP training recipe to ProtoMotions.
+The **saved configuration reads the scene from the host**: its `scene_boxes`
+control publishes an empty scene of fixed capacity (one object, two boxes) in
+the simulator, and the two scene observations compute the box corners in the
+root-heading frame inside the policy from the joint readings and the torso
+anchor pose, with the same forward-kinematics kernel as the goal. Running
+`deployment/export_to_onnx.py` on this checkpoint therefore produces a graph
+with eight `scene_boxes_*` inputs (box centers, extents and validity, object
+visibility, pose and the per-episode position and yaw bias) beside the sensor
+readings; a host with no scene feeds the empty scene (every box invalid and
+hidden), and a scene-capable host feeds its object measurements. The native
+preset still plays on empty terrain and rejects a furniture scene file rather
+than silently ignoring it.
+
+This distinction is intentional: these weights support scenes and the export
+takes them, but this package does not add a scene provider or a SAMP training
+recipe to ProtoMotions.
 
 ## Limitations
 

G1/experiment_config.py

@@ -5,8 +5,9 @@
 
 This recipe preserves the trained module order, normalization and five-body
 order. It is an inference qualification recipe, not a reproduction of the
-four-group training run. Scene tokens are explicitly hidden on this empty
-terrain; furniture interaction needs its own scene observations.
+four-group training run. The scene stream is published empty on this flat
+terrain and read from the host at deployment; furniture interaction in the
+simulator needs a scene-library-backed control.
 """
 
 from argparse import Namespace
@@ -15,10 +16,6 @@ from typing import Any, List, Optional, Sequence
 from examples.experiments.masked_mimic.deployable_g1_observations import (
     configure_observations,
 )
-from examples.inference.metatron import (
-    empty_scene_corners,
-    empty_scene_mask,
-)
 from protomotions.agents.common.config import (
     MLPLayerConfig,
     MLPWithConcatConfig,
@@ -51,6 +48,7 @@ from protomotions.envs.control.masked_mimic_control import (
     MaskedMimicStyleConfig,
     MaskedMimicVelocityCommandConfig,
 )
+from protomotions.envs.control.scene_boxes_control import SceneBoxesControlConfig
 from protomotions.envs.mdp_component import MdpComponent
 from protomotions.envs.motion_manager.config import MimicMotionManagerConfig
 from protomotions.envs.obs.cast import to_float
@@ -149,7 +147,10 @@ def env_config(robot_cfg: RobotConfig, args: Namespace) -> EnvConfig:
                 anchor_pose_submasking=True,
                 style=MaskedMimicStyleConfig(),
                 velocity_command=MaskedMimicVelocityCommandConfig(),
-            )
+            ),
+            # The scene the policy was trained to read: one object of two boxes,
+            # published empty here and filled by a scene-capable host.
+            "scene_boxes": SceneBoxesControlConfig(),
         },
         observation_components={
             "max_coords_obs": MdpComponent.from_kernel(
@@ -175,11 +176,9 @@ def env_config(robot_cfg: RobotConfig, args: Namespace) -> EnvConfig:
                 to_float,
                 x=EnvContext.current_action,
             ),
-            "scene_box_corners": MdpComponent.from_kernel(empty_scene_corners),
-            "scene_box_mask": MdpComponent.from_kernel(empty_scene_mask),
         },
     )
-    configure_observations(robot_cfg, cfg)
+    configure_observations(robot_cfg, cfg, scene_boxes=True)
     for name, path in (
         ("poses", EnvContext.masked_mimic.style_pose_samples),
         ("velocities", EnvContext.masked_mimic.style_velocity_samples),

G1/resolved_configs.json

@@ -1,6 +1,6 @@
 {
   "format_version": 1,
-  "protomotions": "source",
+  "protomotions": "3.1",
   "configs": {
     "robot": {
       "$type": "protomotions.robot_configs.g1.G1RobotConfig",
@@ -464,6 +464,12 @@
             "horizon_seconds": 0.5,
             "probability": 0.5
           }
+        },
+        "scene_boxes": {
+          "$type": "protomotions.envs.control.scene_boxes_control.SceneBoxesControlConfig",
+          "_target_": "protomotions.envs.control.scene_boxes_control.SceneBoxesControl",
+          "max_objects": 1,
+          "max_boxes_per_object": 2
         }
       },
       "termination_components": {},
@@ -569,20 +575,6 @@
           },
           "params": {}
         },
-        "scene_box_corners": {
-          "$kernel": "examples.inference.metatron.empty_scene_corners",
-          "bindings": {
-            "dof_pos": "current.dof_pos"
-          },
-          "params": {}
-        },
-        "scene_box_mask": {
-          "$kernel": "examples.inference.metatron.empty_scene_mask",
-          "bindings": {
-            "dof_pos": "current.dof_pos"
-          },
-          "params": {}
-        },
         "fk_max_coords_obs": {
           "$kernel_object": "protomotions.utils.baked_fk.BakedAnchorFK",
           "init": {
@@ -718,6 +710,45 @@
           },
           "params": {}
         },
+        "scene_box_corners": {
+          "$kernel": "protomotions.envs.obs.scene_boxes.compute_fk_scene_box_corners",
+          "bindings": {
+            "object_pos": "scene_boxes.object_pos",
+            "object_rot": "scene_boxes.object_rot",
+            "centers": "scene_boxes.centers",
+            "extents": "scene_boxes.extents",
+            "valid": "scene_boxes.valid",
+            "position_offset": "scene_boxes.position_offset",
+            "yaw_offset": "scene_boxes.yaw_offset",
+            "dof_pos": "noisy.dof_pos",
+            "anchor_pos": "noisy.anchor_pos",
+            "anchor_rot": "noisy.anchor_rot",
+            "anchor_body_index": "noisy.anchor_idx"
+          },
+          "params": {
+            "fk": {
+              "$kernel_object": "protomotions.utils.baked_fk.BakedAnchorFK",
+              "init": {
+                "anchor_idx": 16,
+                "trackable_body_indices": [
+                  16,
+                  13,
+                  7,
+                  24,
+                  32
+                ]
+              }
+            }
+          }
+        },
+        "scene_box_mask": {
+          "$kernel": "protomotions.envs.obs.scene_boxes.compute_scene_box_mask",
+          "bindings": {
+            "visible": "scene_boxes.visible",
+            "valid": "scene_boxes.valid"
+          },
+          "params": {}
+        },
         "masked_mimic_style_poses": {
           "$kernel": "protomotions.envs.obs.cast.to_float",
           "bindings": {

config.json

@@ -12,8 +12,8 @@
         },
         {
           "path": "G1/MODEL_CARD.md",
-          "size": 7498,
-          "sha256": "b32cfa63ff4151113db965bf5af898451d1095e577ccd07ea8d2975c6590d692"
+          "size": 8057,
+          "sha256": "b620ad3af7ef46041a8139e983b53d6ad9d92cd0a095282efa0c5a3c53e4a95f"
         },
         {
           "path": "G1/NOTICE.txt",
@@ -32,8 +32,8 @@
         },
         {
           "path": "G1/experiment_config.py",
-          "size": 20409,
-          "sha256": "90c0a6411f7717e8e699ebb419fabb66000010445c92464c7c05c60fc160bb06"
+          "size": 20529,
+          "sha256": "51784e3154d739807e2a8935e74eac05df2af0e25408a5229dd153cd8edaa1df"
         },
         {
           "path": "G1/inference_last.ckpt",
@@ -47,8 +47,8 @@
         },
         {
           "path": "G1/resolved_configs.json",
-          "size": 91797,
-          "sha256": "3919e9f4a30072cd708995293d3301388b4248a4c39a9604668b5dcc532bc5e0"
+          "size": 93009,
+          "sha256": "7fb1e13884292976746647593451541905cf2018849643ddb57f9d814baddc3d"
         }
       ]
     }