Refactor teleoperation commands to actions in mujoco_server.py and update protocol documentation

- Renamed `teleop_command` to `teleop_action` and `command` to `action` for consistency across the codebase.
- Updated message handling to support new action-based velocity control for UR5 robots, including backward compatibility for legacy message types.
- Enhanced the README.md and PROTOCOL_CHANGES.md to reflect the new action structure and provide clear migration guidance for developers.
- Introduced a new `protocol_types.py` file with TypeScript-style type definitions for all protocol messages, improving type safety and IDE support.
- Updated tests to validate the new action message structure and ensure backward compatibility with existing command formats.

README.md

@@ -238,7 +238,7 @@ docker run --gpus all -p 3004:3004 \
 - **Joint Mode**: Use J1-J6 sliders to control individual joints
 - **Gripper**: Open/Close buttons for Robotiq gripper
 - **Home**: Return to home position (joint mode)
-- **Keyboard Teleop**: With the UR5 selected, W/A/S/D jogs the tool in the XY plane, R/F nudges along Z, and every keystroke streams a `teleop_command` event that mirrors into the `command` field so the browser panel and any trainer see the same velocity delta.
+- **Keyboard Teleop**: With the UR5 selected, W/A/S/D jogs the tool in the XY plane, R/F nudges along Z, and every keystroke streams a `teleop_action` event so the browser panel and any trainer see the same velocity delta.
 
 ### Common Controls
 - **Mouse drag**: Rotate camera
@@ -332,7 +332,7 @@ docker run --gpus all -p 3004:3004 \
 
 1. **User Input** → Browser captures keyboard/button events
 2. **WebSocket** → Commands sent as `{vx, vy, vyaw}` velocity targets
-3. **Controller** → Converts velocity commands to joint position targets
+3. **Controller** → Converts velocity actions to joint position targets
 4. **MuJoCo** → Simulates physics at 500Hz (0.002s timestep)
 5. **Renderer** → Captures frames at 60 FPS (native) or 30 FPS (Docker)
 6. **MJPEG Stream** → Frames encoded as JPEG and streamed to browser
@@ -340,7 +340,7 @@ docker run --gpus all -p 3004:3004 \
 
 ### Controller Architecture
 
-**G1 (Humanoid)**: Uses a pre-trained RL policy (PyTorch) that maps observations (body orientation, joint positions/velocities, commands) to joint position targets.
+**G1 (Humanoid)**: Uses a pre-trained RL policy (PyTorch) that maps observations (body orientation, joint positions/velocities, actions) to joint position targets.
 
 **Spot (Quadruped)**: Uses gait-based controllers:
 - **MPC Gait**: Phase-based trot with feedback balance control
@@ -378,7 +378,7 @@ Connect using standard WebSocket:
 const ws = new WebSocket('ws://localhost:3004/nova-sim/api/v1/ws');
 
 // Send message
-ws.send(JSON.stringify({type: 'command', data: {vx: 0.5, vy: 0, vyaw: 0}}));
+ws.send(JSON.stringify({type: 'action', data: {vx: 0.5, vy: 0, vyaw: 0}}));
 
 // Receive messages
 ws.onmessage = (event) => {
@@ -389,7 +389,7 @@ ws.onmessage = (event) => {
   };
  ```
 
-Nova-Sim uses `/ws` as the shared control channel for the browser UI, trainers, and any RL clients. Every UI interaction (teleop, camera controls, robot switching) and the trainer handshake/notifications flows through this single socket; the UI `state` messages shown below now also carry the action deltas, integrated reward, and trainer connection status that RL agents need.
+Nova-Sim uses `/ws` as the shared control channel for the browser UI, trainers, and any RL clients. Every UI interaction (teleop, camera controls, robot switching) and the trainer handshake/notifications flows through this single socket; the UI `state` messages shown below now also carry the action velocities, integrated reward, and trainer connection status that RL agents need.
 
 ### HTTP Endpoints
 
@@ -398,7 +398,7 @@ Nova-Sim provides a minimal HTTP API for static information:
 | Endpoint | Method | Description |
 |----------|--------|-------------|
 | `/env` | `GET` | Returns static environment information: robot, scene, has_gripper, action_space, observation_space, camera_feeds |
-| `/metadata` | `GET` | Returns available robots, scenes, commands, and system configuration |
+| `/metadata` | `GET` | Returns available robots, scenes, actions, and system configuration |
 | `/video_feed` | `GET` | MJPEG video stream of the main camera |
 | `/camera/<name>/video_feed` | `GET` | MJPEG video stream of auxiliary cameras (e.g., aux_top, aux_side) |
 
@@ -421,18 +421,28 @@ Nova-Sim provides a minimal HTTP API for static information:
 
 The `/env` endpoint returns scene-specific information including camera feeds available for the current robot/scene configuration.
 
-All dynamic operations (reset, switching robots, sending commands) are performed via WebSocket messages. Training data (observations, rewards, etc.) come from the `/ws` state stream.
+All dynamic operations (reset, switching robots, sending actions) are performed via WebSocket messages. Training data (observations, rewards, etc.) come from the `/ws` state stream.
 
 ### Client → Server WebSocket Messages
 
-**`command`** - Send velocity commands to locomotion robots:
+**`action`** - Send velocity actions to all robots:
 ```json
-{"type": "command", "data": {"vx": 0.5, "vy": 0.0, "vyaw": 0.0}}
+{"type": "action", "data": {"vx": 0.5, "vy": 0.0, "vyaw": 0.0}}
 ```
+
+For locomotion robots (G1, Spot):
 - `vx`: Forward/backward velocity [-1, 1]
 - `vy`: Left/right strafe velocity [-1, 1]
 - `vyaw`: Turn rate [-1, 1]
 
+For robot arms (UR5):
+- `vx`, `vy`, `vz`: Cartesian translation velocities (m/s)
+- `vrx`, `vry`, `vrz`: Cartesian rotation velocities (rad/s)
+- `j1`-`j6`: Joint velocities (rad/s)
+- `gripper`: Gripper position [0-255]
+
+**Note:** Old message type `command` is still accepted for backward compatibility.
+
 **`reset`** - Reset the environment:
 ```json
 {"type": "reset", "data": {"seed": 42}}
@@ -451,12 +461,13 @@ All dynamic operations (reset, switching robots, sending commands) are performed
 {"type": "home"}
 ```
 
-**`teleop_command`** - Send Cartesian teleop delta (UR5) or velocity (locomotion):
+**`teleop_action`** - Send teleoperation action (primarily for UR5 keyboard control):
 ```json
-{"type": "teleop_command", "data": {"vx": 0.01, "vy": 0.0, "vz": 0.0}}
+{"type": "teleop_action", "data": {"vx": 0.01, "vy": 0.0, "vz": 0.0}}
 ```
-- `vx`, `vy`, `vz`: Velocity/delta values
+- For UR5: `vx`, `vy`, `vz` represent Cartesian velocity deltas (m/s)
 - Backward compatible: old `dx`, `dy`, `dz` format is auto-mapped to `vx`, `vy`, `vz`
+- **Note:** Old message type `teleop_command` is still accepted for backward compatibility.
 
 **`camera`:**
 ```json
@@ -532,7 +543,7 @@ For locomotion robots (G1, Spot):
     },
     "steps": 1234,
     "reward": 0.0,
-    "teleop_command": {"vx": 0.5, "vy": 0.0, "vz": 0.0, "vyaw": 0.0, "gripper": 0.0},
+    "teleop_action": {"vx": 0.5, "vy": 0.0, "vz": 0.0, "vyaw": 0.0, "vrx": 0.0, "vry": 0.0, "vrz": 0.0, "j1": 0.0, "j2": 0.0, "j3": 0.0, "j4": 0.0, "j5": 0.0, "j6": 0.0, "gripper": 0.0},
     "trainer_connected": true
   }
 }
@@ -555,7 +566,7 @@ For robot arm (UR5):
     "control_mode": "ik",
     "steps": 1234,
     "reward": -0.25,
-    "teleop_command": {"vx": 0.02, "vy": 0.0, "vz": 0.0, "vyaw": 0.0, "gripper": 128.0},
+    "teleop_action": {"vx": 0.02, "vy": 0.0, "vz": 0.0, "vyaw": 0.0, "vrx": 0.0, "vry": 0.0, "vrz": 0.0, "j1": 0.0, "j2": 0.0, "j3": 0.0, "j4": 0.0, "j5": 0.0, "j6": 0.0, "gripper": 128.0},
     "trainer_connected": true,
     "nova_api": {
       "connected": true,
@@ -571,10 +582,13 @@ For robot arm (UR5):
 - `observation`: Contains robot-specific sensor data and state information
 - `steps`: Number of simulation steps since last reset
 - `reward`: The integrated task reward from the simulator that remote trainers can consume
-- `teleop_command`: The canonical velocity/delta stream that drives locomotion or arm movement; the UI and every trainer should read this field as the unified action record. Always present with zero values when idle
-  - Common fields: `vx` (forward/back or X-axis), `vy` (strafe or Y-axis), `vz` (vertical for UR5), `vyaw` (rotation for locomotion), `gripper` (0-255 for UR5, 0 for others)
-  - Locomotion robots: Use `vx`, `vy`, `vyaw` (vz and gripper are 0)
-  - UR5: Use `vx`, `vy`, `vz` for Cartesian movement, `gripper` for gripper control (vyaw is 0)
+- `teleop_action`: The canonical action/velocity stream that drives locomotion or arm movement; the UI and every trainer should read this field as the unified action record. Always present with zero values when idle
+  - Cartesian velocities: `vx` (forward/back or X-axis), `vy` (strafe or Y-axis), `vz` (vertical for UR5), `vyaw` (rotation for locomotion)
+  - Cartesian rotation velocities (UR5 only): `vrx`, `vry`, `vrz` (rad/s)
+  - Joint velocities (UR5 only): `j1`, `j2`, `j3`, `j4`, `j5`, `j6` (rad/s)
+  - Gripper: `gripper` (0-255 for UR5, 0 for others)
+  - Locomotion robots: Use `vx`, `vy`, `vyaw` (other fields are 0)
+  - UR5: Use `vx`/`vy`/`vz` for Cartesian translation, `vrx`/`vry`/`vrz` for rotation, `j1`-`j6` for joint velocities, and `gripper` for gripper control
 - `trainer_connected`: Whether a trainer handshake is active on `/ws` (useful for status LEDs)
 
 *Locomotion observation fields (inside `observation`):*
@@ -603,7 +617,7 @@ For robot arm (UR5):
 
 Every `/ws` client receives a `state` message roughly every 100 ms. The examples above show the locomotion (`spot`) and arm (`ur5`) payloads; the payload also now includes:
 
-- `teleop_command`: The latest command/teleoperation stream (`vx`, `vy`, `vz`, `vyaw`) so trainers and the UI read a single canonical action payload. Always present with zero values when idle.
+- `teleop_action`: The latest action/teleoperation stream (includes `vx`, `vy`, `vz`, `vyaw`, `vrx`, `vry`, `vrz`, `j1`-`j6`, `gripper`) so trainers and the UI read a single canonical action payload. Always present with zero values when idle.
 - `reward`: The integrated task reward that trainers can consume without sending a separate `step`.
 - `trainer_connected`: Whether a trainer handshake is active on `/ws` (used to update the UI indicator).
 
@@ -615,7 +629,7 @@ Trainers announce themselves by sending a `trainer_identity` payload when the so
 |----------|--------|-------------|
 | `/nova-sim/api/v1` | GET | Web interface (HTML/JS) |
 | `/nova-sim/api/v1/env` | GET | Static environment info (robot, scene, spaces, camera feeds) |
-| `/nova-sim/api/v1/metadata` | GET | Available robots, scenes, commands, and system configuration |
+| `/nova-sim/api/v1/metadata` | GET | Available robots, scenes, actions, and system configuration |
 | `/nova-sim/api/v1/video_feed` | GET | MJPEG video stream (main camera) |
 | `/nova-sim/api/v1/camera/<name>/video_feed` | GET | MJPEG video stream (auxiliary cameras) |
 
@@ -628,7 +642,7 @@ Trainers announce themselves by sending a `trainer_identity` payload when the so
 
 ### Metadata & Camera Feeds
 
-- `GET /nova-sim/api/v1/metadata` returns JSON describing every available robot/scene pair and the supported commands
+- `GET /nova-sim/api/v1/metadata` returns JSON describing every available robot/scene pair and the supported actions
 - `GET /nova-sim/api/v1/env` returns scene-specific camera feeds - the `camera_feeds` array lists all available video streams for the current robot/scene configuration including the main camera and any auxiliary cameras (e.g., `aux_top`, `aux_side`, `aux_flange`)
 - `GET /nova-sim/api/v1/camera/<name>/video_feed` streams MJPEG for a specific camera feed
 - `pytest tests/` exercises the HTTP metadata/video endpoints, the `/ws` control socket, and every camera feed. Keep Nova-Sim running at `http://localhost:3004` when you run it so the suite can talk to the live server.
@@ -751,7 +765,7 @@ env = UR5Env(
 
 **Note**: When state streaming is enabled, the simulation becomes a read-only digital twin that displays the real robot's movements. Local target controls and IK computations are ignored since the robot is controlled by external hardware/software.
 
-**Sending Commands to Robot**: To send motion commands to the real robot through Nova API, use Nova's motion execution endpoints directly (not currently implemented in the web UI). The simulation is designed to visualize robot state, not to command it.
+**Sending Actions to Robot**: To send motion actions to the real robot through Nova API, use Nova's motion execution endpoints directly (not currently implemented in the web UI). The simulation is designed to visualize robot state, not to control it.
 
 ### Environment Variables Reference
 
@@ -873,10 +887,10 @@ This project uses models from:
 - [unitree_mujoco](https://github.com/unitreerobotics/unitree_mujoco) (BSD-3-Clause)
 - [unitree_rl_gym](https://github.com/unitreerobotics/unitree_rl_gym)
 - [Quadruped-PyMPC](https://github.com/iit-DLSLab/Quadruped-PyMPC) (BSD-3-Clause)
-**`teleop_command`:**
+**`teleop_action`:**
 ```json
-{"type": "teleop_command", "data": {"vx": 0.01, "vy": 0.0, "vz": -0.01}}
+{"type": "teleop_action", "data": {"vx": 0.01, "vy": 0.0, "vz": -0.01}}
 ```
 - `vx`, `vy`, `vz`: Velocity/delta values for UR5 Cartesian movement (WASD + RF from the UI) or locomotion robot velocity (`vx`, `vy`, `vyaw`)
-- These values appear in the `teleop_command` field of each `/ws` `state` broadcast, which is the canonical action stream for both the UI and any RL trainer
+- These values appear in the `teleop_action` field of each `/ws` `state` broadcast, which is the canonical action stream for both the UI and any RL trainer
 - The field always contains zero values when idle (never null)

mujoco_server.py

@@ -100,9 +100,9 @@ episode_control_state = {
 }
 episode_control_lock = threading.Lock()
 
-# Latest teleoperation command (for trainer state)
+# Latest teleoperation action (for trainer state)
 # Initialize with zero values instead of None so it's always a dict
-last_teleop_command: dict[str, Any] = {
+last_teleop_action: dict[str, Any] = {
     "vx": 0.0,
     "vy": 0.0,
     "vz": 0.0,
@@ -216,13 +216,13 @@ DEFAULT_SCENES = {
     "spot": "scene",
 }
 
-AVAILABLE_COMMANDS = [
-    "command",
+AVAILABLE_ACTIONS = [
+    "action",
     "reset",
     "switch_robot",
     "camera",
     "camera_follow",
-    "teleop_command",
+    "teleop_action",
     "start_jog",
     "stop_jog",
     "arm_target",
@@ -531,19 +531,19 @@ def switch_robot(robot_type, scene_name=None):
     current_scene = active_scene
     env.reset()
 
-    # Initialize gripper value in teleop_command for UR5
+    # Initialize gripper value in teleop_action for UR5
     with teleop_lock:
         if robot_type in ("ur5", "ur5_t_push"):
             has_gripper = getattr(env, "has_gripper", False)
             if has_gripper:
                 # Get current gripper value from env
                 gripper_val = getattr(env, "get_gripper", lambda: 128)()
-                last_teleop_command["gripper"] = float(gripper_val)
+                last_teleop_action["gripper"] = float(gripper_val)
             else:
-                last_teleop_command["gripper"] = 0.0
+                last_teleop_action["gripper"] = 0.0
         else:
             # Locomotion robots don't have grippers
-            last_teleop_command["gripper"] = 0.0
+            last_teleop_action["gripper"] = 0.0
 
     # Create new renderer
     renderer = mujoco.Renderer(env.model, height=env.height, width=env.width)
@@ -571,7 +571,7 @@ def broadcast_state():
             "vz": 0.0,
         }
         with teleop_lock:
-            teleop_snapshot = last_teleop_command.copy()
+            teleop_snapshot = last_teleop_action.copy()
             # Debug: print non-zero jogging values
             non_zero = {k: v for k, v in teleop_snapshot.items() if v != 0.0}
             if non_zero:
@@ -628,7 +628,7 @@ def broadcast_state():
                     'control_mode': control_mode,
                     'steps': int(steps),
                     'reward': reward_value,
-                    'teleop_command': teleop_snapshot,
+                    'teleop_action': teleop_snapshot,
                     'nova_api': {
                         'connected': nova_connected,
                         'state_streaming': nova_state_streaming,
@@ -655,7 +655,7 @@ def broadcast_state():
                     },
                     'steps': int(steps),
                     'reward': reward_value,
-                    'teleop_command': teleop_snapshot,
+                    'teleop_action': teleop_snapshot,
                     'trainer_connected': trainer_connected
                 }
             })
@@ -669,8 +669,8 @@ def broadcast_state():
                 # Print actual JSON being sent
                 import json as json_module
                 parsed = json_module.loads(state_msg)
-                actual_teleop = parsed.get('data', {}).get('teleop_command', {})
-                print(f"[Broadcast] Actual JSON teleop_command j3 = {actual_teleop.get('j3', 'MISSING')}")
+                actual_teleop = parsed.get('data', {}).get('teleop_action', {})
+                print(f"[Broadcast] Actual JSON teleop_action j3 = {actual_teleop.get('j3', 'MISSING')}")
             for ws in ws_clients:
                 try:
                     ws.send(state_msg)
@@ -963,7 +963,7 @@ def generate_overlay_frames(name: str):
 
 def handle_ws_message(ws, data):
     """Handle incoming WebSocket message."""
-    global needs_robot_switch, camera_follow, last_teleop_command
+    global needs_robot_switch, camera_follow, last_teleop_action
 
     msg_type = data.get('type')
     print(f"[WS] Received message type: {msg_type}")
@@ -972,21 +972,104 @@ def handle_ws_message(ws, data):
         _handle_trainer_message(ws, data)
         return
 
+    # Backward compatibility: map old message types to new ones
     if msg_type == 'command':
+        msg_type = 'action'
+        data['type'] = 'action'
+    elif msg_type == 'teleop_command':
+        msg_type = 'teleop_action'
+        data['type'] = 'teleop_action'
+
+    if msg_type == 'action':
         payload = data.get('data', {})
         vx = payload.get('vx', 0.0)
         vy = payload.get('vy', 0.0)
+        vz = payload.get('vz', 0.0)
         vyaw = payload.get('vyaw', 0.0)
+        vrx = payload.get('vrx', 0.0)
+        vry = payload.get('vry', 0.0)
+        vrz = payload.get('vrz', 0.0)
+        j1 = payload.get('j1', 0.0)
+        j2 = payload.get('j2', 0.0)
+        j3 = payload.get('j3', 0.0)
+        j4 = payload.get('j4', 0.0)
+        j5 = payload.get('j5', 0.0)
+        j6 = payload.get('j6', 0.0)
+        gripper = payload.get('gripper', None)
+
         with mujoco_lock:
             if env is not None:
-                env.set_command(vx, vy, vyaw)
+                # For UR5: translate velocity actions to jogging commands
+                if current_robot in ("ur5", "ur5_t_push"):
+                    # Handle gripper if specified
+                    if gripper is not None:
+                        env.set_gripper(float(gripper))
+
+                    # Check which type of velocity is active (mutually exclusive)
+                    joint_velocities = [j1, j2, j3, j4, j5, j6]
+                    cartesian_translation = [vx, vy, vz]
+                    cartesian_rotation = [vrx, vry, vrz]
+
+                    # Find which velocity mode is active
+                    active_joint = None
+                    for i, vel in enumerate(joint_velocities):
+                        if abs(vel) > 0.001:  # Threshold to ignore noise
+                            active_joint = (i + 1, vel)
+                            break
+
+                    active_translation = None
+                    for i, (axis, vel) in enumerate(zip(['x', 'y', 'z'], cartesian_translation)):
+                        if abs(vel) > 0.001:
+                            active_translation = (axis, vel)
+                            break
+
+                    active_rotation = None
+                    for i, (axis, vel) in enumerate(zip(['x', 'y', 'z'], cartesian_rotation)):
+                        if abs(vel) > 0.001:
+                            active_rotation = (axis, vel)
+                            break
+
+                    # Apply the appropriate jogging mode
+                    if active_joint:
+                        joint, velocity = active_joint
+                        direction = '+' if velocity > 0 else '-'
+                        env.start_jog('joint', joint=joint, direction=direction, velocity=abs(velocity))
+                    elif active_translation:
+                        axis, velocity = active_translation
+                        direction = '+' if velocity > 0 else '-'
+                        # Convert m/s to mm/s
+                        velocity_mm_s = abs(velocity) * 1000.0
+                        env.start_jog('cartesian_translation', axis=axis, direction=direction,
+                                     velocity=velocity_mm_s, tcp_id='Flange', coord_system_id='world')
+                    elif active_rotation:
+                        axis, velocity = active_rotation
+                        direction = '+' if velocity > 0 else '-'
+                        env.start_jog('cartesian_rotation', axis=axis, direction=direction,
+                                     velocity=abs(velocity), tcp_id='Flange', coord_system_id='world')
+                    else:
+                        # No active velocity - stop jogging
+                        env.stop_jog()
+                else:
+                    # For locomotion robots: use vx, vy, vyaw
+                    env.set_command(vx, vy, vyaw)
+
         with teleop_lock:
-            # Update locomotion velocities, preserve other fields
-            last_teleop_command["vx"] = float(vx)
-            last_teleop_command["vy"] = float(vy)
-            last_teleop_command["vz"] = 0.0
-            last_teleop_command["vyaw"] = float(vyaw)
-            # Note: Don't clear jogging velocities here - they should only be cleared by stop_jog
+            # Update all velocity fields in teleop_action
+            last_teleop_action["vx"] = float(vx)
+            last_teleop_action["vy"] = float(vy)
+            last_teleop_action["vz"] = float(vz)
+            last_teleop_action["vyaw"] = float(vyaw)
+            last_teleop_action["vrx"] = float(vrx)
+            last_teleop_action["vry"] = float(vry)
+            last_teleop_action["vrz"] = float(vrz)
+            last_teleop_action["j1"] = float(j1)
+            last_teleop_action["j2"] = float(j2)
+            last_teleop_action["j3"] = float(j3)
+            last_teleop_action["j4"] = float(j4)
+            last_teleop_action["j5"] = float(j5)
+            last_teleop_action["j6"] = float(j6)
+            if gripper is not None:
+                last_teleop_action["gripper"] = float(gripper)
 
     elif msg_type == 'reset':
         payload = data.get('data', {})
@@ -1197,7 +1280,7 @@ def handle_ws_message(ws, data):
         payload = data.get('data', {})
         camera_follow = payload.get('follow', True)
 
-    elif msg_type == 'teleop_command':
+    elif msg_type == 'teleop_action':
         payload = data.get('data', {})
         # Accept both old format (dx/dy/dz) and new format (vx/vy/vz)
         vx = float(payload.get('vx', payload.get('dx', 0.0)))
@@ -1220,19 +1303,19 @@ def handle_ws_message(ws, data):
 
         with teleop_lock:
             # Update UR5 Cartesian velocities, preserve other fields
-            last_teleop_command["vx"] = vx
-            last_teleop_command["vy"] = vy
-            last_teleop_command["vz"] = vz
-            last_teleop_command["vyaw"] = 0.0
+            last_teleop_action["vx"] = vx
+            last_teleop_action["vy"] = vy
+            last_teleop_action["vz"] = vz
+            last_teleop_action["vyaw"] = 0.0
             # Clear jogging velocities when using teleop (jogging and teleop are mutually exclusive)
             for j in range(1, 7):
-                last_teleop_command[f"j{j}"] = 0.0
-            last_teleop_command["vrx"] = 0.0
-            last_teleop_command["vry"] = 0.0
-            last_teleop_command["vrz"] = 0.0
+                last_teleop_action[f"j{j}"] = 0.0
+            last_teleop_action["vrx"] = 0.0
+            last_teleop_action["vry"] = 0.0
+            last_teleop_action["vrz"] = 0.0
 
         broadcast_to_trainer(
-            "teleop_command",
+            "action_update",
             {
                 "robot": current_robot,
                 "scene": getattr(env, "scene_name", None) if env is not None else None,
@@ -1270,9 +1353,9 @@ def handle_ws_message(ws, data):
         with mujoco_lock:
             if env is not None and current_robot in ("ur5", "ur5_t_push"):
                 env.set_gripper(value)
-        # Update teleop_command with gripper value
+        # Update teleop_action with gripper value
         with teleop_lock:
-            last_teleop_command["gripper"] = float(value)
+            last_teleop_action["gripper"] = float(value)
 
     elif msg_type == 'control_mode':
         payload = data.get('data', {})
@@ -1324,7 +1407,7 @@ def handle_ws_message(ws, data):
                 if not success:
                     print(f"[Server] Failed to start jog: {jog_type}, {jog_params}")
 
-        # Update teleop_command to reflect jogging state
+        # Update teleop_action to reflect jogging state
         with teleop_lock:
             if jog_type == 'joint':
                 # Joint jogging: {"j1": vel, "j2": vel, ...}
@@ -1337,16 +1420,16 @@ def handle_ws_message(ws, data):
 
                 # Reset all joint velocities to 0
                 for j in range(1, 7):
-                    last_teleop_command[f"j{j}"] = 0.0
+                    last_teleop_action[f"j{j}"] = 0.0
                 # Set active joint velocity
-                last_teleop_command[f"j{joint}"] = float(vel_value)
+                last_teleop_action[f"j{joint}"] = float(vel_value)
                 # Clear Cartesian velocities
-                last_teleop_command["vx"] = 0.0
-                last_teleop_command["vy"] = 0.0
-                last_teleop_command["vz"] = 0.0
-                last_teleop_command["vrx"] = 0.0
-                last_teleop_command["vry"] = 0.0
-                last_teleop_command["vrz"] = 0.0
+                last_teleop_action["vx"] = 0.0
+                last_teleop_action["vy"] = 0.0
+                last_teleop_action["vz"] = 0.0
+                last_teleop_action["vrx"] = 0.0
+                last_teleop_action["vry"] = 0.0
+                last_teleop_action["vrz"] = 0.0
 
             elif jog_type == 'cartesian_translation':
                 # Cartesian translation: {"vx": vel, "vy": vel, "vz": vel}
@@ -1356,17 +1439,17 @@ def handle_ws_message(ws, data):
                 vel_value = velocity if direction == '+' else -velocity
 
                 # Reset Cartesian velocities
-                last_teleop_command["vx"] = 0.0
-                last_teleop_command["vy"] = 0.0
-                last_teleop_command["vz"] = 0.0
-                last_teleop_command["vrx"] = 0.0
-                last_teleop_command["vry"] = 0.0
-                last_teleop_command["vrz"] = 0.0
+                last_teleop_action["vx"] = 0.0
+                last_teleop_action["vy"] = 0.0
+                last_teleop_action["vz"] = 0.0
+                last_teleop_action["vrx"] = 0.0
+                last_teleop_action["vry"] = 0.0
+                last_teleop_action["vrz"] = 0.0
                 # Set active axis velocity
-                last_teleop_command[f"v{axis}"] = float(vel_value)
+                last_teleop_action[f"v{axis}"] = float(vel_value)
                 # Clear joint velocities
                 for j in range(1, 7):
-                    last_teleop_command[f"j{j}"] = 0.0
+                    last_teleop_action[f"j{j}"] = 0.0
 
             elif jog_type == 'cartesian_rotation':
                 # Cartesian rotation: {"vrx": vel, "vry": vel, "vrz": vel}
@@ -1376,33 +1459,33 @@ def handle_ws_message(ws, data):
                 vel_value = velocity if direction == '+' else -velocity
 
                 # Reset Cartesian velocities
-                last_teleop_command["vx"] = 0.0
-                last_teleop_command["vy"] = 0.0
-                last_teleop_command["vz"] = 0.0
-                last_teleop_command["vrx"] = 0.0
-                last_teleop_command["vry"] = 0.0
-                last_teleop_command["vrz"] = 0.0
+                last_teleop_action["vx"] = 0.0
+                last_teleop_action["vy"] = 0.0
+                last_teleop_action["vz"] = 0.0
+                last_teleop_action["vrx"] = 0.0
+                last_teleop_action["vry"] = 0.0
+                last_teleop_action["vrz"] = 0.0
                 # Set active rotation velocity
-                last_teleop_command[f"vr{axis}"] = float(vel_value)
+                last_teleop_action[f"vr{axis}"] = float(vel_value)
                 # Clear joint velocities
                 for j in range(1, 7):
-                    last_teleop_command[f"j{j}"] = 0.0
+                    last_teleop_action[f"j{j}"] = 0.0
 
     elif msg_type == 'stop_jog':
         with mujoco_lock:
             if env is not None and current_robot in ("ur5", "ur5_t_push"):
                 env.stop_jog()
 
-        # Clear all jogging velocities in teleop_command
+        # Clear all jogging velocities in teleop_action
         with teleop_lock:
-            last_teleop_command["vx"] = 0.0
-            last_teleop_command["vy"] = 0.0
-            last_teleop_command["vz"] = 0.0
-            last_teleop_command["vrx"] = 0.0
-            last_teleop_command["vry"] = 0.0
-            last_teleop_command["vrz"] = 0.0
+            last_teleop_action["vx"] = 0.0
+            last_teleop_action["vy"] = 0.0
+            last_teleop_action["vz"] = 0.0
+            last_teleop_action["vrx"] = 0.0
+            last_teleop_action["vry"] = 0.0
+            last_teleop_action["vrz"] = 0.0
             for j in range(1, 7):
-                last_teleop_command[f"j{j}"] = 0.0
+                last_teleop_action[f"j{j}"] = 0.0
 
     elif msg_type == 'homing':
         with mujoco_lock:
@@ -2858,7 +2941,7 @@ def index():
                                 }
 
                                 // Update teleop command display - only show non-zero values
-                                const armTeleop = data.teleop_command;
+                                const armTeleop = data.teleop_action;
                                 const armTeleopDisplayEl = document.getElementById('arm_teleop_display');
                                 if (armTeleop && armTeleopDisplayEl) {
                                     const parts = [];
@@ -2913,7 +2996,7 @@ def index():
                                 }
 
                                 // Update teleop command display - only show non-zero values
-                                const locoTeleop = data.teleop_command || {};
+                                const locoTeleop = data.teleop_action || {};
                                 const locoTeleopDisplayEl = document.getElementById('loco_teleop_display');
                                 if (locoTeleopDisplayEl) {
                                     const parts = [];
@@ -3428,7 +3511,7 @@ def index():
                 }
 
                 lastTeleopCommand = {dx, dy, dz};
-                send('teleop_command', {dx, dy, dz});
+                send('teleop_action', {dx, dy, dz});
                 if (teleopStatus) {
                     teleopStatus.innerText = `UI teleop → dx: ${dx.toFixed(3)} m, dy: ${dy.toFixed(3)} m, dz: ${dz.toFixed(3)} m`;
                 }
@@ -3670,7 +3753,7 @@ def metadata():
                 pass
     response = {
         "robots": robots_meta,
-        "commands": AVAILABLE_COMMANDS,
+        "actions": AVAILABLE_ACTIONS,
         "nova_api": {
             "preconfigured": NOVA_CREDENTIALS_PRESENT,
         },

protocol_types.py

@@ -0,0 +1,456 @@
+"""
+Protocol type definitions for Nova-Sim WebSocket API.
+
+This module defines the structure of all WebSocket messages exchanged between
+clients and the server, including both request and response messages.
+"""
+
+from typing import TypedDict, Literal, Optional, Union, List
+
+
+# ============================================================================
+# Common Types
+# ============================================================================
+
+RobotType = Literal["g1", "spot", "ur5", "ur5_t_push"]
+SceneType = Optional[str]
+ControlMode = Literal["ik", "joint"]
+
+
+# ============================================================================
+# Action Messages (Client -> Server)
+# ============================================================================
+
+class ActionData(TypedDict, total=False):
+    """Velocity-based action commands for robot control.
+
+    For locomotion robots (G1, Spot):
+        - vx: Forward/backward velocity [-1, 1]
+        - vy: Left/right strafe velocity [-1, 1]
+        - vyaw: Turn rate [-1, 1]
+
+    For robot arms (UR5):
+        - vx, vy, vz: Cartesian translation velocities (m/s)
+        - vrx, vry, vrz: Cartesian rotation velocities (rad/s)
+        - j1-j6: Joint velocities (rad/s)
+        - gripper: Gripper position [0-255]
+    """
+    # Cartesian translation velocities (m/s for UR5, normalized for locomotion)
+    vx: float
+    vy: float
+    vz: float
+
+    # Rotation velocity (rad/s for UR5, normalized for locomotion)
+    vyaw: float
+
+    # Cartesian rotation velocities (rad/s, UR5 only)
+    vrx: float
+    vry: float
+    vrz: float
+
+    # Joint velocities (rad/s, UR5 only)
+    j1: float
+    j2: float
+    j3: float
+    j4: float
+    j5: float
+    j6: float
+
+    # Gripper position (0=open, 255=closed, UR5 only)
+    gripper: float
+
+
+class ActionMessage(TypedDict):
+    """Action command message for all robots."""
+    type: Literal["action"]
+    data: ActionData
+
+
+class TeleopActionData(TypedDict, total=False):
+    """Teleoperation action data (backward compatible with old teleop_command)."""
+    vx: float
+    vy: float
+    vz: float
+    # Backward compatibility: accept old dx/dy/dz format
+    dx: float
+    dy: float
+    dz: float
+
+
+class TeleopActionMessage(TypedDict):
+    """Teleoperation action message (UR5 keyboard control)."""
+    type: Literal["teleop_action"]
+    data: TeleopActionData
+
+
+# ============================================================================
+# Other Client -> Server Messages
+# ============================================================================
+
+class ResetData(TypedDict, total=False):
+    """Reset environment data."""
+    seed: Optional[int]
+
+
+class ResetMessage(TypedDict):
+    """Reset environment message."""
+    type: Literal["reset"]
+    data: ResetData
+
+
+class SwitchRobotData(TypedDict):
+    """Switch robot data."""
+    robot: RobotType
+    scene: Optional[str]
+
+
+class SwitchRobotMessage(TypedDict):
+    """Switch robot message."""
+    type: Literal["switch_robot"]
+    data: SwitchRobotData
+
+
+class HomeMessage(TypedDict):
+    """Home robot message (UR5 only)."""
+    type: Literal["home"]
+
+
+class StopHomeMessage(TypedDict):
+    """Stop homing sequence message (UR5 only)."""
+    type: Literal["stop_home"]
+
+
+class CameraRotateData(TypedDict):
+    """Camera rotation data."""
+    action: Literal["rotate"]
+    dx: float
+    dy: float
+
+
+class CameraZoomData(TypedDict):
+    """Camera zoom data."""
+    action: Literal["zoom"]
+    dz: float
+
+
+class CameraPanData(TypedDict):
+    """Camera pan data."""
+    action: Literal["pan"]
+    dx: float
+    dy: float
+
+
+class CameraSetDistanceData(TypedDict):
+    """Camera set distance data."""
+    action: Literal["set_distance"]
+    distance: float
+
+
+CameraData = Union[CameraRotateData, CameraZoomData, CameraPanData, CameraSetDistanceData]
+
+
+class CameraMessage(TypedDict):
+    """Camera control message."""
+    type: Literal["camera"]
+    data: CameraData
+
+
+class CameraFollowData(TypedDict):
+    """Camera follow mode data."""
+    follow: bool
+
+
+class CameraFollowMessage(TypedDict):
+    """Camera follow mode message."""
+    type: Literal["camera_follow"]
+    data: CameraFollowData
+
+
+# ============================================================================
+# UR5-Specific Client -> Server Messages
+# ============================================================================
+
+class ArmTargetData(TypedDict):
+    """Arm target position data (IK mode)."""
+    x: float
+    y: float
+    z: float
+
+
+class ArmTargetMessage(TypedDict):
+    """Set arm target position message (UR5, IK mode)."""
+    type: Literal["arm_target"]
+    data: ArmTargetData
+
+
+class ArmOrientationData(TypedDict):
+    """Arm target orientation data (IK mode)."""
+    roll: float
+    pitch: float
+    yaw: float
+
+
+class ArmOrientationMessage(TypedDict):
+    """Set arm target orientation message (UR5, IK mode)."""
+    type: Literal["arm_orientation"]
+    data: ArmOrientationData
+
+
+class UseOrientationData(TypedDict):
+    """Toggle orientation control data."""
+    enabled: bool
+
+
+class UseOrientationMessage(TypedDict):
+    """Toggle orientation control message (UR5)."""
+    type: Literal["use_orientation"]
+    data: UseOrientationData
+
+
+class JointPositionsData(TypedDict):
+    """Joint positions data (joint mode)."""
+    positions: List[float]  # Array of 6 joint angles in radians
+
+
+class JointPositionsMessage(TypedDict):
+    """Set joint positions message (UR5, joint mode)."""
+    type: Literal["joint_positions"]
+    data: JointPositionsData
+
+
+class ControlModeData(TypedDict):
+    """Control mode data."""
+    mode: ControlMode
+
+
+class ControlModeMessage(TypedDict):
+    """Set control mode message (UR5)."""
+    type: Literal["control_mode"]
+    data: ControlModeData
+
+
+class GripperData(TypedDict):
+    """Gripper control data."""
+    action: Literal["open", "close"]
+    value: Optional[int]  # 0-255, only used if action is not "open" or "close"
+
+
+class GripperMessage(TypedDict):
+    """Gripper control message (UR5)."""
+    type: Literal["gripper"]
+    data: GripperData
+
+
+class NovaModeSetting(TypedDict):
+    """Nova API mode settings."""
+    state_streaming: bool
+    ik: bool
+
+
+class SetNovaModeData(TypedDict):
+    """Set Nova API mode data."""
+    enabled: Optional[bool]  # Legacy: enable/disable all Nova features
+    # New granular settings:
+    state_streaming: Optional[bool]
+    ik: Optional[bool]
+
+
+class SetNovaModeMessage(TypedDict):
+    """Set Nova API mode message (UR5)."""
+    type: Literal["set_nova_mode"]
+    data: SetNovaModeData
+
+
+# ============================================================================
+# Trainer Messages (Client -> Server)
+# ============================================================================
+
+class TrainerIdentityData(TypedDict):
+    """Trainer identity data."""
+    trainer_id: str
+
+
+class TrainerIdentityMessage(TypedDict):
+    """Trainer identity handshake message."""
+    type: Literal["trainer_identity"]
+    data: TrainerIdentityData
+
+
+class TrainerNotificationData(TypedDict, total=False):
+    """Trainer notification data."""
+    message: str
+    level: Literal["info", "warning", "error"]
+
+
+class TrainerNotificationMessage(TypedDict):
+    """Trainer notification message."""
+    type: Literal["notification"]
+    data: TrainerNotificationData
+
+
+class EpisodeControlData(TypedDict):
+    """Episode control data."""
+    action: Literal["terminate", "truncate"]
+
+
+class EpisodeControlMessage(TypedDict):
+    """Episode control message (trainer only)."""
+    type: Literal["episode_control"]
+    data: EpisodeControlData
+
+
+# ============================================================================
+# Server -> Client Messages
+# ============================================================================
+
+class Position(TypedDict):
+    """3D position."""
+    x: float
+    y: float
+    z: float
+
+
+class Quaternion(TypedDict):
+    """Quaternion orientation."""
+    w: float
+    x: float
+    y: float
+    z: float
+
+
+class EulerAngles(TypedDict):
+    """Euler angles orientation."""
+    roll: float
+    pitch: float
+    yaw: float
+
+
+class LocomotionObservation(TypedDict):
+    """Observation data for locomotion robots (G1, Spot)."""
+    position: Position
+    orientation: Quaternion
+
+
+class UR5Observation(TypedDict):
+    """Observation data for UR5 robot arm."""
+    end_effector: Position
+    ee_orientation: Quaternion
+    ee_target: Position
+    ee_target_orientation: EulerAngles
+    gripper: int  # 0-255
+    joint_positions: List[float]  # 6 joint angles
+    joint_targets: List[float]  # 6 target joint angles
+
+
+Observation = Union[LocomotionObservation, UR5Observation]
+
+
+class NovaApiStatus(TypedDict):
+    """Nova API integration status."""
+    connected: bool
+    state_streaming: bool
+    ik: bool
+
+
+class StateData(TypedDict, total=False):
+    """State broadcast data."""
+    observation: Observation
+    steps: int
+    reward: float
+    teleop_action: ActionData  # Current action/velocity commands
+    trainer_connected: bool
+
+    # UR5-specific fields
+    control_mode: ControlMode
+    nova_api: NovaApiStatus
+
+
+class StateMessage(TypedDict):
+    """State broadcast message."""
+    type: Literal["state"]
+    data: StateData
+
+
+class TrainerStatusData(TypedDict):
+    """Trainer connection status data."""
+    connected: bool
+    trainer_id: Optional[str]
+
+
+class TrainerStatusMessage(TypedDict):
+    """Trainer status broadcast message (to UI clients)."""
+    type: Literal["trainer_status"]
+    data: TrainerStatusData
+
+
+class TrainerNotificationBroadcast(TypedDict):
+    """Trainer notification broadcast (to UI clients)."""
+    type: Literal["trainer_notification"]
+    data: TrainerNotificationData
+
+
+# ============================================================================
+# HTTP Response Types
+# ============================================================================
+
+class CameraFeed(TypedDict):
+    """Camera feed information."""
+    name: str
+    label: str
+    url: str
+
+
+class EnvResponse(TypedDict, total=False):
+    """GET /env response."""
+    robot: RobotType
+    scene: str
+    has_gripper: bool
+    control_mode: ControlMode
+    action_space: dict  # Gym space serialization
+    observation_space: dict  # Gym space serialization
+    camera_feeds: List[CameraFeed]
+    home_pose: List[float]  # Joint angles for home position
+
+
+class CommandInfo(TypedDict):
+    """Command metadata."""
+    name: str
+    description: str
+
+
+class MetadataResponse(TypedDict):
+    """GET /metadata response."""
+    robots: List[str]
+    commands: List[CommandInfo]
+
+
+# ============================================================================
+# Union Types for All Messages
+# ============================================================================
+
+ClientMessage = Union[
+    ActionMessage,
+    TeleopActionMessage,
+    ResetMessage,
+    SwitchRobotMessage,
+    HomeMessage,
+    StopHomeMessage,
+    CameraMessage,
+    CameraFollowMessage,
+    ArmTargetMessage,
+    ArmOrientationMessage,
+    UseOrientationMessage,
+    JointPositionsMessage,
+    ControlModeMessage,
+    GripperMessage,
+    SetNovaModeMessage,
+    TrainerIdentityMessage,
+    TrainerNotificationMessage,
+    EpisodeControlMessage,
+]
+
+ServerMessage = Union[
+    StateMessage,
+    TrainerStatusMessage,
+    TrainerNotificationBroadcast,
+]

tests/test_api.py

@@ -38,7 +38,7 @@ class TestHTTPEndpoints:
 
         data = response.json()
         assert "robots" in data
-        assert "commands" in data
+        assert "actions" in data
         # camera_feeds, current_selection moved to /env endpoint
         assert "camera_feeds" not in data
         assert "overlay_camera_presets" not in data
@@ -225,8 +225,8 @@ class TestWebSocketMessages:
             tolerance = 0.01
             assert final_distance < tolerance, f"Robot did not reach home position: distance={final_distance:.4f}, tolerance={tolerance}"
 
-    def test_teleop_command_message(self, check_server):
-        """Test teleop_command WebSocket message with new vx/vy/vz format."""
+    def test_teleop_action_message(self, check_server):
+        """Test teleop_action WebSocket message with vx/vy/vz format."""
         with connect(WS_URL, timeout=10) as ws:
             # Ensure we're on UR5
             ws.send(json.dumps({
@@ -235,28 +235,28 @@ class TestWebSocketMessages:
             }))
             time.sleep(2)
 
-            # Send teleop command with new format
+            # Send teleop action with new format
             ws.send(json.dumps({
-                "type": "teleop_command",
+                "type": "teleop_action",
                 "data": {"vx": 0.01, "vy": 0.0, "vz": 0.0}
             }))
 
-            # Receive state and check teleop_command
+            # Receive state and check teleop_action
             msg = ws.recv(timeout=2)
             data = json.loads(msg)
             assert data.get("type") == "state"
             state_data = data.get("data", {})
-            teleop = state_data.get("teleop_command")
+            teleop = state_data.get("teleop_action")
 
-            # Should always have teleop_command (never null)
+            # Should always have teleop_action (never null)
             assert teleop is not None
             assert isinstance(teleop, dict)
             assert "vx" in teleop
             assert "vy" in teleop
             assert "vz" in teleop
 
-    def test_command_message(self, check_server):
-        """Test command WebSocket message for locomotion robots."""
+    def test_action_message(self, check_server):
+        """Test action WebSocket message for locomotion robots."""
         with connect(WS_URL, timeout=10) as ws:
             # Switch to Spot
             ws.send(json.dumps({
@@ -265,9 +265,9 @@ class TestWebSocketMessages:
             }))
             time.sleep(2)
 
-            # Send command
+            # Send action
             ws.send(json.dumps({
-                "type": "command",
+                "type": "action",
                 "data": {"vx": 0.5, "vy": 0.0, "vyaw": 0.0}
             }))
 
@@ -276,9 +276,9 @@ class TestWebSocketMessages:
             data = json.loads(msg)
             assert data.get("type") == "state"
             state_data = data.get("data", {})
-            teleop = state_data.get("teleop_command")
+            teleop = state_data.get("teleop_action")
 
-            # Should have teleop_command with values
+            # Should have teleop_action with values
             assert teleop is not None
             assert isinstance(teleop, dict)
 
@@ -309,8 +309,8 @@ class TestStateStructure:
             assert "scene" not in state_data, "scene should not be in state stream"
             assert "has_gripper" not in state_data, "has_gripper should not be in state stream"
 
-    def test_state_always_has_teleop_command(self, check_server):
-        """Verify state always contains teleop_command (never null)."""
+    def test_state_always_has_teleop_action(self, check_server):
+        """Verify state always contains teleop_action (never null)."""
         with connect(WS_URL, timeout=10) as ws:
             # Get multiple state messages
             for _ in range(3):
@@ -318,11 +318,11 @@ class TestStateStructure:
                 data = json.loads(msg)
                 if data.get("type") == "state":
                     state_data = data.get("data", {})
-                    teleop = state_data.get("teleop_command")
+                    teleop = state_data.get("teleop_action")
 
                     # Should never be None
-                    assert teleop is not None, "teleop_command should never be null"
-                    assert isinstance(teleop, dict), "teleop_command should be a dict"
+                    assert teleop is not None, "teleop_action should never be null"
+                    assert isinstance(teleop, dict), "teleop_action should be a dict"
 
                     # Should have expected keys
                     assert "vx" in teleop
@@ -358,7 +358,7 @@ class TestStateStructure:
             assert "control_mode" in state_data
             assert "steps" in state_data
             assert "reward" in state_data
-            assert "teleop_command" in state_data
+            assert "teleop_action" in state_data
             assert "trainer_connected" in state_data
             # Verify old fields are not in state root
             assert "target" not in state_data
@@ -388,7 +388,7 @@ class TestStateStructure:
             assert "orientation" in obs
             assert "steps" in state_data
             assert "reward" in state_data
-            assert "teleop_command" in state_data
+            assert "teleop_action" in state_data
             assert "trainer_connected" in state_data
             # Verify old fields are removed
             assert "base_height" not in state_data
@@ -411,15 +411,57 @@ class TestBackwardCompatibility:
             }))
             time.sleep(2)
 
-            # Send teleop command with old format (dx/dy/dz)
+            # Send teleop action with old format (dx/dy/dz)
             ws.send(json.dumps({
-                "type": "teleop_command",
+                "type": "teleop_action",
                 "data": {"dx": 0.01, "dy": 0.0, "dz": 0.0}
             }))
 
             # Should still work - receive state
             msg = ws.recv(timeout=2)
-            data = json.loads(msg)  # Fixed: was json.dumps, should be json.loads
+            data = json.loads(msg)
+            assert data.get("type") == "state"
+
+    def test_old_command_type_accepted(self, check_server):
+        """Test that old 'command' message type is still accepted."""
+        with connect(WS_URL, timeout=10) as ws:
+            # Switch to Spot
+            ws.send(json.dumps({
+                "type": "switch_robot",
+                "data": {"robot": "spot"}
+            }))
+            time.sleep(2)
+
+            # Send command with old message type
+            ws.send(json.dumps({
+                "type": "command",
+                "data": {"vx": 0.5, "vy": 0.0, "vyaw": 0.0}
+            }))
+
+            # Should still work - receive state
+            msg = ws.recv(timeout=2)
+            data = json.loads(msg)
+            assert data.get("type") == "state"
+
+    def test_old_teleop_command_type_accepted(self, check_server):
+        """Test that old 'teleop_command' message type is still accepted."""
+        with connect(WS_URL, timeout=10) as ws:
+            # Switch to UR5
+            ws.send(json.dumps({
+                "type": "switch_robot",
+                "data": {"robot": "ur5"}
+            }))
+            time.sleep(2)
+
+            # Send teleop with old message type
+            ws.send(json.dumps({
+                "type": "teleop_command",
+                "data": {"vx": 0.01, "vy": 0.0, "vz": 0.0}
+            }))
+
+            # Should still work - receive state
+            msg = ws.recv(timeout=2)
+            data = json.loads(msg)
             assert data.get("type") == "state"
 
 
@@ -452,7 +494,7 @@ class TestRobotMovement:
             # Send multiple teleop commands to move in +X direction
             for _ in range(5):
                 ws.send(json.dumps({
-                    "type": "teleop_command",
+                    "type": "teleop_action",
                     "data": {"vx": 0.02, "vy": 0.0, "vz": 0.0}
                 }))
                 time.sleep(0.1)
@@ -471,8 +513,8 @@ class TestRobotMovement:
             target_x = target.get("x", 0)
             assert target_x > initial_x, f"Expected target X to increase from {initial_x}, got {target_x}"
 
-    def test_spot_command_moves_robot(self, check_server):
-        """Test Spot responds to command messages and position changes."""
+    def test_spot_action_moves_robot(self, check_server):
+        """Test Spot responds to action messages and position changes."""
         with connect(WS_URL, timeout=10) as ws:
             # Switch to Spot
             ws.send(json.dumps({
@@ -494,10 +536,10 @@ class TestRobotMovement:
             initial_pos = obs.get("position", {})
             initial_x = initial_pos.get("x", 0)
 
-            # Send multiple commands to move forward
+            # Send multiple actions to move forward
             for _ in range(10):
                 ws.send(json.dumps({
-                    "type": "command",
+                    "type": "action",
                     "data": {"vx": 0.5, "vy": 0.0, "vyaw": 0.0}
                 }))
                 time.sleep(0.1)
@@ -514,8 +556,8 @@ class TestRobotMovement:
             # Locomotion robots may move slowly or fall, just verify some movement occurred
             assert abs(new_x - initial_x) > 0.001, f"Expected some X movement from {initial_x}, got {new_x}"
 
-    def test_g1_command_moves_robot(self, check_server):
-        """Test G1 responds to command messages and position changes."""
+    def test_g1_action_moves_robot(self, check_server):
+        """Test G1 responds to action messages and position changes."""
         with connect(WS_URL, timeout=10) as ws:
             # Switch to G1
             ws.send(json.dumps({
@@ -537,10 +579,10 @@ class TestRobotMovement:
             initial_pos = obs.get("position", {})
             initial_x = initial_pos.get("x", 0)
 
-            # Send multiple commands to move forward
+            # Send multiple actions to move forward
             for _ in range(10):
                 ws.send(json.dumps({
-                    "type": "command",
+                    "type": "action",
                     "data": {"vx": 0.5, "vy": 0.0, "vyaw": 0.0}
                 }))
                 time.sleep(0.1)
@@ -594,8 +636,65 @@ class TestRobotMovement:
             data = json.loads(msg)
             assert data.get("type") == "state"
 
+    def test_ur5_action_with_joint_velocity(self, check_server):
+        """Test UR5 responds to action messages with joint velocities."""
+        with connect(WS_URL, timeout=10) as ws:
+            # Switch to UR5
+            ws.send(json.dumps({
+                "type": "switch_robot",
+                "data": {"robot": "ur5"}
+            }))
+            time.sleep(2)
+
+            # Reset to clear any previous state
+            ws.send(json.dumps({"type": "reset"}))
+            time.sleep(0.5)
+
+            # Send action with joint 3 velocity set
+            ws.send(json.dumps({
+                "type": "action",
+                "data": {
+                    "vx": 0.0, "vy": 0.0, "vz": 0.0,
+                    "vrx": 0.0, "vry": 0.0, "vrz": 0.0,
+                    "j1": 0.0, "j2": 0.0, "j3": 0.5, "j4": 0.0, "j5": 0.0, "j6": 0.0,
+                    "gripper": 0.0
+                }
+            }))
+
+            # Get state and verify teleop_action has joint velocity
+            time.sleep(0.3)
+            for _ in range(3):
+                msg = ws.recv(timeout=2)
+            data = json.loads(msg)
+            state_data = data.get("data", {})
+            teleop = state_data.get("teleop_action", {})
+
+            # Joint 3 should have velocity 0.5
+            assert abs(teleop.get("j3", 0) - 0.5) < 0.01, f"Expected j3=0.5, got {teleop.get('j3')}"
+
+            # Stop by sending zero velocities
+            ws.send(json.dumps({
+                "type": "action",
+                "data": {
+                    "vx": 0.0, "vy": 0.0, "vz": 0.0,
+                    "vrx": 0.0, "vry": 0.0, "vrz": 0.0,
+                    "j1": 0.0, "j2": 0.0, "j3": 0.0, "j4": 0.0, "j5": 0.0, "j6": 0.0,
+                    "gripper": 0.0
+                }
+            }))
+
+            time.sleep(0.3)
+            for _ in range(3):
+                msg = ws.recv(timeout=2)
+            data = json.loads(msg)
+            state_data = data.get("data", {})
+            teleop = state_data.get("teleop_action", {})
+
+            # All velocities should be zero
+            assert abs(teleop.get("j3", 1)) < 0.01
+
     def test_ur5_joint_jog_teleop(self, check_server):
-        """Test joint jogging updates teleop_command with joint velocities."""
+        """Test joint jogging with start_jog updates teleop_action with joint velocities."""
         with connect(WS_URL, timeout=10) as ws:
             # Switch to UR5
             ws.send(json.dumps({
@@ -618,7 +717,7 @@ class TestRobotMovement:
             }))
             time.sleep(0.2)
 
-            # Start joint jog on joint 3 in positive direction
+            # Start joint jog on joint 3 in positive direction (legacy start_jog still supported)
             ws.send(json.dumps({
                 "type": "start_jog",
                 "data": {
@@ -631,22 +730,22 @@ class TestRobotMovement:
                 }
             }))
 
-            # Get state and verify teleop_command has joint velocities
+            # Get state and verify teleop_action has joint velocities
             # Receive multiple messages to ensure we get a fresh state after the jog command
             time.sleep(0.3)
             for i in range(3):
                 msg = ws.recv(timeout=2)
                 data = json.loads(msg)
-                teleop = data.get("data", {}).get("teleop_command", {})
+                teleop = data.get("data", {}).get("teleop_action", {})
                 obs = data.get("data", {}).get("observation", {})
                 has_ee = 'end_effector' in obs  # UR5 has end_effector, G1/Spot don't
                 print(f"\nDEBUG message {i+1}: robot={'UR5' if has_ee else 'LOCO'}, teleop j3 = {teleop.get('j3', 'MISSING')}")
 
             state_data = data.get("data", {})
-            teleop = state_data.get("teleop_command", {})
+            teleop = state_data.get("teleop_action", {})
 
             # Debug: print actual values received
-            print(f"\nDEBUG: Final teleop_command = {teleop}")
+            print(f"\nDEBUG: Final teleop_action = {teleop}")
 
             # Joint 3 should have velocity 0.5, others should be 0
             assert teleop.get("j3", 0) == 0.5, f"Expected j3=0.5, got {teleop.get('j3')}, full teleop={teleop}"
@@ -665,13 +764,82 @@ class TestRobotMovement:
                 msg = ws.recv(timeout=2)
             data = json.loads(msg)
             state_data = data.get("data", {})
-            teleop = state_data.get("teleop_command", {})
+            teleop = state_data.get("teleop_action", {})
 
             assert teleop.get("j3", 1) == 0.0
             assert teleop.get("vx", 1) == 0.0
 
+    def test_ur5_action_with_cartesian_velocity(self, check_server):
+        """Test UR5 responds to action messages with Cartesian velocities."""
+        with connect(WS_URL, timeout=10) as ws:
+            # Switch to UR5
+            ws.send(json.dumps({
+                "type": "switch_robot",
+                "data": {"robot": "ur5"}
+            }))
+            time.sleep(2)
+
+            # Reset to clear any previous state
+            ws.send(json.dumps({"type": "reset"}))
+            time.sleep(0.5)
+
+            # Send action with vx velocity set (50 mm/s = 0.05 m/s)
+            ws.send(json.dumps({
+                "type": "action",
+                "data": {
+                    "vx": 0.05, "vy": 0.0, "vz": 0.0,
+                    "vrx": 0.0, "vry": 0.0, "vrz": 0.0,
+                    "j1": 0.0, "j2": 0.0, "j3": 0.0, "j4": 0.0, "j5": 0.0, "j6": 0.0,
+                    "gripper": 0.0
+                }
+            }))
+
+            # Get state and verify teleop_action has vx velocity
+            time.sleep(0.3)
+            for _ in range(3):
+                msg = ws.recv(timeout=2)
+            data = json.loads(msg)
+            state_data = data.get("data", {})
+            teleop = state_data.get("teleop_action", {})
+
+            # vx should be 0.05 m/s
+            assert abs(teleop.get("vx", 0) - 0.05) < 0.01, f"Expected vx=0.05, got {teleop.get('vx')}"
+
+            # Send action with rotation velocity
+            ws.send(json.dumps({
+                "type": "action",
+                "data": {
+                    "vx": 0.0, "vy": 0.0, "vz": 0.0,
+                    "vrx": 0.0, "vry": 0.0, "vrz": -0.3,
+                    "j1": 0.0, "j2": 0.0, "j3": 0.0, "j4": 0.0, "j5": 0.0, "j6": 0.0,
+                    "gripper": 0.0
+                }
+            }))
+
+            # Get state and verify teleop_action has vrz velocity
+            time.sleep(0.3)
+            for _ in range(3):
+                msg = ws.recv(timeout=2)
+            data = json.loads(msg)
+            state_data = data.get("data", {})
+            teleop = state_data.get("teleop_action", {})
+
+            # vrz should be -0.3 rad/s
+            assert abs(teleop.get("vrz", 0) + 0.3) < 0.01, f"Expected vrz=-0.3, got {teleop.get('vrz')}"
+
+            # Stop by sending zero velocities
+            ws.send(json.dumps({
+                "type": "action",
+                "data": {
+                    "vx": 0.0, "vy": 0.0, "vz": 0.0,
+                    "vrx": 0.0, "vry": 0.0, "vrz": 0.0,
+                    "j1": 0.0, "j2": 0.0, "j3": 0.0, "j4": 0.0, "j5": 0.0, "j6": 0.0,
+                    "gripper": 0.0
+                }
+            }))
+
     def test_ur5_cartesian_jog_teleop(self, check_server):
-        """Test Cartesian jogging updates teleop_command with Cartesian velocities."""
+        """Test Cartesian jogging with start_jog updates teleop_action with Cartesian velocities."""
         with connect(WS_URL, timeout=10) as ws:
             # Switch to UR5
             ws.send(json.dumps({
@@ -709,14 +877,14 @@ class TestRobotMovement:
                 }
             }))
 
-            # Get state and verify teleop_command has vx velocity
+            # Get state and verify teleop_action has vx velocity
             # Receive multiple messages to ensure we get a fresh state after the jog command
             time.sleep(0.3)
             for _ in range(3):
                 msg = ws.recv(timeout=2)
             data = json.loads(msg)
             state_data = data.get("data", {})
-            teleop = state_data.get("teleop_command", {})
+            teleop = state_data.get("teleop_action", {})
 
             # vx should have velocity 0.05 m/s (50 mm/s), others should be 0
             assert abs(teleop.get("vx", 0) - 0.05) < 0.01, f"Expected vx near 0.05, got {teleop.get('vx')}"
@@ -747,14 +915,14 @@ class TestRobotMovement:
                 }
             }))
 
-            # Get state and verify teleop_command has vrz velocity
+            # Get state and verify teleop_action has vrz velocity
             # Receive multiple messages to ensure we get a fresh state after the jog command
             time.sleep(0.3)
             for _ in range(3):
                 msg = ws.recv(timeout=2)
             data = json.loads(msg)
             state_data = data.get("data", {})
-            teleop = state_data.get("teleop_command", {})
+            teleop = state_data.get("teleop_action", {})
 
             # vrz should have velocity -0.3 rad/s, others should be 0
             assert teleop.get("vrz", 0) == -0.3, f"Expected vrz=-0.3, got {teleop.get('vrz')}"