Georg commited on
Commit
223bbeb
·
1 Parent(s): 11dd56c

Refactor mujoco_server.py and protocol_types.py for external client support

Browse files

- Replaced trainer-specific WebSocket handling with a more generic external client model, allowing for better integration of various client types.
- Updated message types and payload structures to reflect the new client identity and notification system, ensuring backward compatibility with legacy trainer messages.
- Enhanced the broadcast mechanisms to notify all connected clients about their connection status, improving the overall communication flow.
- Removed deprecated trainer-related code and tests, streamlining the server's functionality and focusing on external client interactions.
- Updated README.md to reflect changes in client messaging and connection handling, providing clearer guidance for developers.

README.md CHANGED
@@ -14,9 +14,9 @@ A unified MuJoCo-based robot simulation platform with web interface for multiple
14
  6. [Docker Deployment](#docker-deployment)
15
  7. [Controls](#controls)
16
  8. [Architecture](#architecture)
17
- 9. [Spot Controller Options](#spot-controller-options)
18
- 10. [API](#api)
19
- 11. [Wandelbots Nova API Integration](#wandelbots-nova-api-integration)
20
  12. [License](#license)
21
 
22
  ## Overview
@@ -28,7 +28,7 @@ Nova Sim combines MuJoCo physics, a Flask/WebSocket server, and a browser UI so
28
  - Real-time MuJoCo physics simulation
29
  - Web-based video streaming interface with TypeScript frontend
30
  - WebSocket-based state/command communication
31
- - **One-command setup**: `make dev` starts both backend and frontend with hot reload
32
  - Gym-style WebSocket API for RL/IL clients
33
  - Interactive camera controls (rotate, zoom, pan)
34
  - Robot switching without restart
@@ -113,39 +113,7 @@ nova_sim/
113
 
114
  ## Quick Start
115
 
116
- ### Using Makefile (Recommended)
117
-
118
- The easiest way to get started is using the included Makefile:
119
-
120
- ```bash
121
- # First time setup: install all dependencies
122
- make install
123
-
124
- # Start development servers (backend + frontend with hot reload)
125
- make dev
126
- # Backend: http://localhost:5000
127
- # Frontend dev server: http://localhost:3004
128
- # Access the application at: http://localhost:3004/nova-sim
129
-
130
- # Press Ctrl+C to stop all servers
131
- ```
132
-
133
- **Production build:**
134
- ```bash
135
- make build # Build frontend assets
136
- make prod # Run production server
137
- ```
138
-
139
- **Other commands:**
140
- ```bash
141
- make help # Show all available commands
142
- make clean # Clean build artifacts
143
- make kill # Stop all running servers
144
- ```
145
-
146
- ### Manual Setup (Alternative)
147
-
148
- If you prefer not to use the Makefile:
149
 
150
  ```bash
151
  # Create and activate a virtualenv
@@ -153,7 +121,7 @@ python3 -m venv .venv
153
  source .venv/bin/activate
154
 
155
  # Install Python dependencies
156
- pip install mujoco gymnasium flask flask-sock opencv-python torch numpy
157
 
158
  # Install frontend dependencies
159
  cd frontend && npm install && cd ..
@@ -171,32 +139,35 @@ python mujoco_server.py
171
 
172
  ## Docker Deployment
173
 
174
- ### Quick Start with Makefile
175
 
176
  ```bash
177
- # Build and run in Docker (CPU mode)
178
- make docker
179
 
180
  # Or with GPU acceleration (requires NVIDIA GPU)
181
- make docker-gpu
182
-
183
- # View logs
184
- make docker-logs
185
 
186
- # Stop containers
187
- make docker-down
188
  ```
189
 
190
- ### Manual Docker Commands
191
 
192
  ```bash
193
- # Build and run (CPU/software rendering)
194
  docker-compose up --build
195
 
196
- # Or with docker run
197
- docker run -p 3004:5000 nova-sim
198
 
199
- # Access at: http://localhost:3004/nova-sim
 
 
 
 
 
 
 
200
  ```
201
 
202
  ### Configuration & Tuning
@@ -415,23 +386,6 @@ docker run --gpus all -p 3004:5000 \
415
 
416
  All controllers output joint position targets; MuJoCo's built-in PD control tracks these targets.
417
 
418
- ## Spot Controller Options
419
-
420
- The Spot robot supports multiple controller types via the `controller_type` parameter:
421
-
422
- ```python
423
- from spot_env import SpotEnv
424
-
425
- # MPC Gait - Feedback-based balance control (default, no extra deps)
426
- env = SpotEnv(controller_type='mpc_gait')
427
-
428
- # PyMPC Gait - Uses standalone gait generator (no external dependencies needed)
429
- env = SpotEnv(controller_type='pympc')
430
-
431
- # Trot Gait - Simple open-loop trot pattern
432
- env = SpotEnv(controller_type='trot')
433
- ```
434
-
435
  ## API
436
 
437
  All endpoints are prefixed with `/nova-sim/api/v1`.
@@ -527,6 +481,11 @@ For robot arms (UR5):
527
  {"type": "home"}
528
  ```
529
 
 
 
 
 
 
530
  **`teleop_action`** - Send teleoperation action (primarily for UR5 keyboard control):
531
  ```json
532
  {"type": "teleop_action", "data": {"vx": 0.01, "vy": 0.0, "vz": 0.0}}
@@ -594,6 +553,37 @@ For robot arms (UR5):
594
  ```
595
  - `action`: `"open"` or `"close"`
596
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
597
  #### Server → Client Messages
598
 
599
  **`state`** (broadcast at ~10 Hz):
@@ -647,15 +637,15 @@ For robot arm (UR5):
647
  *Common fields (all robots):*
648
  - `observation`: Contains robot-specific sensor data and state information
649
  - `steps`: Number of simulation steps since last reset
650
- - `reward`: The integrated task reward from the simulator that remote trainers can consume
651
- - `teleop_action`: The canonical action/velocity stream that drives locomotion or arm movement; the UI and every trainer should read this field as the unified action record. Always present with zero values when idle
652
  - Cartesian velocities: `vx` (forward/back or X-axis), `vy` (strafe or Y-axis), `vz` (vertical for UR5), `vyaw` (rotation for locomotion)
653
  - Cartesian rotation velocities (UR5 only): `vrx`, `vry`, `vrz` (rad/s)
654
  - Joint velocities (UR5 only): `j1`, `j2`, `j3`, `j4`, `j5`, `j6` (rad/s)
655
  - Gripper: `gripper` (0-255 for UR5, 0 for others)
656
  - Locomotion robots: Use `vx`, `vy`, `vyaw` (other fields are 0)
657
  - UR5: Use `vx`/`vy`/`vz` for Cartesian translation, `vrx`/`vry`/`vrz` for rotation, `j1`-`j6` for joint velocities, and `gripper` for gripper control
658
- - `trainer_connected`: Whether a trainer handshake is active on `/ws` (useful for status LEDs)
659
 
660
  *Locomotion observation fields (inside `observation`):*
661
  - `position`: Robot base position in world coordinates (x, y, z) in meters
@@ -679,15 +669,39 @@ For robot arm (UR5):
679
 
680
  **Note:** Static environment information (robot name, scene name, has_gripper, action/observation spaces, camera feeds) has been moved to the `GET /env` endpoint and is no longer included in the state stream.
681
 
682
- ### State broadcasts and trainer notifications
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
683
 
684
  Every `/ws` client receives a `state` message roughly every 100 ms. The examples above show the locomotion (`spot`) and arm (`ur5`) payloads; the payload also now includes:
685
 
686
- - `teleop_action`: The latest action/teleoperation stream (includes `vx`, `vy`, `vz`, `vyaw`, `vrx`, `vry`, `vrz`, `j1`-`j6`, `gripper`) so trainers and the UI read a single canonical action payload. Always present with zero values when idle.
687
- - `reward`: The integrated task reward that trainers can consume without sending a separate `step`.
688
- - `trainer_connected`: Whether a trainer handshake is active on `/ws` (used to update the UI indicator).
689
 
690
- Trainers announce themselves by sending a `trainer_identity` payload when the socket opens. The server mirrors that information into the `trainer_status` broadcasts (`trainer_status` messages flow to every UI client) and lets trainers emit `notification` payloads that the UI receives as `trainer_notification` events.
691
 
692
  ### HTTP Endpoints
693
 
@@ -946,68 +960,6 @@ The Nova API integration is implemented in:
946
 
947
  The tests assume the server is accessible via `http://localhost:3004/nova-sim/api/v1` and will skip automatically if the API is unreachable.
948
 
949
- ## Frontend Development
950
-
951
- The Nova Sim web UI is built with TypeScript, Vite, and vanilla JavaScript (no framework dependencies).
952
-
953
- ### Development Setup
954
-
955
- 1. **Install Node.js dependencies:**
956
- ```bash
957
- cd frontend
958
- npm install
959
- ```
960
-
961
- 2. **Start development server with hot reload:**
962
- ```bash
963
- # Terminal 1: Start Flask backend
964
- python mujoco_server.py
965
-
966
- # Terminal 2: Start Vite dev server
967
- cd frontend
968
- npm run dev
969
- ```
970
-
971
- 3. **Access the development UI:**
972
- - Open [http://localhost:3004/nova-sim](http://localhost:3004/nova-sim)
973
- - All API/WebSocket calls are automatically proxied to Flask at `:5000`
974
- - Changes to TypeScript/CSS auto-reload in the browser
975
-
976
- ### Production Build
977
-
978
- ```bash
979
- cd frontend
980
- npm run build
981
- ```
982
-
983
- This compiles TypeScript and bundles all assets to `frontend/dist/`. The Flask server automatically serves these built files from the `/static` route.
984
-
985
- ### Frontend Structure
986
-
987
- - **`src/main.ts`**: Main application entry point with WebSocket handling and UI logic
988
- - **`src/api/client.ts`**: Type-safe WebSocket client for server communication
989
- - **`src/types/protocol.ts`**: TypeScript definitions mirroring Python protocol types
990
- - **`src/styles.css`**: All CSS styles (Wandelbots design system)
991
- - **`src/index.html`**: HTML template with viewport structure
992
-
993
- ### Type Safety
994
-
995
- The frontend uses TypeScript for compile-time type checking:
996
-
997
- ```bash
998
- # Run type checker
999
- cd frontend
1000
- npm run type-check
1001
- ```
1002
-
1003
- All WebSocket messages are strongly typed to match the Python protocol definitions in `protocol_types.py`.
1004
-
1005
- ### Modifying the UI
1006
-
1007
- - **Styles**: Edit `frontend/src/styles.css`
1008
- - **Behavior**: Edit `frontend/src/main.ts`
1009
- - **Protocol**: Update both `protocol_types.py` (Python) and `frontend/src/types/protocol.ts` (TypeScript)
1010
-
1011
  ## License
1012
 
1013
  This project uses models from:
 
14
  6. [Docker Deployment](#docker-deployment)
15
  7. [Controls](#controls)
16
  8. [Architecture](#architecture)
17
+ 9. [API](#api)
18
+ 10. [Wandelbots Nova API Integration](#wandelbots-nova-api-integration)
19
+ 11. [Testing](#testing)
20
  12. [License](#license)
21
 
22
  ## Overview
 
28
  - Real-time MuJoCo physics simulation
29
  - Web-based video streaming interface with TypeScript frontend
30
  - WebSocket-based state/command communication
31
+ - Simple venv-based setup with Python and npm
32
  - Gym-style WebSocket API for RL/IL clients
33
  - Interactive camera controls (rotate, zoom, pan)
34
  - Robot switching without restart
 
113
 
114
  ## Quick Start
115
 
116
+ ### Setup
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
117
 
118
  ```bash
119
  # Create and activate a virtualenv
 
121
  source .venv/bin/activate
122
 
123
  # Install Python dependencies
124
+ pip install -r requirements.txt
125
 
126
  # Install frontend dependencies
127
  cd frontend && npm install && cd ..
 
139
 
140
  ## Docker Deployment
141
 
142
+ ### Quick Start
143
 
144
  ```bash
145
+ # Build and run (CPU/software rendering)
146
+ docker-compose up --build
147
 
148
  # Or with GPU acceleration (requires NVIDIA GPU)
149
+ docker-compose -f docker-compose.gpu.yml up --build
 
 
 
150
 
151
+ # Access at: http://localhost:3004/nova-sim
 
152
  ```
153
 
154
+ ### Docker Commands
155
 
156
  ```bash
157
+ # Build and run (CPU mode)
158
  docker-compose up --build
159
 
160
+ # Run with GPU support
161
+ docker-compose -f docker-compose.gpu.yml up --build
162
 
163
+ # View logs
164
+ docker-compose logs -f
165
+
166
+ # Stop containers
167
+ docker-compose down
168
+
169
+ # Or with docker run directly
170
+ docker run -p 3004:5000 nova-sim
171
  ```
172
 
173
  ### Configuration & Tuning
 
386
 
387
  All controllers output joint position targets; MuJoCo's built-in PD control tracks these targets.
388
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
389
  ## API
390
 
391
  All endpoints are prefixed with `/nova-sim/api/v1`.
 
481
  {"type": "home"}
482
  ```
483
 
484
+ **`stop_home`** - Stop homing sequence (UR5 only):
485
+ ```json
486
+ {"type": "stop_home"}
487
+ ```
488
+
489
  **`teleop_action`** - Send teleoperation action (primarily for UR5 keyboard control):
490
  ```json
491
  {"type": "teleop_action", "data": {"vx": 0.01, "vy": 0.0, "vz": 0.0}}
 
553
  ```
554
  - `action`: `"open"` or `"close"`
555
 
556
+ **`set_nova_mode`** (Configure Nova API integration):
557
+ ```json
558
+ {"type": "set_nova_mode", "data": {"state_streaming": true, "ik": false}}
559
+ ```
560
+ - `enabled`: (Optional, legacy) Enable/disable all Nova features
561
+ - `state_streaming`: (Optional) Enable/disable Nova state streaming
562
+ - `ik`: (Optional) Enable/disable Nova IK computation
563
+
564
+ #### Client Identity & Notification Messages
565
+
566
+ **`client_identity`** (Client handshake):
567
+ ```json
568
+ {"type": "client_identity", "data": {"client_id": "my_client_v1"}}
569
+ ```
570
+ - `client_id`: Unique identifier for the external client
571
+ - **Note:** Old `trainer_identity` type is still supported for backward compatibility
572
+
573
+ **`client_notification`** (Send client notification to all connected clients):
574
+ ```json
575
+ {"type": "client_notification", "data": {"message": "Starting epoch 5", "level": "info"}}
576
+ ```
577
+ - `message`: Notification text
578
+ - `level`: `"info"`, `"warning"`, or `"error"`
579
+ - **Note:** Old `notification` type is still supported for backward compatibility
580
+
581
+ **`episode_control`** (Control episode state):
582
+ ```json
583
+ {"type": "episode_control", "data": {"action": "terminate"}}
584
+ ```
585
+ - `action`: `"terminate"` (episode ended successfully) or `"truncate"` (episode cut short)
586
+
587
  #### Server → Client Messages
588
 
589
  **`state`** (broadcast at ~10 Hz):
 
637
  *Common fields (all robots):*
638
  - `observation`: Contains robot-specific sensor data and state information
639
  - `steps`: Number of simulation steps since last reset
640
+ - `reward`: The integrated task reward from the simulator that external clients can consume
641
+ - `teleop_action`: The canonical action/velocity stream that drives locomotion or arm movement; the UI and every external client should read this field as the unified action record. Always present with zero values when idle
642
  - Cartesian velocities: `vx` (forward/back or X-axis), `vy` (strafe or Y-axis), `vz` (vertical for UR5), `vyaw` (rotation for locomotion)
643
  - Cartesian rotation velocities (UR5 only): `vrx`, `vry`, `vrz` (rad/s)
644
  - Joint velocities (UR5 only): `j1`, `j2`, `j3`, `j4`, `j5`, `j6` (rad/s)
645
  - Gripper: `gripper` (0-255 for UR5, 0 for others)
646
  - Locomotion robots: Use `vx`, `vy`, `vyaw` (other fields are 0)
647
  - UR5: Use `vx`/`vy`/`vz` for Cartesian translation, `vrx`/`vry`/`vrz` for rotation, `j1`-`j6` for joint velocities, and `gripper` for gripper control
648
+ - `connected_clients`: List of connected external client IDs (e.g., `["trainer_v1", "monitor"]`)
649
 
650
  *Locomotion observation fields (inside `observation`):*
651
  - `position`: Robot base position in world coordinates (x, y, z) in meters
 
669
 
670
  **Note:** Static environment information (robot name, scene name, has_gripper, action/observation spaces, camera feeds) has been moved to the `GET /env` endpoint and is no longer included in the state stream.
671
 
672
+ **`connected_clients`** (broadcast to all clients):
673
+ ```json
674
+ {
675
+ "type": "connected_clients",
676
+ "data": {
677
+ "clients": ["trainer_v1", "monitor"]
678
+ }
679
+ }
680
+ ```
681
+ - `clients`: Array of connected external client IDs
682
+
683
+ **`client_notification`** (broadcast to all clients):
684
+ ```json
685
+ {
686
+ "type": "client_notification",
687
+ "data": {
688
+ "message": "Starting epoch 5",
689
+ "level": "info"
690
+ }
691
+ }
692
+ ```
693
+ - `message`: Notification text from external client
694
+ - `level`: `"info"`, `"warning"`, or `"error"`
695
+
696
+ ### State broadcasts and client notifications
697
 
698
  Every `/ws` client receives a `state` message roughly every 100 ms. The examples above show the locomotion (`spot`) and arm (`ur5`) payloads; the payload also now includes:
699
 
700
+ - `teleop_action`: The latest action/teleoperation stream (includes `vx`, `vy`, `vz`, `vyaw`, `vrx`, `vry`, `vrz`, `j1`-`j6`, `gripper`) so external clients and the UI read a single canonical action payload. Always present with zero values when idle.
701
+ - `reward`: The integrated task reward that external clients can consume without sending a separate `step`.
702
+ - `connected_clients`: Array of connected external client IDs (used to display which clients are connected).
703
 
704
+ External clients (trainers, monitors, etc.) announce themselves by sending a `client_identity` payload when the socket opens. The server mirrors that information into the `connected_clients` broadcasts (`connected_clients` messages flow to all clients) and lets external clients emit `client_notification` payloads that all clients receive.
705
 
706
  ### HTTP Endpoints
707
 
 
960
 
961
  The tests assume the server is accessible via `http://localhost:3004/nova-sim/api/v1` and will skip automatically if the API is unreachable.
962
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
963
  ## License
964
 
965
  This project uses models from:
examples/ur5_nova_api_example.py DELETED
@@ -1,155 +0,0 @@
1
- #!/usr/bin/env python3
2
- """Example script demonstrating Nova API integration with UR5 environment.
3
-
4
- This script shows how to use the Wandelbots Nova API for robot state streaming
5
- and inverse kinematics with the UR5 simulation environment.
6
-
7
- Requirements:
8
- - Nova instance credentials in .env file
9
- - websockets: pip install websockets
10
- - python-dotenv: pip install python-dotenv
11
-
12
- Usage:
13
- python examples/ur5_nova_api_example.py [--mode MODE]
14
-
15
- Modes:
16
- state_only: Enable state streaming only
17
- ik_only: Enable Nova IK only
18
- both: Enable both state streaming and Nova IK (default)
19
- none: Disable Nova API (local simulation only)
20
- """
21
-
22
- import argparse
23
- import sys
24
- from pathlib import Path
25
-
26
- # Add parent directory to path
27
- sys.path.insert(0, str(Path(__file__).parent.parent.parent))
28
-
29
- try:
30
- from dotenv import load_dotenv
31
- except ImportError:
32
- print("Warning: python-dotenv not installed. Install with: pip install python-dotenv")
33
- print("Continuing without .env file support...")
34
- load_dotenv = None
35
-
36
- from nova_sim.robots.ur5.ur5_env import UR5Env
37
-
38
-
39
- def main():
40
- parser = argparse.ArgumentParser(
41
- description="UR5 Nova API Integration Example",
42
- formatter_class=argparse.RawDescriptionHelpFormatter,
43
- epilog=__doc__
44
- )
45
- parser.add_argument(
46
- "--mode",
47
- choices=["state_only", "ik_only", "both", "none"],
48
- default="both",
49
- help="Nova API integration mode"
50
- )
51
- parser.add_argument(
52
- "--steps",
53
- type=int,
54
- default=1000,
55
- help="Number of simulation steps to run"
56
- )
57
- parser.add_argument(
58
- "--no-render",
59
- action="store_true",
60
- help="Disable rendering"
61
- )
62
- args = parser.parse_args()
63
-
64
- # Load environment variables
65
- if load_dotenv is not None:
66
- env_path = Path(__file__).parent.parent / ".env"
67
- if env_path.exists():
68
- load_dotenv(env_path)
69
- print(f"Loaded environment variables from {env_path}")
70
- else:
71
- print(f"Warning: .env file not found at {env_path}")
72
- print("Copy .env.example to .env and fill in your Nova credentials")
73
-
74
- # Configure Nova API based on mode
75
- nova_config = None
76
- if args.mode != "none":
77
- use_state = args.mode in ("state_only", "both")
78
- use_ik = args.mode in ("ik_only", "both")
79
-
80
- nova_config = {
81
- "use_state_stream": use_state,
82
- "use_ik": use_ik
83
- }
84
-
85
- print(f"\nNova API Integration Mode: {args.mode}")
86
- print(f" State Streaming: {'enabled' if use_state else 'disabled'}")
87
- print(f" Nova IK: {'enabled' if use_ik else 'disabled'}")
88
- else:
89
- print("\nNova API Integration: disabled (local simulation only)")
90
-
91
- # Create environment
92
- print("\nCreating UR5 environment...")
93
- try:
94
- env = UR5Env(
95
- render_mode="human" if not args.no_render else None,
96
- scene_name="scene",
97
- nova_api_config=nova_config
98
- )
99
- except Exception as e:
100
- print(f"Error creating environment: {e}")
101
- print("\nMake sure your .env file is configured correctly with Nova credentials.")
102
- return 1
103
-
104
- print("Environment created successfully!")
105
-
106
- # Reset environment
107
- print("\nResetting environment...")
108
- obs, info = env.reset()
109
- print(f"Initial observation shape: {obs.shape}")
110
-
111
- # Get initial state
112
- joint_pos = env.get_joint_positions()
113
- ee_pos = env.get_end_effector_pos()
114
- print(f"Initial joint positions: {joint_pos}")
115
- print(f"Initial end-effector position: {ee_pos}")
116
-
117
- # Set a target position
118
- print("\nSetting target position...")
119
- target_pos = [0.4, 0.2, 0.6]
120
- env.set_target(*target_pos)
121
- print(f"Target position: {target_pos}")
122
-
123
- # Run simulation
124
- print(f"\nRunning simulation for {args.steps} steps...")
125
- print("Press Ctrl+C to stop early")
126
-
127
- try:
128
- for step in range(args.steps):
129
- # Step with controller (uses IK to reach target)
130
- obs = env.step_with_controller(dt=0.002)
131
-
132
- # Render
133
- if not args.no_render:
134
- env.render()
135
-
136
- # Print progress every 100 steps
137
- if (step + 1) % 100 == 0:
138
- ee_pos = env.get_end_effector_pos()
139
- distance = ((ee_pos - target_pos) ** 2).sum() ** 0.5
140
- print(f"Step {step + 1}/{args.steps} - Distance to target: {distance:.4f}m")
141
-
142
- except KeyboardInterrupt:
143
- print("\nSimulation interrupted by user")
144
-
145
- finally:
146
- # Clean up
147
- print("\nClosing environment...")
148
- env.close()
149
- print("Done!")
150
-
151
- return 0
152
-
153
-
154
- if __name__ == "__main__":
155
- sys.exit(main())
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
frontend/index.html ADDED
File without changes
mujoco_server.py CHANGED
@@ -139,10 +139,10 @@ homing_lock = threading.Lock()
139
  # WebSocket clients
140
  ws_clients = set()
141
  ws_clients_lock = threading.Lock()
142
- trainer_ws_clients = set()
143
- trainer_ws_clients_lock = threading.Lock()
144
- trainer_client_metadata: dict = {}
145
- trainer_client_metadata_lock = threading.Lock()
146
  # Camera state for orbit controls
147
  cam = mujoco.MjvCamera()
148
  cam.azimuth = 135
@@ -580,8 +580,8 @@ def broadcast_state():
580
  if hasattr(env, "get_task_reward"):
581
  reward_value = env.get_task_reward()
582
 
583
- with trainer_ws_clients_lock:
584
- trainer_connected = len(trainer_ws_clients) > 0
585
 
586
  # UR5 has different state structure
587
  if current_robot in ("ur5", "ur5_t_push"):
@@ -636,7 +636,7 @@ def broadcast_state():
636
  'available': nova_available,
637
  'enabled': nova_enabled_pref
638
  },
639
- 'trainer_connected': trainer_connected
640
  }
641
  })
642
  else:
@@ -656,7 +656,7 @@ def broadcast_state():
656
  'steps': int(steps),
657
  'reward': reward_value,
658
  'teleop_action': teleop_snapshot,
659
- 'trainer_connected': trainer_connected
660
  }
661
  })
662
 
@@ -681,64 +681,58 @@ def broadcast_state():
681
  ws_clients.difference_update(dead_clients)
682
 
683
 
684
- def _handle_trainer_message(ws, data):
685
- """Process message payloads originating from trainers."""
686
  msg_type = data.get("type")
687
- if msg_type == "trainer_identity":
688
  payload = data.get("data", {}) or {}
689
- identity = payload.get("trainer_id") or payload.get("trainer_name") or payload.get("name") or "trainer"
690
- with trainer_ws_clients_lock:
691
- trainer_ws_clients.add(ws)
692
- _register_trainer_client(ws)
693
- _set_trainer_identity(ws, identity)
694
  broadcast_state()
695
- broadcast_trainer_connection_status()
696
  return
697
- if msg_type == "notification":
698
  payload = data.get("data", {})
699
  payload.setdefault("timestamp", time.time())
700
- broadcast_notification_to_ui(payload)
701
 
702
 
703
- def _build_trainer_status_payload():
704
- """Build a summary payload describing connected trainer clients."""
705
- with trainer_ws_clients_lock:
706
- connection_count = len(trainer_ws_clients)
707
- connected = connection_count > 0
708
- with trainer_client_metadata_lock:
709
- identities = [info.get("identity") for info in trainer_client_metadata.values() if info.get("identity")]
710
  payload = {
711
- "connected": connected,
712
- "trainer_count": connection_count,
713
  }
714
- if identities:
715
- payload["identities"] = identities
716
  return payload
717
 
718
 
719
- def _register_trainer_client(ws):
720
- with trainer_client_metadata_lock:
721
- trainer_client_metadata[ws] = {"identity": None, "connected_at": time.time()}
722
 
723
 
724
- def _unregister_trainer_client(ws):
725
- with trainer_client_metadata_lock:
726
- trainer_client_metadata.pop(ws, None)
727
 
728
 
729
- def _set_trainer_identity(ws, identity: str):
730
- with trainer_client_metadata_lock:
731
- info = trainer_client_metadata.get(ws)
732
  if not info:
733
  return
734
  info["identity"] = identity
735
  info["last_seen"] = time.time()
736
 
737
 
738
- def broadcast_trainer_connection_status():
739
- """Notify UI clients about trainer connection state."""
740
- payload = _build_trainer_status_payload()
741
- message = json.dumps({"type": "trainer_status", "data": payload})
742
  with ws_clients_lock:
743
  dead_clients = set()
744
  for client in ws_clients:
@@ -748,22 +742,22 @@ def broadcast_trainer_connection_status():
748
  dead_clients.add(client)
749
  ws_clients.difference_update(dead_clients)
750
 
751
- def broadcast_to_trainer(event_type: str, payload: dict):
752
- """Broadcast a structured event to connected trainer WebSocket clients."""
753
  message = json.dumps({"type": event_type, "data": payload})
754
- with trainer_ws_clients_lock:
755
  dead_clients = set()
756
- for ws in trainer_ws_clients:
757
  try:
758
  ws.send(message)
759
  except Exception:
760
  dead_clients.add(ws)
761
- trainer_ws_clients.difference_update(dead_clients)
762
 
763
 
764
- def broadcast_notification_to_ui(payload: dict):
765
- """Send a notification message (from trainer) to every UI client."""
766
- notification = json.dumps({"type": "trainer_notification", "data": payload})
767
  with ws_clients_lock:
768
  dead_clients = set()
769
  for ws in ws_clients:
@@ -798,7 +792,7 @@ def _signal_episode_control(action: str):
798
  "source": "ui",
799
  "timestamp": time.time(),
800
  }
801
- broadcast_to_trainer("episode_event", payload)
802
  # Reset UI environment so the user can continue without manual reset
803
  with mujoco_lock:
804
  if env is not None:
@@ -968,8 +962,8 @@ def handle_ws_message(ws, data):
968
  msg_type = data.get('type')
969
  print(f"[WS] Received message type: {msg_type}")
970
 
971
- if msg_type in ("trainer_identity", "notification"):
972
- _handle_trainer_message(ws, data)
973
  return
974
 
975
  # Backward compatibility: map old message types to new ones
@@ -1314,7 +1308,7 @@ def handle_ws_message(ws, data):
1314
  last_teleop_action["vry"] = 0.0
1315
  last_teleop_action["vrz"] = 0.0
1316
 
1317
- broadcast_to_trainer(
1318
  "action_update",
1319
  {
1320
  "robot": current_robot,
@@ -1501,13 +1495,13 @@ def websocket_handler(ws):
1501
  # Unregister client
1502
  with ws_clients_lock:
1503
  ws_clients.discard(ws)
1504
- with trainer_ws_clients_lock:
1505
- was_trainer = ws in trainer_ws_clients
1506
- trainer_ws_clients.discard(ws)
1507
- if was_trainer:
1508
- _unregister_trainer_client(ws)
1509
  broadcast_state()
1510
- broadcast_trainer_connection_status()
1511
  print('WebSocket client disconnected')
1512
 
1513
 
@@ -2000,11 +1994,11 @@ def index():
2000
  </div>
2001
  </div>
2002
  </div>
2003
- <div id="trainer_status_card" class="status-card disconnected">
2004
- <strong>Trainer Connection</strong>
2005
- <div class="trainer-status disconnected" id="trainer_status_indicator">
2006
- <span class="trainer-status-dot" id="trainer_status_dot" aria-hidden="true"></span>
2007
- <span class="status-card-text" id="trainer_status_text">Trainer: Disconnected</span>
2008
  </div>
2009
  </div>
2010
  </div>
@@ -2259,9 +2253,10 @@ def index():
2259
  const robotSelect = document.getElementById('robot_select');
2260
  const sceneLabel = document.getElementById('scene_label');
2261
  const overlayTiles = document.getElementById('overlay_tiles');
2262
- const trainerStatusCard = document.getElementById('trainer_status_card');
2263
- const trainerStatusIndicator = document.getElementById('trainer_status_indicator');
2264
- const trainerStatusText = document.getElementById('trainer_status_text');
 
2265
  const viewportImage = document.querySelector('.video-container img');
2266
  const robotTitle = document.getElementById('robot_title');
2267
  const robotInfo = document.getElementById('robot_info');
@@ -2339,22 +2334,29 @@ def index():
2339
  const NOVA_JOINT_VELOCITY = 0.5;
2340
  let novaVelocitiesConfigured = false;
2341
 
2342
- function updateTrainerStatus(connected) {
2343
- if (!trainerStatusCard || !trainerStatusText) {
2344
  return;
2345
  }
2346
- trainerStatusText.innerText = connected ? 'Trainer: Connected' : 'Trainer: Disconnected';
2347
- trainerStatusCard.classList.toggle('connected', connected);
2348
- trainerStatusCard.classList.toggle('disconnected', !connected);
2349
- if (trainerStatusIndicator) {
2350
- trainerStatusIndicator.classList.toggle('connected', connected);
2351
- trainerStatusIndicator.classList.toggle('disconnected', !connected);
2352
- }
2353
- if (connected) {
2354
- configureNovaVelocities();
 
2355
  }
2356
  }
2357
 
 
 
 
 
 
 
2358
  function configureNovaVelocities() {
2359
  if (novaVelocitiesConfigured) {
2360
  return;
@@ -2682,7 +2684,11 @@ def index():
2682
  currentScene = data.scene;
2683
  }
2684
 
2685
- if (typeof data.trainer_connected === 'boolean') {
 
 
 
 
2686
  updateTrainerStatus(data.trainer_connected);
2687
  }
2688
 
@@ -2927,13 +2933,19 @@ def index():
2927
 
2928
  stepVal.innerText = data.steps;
2929
  }
 
 
 
 
 
2930
  } else if (msg.type === 'trainer_status') {
 
2931
  const payload = msg.data || {};
2932
  if (typeof payload.connected === 'boolean') {
2933
  updateTrainerStatus(payload.connected);
2934
  }
2935
- } else if (msg.type === 'trainer_notification') {
2936
- showTrainerNotification(msg.data);
2937
  }
2938
  } catch (e) {
2939
  console.error('Error parsing message:', e);
@@ -2965,7 +2977,7 @@ def index():
2965
  }
2966
  }
2967
 
2968
- function showTrainerNotification(payload) {
2969
  if (!notificationList || !payload) {
2970
  return;
2971
  }
 
139
  # WebSocket clients
140
  ws_clients = set()
141
  ws_clients_lock = threading.Lock()
142
+ external_ws_clients = set() # External clients (not UI) that can send notifications
143
+ external_ws_clients_lock = threading.Lock()
144
+ client_metadata: dict = {} # Metadata for all external clients
145
+ client_metadata_lock = threading.Lock()
146
  # Camera state for orbit controls
147
  cam = mujoco.MjvCamera()
148
  cam.azimuth = 135
 
580
  if hasattr(env, "get_task_reward"):
581
  reward_value = env.get_task_reward()
582
 
583
+ with external_ws_clients_lock:
584
+ connected_clients = [info.get("identity", "client") for ws, info in list(client_metadata.items()) if ws in external_ws_clients]
585
 
586
  # UR5 has different state structure
587
  if current_robot in ("ur5", "ur5_t_push"):
 
636
  'available': nova_available,
637
  'enabled': nova_enabled_pref
638
  },
639
+ 'connected_clients': connected_clients
640
  }
641
  })
642
  else:
 
656
  'steps': int(steps),
657
  'reward': reward_value,
658
  'teleop_action': teleop_snapshot,
659
+ 'connected_clients': connected_clients
660
  }
661
  })
662
 
 
681
  ws_clients.difference_update(dead_clients)
682
 
683
 
684
+ def _handle_external_client_message(ws, data):
685
+ """Process message payloads originating from external clients."""
686
  msg_type = data.get("type")
687
+ if msg_type in ("client_identity", "trainer_identity"): # Support old name for backward compat
688
  payload = data.get("data", {}) or {}
689
+ identity = payload.get("client_id") or payload.get("trainer_id") or payload.get("name") or "client"
690
+ with external_ws_clients_lock:
691
+ external_ws_clients.add(ws)
692
+ _register_external_client(ws)
693
+ _set_client_identity(ws, identity)
694
  broadcast_state()
695
+ broadcast_connected_clients_status()
696
  return
697
+ if msg_type in ("client_notification", "notification"): # Support old name for backward compat
698
  payload = data.get("data", {})
699
  payload.setdefault("timestamp", time.time())
700
+ broadcast_notification_to_all(payload)
701
 
702
 
703
+ def _build_connected_clients_payload():
704
+ """Build a summary payload describing connected external clients."""
705
+ with client_metadata_lock:
706
+ identities = [info.get("identity", "client") for ws, info in client_metadata.items() if ws in external_ws_clients and info.get("identity")]
 
 
 
707
  payload = {
708
+ "clients": identities,
 
709
  }
 
 
710
  return payload
711
 
712
 
713
+ def _register_external_client(ws):
714
+ with client_metadata_lock:
715
+ client_metadata[ws] = {"identity": None, "connected_at": time.time()}
716
 
717
 
718
+ def _unregister_external_client(ws):
719
+ with client_metadata_lock:
720
+ client_metadata.pop(ws, None)
721
 
722
 
723
+ def _set_client_identity(ws, identity: str):
724
+ with client_metadata_lock:
725
+ info = client_metadata.get(ws)
726
  if not info:
727
  return
728
  info["identity"] = identity
729
  info["last_seen"] = time.time()
730
 
731
 
732
+ def broadcast_connected_clients_status():
733
+ """Notify all clients about connected external clients."""
734
+ payload = _build_connected_clients_payload()
735
+ message = json.dumps({"type": "connected_clients", "data": payload})
736
  with ws_clients_lock:
737
  dead_clients = set()
738
  for client in ws_clients:
 
742
  dead_clients.add(client)
743
  ws_clients.difference_update(dead_clients)
744
 
745
+ def broadcast_to_external_clients(event_type: str, payload: dict):
746
+ """Broadcast a structured event to connected external WebSocket clients."""
747
  message = json.dumps({"type": event_type, "data": payload})
748
+ with external_ws_clients_lock:
749
  dead_clients = set()
750
+ for ws in external_ws_clients:
751
  try:
752
  ws.send(message)
753
  except Exception:
754
  dead_clients.add(ws)
755
+ external_ws_clients.difference_update(dead_clients)
756
 
757
 
758
+ def broadcast_notification_to_all(payload: dict):
759
+ """Send a notification message (from external client) to all clients."""
760
+ notification = json.dumps({"type": "client_notification", "data": payload})
761
  with ws_clients_lock:
762
  dead_clients = set()
763
  for ws in ws_clients:
 
792
  "source": "ui",
793
  "timestamp": time.time(),
794
  }
795
+ broadcast_to_external_clients("episode_event", payload)
796
  # Reset UI environment so the user can continue without manual reset
797
  with mujoco_lock:
798
  if env is not None:
 
962
  msg_type = data.get('type')
963
  print(f"[WS] Received message type: {msg_type}")
964
 
965
+ if msg_type in ("client_identity", "trainer_identity", "client_notification", "notification"):
966
+ _handle_external_client_message(ws, data)
967
  return
968
 
969
  # Backward compatibility: map old message types to new ones
 
1308
  last_teleop_action["vry"] = 0.0
1309
  last_teleop_action["vrz"] = 0.0
1310
 
1311
+ broadcast_to_external_clients(
1312
  "action_update",
1313
  {
1314
  "robot": current_robot,
 
1495
  # Unregister client
1496
  with ws_clients_lock:
1497
  ws_clients.discard(ws)
1498
+ with external_ws_clients_lock:
1499
+ was_external = ws in external_ws_clients
1500
+ external_ws_clients.discard(ws)
1501
+ if was_external:
1502
+ _unregister_external_client(ws)
1503
  broadcast_state()
1504
+ broadcast_connected_clients_status()
1505
  print('WebSocket client disconnected')
1506
 
1507
 
 
1994
  </div>
1995
  </div>
1996
  </div>
1997
+ <div id="clients_status_card" class="status-card">
1998
+ <strong>Connected Clients</strong>
1999
+ <div class="clients-status" id="clients_status_indicator">
2000
+ <span class="status-card-text" id="clients_status_text">No clients connected</span>
2001
+ <ul id="clients_list" style="margin: 0.5em 0 0 0; padding-left: 1.5em; font-size: 0.9em;"></ul>
2002
  </div>
2003
  </div>
2004
  </div>
 
2253
  const robotSelect = document.getElementById('robot_select');
2254
  const sceneLabel = document.getElementById('scene_label');
2255
  const overlayTiles = document.getElementById('overlay_tiles');
2256
+ const clientsStatusCard = document.getElementById('clients_status_card');
2257
+ const clientsStatusIndicator = document.getElementById('clients_status_indicator');
2258
+ const clientsStatusText = document.getElementById('clients_status_text');
2259
+ const clientsList = document.getElementById('clients_list');
2260
  const viewportImage = document.querySelector('.video-container img');
2261
  const robotTitle = document.getElementById('robot_title');
2262
  const robotInfo = document.getElementById('robot_info');
 
2334
  const NOVA_JOINT_VELOCITY = 0.5;
2335
  let novaVelocitiesConfigured = false;
2336
 
2337
+ function updateConnectedClients(clients) {
2338
+ if (!clientsStatusCard || !clientsStatusText || !clientsList) {
2339
  return;
2340
  }
2341
+
2342
+ // Update text and list
2343
+ if (!clients || clients.length === 0) {
2344
+ clientsStatusText.innerText = "No clients connected";
2345
+ clientsList.innerHTML = "";
2346
+ clientsList.style.display = "none";
2347
+ } else {
2348
+ clientsStatusText.innerText = `${clients.length} client${clients.length > 1 ? 's' : ''} connected`;
2349
+ clientsList.innerHTML = clients.map(id => `<li>${id}</li>`).join('');
2350
+ clientsList.style.display = "block";
2351
  }
2352
  }
2353
 
2354
+ // Legacy function for backward compatibility with old state messages
2355
+ function updateTrainerStatus(connected) {
2356
+ // Deprecated: convert to new format
2357
+ updateConnectedClients(connected ? ["legacy-client"] : []);
2358
+ }
2359
+
2360
  function configureNovaVelocities() {
2361
  if (novaVelocitiesConfigured) {
2362
  return;
 
2684
  currentScene = data.scene;
2685
  }
2686
 
2687
+ // Handle both new and legacy client status formats
2688
+ if (data.connected_clients && Array.isArray(data.connected_clients)) {
2689
+ updateConnectedClients(data.connected_clients);
2690
+ } else if (typeof data.trainer_connected === 'boolean') {
2691
+ // Legacy format
2692
  updateTrainerStatus(data.trainer_connected);
2693
  }
2694
 
 
2933
 
2934
  stepVal.innerText = data.steps;
2935
  }
2936
+ } else if (msg.type === 'connected_clients') {
2937
+ const payload = msg.data || {};
2938
+ if (payload.clients && Array.isArray(payload.clients)) {
2939
+ updateConnectedClients(payload.clients);
2940
+ }
2941
  } else if (msg.type === 'trainer_status') {
2942
+ // Legacy support
2943
  const payload = msg.data || {};
2944
  if (typeof payload.connected === 'boolean') {
2945
  updateTrainerStatus(payload.connected);
2946
  }
2947
+ } else if (msg.type === 'client_notification' || msg.type === 'trainer_notification') {
2948
+ showClientNotification(msg.data);
2949
  }
2950
  } catch (e) {
2951
  console.error('Error parsing message:', e);
 
2977
  }
2978
  }
2979
 
2980
+ function showClientNotification(payload) {
2981
  if (!notificationList || !payload) {
2982
  return;
2983
  }
protocol_types.py CHANGED
@@ -262,30 +262,30 @@ class SetNovaModeMessage(TypedDict):
262
 
263
 
264
  # ============================================================================
265
- # Trainer Messages (Client -> Server)
266
  # ============================================================================
267
 
268
- class TrainerIdentityData(TypedDict):
269
- """Trainer identity data."""
270
- trainer_id: str
271
 
272
 
273
- class TrainerIdentityMessage(TypedDict):
274
- """Trainer identity handshake message."""
275
- type: Literal["trainer_identity"]
276
- data: TrainerIdentityData
277
 
278
 
279
- class TrainerNotificationData(TypedDict, total=False):
280
- """Trainer notification data."""
281
  message: str
282
  level: Literal["info", "warning", "error"]
283
 
284
 
285
- class TrainerNotificationMessage(TypedDict):
286
- """Trainer notification message."""
287
- type: Literal["notification"]
288
- data: TrainerNotificationData
289
 
290
 
291
  class EpisodeControlData(TypedDict):
@@ -358,7 +358,7 @@ class StateData(TypedDict, total=False):
358
  steps: int
359
  reward: float
360
  teleop_action: ActionData # Current action/velocity commands
361
- trainer_connected: bool
362
 
363
  # UR5-specific fields
364
  control_mode: ControlMode
@@ -371,22 +371,21 @@ class StateMessage(TypedDict):
371
  data: StateData
372
 
373
 
374
- class TrainerStatusData(TypedDict):
375
- """Trainer connection status data."""
376
- connected: bool
377
- trainer_id: Optional[str]
378
 
379
 
380
- class TrainerStatusMessage(TypedDict):
381
- """Trainer status broadcast message (to UI clients)."""
382
- type: Literal["trainer_status"]
383
- data: TrainerStatusData
384
 
385
 
386
- class TrainerNotificationBroadcast(TypedDict):
387
- """Trainer notification broadcast (to UI clients)."""
388
- type: Literal["trainer_notification"]
389
- data: TrainerNotificationData
390
 
391
 
392
  # ============================================================================
@@ -444,13 +443,13 @@ ClientMessage = Union[
444
  ControlModeMessage,
445
  GripperMessage,
446
  SetNovaModeMessage,
447
- TrainerIdentityMessage,
448
- TrainerNotificationMessage,
449
  EpisodeControlMessage,
450
  ]
451
 
452
  ServerMessage = Union[
453
  StateMessage,
454
- TrainerStatusMessage,
455
- TrainerNotificationBroadcast,
456
  ]
 
262
 
263
 
264
  # ============================================================================
265
+ # Client Identity & Notification Messages (Client -> Server)
266
  # ============================================================================
267
 
268
+ class ClientIdentityData(TypedDict):
269
+ """Client identity data."""
270
+ client_id: str
271
 
272
 
273
+ class ClientIdentityMessage(TypedDict):
274
+ """Client identity handshake message."""
275
+ type: Literal["client_identity"]
276
+ data: ClientIdentityData
277
 
278
 
279
+ class ClientNotificationData(TypedDict, total=False):
280
+ """Client notification data."""
281
  message: str
282
  level: Literal["info", "warning", "error"]
283
 
284
 
285
+ class ClientNotificationMessage(TypedDict):
286
+ """Client notification message."""
287
+ type: Literal["client_notification"]
288
+ data: ClientNotificationData
289
 
290
 
291
  class EpisodeControlData(TypedDict):
 
358
  steps: int
359
  reward: float
360
  teleop_action: ActionData # Current action/velocity commands
361
+ connected_clients: List[str] # List of connected client IDs
362
 
363
  # UR5-specific fields
364
  control_mode: ControlMode
 
371
  data: StateData
372
 
373
 
374
+ class ConnectedClientsData(TypedDict):
375
+ """Connected clients status data."""
376
+ clients: List[str] # List of connected client IDs
 
377
 
378
 
379
+ class ConnectedClientsMessage(TypedDict):
380
+ """Connected clients broadcast message (to all clients)."""
381
+ type: Literal["connected_clients"]
382
+ data: ConnectedClientsData
383
 
384
 
385
+ class ClientNotificationBroadcast(TypedDict):
386
+ """Client notification broadcast (to all clients)."""
387
+ type: Literal["client_notification"]
388
+ data: ClientNotificationData
389
 
390
 
391
  # ============================================================================
 
443
  ControlModeMessage,
444
  GripperMessage,
445
  SetNovaModeMessage,
446
+ ClientIdentityMessage,
447
+ ClientNotificationMessage,
448
  EpisodeControlMessage,
449
  ]
450
 
451
  ServerMessage = Union[
452
  StateMessage,
453
+ ConnectedClientsMessage,
454
+ ClientNotificationBroadcast,
455
  ]
templates/index.html DELETED
@@ -1,41 +0,0 @@
1
- <!DOCTYPE html>
2
- <html>
3
- <head>
4
- <title>Nova Sim - Wandelbots Robot Simulator</title>
5
- <meta charset="utf-8">
6
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
7
- <script type="module" crossorigin src="/nova-sim/assets/index-CwMJ-B8R.js"></script>
8
- <link rel="stylesheet" crossorigin href="/nova-sim/assets/index-DoqnwcEU.css">
9
- </head>
10
- <body>
11
- <!-- Video viewport -->
12
- <div class="video-container" id="viewport">
13
- <img src="/nova-sim/api/v1/video_feed" draggable="false" alt="Robot simulation video feed">
14
- </div>
15
-
16
- <!-- Main control overlay -->
17
- <div class="overlay" id="main_overlay">
18
- <!-- Content will be generated by JavaScript -->
19
- </div>
20
-
21
- <!-- State panel -->
22
- <div class="state-panel" id="state_panel">
23
- <!-- Content will be generated by JavaScript -->
24
- </div>
25
-
26
- <!-- Camera panel -->
27
- <div class="camera-panel" id="camera_panel">
28
- <!-- Content will be generated by JavaScript -->
29
- </div>
30
-
31
- <!-- Overlay tiles for auxiliary camera feeds -->
32
- <div class="overlay-tiles" id="overlay_tiles">
33
- <!-- Generated dynamically -->
34
- </div>
35
-
36
- <!-- Notification area -->
37
- <div class="rl-notifications" id="rl_notifications">
38
- <!-- Toast notifications appear here -->
39
- </div>
40
- </body>
41
- </html>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
test_home.py DELETED
@@ -1,97 +0,0 @@
1
- #!/usr/bin/env python3
2
- """
3
- Test the home function by sending continuous home commands.
4
- Simulates the UI button press behavior.
5
- """
6
-
7
- import time
8
- import json
9
- from websocket import create_connection
10
- import threading
11
-
12
- WS_URL = "ws://localhost:3004/nova-sim/api/v1/ws"
13
-
14
- print("=" * 80)
15
- print("HOME FUNCTION TEST")
16
- print("=" * 80)
17
-
18
- # Connect to WebSocket
19
- print("\n1. Connecting to WebSocket...")
20
- ws = create_connection(WS_URL)
21
- print(" ✓ Connected")
22
-
23
- # Switch to UR5
24
- print("\n2. Switching to UR5...")
25
- ws.send(json.dumps({"type": "switch_robot", "data": {"robot": "ur5", "scene": None}}))
26
- time.sleep(2)
27
- print(" ✓ Switched to UR5")
28
-
29
- # Jog some joints away from home position
30
- print("\n3. Moving joints away from home...")
31
- for joint, direction, duration in [(1, "+", 2), (3, "-", 2), (6, "+", 3)]:
32
- ws.send(json.dumps({
33
- "type": "start_jog",
34
- "data": {"jog_type": "joint", "params": {"joint": joint, "direction": direction, "velocity": 0.4}}
35
- }))
36
- time.sleep(duration)
37
- ws.send(json.dumps({"type": "stop_jog"}))
38
- time.sleep(0.5)
39
- print(" ✓ Joints moved away from home")
40
-
41
- # Send continuous home messages (simulating button press)
42
- print("\n4. Starting HOME sequence (sending messages every 100ms)...")
43
- print(" Press Ctrl+C to stop or wait 60 seconds")
44
- print("=" * 80)
45
- print("\n📺 WATCH SERVER CONSOLE FOR:")
46
- print(" 🏠 Starting homing sequence for X joints")
47
- print(" → Homing joint X...")
48
- print(" J1: err=..., dir=..., vel=...")
49
- print(" ✓ Joint X reached target")
50
- print(" ✓ Homing sequence complete!")
51
- print("")
52
-
53
- stop_homing = False
54
- message_count = 0
55
-
56
- def send_home_messages():
57
- """Send home messages every 100ms while running"""
58
- global message_count
59
- while not stop_homing:
60
- ws.send(json.dumps({"type": "home"}))
61
- message_count += 1
62
- time.sleep(0.1)
63
-
64
- # Start sending home messages in background
65
- home_thread = threading.Thread(target=send_home_messages, daemon=True)
66
- home_thread.start()
67
-
68
- try:
69
- # Wait for homing to complete (up to 60 seconds)
70
- for i in range(12):
71
- time.sleep(5)
72
- print(f" {(i+1)*5}s - sent {message_count} home messages")
73
-
74
- print("\n Timeout reached (60 seconds)")
75
-
76
- except KeyboardInterrupt:
77
- print("\n\n Interrupted by user")
78
-
79
- # Stop homing
80
- print("\n5. Stopping homing...")
81
- stop_homing = True
82
- time.sleep(0.2)
83
- ws.send(json.dumps({"type": "stop_home"}))
84
- print(f" ✓ Sent stop_home message (total: {message_count} home messages)")
85
-
86
- # Close connection
87
- ws.close()
88
-
89
- print("\n" + "=" * 80)
90
- print("TEST COMPLETE")
91
- print("=" * 80)
92
- print("\nCheck the server console to verify:")
93
- print(" ✓ Sequential homing (one joint after another)")
94
- print(" ✓ Joints stopped when error <= 0.01 rad")
95
- print(" ✓ Overshoot detection and velocity halving (if applicable)")
96
- print(" ✓ 'Homing sequence complete!' was printed")
97
- print("=" * 80)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
tests/test_api.py CHANGED
@@ -80,10 +80,10 @@ class TestWebSocketMessages:
80
  def test_websocket_connection(self, check_server):
81
  """Test basic WebSocket connection."""
82
  with connect(WS_URL, timeout=10) as ws:
83
- # Send trainer identity
84
  ws.send(json.dumps({
85
- "type": "trainer_identity",
86
- "data": {"trainer_id": "test-trainer"}
87
  }))
88
 
89
  # Should receive state messages
@@ -359,7 +359,7 @@ class TestStateStructure:
359
  assert "steps" in state_data
360
  assert "reward" in state_data
361
  assert "teleop_action" in state_data
362
- assert "trainer_connected" in state_data
363
  # Verify old fields are not in state root
364
  assert "target" not in state_data
365
  assert "target_orientation" not in state_data
@@ -389,7 +389,7 @@ class TestStateStructure:
389
  assert "steps" in state_data
390
  assert "reward" in state_data
391
  assert "teleop_action" in state_data
392
- assert "trainer_connected" in state_data
393
  # Verify old fields are removed
394
  assert "base_height" not in state_data
395
  assert "upright" not in state_data
 
80
  def test_websocket_connection(self, check_server):
81
  """Test basic WebSocket connection."""
82
  with connect(WS_URL, timeout=10) as ws:
83
+ # Send client identity (old trainer_identity still works for backward compat)
84
  ws.send(json.dumps({
85
+ "type": "client_identity",
86
+ "data": {"client_id": "test-client"}
87
  }))
88
 
89
  # Should receive state messages
 
359
  assert "steps" in state_data
360
  assert "reward" in state_data
361
  assert "teleop_action" in state_data
362
+ assert "connected_clients" in state_data
363
  # Verify old fields are not in state root
364
  assert "target" not in state_data
365
  assert "target_orientation" not in state_data
 
389
  assert "steps" in state_data
390
  assert "reward" in state_data
391
  assert "teleop_action" in state_data
392
+ assert "connected_clients" in state_data
393
  # Verify old fields are removed
394
  assert "base_height" not in state_data
395
  assert "upright" not in state_data