Georg commited on
Commit
1227fda
·
1 Parent(s): d5b839c

Update README.md with project outline and detailed sections; add .env.off.local to .gitignore for environment configuration

Browse files
Files changed (2) hide show
  1. .gitignore +1 -0
  2. README.md +87 -67
.gitignore CHANGED
@@ -46,3 +46,4 @@ Thumbs.db
46
  # MuJoCo compiled
47
  *.mjb
48
  .env.local
 
 
46
  # MuJoCo compiled
47
  *.mjb
48
  .env.local
49
+ .env.off.local
README.md CHANGED
@@ -5,6 +5,34 @@ A unified MuJoCo-based robot simulation platform with web interface for multiple
5
  ![image](screenshot_ur.png)
6
  ![image](screenshot.png)
7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  ## Supported Robots
9
 
10
  ### Unitree G1 (Humanoid)
@@ -29,15 +57,43 @@ A unified MuJoCo-based robot simulation platform with web interface for multiple
29
  - Full 6-DOF IK with orientation control (can be toggled on/off)
30
  - **Optional [Wandelbots Nova API](#wandelbots-nova-api-integration) integration** for real robot state streaming and cloud-based IK
31
 
32
- ## Features
33
 
34
- - Real-time MuJoCo physics simulation
35
- - Web-based video streaming interface
36
- - WebSocket-based state/command communication
37
- - Gym-style WebSocket API for RL/IL clients
38
- - Interactive camera controls (rotate, zoom, pan)
39
- - Robot switching without restart
40
- - Keyboard and button controls for locomotion
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
 
42
  ## Quick Start
43
 
@@ -68,22 +124,9 @@ python mujoco_server.py
68
  If you see `ModuleNotFoundError: No module named 'cv2'`, make sure the venv is activated
69
  and `opencv-python` is installed in it.
70
 
71
- ## Gym WebSocket API (RL/IL)
72
 
73
- The gym-style API is exposed at `ws://localhost:3004/nova-sim/api/v1/gym/ws`.
74
- It supports `reset`, `step`, `configure`, and `get_spaces`.
75
-
76
- Example request payloads:
77
-
78
- ```json
79
- {"type": "configure", "data": {"robot": "ur5_t_push"}}
80
- {"type": "reset"}
81
- {"type": "step", "data": {"action": [0,0,0,0,0,0,0], "render": false}}
82
- ```
83
-
84
- The server responds with `gym_reset`, `gym_step`, `gym_spaces`, or `gym_configured` messages.
85
-
86
- ### Docker
87
 
88
  ```bash
89
  # Build and run (CPU/software rendering)
@@ -93,9 +136,9 @@ docker-compose up --build
93
  docker run -p 3004:3004 nova-sim
94
  ```
95
 
96
- ## Docker Configuration
97
 
98
- ### Performance Optimization
99
 
100
  Docker uses software rendering (OSMesa) which is slower than native GPU rendering. Configure these environment variables to optimize performance:
101
 
@@ -108,7 +151,7 @@ Docker uses software rendering (OSMesa) which is slower than native GPU renderin
108
  | `OMP_NUM_THREADS` | 4 | OpenMP thread count |
109
  | `MKL_NUM_THREADS` | 4 | MKL thread count |
110
 
111
- ### docker-compose.yml (CPU - Default)
112
 
113
  ```yaml
114
  services:
@@ -124,7 +167,7 @@ services:
124
  - TARGET_FPS=30
125
  ```
126
 
127
- ### docker-compose.gpu.yml (NVIDIA GPU)
128
 
129
  For significantly better performance, use GPU acceleration:
130
 
@@ -154,7 +197,7 @@ services:
154
  capabilities: [gpu]
155
  ```
156
 
157
- ### Performance Comparison
158
 
159
  | Mode | Resolution | Expected FPS | Notes |
160
  |------|------------|--------------|-------|
@@ -163,7 +206,7 @@ services:
163
  | Docker + CPU (OSMesa) | 640x360 | 15-20 | Slower, but compatible |
164
  | Docker + CPU (OSMesa) | 320x180 | 30+ | Very low quality |
165
 
166
- ### Custom Docker Run
167
 
168
  ```bash
169
  # Ultra-low resolution for maximum speed
@@ -302,44 +345,6 @@ docker run --gpus all -p 3004:3004 \
302
 
303
  All controllers output joint position targets; MuJoCo's built-in PD control tracks these targets.
304
 
305
- ## Project Structure
306
-
307
- ```
308
- nova_sim/
309
- ├── mujoco_server.py # Main Flask server with WebSocket
310
- ├── Dockerfile # Docker build configuration
311
- ├── docker-compose.yml # CPU/OSMesa configuration
312
- ├── docker-compose.gpu.yml # GPU/EGL configuration
313
- ├── requirements.txt # Python dependencies
314
- ├── robots/
315
- │ ├── g1/ # Unitree G1 humanoid
316
- │ │ ├── g1_env.py # Gymnasium environment
317
- │ │ ├── scene.xml # MuJoCo scene
318
- │ │ ├── g1_29dof.xml # Robot model
319
- │ │ ├── meshes/ # 3D mesh files
320
- │ │ ├── policy/ # RL policy weights
321
- │ │ └── controllers/ # G1 controllers
322
- │ │ ├── rl_policy.py # RL walking policy
323
- │ │ ├── pd_standing.py # Standing controller
324
- │ │ └── keyframe.py # Keyframe controller
325
- │ ├── spot/ # Boston Dynamics Spot quadruped
326
- │ │ ├── spot_env.py # Gymnasium environment
327
- │ │ ├── model/ # MuJoCo model files
328
- │ │ └── controllers/ # Spot controllers
329
- │ │ ├── mpc_gait.py # MPC-inspired gait (default)
330
- │ │ ├── quadruped_pympc_controller.py # PyMPC gait
331
- │ │ ├── trot_gait.py # Simple trot gait
332
- │ │ └── pd_standing.py # Standing controller
333
- │ └── ur5/ # Universal Robots UR5e arm
334
- │ ├── ur5_env.py # Gymnasium environment
335
- │ ├── model/ # MuJoCo model files
336
- │ │ ├── scene.xml # Combined UR5e + Robotiq scene
337
- │ │ └── assets/ # Mesh files
338
- │ └── controllers/
339
- │ └── ik_controller.py # Damped least-squares IK
340
- └── README.md
341
- ```
342
-
343
  ## Spot Controller Options
344
 
345
  The Spot robot supports multiple controller types via the `controller_type` parameter:
@@ -521,6 +526,21 @@ For robot arm (UR5):
521
  - `state_streaming`: Whether using Nova API for robot state streaming (vs. internal)
522
  - `ik`: Whether using Nova API for inverse kinematics (vs. internal)
523
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
524
  ### HTTP Endpoints
525
 
526
  | Endpoint | Method | Description |
 
5
  ![image](screenshot_ur.png)
6
  ![image](screenshot.png)
7
 
8
+ ## Outline
9
+ 1. [Overview](#overview)
10
+ 2. [Highlights](#highlights)
11
+ 3. [Supported Robots](#supported-robots)
12
+ 4. [Project Structure](#project-structure)
13
+ 5. [Quick Start](#quick-start)
14
+ 6. [Docker Deployment](#docker-deployment)
15
+ 7. [Controls](#controls)
16
+ 8. [Architecture](#architecture)
17
+ 9. [Spot Controller Options](#spot-controller-options)
18
+ 10. [API](#api)
19
+ 11. [Wandelbots Nova API Integration](#wandelbots-nova-api-integration)
20
+ 12. [License](#license)
21
+
22
+ ## Overview
23
+
24
+ Nova Sim combines MuJoCo physics, a Flask/WebSocket server, and a browser UI so you can explore locomotion and manipulation robots from a single web interface. The platform streams MJPEG video of the rendered scene, exposes a gym-style API, and lets you switch robots and control inputs without restarting the server.
25
+
26
+ ## Highlights
27
+
28
+ - Real-time MuJoCo physics simulation
29
+ - Web-based video streaming interface
30
+ - WebSocket-based state/command communication
31
+ - Gym-style WebSocket API for RL/IL clients
32
+ - Interactive camera controls (rotate, zoom, pan)
33
+ - Robot switching without restart
34
+ - Keyboard and button controls for locomotion
35
+
36
  ## Supported Robots
37
 
38
  ### Unitree G1 (Humanoid)
 
57
  - Full 6-DOF IK with orientation control (can be toggled on/off)
58
  - **Optional [Wandelbots Nova API](#wandelbots-nova-api-integration) integration** for real robot state streaming and cloud-based IK
59
 
60
+ ## Project Structure
61
 
62
+ ```
63
+ nova_sim/
64
+ ├── mujoco_server.py # Main Flask server with WebSocket
65
+ ├── Dockerfile # Docker build configuration
66
+ ├── docker-compose.yml # CPU/OSMesa configuration
67
+ ├── docker-compose.gpu.yml # GPU/EGL configuration
68
+ ├── requirements.txt # Python dependencies
69
+ ├── robots/
70
+ │ ├── g1/ # Unitree G1 humanoid
71
+ │ │ ├── g1_env.py # Gymnasium environment
72
+ │ │ ├── scene.xml # MuJoCo scene
73
+ │ │ ├── g1_29dof.xml # Robot model
74
+ │ │ ├── meshes/ # 3D mesh files
75
+ │ │ ├── policy/ # RL policy weights
76
+ │ │ └── controllers/ # G1 controllers
77
+ │ │ ├── rl_policy.py # RL walking policy
78
+ │ │ ├── pd_standing.py # Standing controller
79
+ │ │ └── keyframe.py # Keyframe controller
80
+ │ ├── spot/ # Boston Dynamics Spot quadruped
81
+ │ │ ├── spot_env.py # Gymnasium environment
82
+ │ │ ├── model/ # MuJoCo model files
83
+ │ │ └── controllers/ # Spot controllers
84
+ │ │ ├── mpc_gait.py # MPC-inspired gait (default)
85
+ │ │ ├── quadruped_pympc_controller.py # PyMPC gait
86
+ │ │ ├── trot_gait.py # Simple trot gait
87
+ │ │ └── pd_standing.py # Standing controller
88
+ │ └── ur5/ # Universal Robots UR5e arm
89
+ │ ├── ur5_env.py # Gymnasium environment
90
+ │ ├── model/ # MuJoCo model files
91
+ │ │ ├── scene.xml # Combined UR5e + Robotiq scene
92
+ │ │ └── assets/ # Mesh files
93
+ │ └── controllers/
94
+ │ └── ik_controller.py # Damped least-squares IK
95
+ └── README.md
96
+ ```
97
 
98
  ## Quick Start
99
 
 
124
  If you see `ModuleNotFoundError: No module named 'cv2'`, make sure the venv is activated
125
  and `opencv-python` is installed in it.
126
 
127
+ ## Docker Deployment
128
 
129
+ ### Getting Started
 
 
 
 
 
 
 
 
 
 
 
 
 
130
 
131
  ```bash
132
  # Build and run (CPU/software rendering)
 
136
  docker run -p 3004:3004 nova-sim
137
  ```
138
 
139
+ ### Configuration & Tuning
140
 
141
+ #### Performance Optimization
142
 
143
  Docker uses software rendering (OSMesa) which is slower than native GPU rendering. Configure these environment variables to optimize performance:
144
 
 
151
  | `OMP_NUM_THREADS` | 4 | OpenMP thread count |
152
  | `MKL_NUM_THREADS` | 4 | MKL thread count |
153
 
154
+ #### docker-compose.yml (CPU - Default)
155
 
156
  ```yaml
157
  services:
 
167
  - TARGET_FPS=30
168
  ```
169
 
170
+ #### docker-compose.gpu.yml (NVIDIA GPU)
171
 
172
  For significantly better performance, use GPU acceleration:
173
 
 
197
  capabilities: [gpu]
198
  ```
199
 
200
+ #### Performance Comparison
201
 
202
  | Mode | Resolution | Expected FPS | Notes |
203
  |------|------------|--------------|-------|
 
206
  | Docker + CPU (OSMesa) | 640x360 | 15-20 | Slower, but compatible |
207
  | Docker + CPU (OSMesa) | 320x180 | 30+ | Very low quality |
208
 
209
+ #### Custom Docker Run
210
 
211
  ```bash
212
  # Ultra-low resolution for maximum speed
 
345
 
346
  All controllers output joint position targets; MuJoCo's built-in PD control tracks these targets.
347
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
348
  ## Spot Controller Options
349
 
350
  The Spot robot supports multiple controller types via the `controller_type` parameter:
 
526
  - `state_streaming`: Whether using Nova API for robot state streaming (vs. internal)
527
  - `ik`: Whether using Nova API for inverse kinematics (vs. internal)
528
 
529
+ ### Gym WebSocket API (RL/IL)
530
+
531
+ The gym-style API is exposed at `ws://localhost:3004/nova-sim/api/v1/gym/ws`.
532
+ It supports `reset`, `step`, `configure`, and `get_spaces`.
533
+
534
+ Example request payloads:
535
+
536
+ ```json
537
+ {"type": "configure", "data": {"robot": "ur5_t_push"}}
538
+ {"type": "reset"}
539
+ {"type": "step", "data": {"action": [0,0,0,0,0,0,0], "render": false}}
540
+ ```
541
+
542
+ The server responds with `gym_reset`, `gym_step`, `gym_spaces`, or `gym_configured` messages.
543
+
544
  ### HTTP Endpoints
545
 
546
  | Endpoint | Method | Description |