Delete CAMERA_README.md

CAMERA_README.md

@@ -1,256 +0,0 @@
-# Camera System for MuJoCo G1 Simulator
-
-## Overview
-
-The simulator has two cameras defined:
-
-### 1. **`head_camera`** - Robot Ego-View
-- **Location**: Attached to `torso_link` body
-- **Position**: `[0.06, 0.0, 0.45]` relative to torso (6cm forward, 45cm up)
-- **Orientation**: `euler="0 -0.8 -1.57"` (facing forward, slightly tilted down)
-- **FOV**: 90 degrees
-- **Purpose**: First-person view from the robot's perspective (like a head-mounted camera)
-
-### 2. **`global_view`** - Third-Person View  
-- **Location**: Fixed in world coordinates
-- **Position**: `[2.910, -5.040, 3.860]` (behind and above the robot)
-- **Purpose**: External observer view for visualization
-
-## How Camera Publishing Works
-
-The camera system uses a **zero-copy architecture** with three components:
-
-```
-┌─────────────────────┐
-│   MuJoCo Simulator  │
-│  (Main Process)     │
-│                     │
-│  1. Render cameras  │
-│  2. Copy to shmem   │──┐
-└─────────────────────┘  │
-                         │ Shared Memory
-                         │ (fast IPC)
-                    ┌────▼────────────────┐
-                    │  Image Publisher    │
-                    │  (Subprocess)       │
-                    │                     │
-                    │  3. Encode images   │
-                    │  4. ZMQ publish     │
-                    └─────────┬───────────┘
-                              │
-                              │ TCP (ZMQ)
-                              │ port 5555
-                              ▼
-                    ┌─────────────────────┐
-                    │  Your Policy/Client │
-                    │  (Subscribe)        │
-                    └─────────────────────┘
-```
-
-### Key Technologies:
-- **MuJoCo Renderer**: Captures RGB images from virtual cameras
-- **Shared Memory (`multiprocessing.shared_memory`)**: Zero-copy transfer between processes
-- **ZMQ (ZeroMQ)**: Network socket for publishing images (TCP)
-- **No ROS2 required!** Pure Python multiprocessing
-
-## Usage
-
-### Basic Simulation (No Camera Publishing)
-```bash
-python run_sim.py
-```
-
-### With Camera Publishing
-```bash
-# Publish head camera on default port 5555
-python run_sim.py --publish-images
-
-# Publish multiple cameras
-python run_sim.py --publish-images --cameras head_camera global_view
-
-# Custom port
-python run_sim.py --publish-images --camera-port 6000
-```
-
-### Viewing Camera Streams
-
-In a **separate terminal**, run the camera viewer:
-
-```bash
-# Basic usage (default: localhost:5555)
-python view_cameras.py
-
-# Custom host/port
-python view_cameras.py --host 192.168.1.100 --port 6000
-
-# Save images to directory
-python view_cameras.py --save ./camera_recordings
-
-# Adjust display rate
-python view_cameras.py --fps 60
-```
-
-**Keyboard Controls:**
-- `q`: Quit viewer
-- `s`: Save snapshot of current frame
-
-**Example Workflow:**
-```bash
-# Terminal 1: Start simulator with camera publishing
-python run_sim.py --publish-images
-
-# Terminal 2: View the camera feed
-python view_cameras.py
-```
-
-### Receiving Images in Your Code
-
-```python
-import zmq
-import numpy as np
-from gr00t_wbc.control.sensor.sensor_server import ImageMessageSchema
-
-# Connect to camera publisher
-context = zmq.Context()
-socket = context.socket(zmq.SUB)
-socket.connect("tcp://localhost:5555")
-socket.setsockopt(zmq.SUBSCRIBE, b"")  # Subscribe to all messages
-
-while True:
-    # Receive serialized image data
-    data = socket.recv_pyobj()
-    
-    # Decode images
-    if "head_camera" in data:
-        # Decode image (returns numpy array HxWx3 uint8)
-        img = decode_image(data["head_camera"])
-        
-        # Use image for your policy
-        process_observation(img)
-```
-
-## Camera Configuration
-
-Edit `config.yaml` to change camera settings:
-
-```yaml
-IMAGE_DT: 0.033333  # Publishing rate (30 Hz)
-ENABLE_OFFSCREEN: false  # Enable for camera rendering
-MP_START_METHOD: "spawn"  # Multiprocessing method
-```
-
-Or programmatically in `run_sim.py`:
-
-```python
-camera_configs = {
-    "head_camera": {
-        "height": 480,
-        "width": 640
-    },
-    "custom_camera": {
-        "height": 224,
-        "width": 224
-    }
-}
-```
-
-## Adding Custom Cameras
-
-Edit `assets/g1_29dof_with_hand.xml` or `assets/scene_43dof.xml`:
-
-```xml
-<!-- Camera attached to robot body -->
-<body name="torso_link" pos="0 0 0.019">
-  <camera name="my_camera" pos="0.1 0.0 0.5" euler="0 0 0" fovy="60"/>
-</body>
-
-<!-- Camera in world coordinates -->
-<worldbody>
-  <camera name="side_view" pos="0 -3.0 1.5" xyaxes="1 0 0 0 0.5 0.866"/>
-</worldbody>
-```
-
-Then publish it:
-```bash
-python run_sim.py --publish-images --cameras my_camera
-```
-
-## Performance Notes
-
-- **Rendering overhead**: ~5-10ms per camera per frame @ 640x480
-- **Publishing overhead**: ~2-3ms for encoding + network
-- Image publishing runs in **separate subprocess** to not block simulation
-- Uses **shared memory** for fast inter-process image transfer
-- Target: 30 FPS camera publishing while maintaining 500 Hz simulation
-
-## Troubleshooting
-
-### No images received?
-1. Check if offscreen rendering is enabled (`--publish-images` flag)
-2. Verify ZMQ port is not blocked
-3. Check camera exists in scene XML
-
-### Images are delayed?
-- Reduce `IMAGE_DT` in config
-- Lower camera resolution
-- Use fewer cameras
-
-### "Camera not found" error?
-- Verify camera name in XML matches config
-- Check XML syntax is valid
-- Ensure MuJoCo model loads successfully
-
-## Quick Reference
-
-### File Structure
-```
-mujoco_sim_g1/
-├── run_sim.py              # Simulator launcher
-├── view_cameras.py         # Camera viewer (this file!)
-├── config.yaml             # Simulator config
-├── assets/
-│   ├── scene_43dof.xml     # Scene with global_view camera
-│   └── g1_29dof_with_hand.xml  # Robot model with head_camera
-└── sim/
-    ├── base_sim.py         # MuJoCo environment
-    ├── sensor_utils.py     # ZMQ camera server/client
-    └── image_publish_utils.py  # Multiprocessing image publisher
-```
-
-### Camera Definitions
-
-Edit these files to modify cameras:
-
-**`assets/g1_29dof_with_hand.xml`** - Robot-attached cameras:
-```xml
-<body name="torso_link" pos="0 0 0.019">
-  <camera name="head_camera" pos="0.06 0.0 0.45" euler="0 -0.8 -1.57" fovy="90"/>
-</body>
-```
-
-**`assets/scene_43dof.xml`** - World-frame cameras:
-```xml
-<worldbody>
-  <camera name="global_view" pos="2.910 -5.040 3.860" xyaxes="0.866 0.500 0.000 -0.250 0.433 0.866" fovy="45"/>
-</worldbody>
-```
-
-### Complete Example
-
-```bash
-# Terminal 1: Start simulator with camera publishing
-cd mujoco_sim_g1
-python run_sim.py --publish-images --cameras head_camera global_view
-
-# Terminal 2: View cameras in real-time
-python view_cameras.py
-
-# Terminal 3: Use in your policy (Python code)
-from sim.sensor_utils import SensorClient, ImageUtils
-client = SensorClient()
-client.start_client("localhost", 5555)
-data = client.receive_message()
-img = ImageUtils.decode_image(data["head_camera"])
-# img is now numpy array (H, W, 3) in BGR format
-```
-