Comprehensive README update with architecture, troubleshooting, and development history

README.md

@@ -14,9 +14,39 @@ Web-based IDE for writing, verifying, and executing rmscript code on Reachy Mini
 
 ## Architecture
 
-- **Frontend** (`frontend/`): HTML/CSS/JavaScript editor with live compilation and robot control
-- **Backend** (`backend/`): FastAPI server for rmscript compilation and validation
-- **Robot**: Reachy Mini robot daemon running on localhost:8000
+### Components
+
+- **Frontend** (`frontend/`): Pure HTML/CSS/JavaScript web IDE
+  - `index.html`: UI layout with embedded CSS styling
+  - `app.js`: Application logic, compilation API calls, robot communication
+  - Auto-detects environment (file:// for local, web server for production)
+
+- **Backend** (`backend/`): FastAPI server (Python 3.11)
+  - `app.py`: REST API for rmscript compilation and validation
+  - Serves frontend files via static file mounting
+  - Runs on port 8001 (local) or port 7860 (HuggingFace Spaces)
+
+- **Robot Daemon**: Reachy Mini daemon (expected on localhost:8000)
+  - REST API: `/api/move/goto` (with duration support)
+  - WebSocket API: `/api/move/ws/set_target` (streaming, no duration)
+  - State API: `/api/state/*` (robot current state)
+
+### Communication Flow
+
+1. **Script Verification/Compilation**:
+   - Frontend → Backend (`/api/verify` or `/api/compile`)
+   - Backend compiles rmscript using the rmscript package
+   - Returns IR (Intermediate Representation) + errors/warnings
+
+2. **Script Execution** (Local):
+   - Frontend → Robot Daemon (`POST /api/move/goto`)
+   - Each IR action sent with duration and interpolation mode
+   - Proper timing with minimum-jerk interpolation
+
+3. **Script Execution** (HuggingFace Spaces):
+   - Frontend → Robot Daemon (WebSocket `/api/move/ws/set_target`)
+   - Fallback due to HTTPS → HTTP localhost blocking
+   - No duration control - moves are instant
 
 ## Quick Start
 
@@ -77,12 +107,30 @@ uv run python app.py
 To deploy this demo to HuggingFace Spaces:
 
 1. **Prepare the repository structure:**
-   - Copy the `rmscript` and `reachy_mini` directories into the `rmscript_space_demo` directory:
+   - Copy the `rmscript` and `reachy_mini` directories (without `.git` and binary files):
      ```bash
      cd rmscript_space_demo
+
+     # Option 1: Using rsync (recommended - cleaner)
+     rsync -av --exclude='.git' --exclude='.venv' --exclude='__pycache__' \
+           --exclude='*.pyc' ../rmscript/ ./rmscript/
+     rsync -av --exclude='.git' --exclude='.venv' --exclude='__pycache__' \
+           --exclude='*.pyc' ../reachy_mini/ ./reachy_mini/
+
+     # Option 2: Using cp then cleanup
      cp -r ../rmscript ./rmscript
      cp -r ../reachy_mini ./reachy_mini
+     rm -rf ./rmscript/.git ./reachy_mini/.git
+     rm -rf ./rmscript/.venv ./reachy_mini/.venv
+
+     # Remove binary files that HuggingFace Spaces rejects
+     rm -rf ./reachy_mini/src/reachy_mini/assets/*.wav
+     rm -rf ./reachy_mini/src/reachy_mini/daemon/app/dashboard/static/fonts/*.ttf
+     rm -rf ./reachy_mini/docs ./reachy_mini/examples
+     rm -rf ./reachy_mini/src/reachy_mini/descriptions/reachy_mini/mjcf
+     rm -rf ./rmscript/examples ./rmscript/tests
      ```
+
    - Your structure should be:
      ```
      rmscript_space_demo/
@@ -91,8 +139,8 @@ To deploy this demo to HuggingFace Spaces:
        ├── .dockerignore
        ├── backend/
        ├── frontend/
-       ├── rmscript/     ← copied source
-       └── reachy_mini/  ← copied source (dependency)
+       ├── rmscript/     ← copied source (no .git, no binaries)
+       └── reachy_mini/  ← copied source (no .git, no binaries)
      ```
 
 2. **Create a new Space:**
@@ -109,7 +157,211 @@ To deploy this demo to HuggingFace Spaces:
 
 ## Development
 
-To modify the demo:
-- Frontend code: `frontend/app.js`
-- Backend API: `backend/app.py`
-- Styling: `frontend/index.html` (embedded CSS)
+### File Structure
+
+```
+rmscript_space_demo/
+├── Dockerfile              # Docker build config for HuggingFace Spaces
+├── .dockerignore          # Excludes .git, .venv, __pycache__, etc.
+├── README.md              # This file (includes HF Space metadata in YAML)
+├── backend/
+│   ├── app.py            # FastAPI server (compilation + static file serving)
+│   └── pyproject.toml    # Python dependencies
+└── frontend/
+    ├── index.html        # Web IDE UI (with embedded CSS)
+    └── app.js            # Frontend logic (compilation + execution)
+```
+
+### Key Implementation Details
+
+**Matrix to Euler Conversion** (`frontend/app.js:328-355`):
+- Converts 4x4 pose matrices (from rmscript IR) to euler angles (roll, pitch, yaw)
+- Uses XYZ euler convention to match `scipy.spatial.transform.Rotation.as_euler("xyz")`
+- Required because WebSocket API expects `{x, y, z, roll, pitch, yaw}` format
+
+**Environment Detection** (`frontend/app.js:9-11`):
+- Detects `file://` protocol → uses `http://localhost:8001` for backend
+- Detects web server → uses relative URLs (same origin)
+- Script loading path adapts similarly for `app.js` imports
+
+**Execution Strategy** (`frontend/app.js:357-427`):
+- **Tries** `POST /api/move/goto` with duration (works locally)
+- **Catches** fetch failures → falls back to WebSocket (for Spaces)
+- Logs warnings when fallback is used
+
+### Modifying the Demo
+
+**Frontend Changes:**
+- Edit `frontend/app.js` for logic changes
+- Edit `frontend/index.html` for UI/styling changes
+- Test by opening `index.html` directly in browser (local mode)
+
+**Backend Changes:**
+- Edit `backend/app.py` for API changes
+- Run locally: `cd backend && uv run python app.py`
+- Test API at `http://localhost:8001/docs` (FastAPI Swagger UI)
+
+**Deployment:**
+- Changes auto-deploy when pushed to HuggingFace Space repo
+- Build logs visible in Space's "Build" tab
+- Runtime logs visible in Space's "Logs" tab
+
+## Technical Notes
+
+### Browser Security & Mixed Content
+
+**The Problem:**
+- HuggingFace Spaces serve content over HTTPS
+- Robot daemon runs on `http://localhost:8000` (HTTP, not HTTPS)
+- Modern browsers block HTTPS → HTTP requests (mixed content policy)
+- This blocks `fetch()` to the robot's REST API
+
+**Current Solution:**
+- WebSocket connections are allowed (less strict security)
+- Use WebSocket `/ws/set_target` as fallback
+- Limitation: No duration control via WebSocket
+
+**Why We Can't Fix This:**
+- Robot daemon is on user's local machine
+- Space backend runs on HuggingFace servers (can't reach localhost)
+- Can't proxy through Space backend
+- Can't get robot's current state for client-side interpolation
+
+**Alternative Approaches (Not Implemented):**
+1. **Client-side interpolation**: Would need current robot state (blocked by same security issue)
+2. **Local HTTPS proxy**: Too complex for end users
+3. **Run robot daemon on HTTPS**: Requires SSL certificates for localhost
+
+### Deployment Gotchas
+
+**Binary Files:**
+- HuggingFace Spaces reject binary files (WAV, TTF, PNG, etc.)
+- Must remove from `reachy_mini/` and `rmscript/` copies before pushing
+- Already removed: docs/, examples/, assets/, fonts/
+
+**OpenGL Dependencies:**
+- OpenCV (cv2) requires system libraries: `libgl1`, `libglib2.0-0`, etc.
+- Must install in Dockerfile (see lines 8-15)
+- Package name changed from `libgl1-mesa-glx` → `libgl1` in newer Debian
+
+**Git Submodules:**
+- Don't copy `.git` directories from `rmscript/` and `reachy_mini/`
+- Use `rsync --exclude='.git'` or remove after copying
+- Otherwise Space treats them as submodules (causes issues)
+
+## Future Improvements
+
+### Potential Enhancements
+
+1. **Sound & Picture Support:**
+   - Currently IR supports these actions but execution skips them
+   - Would need to implement in `frontend/app.js:executeScript()`
+
+2. **Syntax Highlighting:**
+   - Add CodeMirror or Monaco Editor for better editing experience
+   - Custom rmscript syntax highlighting rules
+
+3. **Save/Load Scripts:**
+   - localStorage for saving scripts in browser
+   - Or integrate with HuggingFace Datasets for cloud storage
+
+4. **Real-time Error Highlighting:**
+   - Show errors inline in editor (like VS Code)
+   - Currently only shows in console panel
+
+5. **Movement Visualization:**
+   - 3D preview of robot movements before execution
+   - Using Three.js or similar
+
+6. **Execution Controls:**
+   - Pause/resume execution
+   - Step through script line-by-line
+   - Stop/cancel running script
+
+### Known Issues
+
+- **Spaces timing limitation**: Cannot be fixed without changing browser security or deployment architecture
+- **Cached IR bug**: Fixed - now always compiles fresh editor content
+- **Euler angle conversion**: Works but could add gimbal lock handling improvements
+
+### Development History & Fixes
+
+**Issues Fixed During Development:**
+
+1. **Robot not executing moves** (2025-11-26):
+   - **Problem**: Robot daemon connected but moves didn't execute
+   - **Cause**: Frontend was extracting translation but hardcoding euler angles to 0
+   - **Fix**: Implemented `matrixToEulerXYZ()` to properly extract rotation from 4x4 matrices
+   - **Location**: `frontend/app.js:328-355`
+
+2. **Modified scripts not executing** (2025-11-26):
+   - **Problem**: Editing script in editor didn't affect execution - same old script ran
+   - **Cause**: `executeScript()` was using cached IR instead of compiling fresh
+   - **Fix**: Always compile editor content before execution
+   - **Location**: `frontend/app.js:285-290`
+
+3. **Local development broken after Space changes** (2025-11-26):
+   - **Problem**: Opening `index.html` locally didn't load `app.js`
+   - **Cause**: Changed script path to `/static/app.js` for Space deployment
+   - **Fix**: Auto-detect environment and load appropriate path
+   - **Location**: `frontend/index.html:320-329`
+
+4. **Binary files rejected during Space deployment** (2025-11-26):
+   - **Problem**: HuggingFace push rejected due to WAV/TTF/PNG files
+   - **Cause**: HuggingFace Spaces don't allow binary files without LFS
+   - **Fix**: Remove docs, examples, assets from copied dependencies
+   - **Commands**: See deployment section step 1
+
+5. **Docker build failing on Spaces** (2025-11-26):
+   - **Problem**: `libgl1-mesa-glx` package not found during build
+   - **Cause**: Package renamed in newer Debian (Trixie)
+   - **Fix**: Changed to `libgl1` in Dockerfile
+   - **Location**: `Dockerfile:10`
+
+6. **Duration not respected in execution** (2025-11-26):
+   - **Problem**: All moves happened instantly regardless of specified duration
+   - **Cause**: Using WebSocket `/ws/set_target` which doesn't support duration
+   - **Fix**: Switch to REST API `POST /api/move/goto` with duration parameter
+   - **Location**: `frontend/app.js:394-405`
+
+7. **Fetch failures on HuggingFace Spaces** (2025-11-26):
+   - **Problem**: "Failed to fetch" errors when executing on Space
+   - **Cause**: Browser mixed content policy blocks HTTPS → HTTP localhost
+   - **Fix**: Fallback to WebSocket when fetch fails (timing limitation accepted)
+   - **Location**: `frontend/app.js:407-426`
+
+## Troubleshooting
+
+**Backend won't start locally:**
+```bash
+# Make sure rmscript is installed
+cd ../rmscript && uv sync
+cd ../rmscript_space_demo/backend
+
+# Install backend deps
+uv sync
+uv run python app.py
+```
+
+**Robot not connecting:**
+- Check robot daemon is running: `curl http://localhost:8000/api/health`
+- Check WebSocket connection in browser console
+- Try reloading the page
+
+**Space build fails:**
+- Check build logs in Space's "Build" tab
+- Common issues: binary files not removed, missing system dependencies
+- Dockerfile may need updating for new Debian versions
+
+**Compilation errors:**
+- Check rmscript syntax in examples
+- Use "Verify" button before "Execute"
+- Check console for detailed error messages
+
+## Credits
+
+Built with:
+- [rmscript](../rmscript/) - Kid-friendly robot programming language
+- [reachy_mini](../reachy_mini/) - Reachy Mini SDK
+- [FastAPI](https://fastapi.tiangolo.com/) - Backend framework
+- [HuggingFace Spaces](https://huggingface.co/spaces) - Deployment platform