feat : offline mic cam check

README.md

@@ -14,106 +14,126 @@ tags:
  - ci
 ---
 
-# Reachy Mini Debug & CI Testbench
-
-A comprehensive web-based debugging and CI validation tool for Reachy Mini.
-
-## Features
-
-### Robot Status
-- Real-time head pose display (roll, pitch, yaw)
-- Detailed motor positions for all 9 motors
-- Connection status indicator
-
-### Movement Control
-- Head position control with sliders (roll, pitch, yaw)
-- Antenna position control
-- Quick actions: Go to Zero, Wake Up, Go to Sleep
-- Adjustable movement duration
-
-### Camera
-- Live camera stream (MJPEG)
-- Capture and save images
-- Download captured images
-- List and manage saved captures
-
-### Audio Recording
-- Record audio from the robot's microphone
-- List saved recordings with metadata
-- Play recordings through the robot's speaker
-- Download recordings as WAV files
-
-### Rotation Validation Test
-Automated test for validating head rotation accuracy:
-1. Moves head to zero position and captures image
-2. Rolls head by specified angle
-3. Captures second image
-4. Computes rotation between images using ORB feature matching
-5. Compares expected vs actual rotation
-6. Returns PASS/FAIL with detailed metrics
-
-## Installation
+# Reachy Mini Testbench
 
-```bash
-pip install -e .
-```
+Debug and CI validation tool for Reachy Mini. Works in two modes depending on daemon status.
 
-## Usage
+**Access:** http://localhost:8042
+
+## Quick Start
 
-### As a Reachy Mini App (daemon on)
-The app registers as a Reachy Mini app and can be launched via the daemon:
 ```bash
+pip install /path/to/reachy_mini_testbench
 reachy-mini app run reachy_mini_testbench
 ```
 
-### Direct Launch
-```bash
-python -m reachy_mini_testbench.main
-```
+---
+
+## Daemon ON Mode
+
+When the daemon backend is running, test robot functionality through the SDK.
+
+| Feature | Description |
+|---------|-------------|
+| **Daemon Status** | Full status: version, state, IP, control loop frequency, errors |
+| **System Stats** | Live CPU/RAM/disk in header (polls every 5s) |
+| **Robot Status** | Current head pose (roll/pitch/yaw) |
+| **Motor Status** | All 9 motor positions |
+| **Movement Control** | Head sliders, antenna control, wake/sleep/zero |
+| **Continuous Movement** | Sinusoidal test motion at 20Hz |
+| **Camera** | Live stream, capture/save images |
+| **Audio** | Record/playback through robot's mic/speaker |
+| **Rotation Validation** | Automated test: captures at zero and rolled positions, computes rotation error |
+
+---
+
+## Daemon OFF Mode
+
+Direct hardware access for diagnostics when daemon is not running.
+
+| Feature | Description |
+|---------|-------------|
+| **Camera Check** | Direct OpenCV capture - verifies camera without daemon/WebRTC |
+| **Microphone Check** | Direct sounddevice recording - shows peak level (dB) |
+| **Motor Scan** | Detect motors on bus (IDs 10-18) |
+| **Configuration Check** | Verify motor configs match expected values |
+| **Baudrate Scanner** | Find misconfigured motors at wrong baudrates |
+| **Motor Reflash** | Reconfigure motor ID, baudrate, settings to preset |
 
-Then open http://localhost:8042 in your browser.
-
-## API Endpoints
-
-### Status
-- `GET /api/status` - Get robot status and head pose
-- `GET /api/motor_status` - Get detailed motor positions
-
-### Movement
-- `POST /api/move_head` - Move head to specified orientation
-- `POST /api/move_antennas` - Move antennas to specified positions
-- `POST /api/go_to_zero` - Move to neutral position
-- `POST /api/wake_up` - Execute wake up behavior
-- `POST /api/go_to_sleep` - Execute sleep behavior
-
-### Camera
-- `GET /api/camera/stream` - MJPEG camera stream
-- `GET /api/camera/capture` - Capture single frame as base64
-- `POST /api/camera/save` - Save current frame to file
-- `GET /api/camera/list` - List saved captures
-- `GET /api/camera/download/{filename}` - Download capture
-- `DELETE /api/camera/delete/{filename}` - Delete capture
-
-### Audio
-- `POST /api/audio/start_recording` - Start recording
-- `POST /api/audio/stop_recording` - Stop and save recording
-- `GET /api/audio/list` - List recordings
-- `GET /api/audio/download/{filename}` - Download recording
-- `POST /api/audio/play/{filename}` - Play recording on robot
-- `DELETE /api/audio/delete/{filename}` - Delete recording
-
-### Validation Tests
-- `POST /api/test/rotation_validation` - Run rotation validation test
-- `GET /api/test/last_rotation_result` - Get last test result
+---
+
+## Compatibility
+
+| Variant | Daemon ON | Daemon OFF |
+|---------|-----------|------------|
+| Lite | All features | All features |
+| Wireless | All features | All features |
+| Simulation | All features | N/A (no hardware) |
+
+---
+
+## What Each Test Validates
+
+| Test | Mode | Validates |
+|------|------|-----------|
+| Camera Check | OFF | Direct camera hardware |
+| Mic Check | OFF | Direct microphone hardware |
+| Motor Scan | OFF | Serial bus communication |
+| Config Check | OFF | Motor EEPROM settings |
+| Head Movement | ON | Kinematics + motor control |
+| Camera Stream | ON | WebRTC/GStreamer pipeline |
+| Audio Recording | ON | Media backend pipeline |
+| Rotation Test | ON | Full vision + motion system |
+
+---
+
+## API Reference
+
+### Connection
+- `GET /api/connection_status` - Check daemon connection
+- `GET /api/system_stats` - CPU/RAM/disk usage
+- `GET /api/daemon_status` - Full daemon status
+
+### Robot (daemon ON)
+- `GET /api/status` - Head pose
+- `GET /api/motor_status` - Motor positions
+- `GET /api/robot_info` - Variant/version info
+- `POST /api/move_head` - Move head
+- `POST /api/move_antennas` - Move antennas
+- `POST /api/go_to_zero` - Neutral position
+- `POST /api/wake_up` / `POST /api/go_to_sleep`
+- `POST /api/continuous_move/start` / `stop`
+
+### Camera (daemon ON)
+- `GET /api/camera/stream` - MJPEG stream
+- `POST /api/camera/save` - Save frame
+- `GET /api/camera/list` / `download/{f}` / `DELETE delete/{f}`
+
+### Audio (daemon ON)
+- `POST /api/audio/start_recording` / `stop_recording`
+- `GET /api/audio/list` / `download/{f}` / `POST play/{f}` / `DELETE delete/{f}`
+
+### Hardware Check (daemon OFF)
+- `GET /api/hardware/camera_check` - Direct camera test
+- `GET /api/hardware/mic_check` - Direct mic test
+- `GET /api/hardware/list_audio_devices` - List audio devices
+
+### Motor Diagnostics (daemon OFF)
+- `GET /api/motors/serial_info` - Serial port info
+- `GET /api/motors/scan` - Scan for motors
+- `GET /api/motors/scan_baudrates` - Scan all baudrates
+- `GET /api/motors/check_all` - Check configurations
+- `POST /api/motors/led` - Control motor LED
+- `GET /api/motors/presets` - Available motor presets
+- `POST /api/motors/reflash` - Reflash motor config
+
+### Tests
+- `POST /api/test/rotation_validation` - Run rotation test
+- `GET /api/test/last_rotation_result` - Last test result
+
+---
 
 ## File Storage
 
 - Recordings: `/tmp/reachy_mini_testbench/recordings/`
 - Captures: `/tmp/reachy_mini_testbench/captures/`
-
-## Dependencies
-
-- reachy-mini
-- opencv-python
-- soundfile
-- scipy

pyproject.toml

@@ -13,7 +13,10 @@ dependencies = [
     "reachy-mini",
     "opencv-python",
     "soundfile",
+    "sounddevice",
     "scipy",
+    "psutil",
+    "aiohttp",
 ]
 keywords = ["reachy-mini-app", "debug", "testing", "ci", "validation"]
 

reachy_mini_testbench/main.py

@@ -25,6 +25,7 @@ from typing import Any
 
 import cv2
 import numpy as np
+import sounddevice as sd
 import soundfile as sf
 import uvicorn
 from fastapi import FastAPI, HTTPException, Response
@@ -1279,6 +1280,154 @@ async def reflash_motor(req: MotorReflashRequest):
         raise HTTPException(500, f"Unexpected error during reflash: {str(e)}")
 
 
+# ==================== Hardware Check Endpoints (work without daemon) ====================
+
+@app.get("/api/hardware/camera_check")
+async def check_camera():
+    """Check camera by capturing a frame directly (no daemon/webrtc).
+
+    Uses OpenCV to directly access the camera device.
+    Returns a base64-encoded JPEG image.
+    """
+    cap = None
+    try:
+        # Try common camera indices (0 is usually the default)
+        for camera_idx in [0, 1, 2]:
+            cap = cv2.VideoCapture(camera_idx)
+            if cap.isOpened():
+                break
+            cap.release()
+            cap = None
+
+        if cap is None:
+            raise HTTPException(503, "No camera found. Tried indices 0, 1, 2.")
+
+        # Let camera warm up
+        for _ in range(5):
+            cap.read()
+
+        # Capture frame
+        ret, frame = cap.read()
+        if not ret or frame is None:
+            raise HTTPException(503, "Camera opened but failed to capture frame")
+
+        # Get camera properties
+        width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
+        height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
+        fps = cap.get(cv2.CAP_PROP_FPS)
+
+        # Encode to JPEG
+        _, buffer = cv2.imencode('.jpg', frame, [cv2.IMWRITE_JPEG_QUALITY, 85])
+        b64_image = base64.b64encode(buffer).decode('utf-8')
+
+        return {
+            "success": True,
+            "camera_index": camera_idx,
+            "resolution": f"{width}x{height}",
+            "fps": fps if fps > 0 else "unknown",
+            "image": b64_image,
+            "timestamp": datetime.now().isoformat()
+        }
+
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.error(f"Camera check error: {e}")
+        raise HTTPException(500, f"Camera check failed: {str(e)}")
+    finally:
+        if cap is not None:
+            cap.release()
+
+
+@app.get("/api/hardware/mic_check")
+async def check_microphone(duration: float = 1.0):
+    """Check microphone by recording a short sample directly (no daemon/webrtc).
+
+    Uses sounddevice for direct microphone access.
+    Returns info about the recording and peak level.
+    """
+    try:
+        # Get default input device info
+        device_info = sd.query_devices(kind='input')
+        device_name = device_info['name']
+        max_channels = device_info['max_input_channels']
+        default_sr = int(device_info['default_samplerate'])
+
+        # Use default sample rate, mono, duration clamped to 0.5-5s
+        duration = max(0.5, min(5.0, duration))
+        samplerate = default_sr
+
+        # Record audio
+        logger.info(f"Recording {duration}s from mic: {device_name}")
+        recording = sd.rec(
+            int(duration * samplerate),
+            samplerate=samplerate,
+            channels=1,
+            dtype='float32'
+        )
+        sd.wait()  # Wait for recording to complete
+
+        # Calculate audio statistics
+        audio_data = recording.flatten()
+        peak_level = float(np.max(np.abs(audio_data)))
+        rms_level = float(np.sqrt(np.mean(audio_data ** 2)))
+
+        # Convert to dB (avoid log of 0)
+        peak_db = 20 * np.log10(peak_level + 1e-10)
+        rms_db = 20 * np.log10(rms_level + 1e-10)
+
+        # Determine if we detected sound (peak above noise floor)
+        sound_detected = peak_db > -40  # -40 dB is typical noise floor
+
+        return {
+            "success": True,
+            "device_name": device_name,
+            "max_channels": max_channels,
+            "samplerate": samplerate,
+            "duration_recorded": duration,
+            "samples": len(audio_data),
+            "peak_level": round(peak_level, 4),
+            "peak_db": round(peak_db, 1),
+            "rms_level": round(rms_level, 4),
+            "rms_db": round(rms_db, 1),
+            "sound_detected": sound_detected,
+            "timestamp": datetime.now().isoformat()
+        }
+
+    except sd.PortAudioError as e:
+        logger.error(f"Microphone access error: {e}")
+        raise HTTPException(503, f"No microphone available: {str(e)}")
+    except Exception as e:
+        logger.error(f"Mic check error: {e}")
+        raise HTTPException(500, f"Microphone check failed: {str(e)}")
+
+
+@app.get("/api/hardware/list_audio_devices")
+async def list_audio_devices():
+    """List all available audio input devices."""
+    try:
+        devices = sd.query_devices()
+        input_devices = []
+
+        for idx, dev in enumerate(devices):
+            if dev['max_input_channels'] > 0:
+                input_devices.append({
+                    "index": idx,
+                    "name": dev['name'],
+                    "channels": dev['max_input_channels'],
+                    "default_samplerate": int(dev['default_samplerate']),
+                    "is_default": idx == sd.default.device[0]  # default input device
+                })
+
+        return {
+            "input_devices": input_devices,
+            "default_input_index": sd.default.device[0]
+        }
+    except Exception as e:
+        logger.error(f"List audio devices error: {e}")
+        raise HTTPException(500, f"Failed to list audio devices: {str(e)}")
+
+
 # ==================== ReachyMiniApp Wrapper for App Manager ====================
 
 from reachy_mini.apps.app import ReachyMiniApp

reachy_mini_testbench/static/index.html

@@ -174,6 +174,47 @@
             </div>
         </section>
 
+        <!-- Hardware Check Panel (daemon-off only) -->
+        <section class="panel daemon-off-only" id="hardware-check-panel">
+            <h2>Hardware Check</h2>
+            <div class="panel-content">
+                <p class="panel-description">
+                    Direct camera and microphone access (no daemon/webrtc).
+                    Use this to verify hardware works independently.
+                </p>
+
+                <div class="control-group">
+                    <h3>Camera Check</h3>
+                    <button onclick="checkCamera()" class="btn btn-primary" id="camera-check-btn">Check Camera</button>
+                    <div id="camera-check-result" class="hardware-check-result hidden">
+                        <div class="camera-preview-container">
+                            <img id="camera-check-preview" src="" alt="Camera preview">
+                        </div>
+                        <div class="hardware-check-info" id="camera-check-info">
+                            <!-- Info will be populated by JS -->
+                        </div>
+                    </div>
+                </div>
+
+                <div class="control-group">
+                    <h3>Microphone Check</h3>
+                    <p class="panel-description">Records 1 second of audio and shows level</p>
+                    <button onclick="checkMicrophone()" class="btn btn-primary" id="mic-check-btn">Check Microphone</button>
+                    <div id="mic-check-result" class="hardware-check-result hidden">
+                        <div class="mic-level-container">
+                            <div class="mic-level-bar">
+                                <div class="mic-level-fill" id="mic-level-fill"></div>
+                            </div>
+                            <span class="mic-level-value" id="mic-level-value">-- dB</span>
+                        </div>
+                        <div class="hardware-check-info" id="mic-check-info">
+                            <!-- Info will be populated by JS -->
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </section>
+
         <!-- Motor Diagnostics Panel (daemon-off only) -->
         <section class="panel daemon-off-only" id="diagnostics-panel">
             <h2>Motor Diagnostics</h2>

reachy_mini_testbench/static/main.js

@@ -847,6 +847,157 @@ async function reflashMotor() {
     }
 }
 
+// ==================== Hardware Check Functions ====================
+
+async function checkCamera() {
+    const btn = document.getElementById('camera-check-btn');
+    const resultDiv = document.getElementById('camera-check-result');
+    const previewImg = document.getElementById('camera-check-preview');
+    const infoDiv = document.getElementById('camera-check-info');
+
+    try {
+        btn.disabled = true;
+        btn.textContent = 'Checking...';
+        resultDiv.classList.add('hidden');
+        showToast('Checking camera...', 'info');
+
+        const data = await apiCall('/api/hardware/camera_check');
+
+        // Show result
+        resultDiv.classList.remove('hidden');
+
+        // Set image
+        previewImg.src = `data:image/jpeg;base64,${data.image}`;
+
+        // Build info HTML
+        infoDiv.innerHTML = `
+            <div class="check-info-item ok">
+                <span class="label">Status</span>
+                <span class="value">Camera Working</span>
+            </div>
+            <div class="check-info-item">
+                <span class="label">Device Index</span>
+                <span class="value">${data.camera_index}</span>
+            </div>
+            <div class="check-info-item">
+                <span class="label">Resolution</span>
+                <span class="value">${data.resolution}</span>
+            </div>
+            <div class="check-info-item">
+                <span class="label">FPS</span>
+                <span class="value">${typeof data.fps === 'number' ? data.fps.toFixed(0) : data.fps}</span>
+            </div>
+        `;
+
+        showToast('Camera check passed!', 'success');
+
+    } catch (error) {
+        resultDiv.classList.remove('hidden');
+        previewImg.src = '';
+        infoDiv.innerHTML = `
+            <div class="check-info-item error">
+                <span class="label">Status</span>
+                <span class="value">Camera Failed</span>
+            </div>
+            <div class="check-info-item error">
+                <span class="label">Error</span>
+                <span class="value">${error.message}</span>
+            </div>
+        `;
+        console.error('Camera check failed:', error);
+    } finally {
+        btn.disabled = false;
+        btn.textContent = 'Check Camera';
+    }
+}
+
+async function checkMicrophone() {
+    const btn = document.getElementById('mic-check-btn');
+    const resultDiv = document.getElementById('mic-check-result');
+    const levelFill = document.getElementById('mic-level-fill');
+    const levelValue = document.getElementById('mic-level-value');
+    const infoDiv = document.getElementById('mic-check-info');
+
+    try {
+        btn.disabled = true;
+        btn.textContent = 'Recording...';
+        resultDiv.classList.add('hidden');
+        showToast('Recording from microphone...', 'info');
+
+        const data = await apiCall('/api/hardware/mic_check?duration=1.0');
+
+        // Show result
+        resultDiv.classList.remove('hidden');
+
+        // Calculate level bar width (map -60dB to 0dB to 0% to 100%)
+        const peakDb = data.peak_db;
+        const barPercent = Math.max(0, Math.min(100, ((peakDb + 60) / 60) * 100));
+        levelFill.style.width = `${barPercent}%`;
+
+        // Color based on level
+        if (data.sound_detected) {
+            levelFill.className = 'mic-level-fill detected';
+        } else {
+            levelFill.className = 'mic-level-fill quiet';
+        }
+
+        levelValue.textContent = `${peakDb.toFixed(1)} dB`;
+
+        // Build info HTML
+        const soundStatus = data.sound_detected ? 'Sound Detected' : 'Very Quiet/No Sound';
+        const statusClass = data.sound_detected ? 'ok' : 'warning';
+
+        infoDiv.innerHTML = `
+            <div class="check-info-item ${statusClass}">
+                <span class="label">Status</span>
+                <span class="value">${soundStatus}</span>
+            </div>
+            <div class="check-info-item">
+                <span class="label">Device</span>
+                <span class="value">${data.device_name}</span>
+            </div>
+            <div class="check-info-item">
+                <span class="label">Sample Rate</span>
+                <span class="value">${data.samplerate} Hz</span>
+            </div>
+            <div class="check-info-item">
+                <span class="label">Peak Level</span>
+                <span class="value">${data.peak_db.toFixed(1)} dB</span>
+            </div>
+            <div class="check-info-item">
+                <span class="label">RMS Level</span>
+                <span class="value">${data.rms_db.toFixed(1)} dB</span>
+            </div>
+        `;
+
+        if (data.sound_detected) {
+            showToast('Microphone check passed - sound detected!', 'success');
+        } else {
+            showToast('Microphone working but very quiet', 'warning');
+        }
+
+    } catch (error) {
+        resultDiv.classList.remove('hidden');
+        levelFill.style.width = '0%';
+        levelFill.className = 'mic-level-fill';
+        levelValue.textContent = '-- dB';
+        infoDiv.innerHTML = `
+            <div class="check-info-item error">
+                <span class="label">Status</span>
+                <span class="value">Microphone Failed</span>
+            </div>
+            <div class="check-info-item error">
+                <span class="label">Error</span>
+                <span class="value">${error.message}</span>
+            </div>
+        `;
+        console.error('Microphone check failed:', error);
+    } finally {
+        btn.disabled = false;
+        btn.textContent = 'Check Microphone';
+    }
+}
+
 // ==================== Connection Status ====================
 
 async function fetchConnectionStatus() {

reachy_mini_testbench/static/style.css

@@ -852,6 +852,125 @@ main {
     }
 }
 
+/* ==================== Hardware Check ====================  */
+.hardware-check-result {
+    margin-top: 1rem;
+    padding: 1rem;
+    background: var(--bg-panel);
+    border-radius: 8px;
+}
+
+.hardware-check-result.hidden {
+    display: none;
+}
+
+.camera-preview-container {
+    margin-bottom: 1rem;
+    background: #000;
+    border-radius: 6px;
+    overflow: hidden;
+    aspect-ratio: 4/3;
+}
+
+.camera-preview-container img {
+    width: 100%;
+    height: 100%;
+    object-fit: contain;
+}
+
+.hardware-check-info {
+    display: grid;
+    grid-template-columns: repeat(2, 1fr);
+    gap: 0.5rem;
+}
+
+.check-info-item {
+    display: flex;
+    flex-direction: column;
+    padding: 0.5rem 0.75rem;
+    background: var(--bg-secondary);
+    border-radius: 4px;
+    border-left: 3px solid var(--border);
+}
+
+.check-info-item.ok {
+    border-left-color: var(--success);
+}
+
+.check-info-item.warning {
+    border-left-color: var(--warning);
+}
+
+.check-info-item.error {
+    border-left-color: var(--error);
+}
+
+.check-info-item .label {
+    font-size: 0.65rem;
+    color: var(--text-secondary);
+    text-transform: uppercase;
+    margin-bottom: 0.2rem;
+}
+
+.check-info-item .value {
+    font-size: 0.85rem;
+    font-weight: 500;
+    color: var(--text-primary);
+}
+
+.check-info-item.ok .value {
+    color: var(--success);
+}
+
+.check-info-item.warning .value {
+    color: var(--warning);
+}
+
+.check-info-item.error .value {
+    color: var(--error);
+}
+
+/* Microphone Level Bar */
+.mic-level-container {
+    display: flex;
+    align-items: center;
+    gap: 1rem;
+    margin-bottom: 1rem;
+}
+
+.mic-level-bar {
+    flex: 1;
+    height: 24px;
+    background: var(--bg-secondary);
+    border-radius: 12px;
+    overflow: hidden;
+    border: 1px solid var(--border);
+}
+
+.mic-level-fill {
+    height: 100%;
+    width: 0%;
+    background: var(--border);
+    transition: width 0.3s ease, background 0.3s ease;
+    border-radius: 12px;
+}
+
+.mic-level-fill.detected {
+    background: linear-gradient(90deg, var(--success), #22c55e);
+}
+
+.mic-level-fill.quiet {
+    background: linear-gradient(90deg, var(--warning), #f59e0b);
+}
+
+.mic-level-value {
+    font-size: 1rem;
+    font-weight: 600;
+    color: var(--text-primary);
+    min-width: 70px;
+    text-align: right;
+}
+
 /* ==================== Motor Diagnostics ====================  */
 .serial-info {
     display: flex;