docs/ARCHITECTURE.md

Architecture

Reachy Mini I Spy runs as one application on the same machine as the Reachy daemon.

Deployment profiles

Reachy Mini Lite

Reachy Mini Control and the daemon run on the connected Mac/PC. The I Spy app, camera-frame processing and direct provider calls run on that same host. The Lite robot supplies camera, microphones, speaker and motion hardware over its normal USB connection.

Reachy Mini Wireless

The daemon and I Spy app run on Reachy's onboard Raspberry Pi CM4. Cloud provider calls leave directly from the app; local ONNX detection and offline speech run on the CM4. No second server is required.

Application boundary

The reachy_mini_i_spy package is installed through the official reachy_mini_apps entry point. It owns:

• game settings and camera-opt-in state;
• owner-only provider credentials;
• fixed provider URL, models, prompts and response schemas;
• deterministic game and generation state;
• bounded head/base motion and final folding;
• transient frame capture;
• local UI and status endpoints;
• moderation, audio playback and immediate Stop handling.

The UI cannot configure arbitrary URLs, models, prompts, voices or tools. Provider credentials are never returned through the status API.

Direct cloud data flow

Camera opt-in
      │
      ▼
App host: bounded motion + up to three transient JPEGs
      │ fixed HTTPS endpoint, fixed schema, owner-only API key
      ▼
OpenAI: moderation + fixed vision/TTS operations
      │ untrusted bounded result
      ▼
App host: deterministic validation + clue/guess state + motion/audio
      │
      └── Stop/reveal/error → invalidate generation, reject late results,
          stop audio, revoke camera, clear target, fold, disable motors

Frames and guesses are processed in memory. The app does not intentionally persist gameplay media or transcripts.

Local-mode data flow

Camera opt-in
      │
      ▼
App host: bounded motion + up to three transient JPEGs
      │ no network inference
      ▼
Bundled ONNX detector: boxes + COCO labels + confidence
      │ untrusted bounded detections
      ▼
Deterministic allowlist + stability + colour + hints + aliases
      │
      ▼
Pinned sherpa-onnx English/Dutch speech → Reachy speaker

The local path replaces both cloud vision and cloud speech synthesis backends, not the safety state machine:

• ONNX object detection executes on the daemon host;
• object classes are restricted to an explicit child-safe allowlist;
• stability across viewpoints, bounding-box rules and colour extraction are deterministic;
• hints and guess aliases come from reviewed English/Dutch tables;
• Stop and generation rules remain unchanged;
• speech uses Apache-2.0 sherpa-onnx with pinned, checksummed English public-domain and Dutch CC0 voice datasets;
• voice archives are installed atomically after path-traversal and size checks;
• model engines are cached in memory after their first load.

Version 0.2.0 local mode is clean-install validated and physically accepted on Wireless. The redesigned Lite-host profile remains explicitly marked as awaiting same-version physical acceptance.

Origin and related project

Reachy Mini I Spy originated as a focused extraction of the I Spy experience developed in Reachy Mini Hermes. It now owns its runtime, provider integration, safety contract, releases and acceptance lifecycle.

Reachy Mini Hermes retains a separate integrated Kids Mode implementation with different frame counts, camera choreography and alternating roles. The two projects may share safety principles, but neither inherits the other's release or physical acceptance.