OPERATIONS.md

Operations runbook

This runbook covers private Reachy Mini deployments using the Reachy daemon, a Hermes host, and the companion bridge.

Service map

Component	Default location	Port/service
Reachy daemon	Reachy/Pi	8000, reachy-mini-daemon.service
Reachy Hermes settings	Reachy app process	8042
Hermes API Server	Hermes host	8642, Hermes gateway
Reachy companion bridge	Hermes host	8643, hermes-reachy-bridge.service

Pre-deployment checks

On the development/Hermes host:

uv run ruff check .
uv run pytest
uv build --wheel

Confirm no production configuration, .env, captured audio, or provider key is included in the wheel or Git diff.

On Reachy:

systemctl is-active reachy-mini-daemon.service
curl -fsS http://127.0.0.1:8000/api/daemon/status

Stop any running app before replacing its installed package:

curl -X POST http://REACHY_HOST:8000/api/apps/stop-current-app

Wheel deployment

Copy the wheel to Reachy, then install it into the same Python environment used by the Reachy daemon:

scp dist/reachy_mini_hermes-*.whl REACHY_HOST:/tmp/

uv pip install \
  --python /path/to/reachy_mini/.venv/bin/python \
  --reinstall \
  --no-deps \
  /tmp/reachy_mini_hermes-*.whl

--no-deps is appropriate only after verifying that the Reachy environment already satisfies the package requirements. In particular, Realtime mode needs websockets>=15,<17.

Start the app:

curl -X POST http://REACHY_HOST:8000/api/apps/start-app/reachy_mini_hermes

Wait for both conditions:

curl -fsS http://REACHY_HOST:8000/api/apps/current-app-status
curl -fsS http://REACHY_HOST:8042/api/status

The app manager may report running several seconds before the settings server and media pipeline are ready.

Companion bridge deployment

The bridge normally runs directly from the checked-out repository. After bridge code or Hermes-host credentials change:

systemctl --user restart hermes-reachy-bridge.service
systemctl --user status hermes-reachy-bridge.service

Validate with the private bearer token:

curl -H "Authorization: Bearer $API_SERVER_KEY" \
  http://127.0.0.1:8643/health

For Realtime mode, require:

• hermes_api: true;
• realtime_available: true;
• realtime_model: gpt-realtime-2.1.

For Kids Mode, additionally require:

• kids_chat_available: true;
• kids_tts_streaming_available: true;
• successful authenticated /v1/kids/chat moderation/chat and /v1/kids/speech/stream PCM probes.

Health checks

Reachy app

curl -fsS http://REACHY_HOST:8042/api/status

Important fields:

• runtime.state;
• runtime.power_mode;
• runtime.last_error;
• runtime.audio_frames_processed;
• runtime.turns_completed;
• runtime.interruptions;
• config.conversation_mode;
• config.camera_enabled, runtime.camera_captures, and runtime.camera_last_error.

Test one local camera frame without returning its image content:

curl -X POST http://REACHY_HOST:8042/api/camera/test \
  -H 'Content-Type: application/json' \
  -d '{"confirm":"camera"}'

In Standby, audio_frames_processed should increase while daemon motor mode remains disabled. In Meeting or Sleep, the frame count should stop increasing.

Reachy daemon

curl -fsS http://REACHY_HOST:8000/api/daemon/status
systemctl show reachy-mini-daemon.service -p ActiveState -p NRestarts

Check:

• daemon state is running;
• motor mode matches the requested app power state;
• control-loop error count remains zero;
• restart count does not increase unexpectedly.

Logs

journalctl -u reachy-mini-daemon.service -f
journalctl --user -u hermes-reachy-bridge.service -f

Expected startup milestones include:

• app process started;
• settings server listening on 8042;
• Reachy Hermes audio input/output rates logged;
• motors disabled when entering Standby.

Treat tracebacks, Reachy voice runtime failed, repeated WebSocket closures, and increasing daemon restarts as failures. Hardware GPU-device discovery warnings from ONNX Runtime may be harmless on a Pi when CPU inference continues successfully.

Bluetooth and controller checks

This procedure applies only to Reachy Mini Wireless. Reachy Mini Lite and wired-only installations do not expose this Bluetooth controller feature as supported hardware.

1. Verify the adapter and BlueZ service:

   systemctl is-active bluetooth
   bluetoothctl show
   sudo rfkill list bluetooth
   

2. Verify the Reachy app service account belongs to input and can run bluetoothctl show through the target image's BlueZ D-Bus/polkit policy. Do not assume a bluetooth Unix group exists. Restart the Reachy daemon after changing groups.

3. Put the controller in pairing mode and use Robot → Bluetooth gamepad → Scan. Confirm Pair, Trust, and Connect all succeed.

4. Confirm the kernel created /dev/input/js0 (or another js* device) and the app reports Controller ready.

5. With clear space and controller movement enabled, test one input at a time: D-pad look, Cross center, Square Happy, Triangle Surprised, Circle Stop.

6. Enter Meeting, Sleep, and Kids Mode and confirm movement inputs are rejected. Disconnect the controller and confirm the UI returns to Waiting/Disconnected without moving Reachy.

7. Do not map power, shutdown, dance, camera, agent, smart-home, or raw joint operations to the controller.

Useful diagnostics:

bluetoothctl devices Paired
bluetoothctl devices Connected
jstest /dev/input/js0
journalctl -u bluetooth -n 100 --no-pager

Power controls

Use the settings UI where practical. API equivalents:

curl -X POST http://REACHY_HOST:8042/api/power \
  -H 'Content-Type: application/json' \
  -d '{"mode":"standby"}'

curl -X POST http://REACHY_HOST:8042/api/power \
  -H 'Content-Type: application/json' \
  -d '{"mode":"meeting","duration_minutes":60}'

curl -X POST http://REACHY_HOST:8042/api/power \
  -H 'Content-Type: application/json' \
  -d '{"mode":"sleep"}'

Stop only the voice app:

curl -X POST http://REACHY_HOST:8042/api/app-off \
  -H 'Content-Type: application/json' \
  -d '{"confirm":"off"}'

The Pi shutdown endpoint intentionally requires {"confirm":"shutdown"}. Do not call it as a routine health test.

Cooling-maintenance acceptance checklist

After installing or changing a heatsink/fan:

1. Inspect that no cable, camera ribbon, speaker lead, or motor path is pinched.

2. Confirm the heatsink does not contact exposed components or obstruct Reachy's movement.

3. Power on the Pi and verify the fan physically spins under its configured trigger condition.

4. Check current temperature:

   vcgencmd measure_temp
   cat /sys/class/thermal/thermal_zone0/temp
   

5. Check Raspberry Pi throttling history:

   vcgencmd get_throttled
   

   A clean result is throttled=0x0. Nonzero values can include historical undervoltage or thermal events; decode them before concluding that the current state is bad.

6. Start the Reachy daemon and voice app.

7. Confirm daemon restart count, control-loop errors, motor mode, and app status.

8. Leave the robot operating long enough to observe steady-state temperature.

9. Perform the human audio acceptance sequence below.

Human audio acceptance

1. Leave Reachy in Standby and confirm motors are relaxed.
2. Say “Hey Hermes” once at normal speaking volume and distance; repeat the initial-wake check with “Okay Nabu” and “Hey Reachy.”
3. Ask a simple social question; verify the native Realtime response begins promptly.
4. Interrupt Reachy naturally while it is speaking; verify playback clears and the new turn is heard. In pipeline mode, confirm each configured wake phrase can also interrupt playback.
5. Ask a non-consequential Hermes tool question, such as checking a sensor state.
6. With on-demand camera enabled, ask “What do you see?” and verify one camera capture is logged.
7. Start a supervised Kids Mode session, verify the parent lock, one moderated child turn, Flash PCM streaming, parent stop, safe fold, and continued transcript/status redaction.
8. Verify the final answers match the fresh image or Hermes tool result rather than an unverified claim.
9. Exercise Meeting, Standby, and Sleep from the UI.
10. Review logs for tracebacks and record temperature after the test.

Soak test

Before declaring a deployment stable:

• issue at least 30 app-status requests;
• issue at least 30 daemon-status requests;
• ping Reachy at least 20 times and report packet loss;
• confirm zero new daemon or bridge restarts;
• confirm zero new runtime tracebacks;
• verify the final power state and motor mode.

Rollback

Keep the previous known-good wheel until acceptance passes.

curl -X POST http://REACHY_HOST:8000/api/apps/stop-current-app
uv pip install --python /path/to/reachy_mini/.venv/bin/python \
  --reinstall --no-deps /path/to/previous/reachy_mini_hermes.whl
curl -X POST http://REACHY_HOST:8000/api/apps/start-app/reachy_mini_hermes

The user configuration is stored separately from the wheel, so rollback normally preserves settings. If a future release changes the configuration schema incompatibly, back up ~/.local/share/reachy_mini_hermes/config.json before deployment.