agents.local.md

agents.local.md — Sam robot & session config

Read by coding agents (per Pollen's AGENTS.md convention) before working here.

App

• Name: Sam — 0G medication reminder companion for Reachy Mini.
• Path: Python on-robot app (ReachyMiniApp subclass at mr_reachy/main.py:MrReachy).
• Plan: see plan.md.

Environment

• Python: 3.12 via uv (.venv/). reachy-mini[mujoco]==1.7.3.
  • The system python3 is Homebrew 3.14 and is too new for the mujoco deps — always use .venv.
• Run the sim daemon with --no-media on macOS: the GStreamer media server hangs headless waiting on a camera/mic TCC permission dialog that never appears.

  source .venv/bin/activate
  reachy-mini-daemon --sim --headless --no-media --no-preload-datasets
  

  Use --no-headless (drop --headless) to open the MuJoCo viewer for demos.
• Audio is handled in-app via a backend (io_backends.py):
  • LocalBackend (sim/dev): laptop mic (sounddevice VAD) + macOS say/espeak.
  • RobotBackend (demo, Wireless): robot onboard mic/speaker/camera via ReachyMini.media (start_recording/get_audio_sample, play_sound, get_frame) + antenna push-to-talk. Speech is synthesized on-device with Piper (tts.py); set PIPER_MODEL. 0G has no TTS.
• Demo station = Reachy Mini Wireless, app runs ON the Pi (Install to Robot from HF Space). Robot needs internet to reach compute-network-*.integratenetwork.work. Tune MR_ANTENNA_THRESHOLD (rad). RobotBackend is not yet tested on physical hardware — unknowns: antenna press threshold, whether media.play_sound blocks, mic samplerate.

0G Compute (mainnet, chain 16661)

• Secrets live in .env (gitignored). Three services, each its own host + app-sk- key.
• Funded sub-accounts: chat (GLM-5) and STT (Whisper). Vision is NOT funded — the code path exists but OG_VISION_* is blank, so vision is disabled until funded: 0g-compute-cli transfer-fund ... && 0g-compute-cli inference get-secret --provider <vision>.
• Mainnet = real funds. Every chat/STT call spends a tiny amount of real 0G.

Sam medication memory

• Local JSON is the operational memory and first read/write path (SAM_MEMORY_PATH).
• 0G Storage is the durable sync target (OG_STORAGE_*). Sync is background and retryable: failed uploads leave pending_sync=true in the local memory file.
• Safety stance: Sam is reminder-only. It stores user-provided instructions but does not prescribe, change dosage, verify ingestion, or provide medical advice.

Safety envelope (SDK clamps; gestures stay well inside)

head pitch/roll +/-40 deg, head yaw +/-180 deg, body yaw +/-160 deg, |head-body yaw| <= 65 deg.

Publishing gotcha

reachy-mini-app-assistant check . spawns a nested temp venv. If you run it with the project .venv activated, the inherited VIRTUAL_ENV corrupts that nested venv and the install step fails. Run it deactivated:

env -u VIRTUAL_ENV .venv/bin/reachy-mini-app-assistant check .

With that, the app passes all checks (entry point mr-reachy = mr_reachy.main:MrReachy).

Quick test (real 0G, no robot daemon required)

source .venv/bin/activate
python -m mr_reachy --health-check --no-robot
python -m mr_reachy --once "I am so happy" --no-robot --no-speak
python -m mr_reachy --once "Sam, I need to take metformin three times a day for five days" --no-robot --no-speak