Spaces:
Running
Running
feat: Suppress macOS dylib warnings during startup and update tool documentation for profiles and core tools.
Browse files- AGENT.md +1 -1
- frontend/AGENT.md +38 -21
- frontend/src/registry/AGENT.md +6 -4
- src/AGENT.md +42 -13
- src/reachy_mini_conversation_app/main.py +35 -7
- src/reachy_mini_conversation_app/profiles/AGENT.md +13 -6
- src/reachy_mini_conversation_app/prompts/AGENT.md +14 -5
- src/reachy_mini_conversation_app/tools/AGENT.md +24 -18
- tests/AGENT.md +20 -10
AGENT.md
CHANGED
|
@@ -24,7 +24,7 @@ reachy_mini_minder/
|
|
| 24 |
β βββ src/hooks/ # Data hooks (see hooks/AGENT.md)
|
| 25 |
β βββ src/registry/ # GenUI components (see registry/AGENT.md)
|
| 26 |
βββ tests/ # Test suite (see tests/AGENT.md)
|
| 27 |
-
βββ documentation/ #
|
| 28 |
βββ .agent/ # Agent rules and skills
|
| 29 |
βββ pyproject.toml # Package configuration
|
| 30 |
```
|
|
|
|
| 24 |
β βββ src/hooks/ # Data hooks (see hooks/AGENT.md)
|
| 25 |
β βββ src/registry/ # GenUI components (see registry/AGENT.md)
|
| 26 |
βββ tests/ # Test suite (see tests/AGENT.md)
|
| 27 |
+
βββ documentation/ # Docs, design system, case studies, guides
|
| 28 |
βββ .agent/ # Agent rules and skills
|
| 29 |
βββ pyproject.toml # Package configuration
|
| 30 |
```
|
frontend/AGENT.md
CHANGED
|
@@ -49,29 +49,46 @@ The frontend has **no direct database access**. All state flows through these in
|
|
| 49 |
| `ui.dismiss` | `emit_ui_dismiss()` | `useConversation` |
|
| 50 |
| `ui.navigate` | `emit_ui_navigate()` | `useConversation` β `ChatInterface` |
|
| 51 |
| `ui.settings_updated` | `emit_settings_updated()` | `useConversation` β `SettingsPanel` |
|
|
|
|
| 52 |
| `session.wakeword_detected` / `.started` / `.ended` | `emit_session_event()` | `useConversation` β `useSession` (via DOM event) |
|
| 53 |
| `cost.update` | `emit_cost_event()` | `useObservability` |
|
| 54 |
| `camera.frame` | `emit_camera_frame()` | `useConversation` β `CameraView` |
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
|
| 60 |
-
|
|
| 61 |
-
| `/api/session`
|
| 62 |
-
| `/api/
|
| 63 |
-
| `/api/
|
| 64 |
-
| `/api/
|
| 65 |
-
| `/api/
|
| 66 |
-
| `/api/
|
| 67 |
-
| `/api/
|
| 68 |
-
| `/api/
|
| 69 |
-
| `/api/
|
| 70 |
-
| `/api/
|
| 71 |
-
| `/api/
|
| 72 |
-
| `/api/
|
| 73 |
-
| `/api/
|
| 74 |
-
| `/api/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
|
| 76 |
### External Dependencies
|
| 77 |
|
|
@@ -86,7 +103,7 @@ The frontend has **no direct database access**. All state flows through these in
|
|
| 86 |
|
| 87 |
## When Adding Features
|
| 88 |
|
| 89 |
-
1. If the feature needs backend data β add REST endpoint in `console.py` β consume in a hook
|
| 90 |
2. If the feature needs real-time updates β add `emit_*()` in `stream_api.py` β handle in `useConversation`
|
| 91 |
3. If the feature renders LLM-driven UI β add GenUI component in `registry/` β trigger via `emit_ui_component()` in a backend tool
|
| 92 |
4. Update this `AGENT.md` tables when adding new endpoints or events
|
|
|
|
| 49 |
| `ui.dismiss` | `emit_ui_dismiss()` | `useConversation` |
|
| 50 |
| `ui.navigate` | `emit_ui_navigate()` | `useConversation` β `ChatInterface` |
|
| 51 |
| `ui.settings_updated` | `emit_settings_updated()` | `useConversation` β `SettingsPanel` |
|
| 52 |
+
| `ui.dashboard_updated` | `emit_dashboard_updated()` | `useConversation` β `DashboardGrid` |
|
| 53 |
| `session.wakeword_detected` / `.started` / `.ended` | `emit_session_event()` | `useConversation` β `useSession` (via DOM event) |
|
| 54 |
| `cost.update` | `emit_cost_event()` | `useObservability` |
|
| 55 |
| `camera.frame` | `emit_camera_frame()` | `useConversation` β `CameraView` |
|
| 56 |
+
|
| 57 |
+
#### REST Endpoints β `console.py` (api_app)
|
| 58 |
+
|
| 59 |
+
| Endpoint | Method | Hook/Component |
|
| 60 |
+
| -------------------------- | ---------- | ----------------- |
|
| 61 |
+
| `/api/session` | GET/POST | `useSession` |
|
| 62 |
+
| `/api/session/inject-text` | POST | `useConversation` |
|
| 63 |
+
| `/api/listening` | GET/POST | `useListening` |
|
| 64 |
+
| `/api/provider` | GET/POST | `useSettings` |
|
| 65 |
+
| `/api/profile` | GET/PUT | `SettingsPanel` |
|
| 66 |
+
| `/api/medications` | GET | `SettingsPanel` |
|
| 67 |
+
| `/api/wakeword` | GET/PUT | `SettingsPanel` |
|
| 68 |
+
| `/api/consent` | GET/POST | `useConsent` |
|
| 69 |
+
| `/api/conversation-log` | GET/DELETE | `SettingsPanel` |
|
| 70 |
+
| `/api/missed-doses` | GET | (missed dose UI) |
|
| 71 |
+
| `/api/missed-doses/notify` | POST | (missed dose UI) |
|
| 72 |
+
| `/api/appointment-export` | GET | `ReportsPanel` |
|
| 73 |
+
| `/api/move-head` | POST | `CameraView` |
|
| 74 |
+
| `/api/data` | DELETE | `SettingsPanel` |
|
| 75 |
+
| `/api/langgraph` | DELETE | `SettingsPanel` |
|
| 76 |
+
|
| 77 |
+
#### REST Endpoints β `stream_api.py` (stream router, prefixed `/api/stream/`)
|
| 78 |
+
|
| 79 |
+
| Endpoint | Method | Hook/Component |
|
| 80 |
+
| ----------------------------- | ------ | -------------------- |
|
| 81 |
+
| `/api/stream/ws` | WS | `useConversation` |
|
| 82 |
+
| `/api/stream/dashboard-stats` | GET | `useDashboardData` |
|
| 83 |
+
| `/api/stream/observability` | GET | `useObservability` |
|
| 84 |
+
| `/api/stream/cost-summary` | GET | `useObservability` |
|
| 85 |
+
| `/api/stream/session-log` | GET | `ObservabilityPanel` |
|
| 86 |
+
| `/api/stream/status` | GET | (health check) |
|
| 87 |
+
| `/api/stream/reset-db` | POST | `SettingsPanel` |
|
| 88 |
+
| `/api/stream/clear-neo4j` | POST | `SettingsPanel` |
|
| 89 |
+
| `/api/stream/emit-component` | POST | (dev tooling) |
|
| 90 |
+
| `/api/stream/reports` | GET | `ReportsPanel` |
|
| 91 |
+
| `/api/stream/reports/{id}` | GET | `ReportsPanel` |
|
| 92 |
|
| 93 |
### External Dependencies
|
| 94 |
|
|
|
|
| 103 |
|
| 104 |
## When Adding Features
|
| 105 |
|
| 106 |
+
1. If the feature needs backend data β add REST endpoint in `console.py` or `stream_api.py` β consume in a hook
|
| 107 |
2. If the feature needs real-time updates β add `emit_*()` in `stream_api.py` β handle in `useConversation`
|
| 108 |
3. If the feature renders LLM-driven UI β add GenUI component in `registry/` β trigger via `emit_ui_component()` in a backend tool
|
| 109 |
4. Update this `AGENT.md` tables when adding new endpoints or events
|
frontend/src/registry/AGENT.md
CHANGED
|
@@ -12,24 +12,26 @@ Components are registered in `index.tsx` and rendered by `ComponentOverlay` / `C
|
|
| 12 |
|
| 13 |
| GenUI Component | Triggered By (Backend Tool) | Tool Location |
|
| 14 |
| -------------------------------------- | ------------------------------------------------------------ | ---------------------------------------------------------------------------------- |
|
| 15 |
-
| `OnboardingProgress` | `show_onboarding_step`
|
|
|
|
| 16 |
| `OnboardingSummary` | `complete_onboarding` | `profiles/_locked_profile/complete_onboarding.py` |
|
| 17 |
| `ProfileCard` | `complete_onboarding` | `profiles/_locked_profile/complete_onboarding.py` |
|
| 18 |
-
| `MedicationCard` | `setup_medication`
|
| 19 |
| `MedLog` | `start_medication_entry`, `update_medication_entry` | `profiles/_locked_profile/start_medication_entry.py`, `update_medication_entry.py` |
|
| 20 |
| `HeadacheLog` | `start_headache_entry`, `update_headache_entry` | `profiles/_locked_profile/start_headache_entry.py`, `update_headache_entry.py` |
|
| 21 |
| `ConfirmationCard` | `save_current_entry` | `profiles/_locked_profile/save_current_entry.py` |
|
| 22 |
| `EntryCard` | `entry_control` | `profiles/_locked_profile/entry_control.py` |
|
| 23 |
| `HistoryTimeline` | `get_recent_entries` | `profiles/_locked_profile/get_recent_entries.py` |
|
| 24 |
-
| `MyMedsList` | `get_my_medications`
|
| 25 |
| `OptionsPicker` | `entry_control` | `profiles/_locked_profile/entry_control.py` |
|
| 26 |
| `QuickReply` | `log_entry` | `profiles/_locked_profile/log_entry.py` |
|
| 27 |
| `ReportPreview` / `ReportConfirmation` | `send_report` | `profiles/_locked_profile/send_report.py` |
|
|
|
|
| 28 |
| `SessionSummary` | `session_summary` tool | `tools/session_summary.py` |
|
| 29 |
| `DailySnapshot` | `check_medication` | `tools/check_medication.py` |
|
| 30 |
| `CheckInCard` | `check_medication` | `tools/check_medication.py` |
|
| 31 |
| `MedStatus` | `check_medication`, `log_entry` | `tools/check_medication.py`, `profiles/_locked_profile/log_entry.py` |
|
| 32 |
-
| `MusicPicker` | `media_control`
|
| 33 |
| `NowPlaying` | `media_control`, `resume_music`, `pause_music`, `stop_music` | `tools/media_control.py`, `resume_music.py`, etc. |
|
| 34 |
| `VolumeControl` | `set_volume` | `tools/set_volume.py` |
|
| 35 |
| `SettingsConfirmation` | `update_settings` | `profiles/_locked_profile/update_settings.py` |
|
|
|
|
| 12 |
|
| 13 |
| GenUI Component | Triggered By (Backend Tool) | Tool Location |
|
| 14 |
| -------------------------------------- | ------------------------------------------------------------ | ---------------------------------------------------------------------------------- |
|
| 15 |
+
| `OnboardingProgress` | `show_onboarding_step`, `get_onboarding_status` | `profiles/_locked_profile/show_onboarding_step.py`, `get_onboarding_status.py` |
|
| 16 |
+
| `OnboardingReview` | `show_onboarding_review` | `profiles/_locked_profile/show_onboarding_review.py` |
|
| 17 |
| `OnboardingSummary` | `complete_onboarding` | `profiles/_locked_profile/complete_onboarding.py` |
|
| 18 |
| `ProfileCard` | `complete_onboarding` | `profiles/_locked_profile/complete_onboarding.py` |
|
| 19 |
+
| `MedicationCard` | `setup_medication`, `update_my_medication` | `profiles/_locked_profile/setup_medication.py`, `update_my_medication.py` |
|
| 20 |
| `MedLog` | `start_medication_entry`, `update_medication_entry` | `profiles/_locked_profile/start_medication_entry.py`, `update_medication_entry.py` |
|
| 21 |
| `HeadacheLog` | `start_headache_entry`, `update_headache_entry` | `profiles/_locked_profile/start_headache_entry.py`, `update_headache_entry.py` |
|
| 22 |
| `ConfirmationCard` | `save_current_entry` | `profiles/_locked_profile/save_current_entry.py` |
|
| 23 |
| `EntryCard` | `entry_control` | `profiles/_locked_profile/entry_control.py` |
|
| 24 |
| `HistoryTimeline` | `get_recent_entries` | `profiles/_locked_profile/get_recent_entries.py` |
|
| 25 |
+
| `MyMedsList` | `get_my_medications`, `update_my_medication` | `profiles/_locked_profile/get_my_medications.py`, `update_my_medication.py` |
|
| 26 |
| `OptionsPicker` | `entry_control` | `profiles/_locked_profile/entry_control.py` |
|
| 27 |
| `QuickReply` | `log_entry` | `profiles/_locked_profile/log_entry.py` |
|
| 28 |
| `ReportPreview` / `ReportConfirmation` | `send_report` | `profiles/_locked_profile/send_report.py` |
|
| 29 |
+
| `TrendChart` | `query_health_history` | `tools/query_health_history.py` |
|
| 30 |
| `SessionSummary` | `session_summary` tool | `tools/session_summary.py` |
|
| 31 |
| `DailySnapshot` | `check_medication` | `tools/check_medication.py` |
|
| 32 |
| `CheckInCard` | `check_medication` | `tools/check_medication.py` |
|
| 33 |
| `MedStatus` | `check_medication`, `log_entry` | `tools/check_medication.py`, `profiles/_locked_profile/log_entry.py` |
|
| 34 |
+
| `MusicPicker` | `media_control`, `list_music` | `tools/media_control.py`, `tools/list_music.py` |
|
| 35 |
| `NowPlaying` | `media_control`, `resume_music`, `pause_music`, `stop_music` | `tools/media_control.py`, `resume_music.py`, etc. |
|
| 36 |
| `VolumeControl` | `set_volume` | `tools/set_volume.py` |
|
| 37 |
| `SettingsConfirmation` | `update_settings` | `profiles/_locked_profile/update_settings.py` |
|
src/AGENT.md
CHANGED
|
@@ -8,19 +8,48 @@ Contains the main Python package `reachy_mini_conversation_app`. This is the cor
|
|
| 8 |
|
| 9 |
```
|
| 10 |
src/reachy_mini_conversation_app/
|
| 11 |
-
βββ console.py
|
| 12 |
-
βββ main.py
|
| 13 |
-
βββ stream_api.py
|
| 14 |
-
βββ openai_realtime.py
|
| 15 |
-
βββ
|
| 16 |
-
βββ
|
| 17 |
-
βββ
|
| 18 |
-
βββ
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
β βββ _reachy_mini_minder_locked_profile/ # Clinical tools + GenUI emitters
|
| 20 |
-
βββ
|
| 21 |
-
βββ
|
| 22 |
-
βββ
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
| 24 |
```
|
| 25 |
|
| 26 |
### Key Entry Points
|
|
@@ -53,4 +82,4 @@ python src/reachy_mini_conversation_app/console.py
|
|
| 53 |
|
| 54 |
## Configuration
|
| 55 |
|
| 56 |
-
Database stored at `
|
|
|
|
| 8 |
|
| 9 |
```
|
| 10 |
src/reachy_mini_conversation_app/
|
| 11 |
+
βββ console.py # FastAPI server (API endpoints + audio pipeline)
|
| 12 |
+
βββ main.py # ReachyMiniApp entry point (robot mode)
|
| 13 |
+
βββ stream_api.py # WebSocket streaming + REST endpoints β frontend bridge
|
| 14 |
+
βββ openai_realtime.py # OpenAI Realtime API handler
|
| 15 |
+
βββ realtime_handler.py # Base handler class
|
| 16 |
+
βββ handler_factory.py # Creates OpenAI handler
|
| 17 |
+
βββ database.py # SQLite CRUD operations
|
| 18 |
+
βββ config.py # Application configuration
|
| 19 |
+
βββ schemas.py # Pydantic models / data schemas
|
| 20 |
+
βββ utils.py # Shared utilities
|
| 21 |
+
βββ entry_state.py # Health entry state machine
|
| 22 |
+
βββ prompts.py # Prompt builder/combiner
|
| 23 |
+
βββ session_enrichment.py # Neo4j graph enrichment at session start
|
| 24 |
+
βββ session_summary.py # Session summary generation
|
| 25 |
+
βββ appointment_export.py # Clinical report export
|
| 26 |
+
βββ email_service.py # Email delivery service
|
| 27 |
+
βββ memory_graph.py # Neo4j knowledge graph operations
|
| 28 |
+
βββ memory_notes.py # Session memory notes
|
| 29 |
+
βββ graph_query_engine.py # Neo4j query engine
|
| 30 |
+
βββ pattern_detector.py # Health pattern detection
|
| 31 |
+
βββ missed_dose_detection.py # Missed dose tracking
|
| 32 |
+
βββ pii_guard.py # PII redaction
|
| 33 |
+
βββ checkin_scheduler.py # Proactive check-in scheduling
|
| 34 |
+
βββ computer_audio.py # Host audio routing (music playback)
|
| 35 |
+
βββ wakeword_detector.py # Wakeword detection
|
| 36 |
+
βββ camera_worker.py # Camera frame capture
|
| 37 |
+
βββ moves.py # Robot movement definitions
|
| 38 |
+
βββ dance_emotion_moves.py # Dance & emotion sequences
|
| 39 |
+
βββ headless_personality.py # Headless mode personality
|
| 40 |
+
βββ headless_personality_ui.py # Headless personality UI
|
| 41 |
+
βββ robot_control_api.py # Robot control REST API router
|
| 42 |
+
βββ tools/ # LLM tools (see tools/AGENT.md)
|
| 43 |
+
βββ prompts/ # System prompts (see prompts/AGENT.md)
|
| 44 |
+
βββ profiles/ # Conversation profiles (see profiles/AGENT.md)
|
| 45 |
β βββ _reachy_mini_minder_locked_profile/ # Clinical tools + GenUI emitters
|
| 46 |
+
βββ langgraph_agent/ # LangGraph agent integration
|
| 47 |
+
βββ audio/ # Audio utilities
|
| 48 |
+
βββ vision/ # Vision processing
|
| 49 |
+
βββ models/ # Data models
|
| 50 |
+
βββ static/ # Static assets
|
| 51 |
+
βββ templates/ # HTML templates
|
| 52 |
+
βββ images/ # Image assets
|
| 53 |
```
|
| 54 |
|
| 55 |
### Key Entry Points
|
|
|
|
| 82 |
|
| 83 |
## Configuration
|
| 84 |
|
| 85 |
+
Database stored at project root: `mini_minder.db`.
|
src/reachy_mini_conversation_app/main.py
CHANGED
|
@@ -8,6 +8,32 @@ import argparse
|
|
| 8 |
import threading
|
| 9 |
from typing import Optional
|
| 10 |
from pathlib import Path
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
from fastapi import FastAPI
|
| 13 |
|
|
@@ -35,13 +61,15 @@ def run(
|
|
| 35 |
) -> None:
|
| 36 |
"""Run the Reachy Mini conversation app."""
|
| 37 |
# Putting these dependencies here makes the dashboard faster to load when the conversation app is installed
|
| 38 |
-
from
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
|
|
|
|
|
|
| 45 |
|
| 46 |
logger = setup_logger(args.debug)
|
| 47 |
logger.info("Starting Reachy Mini Conversation App")
|
|
|
|
| 8 |
import threading
|
| 9 |
from typing import Optional
|
| 10 |
from pathlib import Path
|
| 11 |
+
from contextlib import contextmanager
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
@contextmanager
|
| 15 |
+
def _suppress_duplicate_dylib_warnings():
|
| 16 |
+
"""Suppress macOS ObjC duplicate class warnings from cv2/av FFmpeg dylibs.
|
| 17 |
+
|
| 18 |
+
Both opencv-python and PyAV (via aiortc) bundle their own FFmpeg libavdevice,
|
| 19 |
+
which registers identical ObjC classes (AVFFrameReceiver, AVFAudioReceiver).
|
| 20 |
+
The warning is cosmetic and harmless, but noisy. We redirect C-level stderr
|
| 21 |
+
(fd 2) to /dev/null during the imports that trigger these registrations.
|
| 22 |
+
"""
|
| 23 |
+
if sys.platform != "darwin":
|
| 24 |
+
yield
|
| 25 |
+
return
|
| 26 |
+
|
| 27 |
+
devnull_fd = os.open(os.devnull, os.O_WRONLY)
|
| 28 |
+
original_stderr_fd = os.dup(2)
|
| 29 |
+
try:
|
| 30 |
+
os.dup2(devnull_fd, 2)
|
| 31 |
+
yield
|
| 32 |
+
finally:
|
| 33 |
+
os.dup2(original_stderr_fd, 2)
|
| 34 |
+
os.close(original_stderr_fd)
|
| 35 |
+
os.close(devnull_fd)
|
| 36 |
+
|
| 37 |
|
| 38 |
from fastapi import FastAPI
|
| 39 |
|
|
|
|
| 61 |
) -> None:
|
| 62 |
"""Run the Reachy Mini conversation app."""
|
| 63 |
# Putting these dependencies here makes the dashboard faster to load when the conversation app is installed
|
| 64 |
+
# Suppress macOS ObjC duplicate class warnings from cv2/av bundled FFmpeg libs
|
| 65 |
+
with _suppress_duplicate_dylib_warnings():
|
| 66 |
+
from reachy_mini_conversation_app.moves import MovementManager
|
| 67 |
+
from reachy_mini_conversation_app.console import LocalStream
|
| 68 |
+
from reachy_mini_conversation_app.handler_factory import create_handler
|
| 69 |
+
from reachy_mini_conversation_app.tools.core_tools import ToolDependencies
|
| 70 |
+
from reachy_mini_conversation_app.audio.head_wobbler import HeadWobbler
|
| 71 |
+
from reachy_mini_conversation_app.database import MiniMinderDB
|
| 72 |
+
from reachy_mini_conversation_app.entry_state import EntryStateManager
|
| 73 |
|
| 74 |
logger = setup_logger(args.debug)
|
| 75 |
logger.info("Starting Reachy Mini Conversation App")
|
src/reachy_mini_conversation_app/profiles/AGENT.md
CHANGED
|
@@ -20,14 +20,18 @@ profiles/
|
|
| 20 |
|
| 21 |
| Tool File | Purpose | Emits GenUI |
|
| 22 |
| ---------------------------- | --------------------------- | ---------------------------------------------------- |
|
| 23 |
-
| `show_onboarding_step.py` | Onboarding wizard
|
|
|
|
|
|
|
| 24 |
| `complete_onboarding.py` | Finish onboarding | `OnboardingSummary`, `ProfileCard` |
|
| 25 |
| `setup_medication.py` | Add medication to schedule | `MedicationCard` |
|
|
|
|
| 26 |
| `start_medication_entry.py` | Begin med log | `MedLog` |
|
| 27 |
| `update_medication_entry.py` | Update med fields | `MedLog` |
|
| 28 |
| `start_headache_entry.py` | Begin headache log | `HeadacheLog` |
|
| 29 |
| `update_headache_entry.py` | Update headache fields | `HeadacheLog` |
|
| 30 |
| `save_current_entry.py` | Save entry to DB | `ConfirmationCard` |
|
|
|
|
| 31 |
| `entry_control.py` | Entry list/selection | `EntryCard`, `OptionsPicker` |
|
| 32 |
| `log_entry.py` | Quick log (med or headache) | `MedStatus`, `QuickReply` |
|
| 33 |
| `get_recent_entries.py` | History view | `HistoryTimeline` |
|
|
@@ -36,18 +40,21 @@ profiles/
|
|
| 36 |
| `update_settings.py` | Voice settings update | `SettingsConfirmation` + `ui.settings_updated` event |
|
| 37 |
| `ui_control.py` | Voice navigation | (emits `ui.navigate` event) |
|
| 38 |
| `dismiss_ui.py` | Close GenUI overlay | (emits `ui.dismiss` event) |
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
## Profile Naming
|
| 41 |
|
| 42 |
- `_locked` suffix: Profile cannot be modified by users
|
| 43 |
- Profile names must match folder names exactly
|
| 44 |
|
| 45 |
-
## Adding New
|
| 46 |
|
| 47 |
-
1. Create
|
| 48 |
-
2.
|
| 49 |
-
3.
|
| 50 |
-
4.
|
| 51 |
|
| 52 |
## Dependencies & Connections
|
| 53 |
|
|
|
|
| 20 |
|
| 21 |
| Tool File | Purpose | Emits GenUI |
|
| 22 |
| ---------------------------- | --------------------------- | ---------------------------------------------------- |
|
| 23 |
+
| `show_onboarding_step.py` | Onboarding wizard step | `OnboardingProgress` |
|
| 24 |
+
| `show_onboarding_review.py` | Onboarding review/edit | `OnboardingReview` |
|
| 25 |
+
| `get_onboarding_status.py` | Check onboarding state | `OnboardingProgress` |
|
| 26 |
| `complete_onboarding.py` | Finish onboarding | `OnboardingSummary`, `ProfileCard` |
|
| 27 |
| `setup_medication.py` | Add medication to schedule | `MedicationCard` |
|
| 28 |
+
| `update_my_medication.py` | Edit existing medication | `MedicationCard`, `MyMedsList` |
|
| 29 |
| `start_medication_entry.py` | Begin med log | `MedLog` |
|
| 30 |
| `update_medication_entry.py` | Update med fields | `MedLog` |
|
| 31 |
| `start_headache_entry.py` | Begin headache log | `HeadacheLog` |
|
| 32 |
| `update_headache_entry.py` | Update headache fields | `HeadacheLog` |
|
| 33 |
| `save_current_entry.py` | Save entry to DB | `ConfirmationCard` |
|
| 34 |
+
| `discard_current_entry.py` | Discard in-progress entry | β |
|
| 35 |
| `entry_control.py` | Entry list/selection | `EntryCard`, `OptionsPicker` |
|
| 36 |
| `log_entry.py` | Quick log (med or headache) | `MedStatus`, `QuickReply` |
|
| 37 |
| `get_recent_entries.py` | History view | `HistoryTimeline` |
|
|
|
|
| 40 |
| `update_settings.py` | Voice settings update | `SettingsConfirmation` + `ui.settings_updated` event |
|
| 41 |
| `ui_control.py` | Voice navigation | (emits `ui.navigate` event) |
|
| 42 |
| `dismiss_ui.py` | Close GenUI overlay | (emits `ui.dismiss` event) |
|
| 43 |
+
| `get_current_datetime.py` | Get current date/time | β |
|
| 44 |
+
| `sweep_look.py` | Robot sweep look animation | β |
|
| 45 |
+
| `custom_tool.py` | Custom tool base/template | β |
|
| 46 |
|
| 47 |
## Profile Naming
|
| 48 |
|
| 49 |
- `_locked` suffix: Profile cannot be modified by users
|
| 50 |
- Profile names must match folder names exactly
|
| 51 |
|
| 52 |
+
## Adding New Profile Tools
|
| 53 |
|
| 54 |
+
1. Create tool `.py` file in the profile directory
|
| 55 |
+
2. Register in `tools.txt`
|
| 56 |
+
3. If tool emits GenUI β create component in `frontend/src/registry/` and register in `index.tsx`
|
| 57 |
+
4. Update this `AGENT.md`
|
| 58 |
|
| 59 |
## Dependencies & Connections
|
| 60 |
|
src/reachy_mini_conversation_app/prompts/AGENT.md
CHANGED
|
@@ -8,10 +8,19 @@ System prompts and personality fragments for the conversation agent. These are l
|
|
| 8 |
|
| 9 |
```
|
| 10 |
prompts/
|
| 11 |
-
βββ default_prompt.txt
|
| 12 |
-
βββ passion_for_lobster_jokes.txt
|
| 13 |
-
βββ behaviors/
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
```
|
| 16 |
|
| 17 |
## Usage
|
|
@@ -35,4 +44,4 @@ Prompts are loaded by `prompts.py` and combined with profile-specific instructio
|
|
| 35 |
- **Used by**: `openai_realtime.py`, `handler_factory.py` β injected into LLM session
|
| 36 |
- **Profile link**: `profiles/_reachy_mini_minder_locked_profile/instructions.txt` contains the main profile prompt
|
| 37 |
- **Dynamic assembly**: `session_enrichment.py` may inject context-aware sections at session start
|
| 38 |
-
- **Frontend impact**: Prompt changes affect LLM
|
|
|
|
| 8 |
|
| 9 |
```
|
| 10 |
prompts/
|
| 11 |
+
βββ default_prompt.txt # Base system prompt
|
| 12 |
+
βββ passion_for_lobster_jokes.txt # Personality fragment
|
| 13 |
+
βββ behaviors/ # Behavioral modifiers
|
| 14 |
+
βββ identities/ # Identity/persona definitions
|
| 15 |
+
βββ sections/ # Modular prompt sections
|
| 16 |
+
βββ core_identity.txt # Core identity definition
|
| 17 |
+
βββ health_logging.txt # Health logging instructions
|
| 18 |
+
βββ music_volume.txt # Music & volume control
|
| 19 |
+
βββ onboarding.txt # Onboarding flow instructions
|
| 20 |
+
βββ reporting.txt # Report generation rules
|
| 21 |
+
βββ safety_rules.txt # Safety & medical guardrails
|
| 22 |
+
βββ tool_budget.txt # Tool call budgets
|
| 23 |
+
βββ ui_navigation.txt # UI navigation rules
|
| 24 |
```
|
| 25 |
|
| 26 |
## Usage
|
|
|
|
| 44 |
- **Used by**: `openai_realtime.py`, `handler_factory.py` β injected into LLM session
|
| 45 |
- **Profile link**: `profiles/_reachy_mini_minder_locked_profile/instructions.txt` contains the main profile prompt
|
| 46 |
- **Dynamic assembly**: `session_enrichment.py` may inject context-aware sections at session start
|
| 47 |
+
- **Frontend impact**: Prompt changes affect LLM behaviour, which indirectly affects tool calls and GenUI output
|
src/reachy_mini_conversation_app/tools/AGENT.md
CHANGED
|
@@ -6,24 +6,30 @@ LLM tool definitions for the Mini Minder conversation agent. These tools are reg
|
|
| 6 |
|
| 7 |
## Structure & Navigation
|
| 8 |
|
| 9 |
-
| File
|
| 10 |
-
| --------------------------
|
| 11 |
-
| `check_medication.py`
|
| 12 |
-
| `
|
| 13 |
-
| `
|
| 14 |
-
| `
|
| 15 |
-
| `
|
| 16 |
-
| `
|
| 17 |
-
| `
|
| 18 |
-
| `
|
| 19 |
-
| `
|
| 20 |
-
| `
|
| 21 |
-
| `
|
| 22 |
-
| `
|
| 23 |
-
| `
|
| 24 |
-
| `
|
| 25 |
-
| `
|
| 26 |
-
| `
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
**Note:** Additional clinical tools live in `profiles/_reachy_mini_minder_locked_profile/` (see `profiles/AGENT.md`).
|
| 29 |
|
|
|
|
| 6 |
|
| 7 |
## Structure & Navigation
|
| 8 |
|
| 9 |
+
| File | Purpose | Emits GenUI | Emits Event |
|
| 10 |
+
| -------------------------- | ---------------------------------- | ------------------------------------------- | ----------------------- |
|
| 11 |
+
| `check_medication.py` | Medication check-in | `DailySnapshot`, `CheckInCard`, `MedStatus` | `ui.component` |
|
| 12 |
+
| `check_health_patterns.py` | Health pattern analysis | β | β |
|
| 13 |
+
| `query_health_history.py` | Query health history | `TrendChart` | `ui.component` |
|
| 14 |
+
| `core_tools.py` | Shared tool utilities | β | β |
|
| 15 |
+
| `media_control.py` | Consolidated play/pause/stop music | `MusicPicker`, `NowPlaying` | `ui.component` |
|
| 16 |
+
| `list_music.py` | List available music | `MusicPicker` | `ui.component` |
|
| 17 |
+
| `play_music.py` | Play a specific track | `NowPlaying` | `ui.component` |
|
| 18 |
+
| `pause_music.py` | Pause playback | `NowPlaying` | `ui.component` |
|
| 19 |
+
| `resume_music.py` | Resume playback | `NowPlaying` | `ui.component` |
|
| 20 |
+
| `stop_music.py` | Stop playback | `NowPlaying` | `ui.component` |
|
| 21 |
+
| `set_volume.py` | Adjust volume | `VolumeControl` | `ui.component` |
|
| 22 |
+
| `session_summary.py` | End-of-session summary | `SessionSummary` | `ui.component` |
|
| 23 |
+
| `end_session.py` | End current session | β | `session.end_requested` |
|
| 24 |
+
| `play_emotion.py` | Trigger robot expressions | β | β |
|
| 25 |
+
| `robot_expression.py` | Robot expression utilities | β | β |
|
| 26 |
+
| `move_head.py` | Control robot head | β | β |
|
| 27 |
+
| `dance.py` | Start dance sequences | β | β |
|
| 28 |
+
| `stop_dance.py` | Cancel ongoing dance | β | β |
|
| 29 |
+
| `stop_emotion.py` | Cancel ongoing emotion | β | β |
|
| 30 |
+
| `camera.py` | Camera control | β | β |
|
| 31 |
+
| `head_tracking.py` | Face/object tracking | β | β |
|
| 32 |
+
| `do_nothing.py` | Explicit no-op | β | β |
|
| 33 |
|
| 34 |
**Note:** Additional clinical tools live in `profiles/_reachy_mini_minder_locked_profile/` (see `profiles/AGENT.md`).
|
| 35 |
|
tests/AGENT.md
CHANGED
|
@@ -6,15 +6,25 @@ Test suite for Mini Minder. Uses pytest with fixtures for database mocking.
|
|
| 6 |
|
| 7 |
## Structure & Navigation
|
| 8 |
|
| 9 |
-
| Test File
|
| 10 |
-
| ----------------------------- | --------------------------------- | ---------------------------------- |
|
| 11 |
-
| `test_database.py`
|
| 12 |
-
| `test_database_onboarding.py`
|
| 13 |
-
| `test_minder_tools.py`
|
| 14 |
-
| `test_onboarding_tools.py`
|
| 15 |
-
| `test_openai_realtime.py`
|
| 16 |
-
| `
|
| 17 |
-
| `
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
## Development Workflow
|
| 20 |
|
|
@@ -34,7 +44,7 @@ pytest tests/ --cov=src/reachy_mini_conversation_app
|
|
| 34 |
- **Test location**: `tests/`
|
| 35 |
- **Fixtures**: `conftest.py` provides `test_db` fixture
|
| 36 |
- **Markers**: None currently defined
|
| 37 |
-
- **Special cases**: Audio tests require fixtures in `tests/audio/`
|
| 38 |
|
| 39 |
## Dependencies & Connections
|
| 40 |
|
|
|
|
| 6 |
|
| 7 |
## Structure & Navigation
|
| 8 |
|
| 9 |
+
| Test File | Tests Module | Key Assertions |
|
| 10 |
+
| ---------------------------------- | --------------------------------------------- | ---------------------------------- |
|
| 11 |
+
| `test_database.py` | `database.py` | CRUD operations, schema migrations |
|
| 12 |
+
| `test_database_onboarding.py` | `database.py` (onboarding) | Profile creation, onboarding flow |
|
| 13 |
+
| `test_minder_tools.py` | `tools/*.py` | Tool schemas, execution, DB writes |
|
| 14 |
+
| `test_onboarding_tools.py` | `profiles/_locked_profile/` tools | Onboarding step progression |
|
| 15 |
+
| `test_openai_realtime.py` | `openai_realtime.py` | OpenAI Realtime session lifecycle |
|
| 16 |
+
| `test_update_settings.py` | `profiles/_locked_profile/update_settings.py` | Settings update tool |
|
| 17 |
+
| `test_memory_graph.py` | `memory_graph.py` | Neo4j knowledge graph ops |
|
| 18 |
+
| `test_memory_notes.py` | `memory_notes.py` | Session memory notes |
|
| 19 |
+
| `test_graph_query_engine.py` | `graph_query_engine.py` | Neo4j query engine |
|
| 20 |
+
| `test_pattern_detector.py` | `pattern_detector.py` | Health pattern detection |
|
| 21 |
+
| `test_pii_guard.py` | `pii_guard.py` | PII redaction |
|
| 22 |
+
| `test_session_enrichment.py` | `session_enrichment.py` | Session enrichment pipeline |
|
| 23 |
+
| `test_session_logging.py` | Session logging | Session log persistence |
|
| 24 |
+
| `test_output_escaping.py` | Output escaping | XSS/injection prevention |
|
| 25 |
+
| `test_robot_control_api.py` | `robot_control_api.py` | Robot control REST API |
|
| 26 |
+
| `test_wakeword_and_end_session.py` | `wakeword_detector.py`, `end_session.py` | Wakeword + session end lifecycle |
|
| 27 |
+
| `conftest.py` | β | Shared fixtures (`test_db`, mocks) |
|
| 28 |
|
| 29 |
## Development Workflow
|
| 30 |
|
|
|
|
| 44 |
- **Test location**: `tests/`
|
| 45 |
- **Fixtures**: `conftest.py` provides `test_db` fixture
|
| 46 |
- **Markers**: None currently defined
|
| 47 |
+
- **Special cases**: Audio tests require fixtures in `tests/audio/`, vision tests in `tests/vision/`
|
| 48 |
|
| 49 |
## Dependencies & Connections
|
| 50 |
|