# Adaptive Proactive Messaging System — Design Doc _Last updated: 2026-04-21_ --- ## Overview The proactive pipeline currently sends one fixed-schedule daily message per user with a cycling topic pattern. This doc describes the full adaptive system discussed to replace it: per-user optimal send time, topic drill-down based on engagement, character weighting, and a feedback loop that adjusts both time and topic automatically. --- ## 1. Trigger architecture change ### Current ``` GitHub Action → daily cron 07:00 UTC → trigger_proactive Space → /proactive/run (all users) ``` ### New ``` GitHub Action → hourly cron '0 * * * *' → trigger_proactive Space → /proactive/run → reads proactive_schedule table → sends only to users whose send_hour_utc == current_hour ``` The GitHub Action fires every hour. Each hour it queries `proactive_schedule` for users scheduled at that UTC hour and runs the pipeline only for those users. **Why hourly trigger instead of per-user cron:** - Simpler infrastructure — one cron, one trigger Space - The per-user variation is handled in the application layer, not the scheduler - Easy to add/remove users without touching the cron --- ## 2. Onboarding — age capture At first login, after collecting name and country, ask the user's age. - Store in `user_profiles.age` (new field, integer) - If user skips: mark as `null`, fall back to default slot - Age is used only for time prediction — not surfaced in conversation --- ## 3. Default send time prediction Age + living country define a best-guess local hour for the first proactive message. If either is missing, use the global fallback slot (19:00 local). ### Age × lifestyle mapping | Age range | Assumed lifestyle | Preferred slots (local) | |-----------|------------------|------------------------| | 13–22 | Student | 16:00–19:00, 21:00–23:00 | | 23–35 | Young professional | 07:30–08:30, 12:30, 20:00–22:00 | | 36–55 | Working adult | 07:00–08:00, 12:00–13:00, 20:00–21:00 | | 56–70 | Pre/post retirement | 08:00–10:00, 15:00–16:00 | | 70+ | Retired | 09:00–10:00, 14:00–15:00 | ### Country → timezone mapping Use `pytz` or a country→UTC offset table to convert local slot → UTC hour stored in `proactive_schedule.send_hour_utc`. ### Fallback If age or country missing: default to 19:00 local (or 19:00 UTC if timezone unknown). --- ## 4. proactive_schedule table (new) ```sql create table proactive_schedule ( user_id uuid primary key references auth.users(id), send_hour_utc smallint not null default 19, -- 0–23 timezone text, -- e.g. 'Australia/Sydney' local_hour smallint, -- cached local equivalent no_response_streak smallint default 0, -- consecutive misses last_sent_at timestamptz, last_responded_at timestamptz, created_at timestamptz default now(), updated_at timestamptz default now() ); ``` --- ## 5. Topic pool — full list The proactive pipeline cycles through these types (replacing the current 3-type pattern): | Type | Source | Description | |------|--------|-------------| | `news` | Tavily search | News on user's current interest topic (see drill-down §6) | | `personal_event` | `life_events` table | Follow up on a specific recorded life event | | `personal_dialogic` | `past_dialogues` / FAISS | Reopen an unresolved topic from chat history | | `story` | `story_chapters_*.json` | Follow up on the current chapter of the user's active story | | `philosophy_thread` | `philosophy_threads_all.json` | Reopen an unresolved philosophical thread | **Character selection (§7)** determines which character sends each message. --- ## 6. News topic drill-down The system starts from the user's broad interests and narrows based on engagement. ### Data model — topic_drill_down (new column in proactive_state or new table) ```json { "current_path": ["sport", "soccer", "Roma"], "engagement_history": { "sport": { "sent": 4, "responded": 3 }, "soccer": { "sent": 2, "responded": 2 }, "Roma": { "sent": 1, "responded": 1 } } } ``` ### Drill-down logic ``` Start: broad interest from user_profile.interests (e.g. "sport") │ ▼ User responds / engages with the content │ ▼ Extract the most specific topic mentioned in the response (LLM call: "What specific sub-topic did the user engage with? e.g. sport → soccer") │ ▼ Save sub-topic as next level in path │ ▼ Next news query uses the more specific topic ("soccer" instead of "sport") │ ▼ Repeat: "soccer" → "Roma" → "Roma transfer news" ``` ### Drill-up logic (if engagement drops) If 2 consecutive messages at a specific level get no response → step back up one level. ``` "Roma" × 2 no response → fall back to "soccer" "soccer" × 2 no response → fall back to "sport" ``` ### Implementation note The drill-down path is stored per-user in `proactive_state.topic_drill_down` (JSONB column). The Tavily query uses `current_path[-1]` (the most specific active topic). --- ## 7. Character selection Track which character the user interacts with most and weight proactive messages toward that character. ### Source `chat_history_short` already stores `character_id` on each assistant message. Count per character over the last N sessions. ### Logic ```python # count assistant messages per character in short-term history counts = Counter( m["character_id"] for m in history if m.get("role") == "assistant" and m.get("character_id") ) preferred_character = counts.most_common(1)[0][0] if counts else "socrates" ``` Store result in `proactive_state.character_id` (already exists). Recalculate on each proactive run (cheap, no extra DB call needed). --- ## 8. Feedback loop — adaptive rules ### 8a. Time adaptation | Signal | Action | |--------|--------| | User responds to proactive message | Record `last_responded_at`; if response time is consistently at a different hour, shift `send_hour_utc` toward that hour (±1h per session) | | No response (after 2h window) | Increment `no_response_streak`; shift `send_hour_utc` +1 | | `no_response_streak` reaches 24 | Full reset: try a completely different slot (±6h from current) | | User responds after a miss | Reset `no_response_streak` to 0 | **Response detection:** A response is counted when a user message is saved to `chat_history_short` within 2 hours of a proactive message's `sent_at` timestamp in `proactive_messages`. ### 8b. Topic adaptation — news drill-down Covered in §6. In summary: - Engagement → drill down (more specific topic) - 2× no engagement → drill up (broader topic) - Engagement score stored in `topic_drill_down.engagement_history` ### 8c. Topic type rotation If a certain message type (`news`, `philosophy_thread`, etc.) consistently gets no response over 3 consecutive sends, deprioritise it by reducing its weight in the TYPE_PATTERN cycling. --- ## 9. How proactive/run changes ```python def run_proactive_for_scheduled_users(current_hour_utc: int): """Called every hour. Only runs for users scheduled at this UTC hour.""" resp = ( supabase.table("proactive_schedule") .select("user_id") .eq("send_hour_utc", current_hour_utc) .execute() ) for row in (resp.data or []): run_proactive_pipeline(row["user_id"], ...) ``` The `/proactive/run` endpoint receives the current UTC hour from the trigger and passes it to `run_proactive_for_scheduled_users`. --- ## 10. Implementation order (suggested) | Step | What | Effort | |------|------|--------| | 1 | Add `age` field to `user_profiles`, ask at onboarding | Small | | 2 | Create `proactive_schedule` table in Supabase | Small | | 3 | Populate schedule on signup using age + country heuristic | Small | | 4 | Change cron to hourly, update `/proactive/run` to filter by hour | Small | | 5 | Add `story` and `philosophy_thread` to topic pool | Medium | | 6 | Add character weighting from chat history | Small | | 7 | Add `topic_drill_down` JSONB to `proactive_state`, wire drill-down logic | Medium | | 8 | Add response detection + time shift feedback loop | Medium | | 9 | Add topic type deprioritisation on 3× miss | Small | Steps 1–4 can ship independently and already improve the system significantly. Steps 5–9 can follow incrementally.