docs/proactive_adaptive_system.md

Adaptive Proactive Messaging System — Design Doc

Last updated: 2026-04-21

---

Overview

The proactive pipeline currently sends one fixed-schedule daily message per user with a cycling topic pattern. This doc describes the full adaptive system discussed to replace it: per-user optimal send time, topic drill-down based on engagement, character weighting, and a feedback loop that adjusts both time and topic automatically.

---

1. Trigger architecture change

Current

GitHub Action → daily cron 07:00 UTC → trigger_proactive Space → /proactive/run (all users)

New

GitHub Action → hourly cron '0 * * * *' → trigger_proactive Space → /proactive/run
    → reads proactive_schedule table
    → sends only to users whose send_hour_utc == current_hour

The GitHub Action fires every hour. Each hour it queries proactive_schedule for users scheduled at that UTC hour and runs the pipeline only for those users.

Why hourly trigger instead of per-user cron:

• Simpler infrastructure — one cron, one trigger Space
• The per-user variation is handled in the application layer, not the scheduler
• Easy to add/remove users without touching the cron

---

2. Onboarding — age capture

At first login, after collecting name and country, ask the user's age.

• Store in user_profiles.age (new field, integer)
• If user skips: mark as null, fall back to default slot
• Age is used only for time prediction — not surfaced in conversation

---

3. Default send time prediction

Age + living country define a best-guess local hour for the first proactive message. If either is missing, use the global fallback slot (19:00 local).

Age × lifestyle mapping

Age range	Assumed lifestyle	Preferred slots (local)
13–22	Student	16:00–19:00, 21:00–23:00
23–35	Young professional	07:30–08:30, 12:30, 20:00–22:00
36–55	Working adult	07:00–08:00, 12:00–13:00, 20:00–21:00
56–70	Pre/post retirement	08:00–10:00, 15:00–16:00
70+	Retired	09:00–10:00, 14:00–15:00

Country → timezone mapping

Use pytz or a country→UTC offset table to convert local slot → UTC hour stored in proactive_schedule.send_hour_utc.

Fallback

If age or country missing: default to 19:00 local (or 19:00 UTC if timezone unknown).

---

4. proactive_schedule table (new)

create table proactive_schedule (
    user_id          uuid primary key references auth.users(id),
    send_hour_utc    smallint not null default 19,  -- 0–23
    timezone         text,                           -- e.g. 'Australia/Sydney'
    local_hour       smallint,                       -- cached local equivalent
    no_response_streak  smallint default 0,          -- consecutive misses
    last_sent_at     timestamptz,
    last_responded_at timestamptz,
    created_at       timestamptz default now(),
    updated_at       timestamptz default now()
);

---

5. Topic pool — full list

The proactive pipeline cycles through these types (replacing the current 3-type pattern):

Type	Source	Description
news	Tavily search	News on user's current interest topic (see drill-down §6)
personal_event	life_events table	Follow up on a specific recorded life event
personal_dialogic	past_dialogues / FAISS	Reopen an unresolved topic from chat history
story	story_chapters_*.json	Follow up on the current chapter of the user's active story
philosophy_thread	philosophy_threads_all.json	Reopen an unresolved philosophical thread

Character selection (§7) determines which character sends each message.

---

6. News topic drill-down

The system starts from the user's broad interests and narrows based on engagement.

Data model — topic_drill_down (new column in proactive_state or new table)

{
  "current_path": ["sport", "soccer", "Roma"],
  "engagement_history": {
    "sport":  { "sent": 4, "responded": 3 },
    "soccer": { "sent": 2, "responded": 2 },
    "Roma":   { "sent": 1, "responded": 1 }
  }
}

Drill-down logic

Start: broad interest from user_profile.interests  (e.g. "sport")
    │
    ▼
User responds / engages with the content
    │
    ▼
Extract the most specific topic mentioned in the response
(LLM call: "What specific sub-topic did the user engage with? e.g. sport → soccer")
    │
    ▼
Save sub-topic as next level in path
    │
    ▼
Next news query uses the more specific topic ("soccer" instead of "sport")
    │
    ▼
Repeat: "soccer" → "Roma" → "Roma transfer news"

Drill-up logic (if engagement drops)

If 2 consecutive messages at a specific level get no response → step back up one level.

"Roma" × 2 no response → fall back to "soccer"
"soccer" × 2 no response → fall back to "sport"

Implementation note

The drill-down path is stored per-user in proactive_state.topic_drill_down (JSONB column). The Tavily query uses current_path[-1] (the most specific active topic).

---

7. Character selection

Track which character the user interacts with most and weight proactive messages toward that character.

Source

chat_history_short already stores character_id on each assistant message. Count per character over the last N sessions.

Logic

# count assistant messages per character in short-term history
counts = Counter(
    m["character_id"]
    for m in history
    if m.get("role") == "assistant" and m.get("character_id")
)
preferred_character = counts.most_common(1)[0][0] if counts else "socrates"

Store result in proactive_state.character_id (already exists). Recalculate on each proactive run (cheap, no extra DB call needed).

---

8. Feedback loop — adaptive rules

8a. Time adaptation

Signal	Action
User responds to proactive message	Record last_responded_at; if response time is consistently at a different hour, shift send_hour_utc toward that hour (±1h per session)
No response (after 2h window)	Increment no_response_streak; shift send_hour_utc +1
no_response_streak reaches 24	Full reset: try a completely different slot (±6h from current)
User responds after a miss	Reset no_response_streak to 0

Response detection: A response is counted when a user message is saved to chat_history_short within 2 hours of a proactive message's sent_at timestamp in proactive_messages.

8b. Topic adaptation — news drill-down

Covered in §6. In summary:

• Engagement → drill down (more specific topic)
• 2× no engagement → drill up (broader topic)
• Engagement score stored in topic_drill_down.engagement_history

8c. Topic type rotation

If a certain message type (news, philosophy_thread, etc.) consistently gets no response over 3 consecutive sends, deprioritise it by reducing its weight in the TYPE_PATTERN cycling.

---

9. How proactive/run changes

def run_proactive_for_scheduled_users(current_hour_utc: int):
    """Called every hour. Only runs for users scheduled at this UTC hour."""
    resp = (
        supabase.table("proactive_schedule")
        .select("user_id")
        .eq("send_hour_utc", current_hour_utc)
        .execute()
    )
    for row in (resp.data or []):
        run_proactive_pipeline(row["user_id"], ...)

The /proactive/run endpoint receives the current UTC hour from the trigger and passes it to run_proactive_for_scheduled_users.

---

10. Implementation order (suggested)

Step	What	Effort
1	Add age field to user_profiles, ask at onboarding	Small
2	Create proactive_schedule table in Supabase	Small
3	Populate schedule on signup using age + country heuristic	Small
4	Change cron to hourly, update /proactive/run to filter by hour	Small
5	Add story and philosophy_thread to topic pool	Medium
6	Add character weighting from chat history	Small
7	Add topic_drill_down JSONB to proactive_state, wire drill-down logic	Medium
8	Add response detection + time shift feedback loop	Medium
9	Add topic type deprioritisation on 3× miss	Small

Steps 1–4 can ship independently and already improve the system significantly. Steps 5–9 can follow incrementally.