docs/ARCHITECTURE.md

# Architecture

Chess Club is a FastAPI + Socket.IO application. Domain logic splits across four named layers; infrastructure is SQLite + optional Redis + Google Gemini API + two chess engines.

---

## The four-layer split

### Soul — `app/agents/soul.py`

The LLM agent. Runs after the engine move every character turn. Receives a structured `SoulInput` bundle: a `BoardSummary` (prose description of the position — not raw FEN), the current `MoodState`, surfaced memories from the Subconscious, the engine move just played, any pending player chat, and recent match history. Returns a `SoulResponse` (Pydantic schema in `app/schemas/agents.py`) containing:

- `speak` — nullable; silence is the default (most moves are silent)
- `emotion` + `emotion_intensity` — drives the character video/indicator in the UI
- `mood_deltas` — ±0.1 per axis, applied to raw mood after the Soul returns
- `note_about_opponent` — queued for post-match memory generation
- `save_memory` — optional `InlineMemoryRequest`; if set, `app/memory/inline_save.py` persists it asynchronously mid-match
- `referenced_memory_ids` — validated as a subset of what the Subconscious surfaced
- `game_action` — controls pre-match room flow (`start_game`, `start_match_vs_kenji`, `propose_goal`); always `"none"` during an active match

Does **not** pick chess moves. LLM failures degrade silently to a neutral empty response so gameplay never stalls.

The Soul also has lightweight variants for agent contexts: `run_agent_soul_in_match()` (agent responding to player chat mid-match), `run_agent_soul_in_match_move()` (agent's own move turn), and `run_agent_soul_for_room()` (agent pre-match room).

### Subconscious — `app/agents/subconscious.py`

Memory retrieval agent. Runs **before** the Soul, concurrently with the engine, on the post-player-move board position (not the post-engine position). Multi-axis scoring across all memories for the character or agent:

| Axis | Weight |
|---|---|
| Semantic similarity (cosine on 384-dim embeddings) | 0.35 |
| Trigger keyword match | 0.25 |
| Opponent relevance | 0.15 |
| Mood alignment | 0.10 |
| Recency penalty | −0.15 |

If the top-1 score beats top-2 by > 0.15, returns the top 5 directly. Otherwise sends the top 8 to Gemini Flash Lite for a structured re-rank with one-sentence retrieval reasons per memory. Per-match cache with a 3-turn TTL keyed by `(last_player_uci, chat_hash, mood_polarity_bucket)` prevents redundant LLM re-ranks on identical context.

Returns 0–5 `SurfacedMemory` objects for the Soul's context window.

### Director — `app/director/`

Deterministic code; no LLM calls. Entry point: `choose_engine_config(character, mood, match_context)` in `app/director/director.py`. Computes:

- **Effective Elo** = `current_elo + 100·confidence − 150·tilt`, clamped to `[floor_elo, max_elo]`
- **Engine selection**: Stockfish beast-mode when `effective_elo > 2100` or `aggression ≥ 9 ∧ confidence > 0.7`; Maia-2 for 1100–1900 range; Stockfish fallback below Maia's range or when Maia-2 isn't installed
- **Time budget**: 0.5s (patience=1) → 5s (patience=10), discounted by aggression, capped at 8s

Mood state (`MoodState(aggression, confidence, tilt, engagement)`, all `[0,1]`) is managed by `app/director/mood.py`. Exponential smoothing with τ = 3 moves. Persisted to Redis keyed by match ID; falls back to in-process dict if Redis is absent. The Soul's `mood_deltas` are applied via `apply_deltas()` then `smooth_mood()` after each character turn — the Soul nudges mood, the Director reads it.

Elo ratchet logic lives in `app/director/elo.py`: ±10% of signed outcome delta per match, clamped ±30; floor rises +25 after 3 consecutive matches where current Elo > floor + 100.

### Body — `app/engine/`

The chess engines. Abstract interface: `ChessEngine` ABC + `EngineConfig` + `MoveResult` in `app/engine/base.py`. Three implementations:

- **`maia2_engine.py`** — Maia-2 rapid neural network, 1100–1900 Elo band, human-like piece preference. Requires `torch` + weights (~300 MB, cached in `MAIA2_CACHE_DIR`).
- **`stockfish_engine.py`** — Stockfish UCI, variable Elo via `UCI_Elo`, returns top-3 candidate moves with centipawn evaluations.
- **`mock_engine.py`** — First legal move, fully deterministic. Always available; used in tests and dev-without-engines.

Engine lifecycle: `app/engine/registry.py`. Board → prose summary: `board_to_english()` in `app/engine/board_abstraction.py`. This function is why the Soul never receives raw FEN — it gets material counts, castling rights, pinned/hanging pieces, king safety flags, and game phase as readable English. Prevents LLM chess hallucinations (the Soul making up moves it claims to play).

Shuffle guard: `app/engine/diversity_guard.py::filter_shuffle_moves()` filters candidates that return a piece to a square it recently left (lookback: 6 plies). Applied to Maia's candidate list before the final move is chosen.

---

## How a single match flows

Two orchestration paths share the Soul/Subconscious/engine infrastructure.

### Human-vs-character

**Key files:** `app/sockets/server.py` (event handler), `app/sockets/room_server.py` (per-match room), `app/matches/streaming.py` (turn loop)

1. Client connects to Socket.IO `/play` namespace with `player_id` cookie; `server.py` authenticates, joins the socket to room `match:<match_id>`.
2. Client emits `make_move {uci}`.
3. `room_server.py` validates the session is a participant (not spectator), acquires `match_lock(match_id)` from `app/concurrency/locks.py`.
4. Calls `apply_player_move_streamed(match, move_input, emitters)` in `app/matches/streaming.py`:
   - Validates move legality against `chess.Board`; raises `IllegalMove` if invalid.
   - Applies player move; emits `player_move_applied`.
   - Emits `agent_thinking` with eta hint: `engine_time_budget + agent_thinking_soul_overhead_seconds` (default 1.5s), rounded to 0.5s. This is a soft estimate, not a hard timeout.
   - Launches two concurrent `asyncio` tasks: `run_subconscious(...)` on the current board, and `engine.get_move(config)`.
   - When Subconscious completes → emits `memory_surfaced` (memory ribbon populates while engine still thinking).
   - When engine completes → emits `agent_move` (board updates).
   - `run_soul(character, soul_input)` with engine move + surfaced memories → emits `agent_chat` (if `speak` is non-null) and `mood_update`.
   - Applies `mood_deltas` from Soul; re-smooths; persists.
   - Checks terminal state; if game over → emits `match_ended`, starts post-match processor.
5. Post-match processor (`app/post_match/processor.py`) runs in a background daemon thread. Five steps: engine analysis → feature extraction → Elo ratchet → memory generation → narrative summary. Status streams back via `asyncio.run_coroutine_threadsafe` to the main event loop (`app/sockets/processor_callback.py`). Clients that reconnected can fall back to polling `GET /api/matches/{id}/post_match_status`.

The `memory_surfaced` → `agent_move` ordering is the headline UX property: the memory ribbon populates during the "thinking" state rather than after the move lands.

### Agent-vs-character

**Key files:** `app/sockets/room_server.py` (deploy trigger), `app/matches/agent_streaming.py` (server-side loop)

The match is fully automated. After the player deploys their agent from the agent room, `room_server.py` creates the match and fires `asyncio.create_task(run_agent_match(...))`. The watching player is a spectator in their own match — they see events but don't make moves.

The loop in `agent_streaming.py` alternates:
- **Agent's turn**: engine picks a move from the agent's Elo configuration; `run_agent_soul_in_match_move()` generates chat. Auto-resign triggers at `RESIGN_THRESHOLD_CP = −500` centipawns for 3 consecutive turns.
- **Character's turn**: the same `_run_engine_and_agents` pipeline imported from `streaming.py` — identical Soul + Subconscious + engine sequence as the human path.

Both sides emit events to the `match:<match_id>` Socket.IO room. `agent_streaming.py` imports private helpers from `streaming.py` (`_run_engine_and_agents`, `_finalize_outcome`, `_persist_engine_move`, etc.) — this is the main coupling between the two flows.

### Pre-match rooms

Pre-match chat (human talking to a character before starting): Socket.IO `/room` namespace, `app/sockets/room_server.py`. The `SoulResponse.game_action` field drives match initiation — when the Soul returns `start_game` (character-side room) or `start_match_vs_kenji` (agent-side room), the server emits `game_started {match_id, redirect_url}` and the browser redirects automatically.

---

## Memory model

### Storage

`Memory` rows (`app/models/memory.py`) carry either `character_id` or `agent_id` — exactly one is non-null. Character memories belong to a character (preset or user-created); agent memories belong to the player's agent. No cross-ownership.

Each memory has: `narrative` (prose, 1–3 sentences), `triggers` (keyword list, ≥3 required), `valence` (float sentiment), `scope` (enum: `character_lore` / `opponent_specific` / `cross_player`), `embedding` (JSON array, 384 floats), `surface_count` (retrieval counter).

### Generation

Three paths:

1. **Preset seeding** (`app/characters/memory_generator.py`): LLM generates 30–50 memories at first startup from backstory, voice, and quirks. Embeddings created inline at save. Configurable via `memory_gen_target` / `memory_gen_min` / `memory_gen_max` in `app/config.py`.

2. **Inline (mid-match)** (`app/memory/inline_save.py`): `SoulResponse.save_memory` carries an `InlineMemoryRequest`. If set, an `asyncio.create_task` fires and persists the memory without blocking the turn. The turn returns before the save completes — this is fire-and-forget.

3. **Post-match** (`app/post_match/memory_gen.py`): LLM generates 1–3 `MATCH_RECAP` / `OPPONENT_SPECIFIC` memories after each game. Validates ≥3 triggers per memory; retries once with explicit feedback if validation fails. Deduplicates against existing memories at 0.92 cosine similarity threshold before persisting.

### Retrieval

`app/agents/subconscious.py` (see above). Vector search is in-Python cosine similarity via NumPy in `app/memory/vector_store.py` — no external vector DB. The interface (`upsert`, `search`, `get_embedding`) is designed to be swapped to sqlite-vec without changing any callers.

---

## Streaming architecture

| Namespace | File | Purpose |
|---|---|---|
| `/room` | `app/sockets/room_server.py` | Pre-match chat (both character rooms and agent rooms) |
| `/play` | `app/sockets/server.py` + `room_server.py` | In-match gameplay, post-match streaming |
| Lobby/matchmaking | `app/sockets/lobby_server.py` | PvP lobby and matchmaking queue |
| Presence | `app/sockets/presence_server.py` | Live online count |

All LLM work follows the same pattern — this is the non-negotiable rule for new features:

1. Receive socket event; validate and authenticate immediately (synchronous, fast).
2. Acquire per-session or per-match lock from `app/concurrency/locks.py`.
3. Launch `asyncio.create_task(...)` for LLM work — **never** `await` the LLM call inline on the event handler.
4. Emit events as work completes.

The synchronous-HTTP path (`app/matches/service.py::apply_player_move`) was replaced in Phase 3b specifically because inline `await` on LLM calls blocked all other socket events. That antipattern must not come back.

---

## Concurrency primitives

`app/concurrency/locks.py` — two dict registries of `asyncio.Lock` objects:

- `chat_session_lock(session_id)` — serializes LLM calls within a single pre-match chat session
- `match_lock(match_id)` — serializes character turns within a match; prevents concurrent `make_move` events from racing

Both are context managers (`async with match_lock(match_id):`). Both are in-process dict-backed.

**Multi-worker swap seam:** The module docstring spells it out: replace the `dict[str, asyncio.Lock]` with a Redis `SET NX EX` pattern using `redis.asyncio`. The call sites don't change — they only see the context manager interface. All state that would break under multi-worker (disconnect registry, spectator counts, Socket.IO room membership) has the same swap point.

Memory note: the dicts grow monotonically — one Lock per unique session/match ID, never freed. At current scale (dozens of concurrent sessions) negligible. At scale, add TTL eviction.

---

## Feature flag system

Three boolean settings in `app/config.py`, loaded via pydantic-settings from env vars:

| `Settings` field | Env var | Default |
|---|---|---|
| `show_player_agents` | `SHOW_PLAYER_AGENTS` | `True` |
| `show_clay_economy` | `SHOW_CLAY_ECONOMY` | `False` |
| `show_agent_autonomy` | `SHOW_AGENT_AUTONOMY` | `False` |

**Route guard pattern** (`app/web/routes.py`):
```python
def _require_player_agents() -> None:
    if not get_settings().show_player_agents:
        raise HTTPException(status_code=404)

# Applied with:
@router.get("/agents", dependencies=[Depends(_require_player_agents)])
```
Routes that should be hidden return 404 (not 403 — the flag makes the feature non-existent, not forbidden).

**Template injection** (module level in `app/web/routes.py`):
```python
templates.env.globals["feature_flags"] = {
    "show_player_agents": _ff.show_player_agents,
    "show_clay_economy": _ff.show_clay_economy,
    "show_agent_autonomy": _ff.show_agent_autonomy,
}
```
Templates access `feature_flags.show_player_agents` to conditionally render nav links, wager panels, and autonomy controls.

**Adding a new flag:** (1) Add a `bool` field to `Settings` in `app/config.py`. (2) Inject into `templates.env.globals` in `app/web/routes.py`. (3) Write a `_require_X()` guard function. (4) Apply `Depends(_require_X)` on hidden routes.

---

## Seam-swap interfaces

These are the three places where Lucas needs to wire Chess Club into the wider Metropolis platform. They're designed to be reimplemented without touching call sites.

### `ClayLedger` — `app/economy/clay_ledger.py`

Current implementation: `SqliteClayLedger` — all wagering in Chess Club's own SQLite DB. Abstract interface:

```python
class ClayLedger:
    def get_balance(self, player_id: str) -> int: ...
    def debit(self, player_id, amount, reason, match_id=None) -> ClayTransaction: ...
    def credit(self, player_id, amount, reason, match_id=None) -> ClayTransaction: ...
    def transfer(self, from_id, to_id, amount, reason, match_id=None) -> tuple[...]: ...
    def transactions_for_player(self, player_id, limit=50, reason=None) -> list[...]: ...
```

All call sites import via `from app.economy.clay_ledger import get_ledger, InsufficientFunds`. `get_ledger()` returns the module-level singleton initialized in `app/main.py`. To swap: subclass `ClayLedger`, implement the five methods, update `get_ledger()`. No other files change.

Two likely replacement implementations:
- **`MetropolisInternalLedger`**: delegates to Bhaven's internal balance API over HTTP
- **`OnChainClayLedger`**: calls a Polygon ERC-20 contract (ABI from PolygonScan, `transfer` → on-chain)

### `PlayerAgent.metropolis_token_id` — `app/models/player_agent.py:31`

Currently `nullable=True`, always null. Intended to bind a player agent to a Metropolis NFT when MetropolisX integration lands:

1. Contract address and ABI from Bhaven
2. On agent creation: `balanceOf(owner_wallet)` → list token IDs → `tokenURI(tokenId)` → fetch metadata JSON → extract `image` and `traits`
3. Store `token_id` as `metropolis_token_id` (String(80)); use metadata `image` as `avatar_image_url`; inject `traits` into personality generation

### Auth seam — `app/auth.py`

Currently cookie-based. Cookie name: `PLAYER_COOKIE` (plain player UUID, no signing by default; `SESSION_SECRET` enables signing). Two entry points used by all route handlers:

- `require_player(request)` — raises 401 (API) or redirects to `/login` (HTML) if unauthenticated
- `get_optional_player(request)` — returns `None` for unauthenticated requests

To swap to JWT/SSO with the Metropolis platform: replace these two functions in `app/auth.py`. No other files need to change.

---

## Tech stack reference

| Package | Min version | Role |
|---|---|---|
| Python | 3.11 | Runtime |
| FastAPI | 0.115 | HTTP framework + dependency injection |
| SQLAlchemy | 2.0 | ORM + session factory |
| Pydantic v2 | 2.7 | Schema validation; structured LLM output |
| Alembic | 1.13 | Schema migrations (Phase 3a+) |
| python-socketio | 5.11 | Real-time Socket.IO server |
| google-genai | 0.3 | Gemini API client (Soul + memory generation) |
| sentence-transformers | 2.7 | Memory embeddings (`all-MiniLM-L6-v2`, 384 dims) |
| chess | 1.10 | Board state, move legality, PGN parsing |
| maia2 | 0.1 | Human-like engine (1100–1900 Elo) |
| Stockfish (Python) | 3.28 | High-skill engine + post-match analysis |
| argon2-cffi | 23.1 | Password hashing (Phase 4 email/password auth) |
| Redis | 5.0 | Mood persistence (optional; in-process fallback when absent) |
| NumPy | 1.24 | Cosine similarity for memory retrieval |

LLM model: configured via `gemini_model` in `app/config.py` (current default: `gemini-3.1-flash-lite-preview`). Change there to upgrade.