docs/QUESTION_GENERATION.md

# How question generation works

The Oracle's golden rule: **the model never decides anything factual.** The
deterministic engine chooses *which* attribute to ask about; the model's only job
is to phrase that attribute as a friendly question. And even that phrasing is done
**once at boot and cached**, so during a game there are no model calls at all.

Files involved: `engine.py` (picks the attribute), `question_maker.py` (phrases &
caches), `llm.py` (runs the model via llama.cpp), `app.py` (the turn loop).

## The turn loop

```mermaid
flowchart TD
    A[answers so far] --> B[engine.filter_candidates]
    B --> C{how many left?}
    C -->|1| G[guess]
    C -->|0| T[discovery / teach]
    C -->|many| D[engine.choose_attribute]
    D --> E[question_maker.make_question  ← instant cache lookup]
    E --> F[ask the player]
```

Each turn, `app.py:next_turn()`:

1. Rebuilds the answer facts and calls `engine.filter_candidates()` to narrow the
   candidate list (pure Python, exact).
2. If 1 candidate remains → guess; if 0 → discovery/teach; otherwise:
3. `engine.choose_attribute()` picks the **unused attribute whose yes/no split is
   closest to 50/50** (maximum information gain — it halves the field each time).
4. `question_maker.make_question(category, attribute)` returns the text for that
   attribute — an instant dictionary lookup.

So the "AI question" the player sees is really: *engine chooses the attribute →
the cache returns the model's phrasing of it.*

## Where the model actually runs: boot-time pre-generation

There are only ~42 possible questions (one per attribute, per category). Calling a
model live every turn would be slow on a CPU Space, so instead `app.py` kicks off a
background thread at startup:

```python
def _boot_warm():
    llm.warmup()                       # download + load the GGUF once
    question_maker.prewarm_questions() # generate & cache all questions
```

`prewarm_questions()` loops over every `category:attribute`, asks the model to
phrase it, and writes the result to a cache file (`questions_cache.json`, stored in
`ORACLE_DATA_DIR` / the bucket so it survives restarts). It logs timing per
question and a total, e.g.:

```
[question_maker] animal:long_tail (3.4s): Does this animal have a long tail?
[question_maker] generated 42 questions in 138.0s (3.3s each)
[question_maker] question cache ready: 42 questions (+42 new), style v3
```

After the first successful boot the cache is full and persisted, so every later
restart is instant.

## The prompt

`question_maker.py` asks for **simple, clear, kid-friendly** yes/no questions. It
passes the attribute's plain-English meaning (from `discovery.ATTR_MEANING`) rather
than the raw attribute key, forbids using the attribute word itself, and gives a
few tone examples. Temperature is low (0.3) for predictable phrasing.

```
Turn the fact below into ONE simple yes/no question for a kids' guessing game.
... The {category} either {meaning} — or not. Ask about exactly that.
```

### Cache versioning

`CACHE_VERSION` (e.g. `"v3"`) is stored in the cache file. Whenever the prompt or
style changes, bumping this constant makes `_cache()` discard the old entries so
they regenerate on the next boot — no manual cleanup needed.

## Running the model: `llm.py`

`question_maker` calls `llm.chat(...)`, which runs the model through the
**llama.cpp** runtime in one of two ways:

- **In-process** (`llama-cpp-python`) — loads the GGUF directly in the Python
  process. This is the path used on the Space (no server needed).
- **HTTP** (`llama-server`) — a fallback for local dev.

`llm.py` also caps threads (CPU Spaces over-report cores), wraps each call in a
timeout, warms the model at boot, and logs the active mode
(`🟢 MODE = IN-PROCESS llama.cpp …`).

## Always-works fallback

If the model is disabled (`ORACLE_QUESTION_LLM=0`), unavailable, or hasn't filled
the cache yet, `make_question()` returns the built-in phrasing from
`engine.ATTR_QUESTIONS` (e.g. *"Is it a root vegetable?"*). The game is therefore
playable with **no model at all** — the model only upgrades the wording.

## TL;DR

| Concern | Owner |
|--------|-------|
| Which attribute to ask | `engine.choose_attribute` (deterministic) |
| Wording of the question | `question_maker` (model, cached at boot) |
| Running the model | `llm.py` (llama.cpp, in-process) |
| If the model is missing | built-in phrasing fallback (instant) |