docs/QUESTION_GENERATION.md

How question generation works

The Oracle's golden rule: the model never decides anything factual. The deterministic engine chooses which attribute to ask about; the model's only job is to phrase that attribute as a friendly question. And even that phrasing is done once at boot and cached, so during a game there are no model calls at all.

Files involved: engine.py (picks the attribute), question_maker.py (phrases & caches), llm.py (runs the model via llama.cpp), app.py (the turn loop).

The turn loop

flowchart TD
    A[answers so far] --> B[engine.filter_candidates]
    B --> C{how many left?}
    C -->|1| G[guess]
    C -->|0| T[discovery / teach]
    C -->|many| D[engine.choose_attribute]
    D --> E[question_maker.make_question  ← instant cache lookup]
    E --> F[ask the player]

Each turn, app.py:next_turn():

1. Rebuilds the answer facts and calls engine.filter_candidates() to narrow the candidate list (pure Python, exact).
2. If 1 candidate remains → guess; if 0 → discovery/teach; otherwise:
3. engine.choose_attribute() picks the unused attribute whose yes/no split is closest to 50/50 (maximum information gain — it halves the field each time).
4. question_maker.make_question(category, attribute) returns the text for that attribute — an instant dictionary lookup.

So the "AI question" the player sees is really: engine chooses the attribute → the cache returns the model's phrasing of it.

Where the model actually runs: boot-time pre-generation

There are only ~42 possible questions (one per attribute, per category). Calling a model live every turn would be slow on a CPU Space, so instead app.py kicks off a background thread at startup:

def _boot_warm():
    llm.warmup()                       # download + load the GGUF once
    question_maker.prewarm_questions() # generate & cache all questions

prewarm_questions() loops over every category:attribute, asks the model to phrase it, and writes the result to a cache file (questions_cache.json, stored in ORACLE_DATA_DIR / the bucket so it survives restarts). It logs timing per question and a total, e.g.:

[question_maker] animal:long_tail (3.4s): Does this animal have a long tail?
[question_maker] generated 42 questions in 138.0s (3.3s each)
[question_maker] question cache ready: 42 questions (+42 new), style v3

After the first successful boot the cache is full and persisted, so every later restart is instant.

The prompt

question_maker.py asks for simple, clear, kid-friendly yes/no questions. It passes the attribute's plain-English meaning (from discovery.ATTR_MEANING) rather than the raw attribute key, forbids using the attribute word itself, and gives a few tone examples. Temperature is low (0.3) for predictable phrasing.

Turn the fact below into ONE simple yes/no question for a kids' guessing game.
... The {category} either {meaning} — or not. Ask about exactly that.

Cache versioning

CACHE_VERSION (e.g. "v3") is stored in the cache file. Whenever the prompt or style changes, bumping this constant makes _cache() discard the old entries so they regenerate on the next boot — no manual cleanup needed.

Running the model: llm.py

question_maker calls llm.chat(...), which runs the model through the llama.cpp runtime in one of two ways:

• In-process (llama-cpp-python) — loads the GGUF directly in the Python process. This is the path used on the Space (no server needed).
• HTTP (llama-server) — a fallback for local dev.

llm.py also caps threads (CPU Spaces over-report cores), wraps each call in a timeout, warms the model at boot, and logs the active mode (🟢 MODE = IN-PROCESS llama.cpp …).

Always-works fallback

If the model is disabled (ORACLE_QUESTION_LLM=0), unavailable, or hasn't filled the cache yet, make_question() returns the built-in phrasing from engine.ATTR_QUESTIONS (e.g. "Is it a root vegetable?"). The game is therefore playable with no model at all — the model only upgrades the wording.

TL;DR

Concern	Owner
Which attribute to ask	engine.choose_attribute (deterministic)
Wording of the question	question_maker (model, cached at boot)
Running the model	llm.py (llama.cpp, in-process)
If the model is missing	built-in phrasing fallback (instant)