initial_docs/docs/EVAL_AND_TESTING.md

Evaluation and Testing

Testing philosophy

The frontend is part of the product. The simulation is only successful if it can be manually played and browser-tested.

Tests should verify real state transitions, not only UI existence.

Every stage must include:

• unit tests for core logic,
• schema/validation tests where relevant,
• frontend tests,
• browser-level interaction tests,
• manual verification instructions,
• regression checks for previous demos.

Manual evaluation target

A five-minute run should produce visible behavior without requiring user interaction.

Target events:

• consumption or attempted consumption,
• inspection/discovery,
• avoidance/fear/conflict,
• rest/shelter/comfort-seeking if resources demand it,
• memory or belief update,
• response to environment change,
• no long stuck loops.

Frontend test requirements

Frontend tests should interact with the app like a user.

They should verify:

• grid renders,
• play/pause/reset works,
• tick count changes,
• resources change over time,
• objects can be placed,
• user can move/speak,
• creature moves/actions update state,
• event log receives real events,
• Mind Lens displays real state,
• notebook/export works,
• model provenance/compliance is visible.

Canvas/SVG visuals must be backed by inspectable state.

If a visual changes but state does not, the feature is not complete.

Stage-level frontend interactions

Each stage should include at least one browser-level test or scripted manual test for the main user-facing behavior added in that stage.

Examples:

• Stage 1: start grid, tick, pause, reset.
• Stage 2: generated map has reachable objects and creature acts in simple preset world.
• Stage 3: user creates object from text, places it, sees schema/provenance.
• Stage 4: slow cognition updates Mind Lens and changes behavior modifiers.
• Stage 5: demo mode runs and exports summary.

Model tests

LLM tests should verify:

• request uses approved model,
• request uses approved provider target,
• fallbacks are disabled,
• strict JSON schema is present,
• invalid output is rejected,
• valid output is accepted,
• frontend never sees API key,
• final-submission mode blocks prototype-only model.

Map generator tests

Map tests should verify:

• creature spawn reachable,
• user spawn reachable,
• at least one useful resource reachable,
• at least one shelter reachable,
• at least one risky/ambiguous object exists,
• hazards do not block all paths,
• map seed is deterministic,
• generated maps vary across seeds.

Memory/event tests

Memory tests should verify:

• events are emitted by real state transitions,
• recent events are derived from event log,
• object memory valence updates from outcomes,
• rolling memory text stays within limit,
• beliefs update from event evidence,
• slow cognition cannot invent direct state changes.

Fast/slow interaction tests

Tests should verify:

• slow directive changes priority modifiers or target bias,
• fast policy may follow safe directive,
• fast policy may reject unsafe directive,
• user “come here” is social signal, not forced movement,
• high fear can override social approach,
• critical hunger can override weak curiosity.

Acceptance standard

A stage is not done unless:

• app runs locally,
• new behavior can be manually verified,
• tests pass,
• frontend interaction is tested,
• debug/Mind Lens state is inspectable,
• docs are updated.