TESTING.md

Testing Guide (ibe-pp)

This project ships with a complete test suite covering cloned apps and full-session flows. Tests never modify oTree core and run against the local project only.

Test Types

• Unit-lite: config and import sanity checks (no DB writes).
• App bots (oTree): app-level flows in each cloned app (tests.py inside the app).
• E2E sessions (pytest): spins up full sessions via otree test <session> using fresh SQLite DBs.
• Matrix/infra checks: participant-count matrix, rooms/labels, static assets.

Layout

• policy_*/tests.py – Bot specs per app:
  • policy_public_goods_defaults, policy_trust_framed, policy_dictator_norms, policy_guess_anchoring, policy_survey_biases
  • Patterns used: expect() to assert payoffs and SubmissionMustFail(...) to test validation failures (see policy_survey_biases).
• payment_info/tests.py – No-op bot to satisfy multi-app sessions.
• tests/test_integration_settings.py – Verifies session configs and app folders.
• tests/test_e2e_bots.py – Runs E2E bots for configured sessions (policy_nudges, anchoring_demo, survey_biases_full).

Survey Biases (policy_survey_biases) modules and tests

• Module A (Information and Views): randomized order order_a ∈ {info_before, info_after} with a comprehension check. Bots cover both branches and include a negative case via SubmissionMustFail for a wrong comprehension answer.
• Module B (Statements and Evidence): randomized order order_b ∈ {support_first, eval_first}. Bots cover both branches.
• Module C (Topics and News): randomized order order_c ∈ {recall_first, opinion_first} with validation requiring ~20 characters for each recall text. Bots assert negative cases (short input) then submit valid responses.
• Results page safely handles optional fields using field_maybe_none(...) to avoid null access during rendering.

Running just the survey biases suite

• make otree-test-survey (fresh SQLite DB) or:
  • source scripts/activate_venv.sh && cd otree-projects/ibe-pp && OTREE_DATABASE_URL=sqlite:///tmp_survey.sqlite3 otree test survey_biases_full
• tests/test_session_matrix_and_rooms.py – Participant-count matrix (valid/invalid), rooms label file presence, and static file presence.
• tests/test_static_http.py – Optional HTTP check for /_static/custom.css when RUN_OTREE_HTTP_TESTS=1.

How To Run

• Quick run (recommended):
  • make test-ibe-pp (pytest over tests/)
  • make test-otree-e2e (serial bot runs across key sessions)
• Manual (from project root):
  • source scripts/activate_venv.sh
  • cd otree-projects/ibe-pp && pytest -q tests
• Single E2E:
  • cd otree-projects/ibe-pp && pytest -q tests/test_e2e_bots.py::test_bots_policy_nudges_sequence
• Optional HTTP static check:
  • RUN_OTREE_HTTP_TESTS=1 cd otree-projects/ibe-pp && pytest -q tests/test_static_http.py

Dependencies

• Added to root requirements.txt: pytest, requests, httpx, starlette==0.14.1.
• Pin rationale: oTree 5.11.4 bots expect Starlette 0.14.x; newer Starlette breaks bot TestClient.
• If running FastAPI tooling simultaneously, consider a dedicated test requirements file or separate venv.

E2E Strategy

• Tests set OTREE_DATABASE_URL=sqlite:///test_db_<name>.sqlite3 to isolate DBs.
• If you run otree test manually and see “delete your database”, remove db.sqlite3 or set OTREE_DATABASE_URL.
• Participant-count matrix asserts success/failure for group-size compatibility.

Adding Tests

• New app: add tests.py alongside __init__.py with a PlayerBot(Bot) and yields for each page.
• Use expect() to assert key state (e.g., payoffs), and SubmissionMustFail(...) for invalid input branches.
• New session: add a test in tests/test_e2e_bots.py invoking run_otree_test('<session_name>').

Common Issues

• Missing dependencies for bots: install requests and httpx (already in requirements.txt).

• Name collisions: cloned apps set unique NAME_IN_URL values; keep this when adding apps.

• Timeouts: prisoner.Introduction now uses get_timeout_seconds; override with session.config['prisoner_intro_timeout'] in settings.py if needed, for example:

  SESSION_CONFIGS = [
      dict(
          name='classic_baseline',
          app_sequence=['sequence_welcome', 'prisoner', 'trust_simple', 'public_goods_simple', 'payment_info'],
          num_demo_participants=6,
          prisoner_intro_timeout=30,  # seconds (optional)
      ),
  ]