docs/TESTING.md

Testing Guide — Color-UX-Access

HF Space containers cannot run a browser. Playwright is used locally for screenshots but tests are source-inspection only — no live browser in CI.

For test setup (venv, uv install), see docs/DEVELOPMENT.md.

---

Testing Philosophy

Tests use a layered approach, from fast to slow:

Category	Marker	What it tests	Duration
Smoke	smoke	Import, module load, CVD config, Gradio apps loadable	<10s
Pipeline	pipeline	Full image→CVD→WCAG report with mocked VLM	<30s
Slow	slow	Real VLM calls, needs HF_TOKEN	60-90s

Source inspection tests verify structural properties that mocks miss — e.g., isinstance(file_obj, bytes) check for GradioFile binary type, Modal app GPU config, Gradio 6 theme placement in launch() not Blocks().

---

Running Tests

uv run pytest -v                     # all tests (smoke + pipeline) — no network, no GPU
uv run pytest -m smoke               # fast: imports + smoke only (<10s)
uv run pytest -m pipeline            # full pipeline with mocked VLM (<30s)
uv run pytest tests/test_app_space.py -v  # single file
uv run pytest tests/test_app_space.py::test_cvd_gallery_generates_10_types -v  # single test

---

Test Structure

tests/
├── conftest.py              Shared fixtures (img_factory, cvd_img_factory, mock_vlm_factory)
├── fixtures/                Synthetic UI screenshots — generated by generate_fixtures.py
│   ├── button_panel_accessible.png
│   ├── button_panel_color_only.png
│   ├── status_badges_accessible.png
│   ├── status_badges_color_only.png
│   ├── form_validation_accessible.png
│   ├── form_validation_color_only.png
│   └── *.png               Plus deuteranopia/protanopia/tritanopia variants per original
├── test_smoke.py            All core imports work, CVD variants correct, Gradio apps loadable
├── test_capture.py          capture module contract: take_screenshot exists + correct signature
├── test_app_space.py        Integration: CVD gallery, WCAG report, Gradio 6 theme launch
├── test_gradio_binary.py    Source inspection: Gradio File binary type handling
└── test_modal_app.py        Source inspection: Modal app structure (GPU, secrets, base URL)

---

Shared Fixtures (from conftest.py)

All fixtures are auto-discovered by pytest — import them directly in tests.

img_factory

def test_something(img_factory):
    img = img_factory(640, 480, seed=42)  # deterministic random PIL Image

cvd_img_factory

def test_cvd_simulation(cvd_img_factory):
    img, simulate = cvd_img_factory(200, 100, seed=42)
    deut = simulate(img, 'deuteranopia')
    assert deut.size == img.size

mock_vlm_factory

def test_report_passes(mock_vlm_factory):
    result = mock_vlm_factory(passes=True)
    assert result['passes'] is True
    assert result['findings'] == []

sample_button_panel

def test_cvd_confuses_buttons(sample_button_panel, cvd_img_factory):
    img = sample_button_panel
    _, simulate = cvd_img_factory()
    deut = simulate(img, 'deuteranopia')
    # Green (Confirm) and red (Cancel) buttons — visually similar for CVD

ten_type_gallery

def test_gallery_types(ten_type_gallery):
    assert len(ten_type_gallery) == 7

---

TDD Pattern: RED → GREEN → REFACTOR

# 1. Write a failing test
uv run pytest tests/test_capture.py::test_take_screenshot_signature -v
# → RED if not yet implemented

# 2. Implement the feature
# create color_ux_access/capture.py with take_screenshot(page, url, timeout=60000)

# 3. Run again — verify GREEN
uv run pytest tests/test_capture.py -v

# 4. Refactor if needed

For new features:

# tests/test_new_feature.py
def test_feature_does_x():
    from color_ux_access.module import function
    result = function(input)
    assert expected_output in result

---

Source Inspection Tests

Tests that verify source code structure rather than runtime behavior:

# tests/test_gradio_binary.py — verify GradioFile bytes handling
def test_analyze_screenshot_handles_bytes():
    source = pathlib.Path("color_ux_access/modal_app.py").read_text()
    assert "isinstance(file_obj, bytes)" in source

# tests/test_modal_app.py — verify Modal app structure
def test_base_url_correct():
    source = pathlib.Path("color_ux_access/modal_app.py").read_text()
    assert "router.huggingface.co/v1" in source

---

CI Integration

No CI configured yet. For now, run uv run pytest before every commit locally.

# .github/workflows/test.yml (future)
name: Test
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v4
        with:
          python-version: "3.12"
      - run: uv sync --extra dev
      - run: uv run pytest -m smoke

---

Fixture Generation

Generate synthetic UI screenshots for local testing:

PYTHONPATH=. python tests/generate_fixtures.py

Each original image gets CVD-simulated variants (deuteranopia, protanopia, tritanopia). Fixture pairs:

• button_panel_color_only.png vs button_panel_accessible.png — CVD-confusable buttons
• status_badges_color_only.png vs status_badges_accessible.png — CVD-invisible distinctions
• form_validation_color_only.png vs form_validation_accessible.png — CVD-invisible error states