tests/e2e_ui/README.md

# E2E UI Testing with Playwright

This directory contains end-to-end (E2E) UI tests for Atom using Playwright Python 1.58.0. The tests validate critical user workflows across authentication, agent chat, canvas presentations, skills, and workflows.

## Overview

**Technology Stack:**
- **Playwright Python 1.58.0**: Browser automation framework (Chromium)
- **pytest-playwright 0.5.2**: Pytest plugin for Playwright
- **pytest-xdist 3.6.1**: Parallel test execution
- **faker 22.7.0**: Realistic test data generation

**Test Infrastructure:**
- API-first authentication (JWT tokens in localStorage, 10-100x faster than UI login)
- Worker-based database isolation for parallel execution
- Page Object Model for maintainable UI abstractions
- Comprehensive fixture suite (auth, database, API, factory)

## Quick Start

### Prerequisites

1. **Docker Desktop** - For test environment (backend, frontend, PostgreSQL)
   ```bash

   docker --version

   ```

2. **Python 3.11+** - For running pytest
   ```bash

   python --version

   ```

3. **Node.js 18+** - For frontend (if running locally)
   ```bash

   node --version

   ```

### Setup

1. **Start E2E Test Environment**
   ```bash

   # Start Docker Compose services (backend, frontend, PostgreSQL)

   ./scripts/start-e2e-env.sh



   # Verify services are running

   curl http://localhost:8001/health/live  # Backend health check

   curl http://localhost:3001              # Frontend (should load)

   ```

2. **Install Dependencies**
   ```bash

   # Install Python dependencies

   pip install -r backend/requirements.txt



   # Install Playwright browsers

   playwright install chromium

   ```

3. **Verify Installation**
   ```bash

   # Check Playwright version

   playwright --version

   # Expected output: Version 1.58.0



   # Run smoke tests to verify setup

   pytest backend/tests/e2e_ui/tests/test_smoke.py -v

   ```

## Running Tests

### Run All E2E Tests
```bash

# Run all E2E tests sequentially

pytest backend/tests/e2e_ui/ -v



# Run with 4 parallel workers (faster)

pytest backend/tests/e2e_ui/ -v -n 4

```

### Run Specific Test Files
```bash

# Run smoke tests only

pytest backend/tests/e2e_ui/tests/test_smoke.py -v



# Run authentication tests

pytest backend/tests/e2e_ui/tests/test_auth_example.py -v



# Run database isolation tests

pytest backend/tests/e2e_ui/tests/test_database_isolation.py -v

```

### Run with Markers
```bash

# Run only E2E tests (skip unit/integration tests)

pytest backend/tests/e2e_ui/ -v -m e2e



# Run authentication tests only

pytest backend/tests/e2e_ui/ -v -m auth



# Skip slow tests

pytest backend/tests/e2e_ui/ -v -m "not slow"

```

### Run with Debugging
```bash

# Run with headful browser (see UI)

pytest backend/tests/e2e_ui/tests/test_smoke.py::test_playwright_browser_launches -v --headed



# Run with Playwright Inspector (debug mode)

pytest backend/tests/e2e_ui/tests/test_smoke.py::test_playwright_browser_launches -v --debug



# Run with screenshots on failure (default)

pytest backend/tests/e2e_ui/tests/test_smoke.py -v --tracing on

```

## Test Structure

### Directory Layout
```

tests/e2e_ui/

├── conftest.py                 # Pytest configuration and fixtures

├── fixtures/                   # Reusable test fixtures

│   ├── auth_fixtures.py        # API-first authentication (JWT in localStorage)

│   ├── database_fixtures.py    # Database session and worker isolation

│   ├── api_fixtures.py         # API setup utilities (setup_test_user, setup_test_project)

│   └── test_data_factory.py    # Factory Boy factories (UserFactory, ProjectFactory)

├── tests/                      # Test files

│   ├── test_smoke.py           # Smoke tests (infrastructure validation)

│   ├── test_auth_example.py    # Authentication test examples

│   ├── test_api_setup_example.py  # API setup test examples

│   └── test_database_isolation.py  # Database isolation tests

└── README.md                   # This file

```

### Fixtures

#### Authentication Fixtures
- **`test_user`**: Creates a test user with UUID v4 email (unique per test)

- **`authenticated_user`**: Creates user and returns (user, JWT token) tuple
- **`authenticated_page`**: Creates Playwright page with JWT token in localStorage (bypasses UI login)

- **`admin_user`**: Creates admin user with elevated permissions

#### Database Fixtures
- **`db_session`**: SQLAlchemy session with worker-specific schema isolation

- **`clean_database`**: Fresh database tables per test (function-scoped)

#### API Fixtures
- **`setup_test_user`**: Creates test user via API
- **`setup_test_project`**: Creates test project via API
- **`api_client_authenticated`**: HTTP client with pre-set Authorization header

#### Factory Fixtures
- **`UserFactory`**: Factory Boy factory for test users
- **`ProjectFactory`**: Factory Boy factory for test projects

### Example Test

```python

import pytest

from playwright.sync_api import Page



def test_user_login_flow(authenticated_page: Page):

    """Test user can access protected route after authentication."""

    # Navigate to dashboard (JWT token already set in localStorage)

    authenticated_page.goto("/dashboard")



    # Verify dashboard loads (no redirect to login)

    assert authenticated_page.locator("h1").contains("Dashboard")



    # Verify user menu shows logged-in user

    authenticated_page.click("button[data-testid='user-menu']")

    assert authenticated_page.locator("text=Logout").is_visible()

```

## Troubleshooting

### Port Conflicts

**Issue**: `Error: listen EADDRINUSE :8001` or `:3001`

**Solution**: Kill process using the port
```bash

# Kill backend on port 8001

lsof -ti:8001 | xargs kill -9



# Kill frontend on port 3001

lsof -ti:3001 | xargs kill -9



# Or use different ports in docker-compose-e2e.yml

```

### Database Connection Issues

**Issue**: `sqlalchemy.exc.OperationalError: could not connect to server`

**Solution**: Verify PostgreSQL container is running
```bash

# Check container status

docker ps | grep postgres



# Restart PostgreSQL container

docker restart atom-e2e-postgres



# Check connection

docker exec atom-e2e-postgres psql -U atom -d atom_test -c "SELECT 1;"

```

### Playwright Browser Not Found

**Issue**: `Executable doesn't exist at /path/to/chromium`

**Solution**: Install Playwright browsers
```bash

playwright install chromium



# Or install all browsers

playwright install

```

### Frontend Not Loading

**Issue**: `Error: connect ECONNREFUSED localhost:3001`

**Solution**: Verify frontend container is running
```bash

# Check container status

docker ps | grep frontend



# View frontend logs

docker logs atom-e2e-frontend



# Restart frontend container

docker restart atom-e2e-frontend

```

### Tests Timing Out

**Issue**: Tests timeout after 30 seconds (Playwright default)

**Solution**: Increase timeout for specific tests
```python

@pytest.mark.timeout(60)

def test_slow_operation(authenticated_page: Page):

    authenticated_page.goto("/slow-page")

    authenticated_page.wait_for_selector("text=Loaded", timeout=30000)

```

### JWT Token Not Working

**Issue**: Tests redirect to login despite authenticated_page fixture



**Solution**: Verify token format and localStorage keys

```python

# Debug: Check localStorage in test

def test_debug_auth(authenticated_page: Page):
    token = authenticated_page.evaluate("() => localStorage.getItem('auth_token')")

    print(f"Token: {token}")  # Should not be None


    # Verify backend accepts token

    response = authenticated_page.request.get("/api/v1/users/me", headers={

        "Authorization": f"Bearer {token}"

    })

    assert response.ok

```


## Best Practices

### 1. Use API-First Setup
Always use `authenticated_page` fixture instead of UI login:
```python

# Good: 10-100x faster

def test_authenticated_access(authenticated_page: Page):

    authenticated_page.goto("/dashboard")

    # Already logged in via JWT token



# Bad: Slow and fragile

def test_ui_login(page: Page):

    page.goto("/login")

    page.fill("input[name='email']", "test@example.com")

    page.fill("input[name='password']", "password")

    page.click("button[type='submit']")

    # Waits for navigation, slower and less reliable

```

### 2. Use data-testid Selectors
Prefer `data-testid` attributes over CSS selectors:
```python

# Good: Resilient to CSS changes

authenticated_page.click("button[data-testid='submit-button']")



# Bad: Breaks when CSS classes change

authenticated_page.click(".btn.btn-primary.submit")

```

### 3. Keep Tests Independent
Each test should create its own data (no shared state):
```python

# Good: Isolated test data

def test_user_can_create_project(authenticated_page: Page, setup_test_project):

    project = setup_test_project(name="My Project")

    authenticated_page.goto(f"/projects/{project['id']}")



# Bad: Relies on data from other tests

def test_user_can_edit_project(authenticated_page: Page):

    # Assumes project was created in previous test

    authenticated_page.goto("/projects/1")  # Fragile!

```

### 4. Use Explicit Waits
Avoid hard-coded sleeps, use Playwright's auto-waiting:
```python

# Good: Waits for element to be ready

authenticated_page.click("button[data-testid='submit']")

authenticated_page.wait_for_selector("text=Success")



# Bad: Arbitrary sleep time

authenticated_page.click("button[data-testid='submit']")

time.sleep(2)  # Flaky!

```

### 5. Run Tests in Parallel
Use pytest-xdist for faster execution:
```bash

# Run with 4 workers

pytest backend/tests/e2e_ui/ -v -n 4



# Each worker gets isolated database schema (gw0, gw1, gw2, gw3)

```

## Performance Targets

- **Per test**: <30 seconds
- **Full suite**: <10 minutes (with 4 parallel workers)
- **Authentication**: <100ms (API-first vs 2-10s UI login)
- **Database isolation**: <50ms per test setup

## CI/CD Integration

To run E2E tests in CI/CD (GitHub Actions, GitLab CI, etc.):

```yaml

# .github/workflows/e2e-tests.yml

name: E2E Tests



on: [push, pull_request]



jobs:

  e2e:

    runs-on: ubuntu-latest

    steps:

      - uses: actions/checkout@v3



      - name: Start E2E environment

        run: ./scripts/start-e2e-env.sh



      - name: Install dependencies

        run: |

          pip install -r backend/requirements.txt

          playwright install chromium



      - name: Run E2E tests

        run: pytest backend/tests/e2e_ui/ -v -n 4



      - name: Upload screenshots

        if: failure()

        uses: actions/upload-artifact@v3

        with:

          name: screenshots

          path: backend/tests/e2e_ui/screenshots/



      - name: Upload videos

        if: failure()

        uses: actions/upload-artifact@v3

        with:

          name: videos

          path: backend/tests/e2e_ui/videos/

```

## Bug Discovery Fixture Reuse

Bug discovery tests (fuzzing, chaos, property tests, browser discovery) reuse fixtures from this directory to avoid duplication and ensure consistency.

**For comprehensive fixture documentation, see:**
- **[Bug Discovery Fixture Reuse Guide](../bug_discovery/FIXTURE_REUSE_GUIDE.md)** - Complete guide to reusing fixtures in bug discovery tests

**Quick Import Reference:**

```python

# Import authentication fixtures (10-100x faster than UI login)

from tests.e2e_ui.fixtures.auth_fixtures import test_user, authenticated_user, authenticated_page



# Import database fixtures (worker-based isolation for parallel execution)

from tests.e2e_ui.fixtures.database_fixtures import db_session



# Import API fixtures (HTTP client with pre-set auth headers)

from tests.e2e_ui.fixtures.api_fixtures import setup_test_user, setup_test_project, api_client_authenticated



# Import factory fixtures (Factory Boy for test data)

from tests.e2e_ui.fixtures.test_data_factory import user_factory, agent_factory, skill_factory



# Import page objects (Page Object Model for maintainable UI tests)

from tests.e2e_ui.pages.page_objects import LoginPage, DashboardPage, ChatPage

```

**Example: Browser Discovery Test Using Fixtures**

```python

from tests.e2e_ui.fixtures.auth_fixtures import authenticated_page

from playwright.sync_api import Page



@pytest.mark.browser

def test_console_errors_on_dashboard(authenticated_page: Page):

    """Discover console errors on dashboard (API-first auth = 10-100x faster)."""

    authenticated_page.goto("http://localhost:3001/dashboard")  # Already authenticated!



    # Check for console errors

    errors = authenticated_page.evaluate("() => window.consoleErrors || []")

    assert len(errors) == 0, f"Console errors: {errors}"

```

**Bug Discovery Test Directories:**
- `backend/tests/fuzzing/` - Atheris fuzzing tests
- `backend/tests/browser_discovery/` - Playwright bug discovery tests
- `backend/tests/chaos/` - Chaos engineering tests
- `backend/tests/property_tests/` - Hypothesis property-based tests

**See Also:**
- [Bug Discovery Templates](../bug_discovery/TEMPLATES/) - Test documentation templates
- [TEST_QUALITY_STANDARDS.md](../../docs/TEST_QUALITY_STANDARDS.md) - Test quality requirements

## Additional Resources

- **Playwright Documentation**: https://playwright.dev/python/
- **pytest-playwright Plugin**: https://pytest-playwright.readthedocs.io/
- **Factory Boy**: https://factoryboy.readthedocs.io/
- **pytest-xdist**: https://pytest-xdist.readthedocs.io/

## Comprehensive Guide

For detailed E2E testing documentation covering all three platforms (web, mobile, desktop), see:
- **[E2E Testing Guide](../../../../docs/E2E_TESTING_GUIDE.md)** - Comprehensive guide with platform-specific patterns, CI/CD integration, troubleshooting, and reference documentation.

## Status

**Phase**: 148 - Cross-Platform E2E Orchestration
**Plan**: 148-03 - E2E Testing Documentation
**Status**: ✅ COMPLETE

**Completed Tasks**:
- ✅ Playwright 1.58.0 installed
- ✅ All Wave 1 fixtures integrated
- ✅ pytest.ini configured for E2E test discovery
- ✅ Smoke test suite created
- ✅ Developer documentation (README.md)
- ✅ Comprehensive E2E testing guide (docs/E2E_TESTING_GUIDE.md)

**Test Coverage**:
- Agent execution: Spawn, chat, streaming, governance (11 tests)
- Canvas presentation: Charts, forms, accessibility trees (46 tests)
- Authentication: Login, logout, session management (8 tests)
- Skills: Skill execution, validation, governance (15 tests)

**Next Steps**:
- Phase 148-04: E2E test execution and CI/CD integration