docs/code-assistant/testing-guide.md

# Testing Guide

Essential testing information for code assistants. For comprehensive testing documentation, see [../development/testing.md](../development/testing.md).

## Test Structure

```
tests/
├── unit/{js,fastapi}/         # Unit tests
├── api/v1/                     # API integration tests (*.test.js)
├── e2e/tests/                  # E2E Playwright tests (*.spec.js)
├── backend-test-runner.js      # API test runner
├── e2e-runner.js              # E2E test runner
└── smart-test-runner.js       # Intelligent test selection
```

## Quick Commands

```bash
# Run tests for changed files only (use this most often)
npm run test:changed

# Check which tests would run without executing them
npm run test:changed -- --dry-run
npm run test:changed -- --names-only  # Output only test file names

# Unit tests
npm run test:unit:js          # JavaScript units
npm run test:unit:fastapi     # Python units

# API tests (local FastAPI server, no containers)
npm run test:api              # All API tests
npm run test:api -- --grep "save"    # Filter by file path pattern
npm run test:api -- --grep "files_serve_caching"  # Run specific test file

# E2E tests (Playwright)
npm run test:e2e              # All E2E tests with local server
npm run test:e2e:headed       # Show browser
npm run test:e2e:debug        # Step-through debugging
npm run test:e2e:debug-failure # Capture debug artifacts on failure (headless)
npm run test:e2e -- --grep "should upload"  # Filter by test name pattern

# Cross-browser E2E tests (real browser engines, no login required)
npm run test:e2e:xmleditor-browsers  # xmlTagSync tests in chromium, firefox, webkit

# Container tests (runs all tests inside container)
npm run test:container                                  # Run with cache
npm run test:container -- --no-cache                    # Rebuild all layers
npm run test:container -- path/to/file.js               # Test specific files
npm run test:container -- --browser firefox             # Use specific browser
npm run test:container -- --browser chromium,firefox,webkit  # Test multiple browsers
```

## Test Types

### Unit Tests

- **Location**: `tests/unit/{js,fastapi}/`
- **Purpose**: Test isolated functions/classes
- **Run**: `npm run test:unit`

#### Python Unit Tests with FastAPI Routes

Python unit tests for backend plugins and custom routes use a combination of dependency injection overrides and mocking.

##### Pattern: Testing FastAPI Routes with Mixed Dependencies

Routes often mix FastAPI dependency injection with direct function calls. Test both using:

1. **Dependency Overrides** (in setUp) - for consistent mocks across all tests
2. **@patch decorators** - for functions called inside routes

**Example:**

```python
import unittest
from unittest.mock import MagicMock, patch
from fastapi import FastAPI
from fastapi.testclient import TestClient

class TestMyRoute(unittest.IsolatedAsyncioTestCase):
    """Test my custom route."""

    def setUp(self):
        """Set up test fixtures."""
        from myapp.routes import router
        from myapp.dependencies import get_auth_manager, get_session_manager

        self.app = FastAPI()
        self.app.include_router(router)

        # Create mocks for injected dependencies
        self.mock_session_manager = MagicMock()
        self.mock_auth_manager = MagicMock()

        # Override dependencies - these work for all tests
        self.app.dependency_overrides[get_session_manager] = lambda: self.mock_session_manager
        self.app.dependency_overrides[get_auth_manager] = lambda: self.mock_auth_manager

        self.client = TestClient(self.app)

    @patch("myapp.config.get_settings")  # Patch functions called inside route
    @patch("myapp.routes.get_db")        # Not dependency-injected, called directly
    def test_my_endpoint(self, mock_get_db, mock_settings):
        """Test endpoint with authentication."""
        # Mock settings (called inside route via import)
        mock_settings_obj = MagicMock()
        mock_settings_obj.session_timeout = 3600
        mock_settings.return_value = mock_settings_obj

        # Configure dependency override mocks (set in setUp)
        self.mock_session_manager.is_session_valid.return_value = True
        mock_user = MagicMock()
        self.mock_auth_manager.get_user_by_session_id.return_value = mock_user

        # Mock direct function calls
        mock_get_db.return_value = MagicMock()

        # Make request
        response = self.client.get(
            "/api/my-endpoint",
            params={"session_id": "valid-session"}
        )

        self.assertEqual(response.status_code, 200)
```

**Key Points:**

- **Dependency overrides** (`self.app.dependency_overrides`) handle FastAPI `Depends()` parameters
- **@patch decorators** handle functions imported and called inside routes (`get_settings()`, `get_db()`)
- **Mock at import location**: Patch where the function is used, not where it's defined
  - Route imports `from fastapi_app.config import get_settings` → patch `"myapp.routes.get_settings"`
  - Route imports function from another module → patch at that import path
- **Order matters**: `@patch` decorators apply bottom-to-top, parameters passed in reverse order

**Common Mistakes:**

- Trying to patch dependency-injected functions with `@patch` instead of using `dependency_overrides`
- Patching at definition location instead of import location
- Forgetting to mock functions called inside routes that aren't dependency-injected
- **Calling functions directly instead of using dependency injection** - Routes should use `Depends(get_db)` and `Depends(get_file_storage)` as parameters, not call `db = get_db()` inside the function body. This ensures proper mocking via `dependency_overrides` and prevents database access failures in CI environments where databases may not exist

**Best Practice - Always Use Dependency Injection:**

Routes should declare all dependencies as parameters using `Depends()`:

```python
# ✅ CORRECT - Use dependency injection
@router.get("/export")
async def export_csv(
    pdf: str = Query(...),
    db=Depends(get_db),
    file_storage=Depends(get_file_storage)
):
    file_repo = FileRepository(db)
    # ... use db and file_storage ...

# ❌ WRONG - Direct function calls
@router.get("/export")
async def export_csv(
    pdf: str = Query(...)
):
    db = get_db()  # Can't be mocked via dependency_overrides
    file_storage = get_file_storage()  # Fails in CI if database doesn't exist
```

This pattern:

- Enables proper mocking via `app.dependency_overrides` in tests
- Prevents CI failures when databases/resources don't exist
- Aligns with FastAPI best practices
- Matches patterns in existing routes (edit_history plugin)

**Authentication Testing Pattern:**

When testing routes that require authentication (via `session_id` query parameter or `X-Session-ID` header), set up default authentication in `setUp()`:

```python
def setUp(self):
    """Set up test fixtures."""
    from fastapi import FastAPI
    from myapp.routes import router
    from fastapi_app.lib.core.dependencies import (
        get_auth_manager,
        get_session_manager,
        get_db,
        get_file_storage,
    )

    self.app = FastAPI()
    self.app.include_router(router)

    # Create mocks for dependencies
    self.mock_session_manager = MagicMock()
    self.mock_auth_manager = MagicMock()
    self.mock_db = MagicMock()
    self.mock_storage = MagicMock()

    # Mock valid authentication by default
    self.mock_session_manager.is_session_valid.return_value = True
    self.mock_auth_manager.get_user_by_session_id.return_value = MagicMock(
        username="testuser",
        groups=["*"]  # Wildcard access
    )

    # Override dependencies
    self.app.dependency_overrides[get_session_manager] = lambda: self.mock_session_manager
    self.app.dependency_overrides[get_auth_manager] = lambda: self.mock_auth_manager
    self.app.dependency_overrides[get_db] = lambda: self.mock_db
    self.app.dependency_overrides[get_file_storage] = lambda: self.mock_storage

    self.client = TestClient(self.app)
```

Then include `session_id` in test requests:

```python
response = self.client.get(
    "/api/my-endpoint",
    params={"session_id": "test-session", "other_param": "value"}
)
```

**Why This Pattern:**

- Routes using `Depends(get_session_manager)` and `Depends(get_auth_manager)` require dependency overrides
- Setting up valid authentication in `setUp()` makes all tests pass authentication by default
- Individual tests can override these defaults to test authentication failures
- Must include `session_id` parameter in requests to satisfy route authentication requirements

**See Also:**

- [fastapi_app/plugins/edit_history/tests/test_edit_history_export.py](../../fastapi_app/plugins/edit_history/tests/test_edit_history_export.py) - Complete example with authentication, authorization, and data mocking
- [fastapi_app/plugins/annotation_history/tests/test_annotation_history.py](../../fastapi_app/plugins/annotation_history/tests/test_annotation_history.py) - Routes requiring authentication via dependency injection

### API Integration Tests

- **Location**: `tests/api/v1/`
- **Naming**: `*.test.js`
- **Purpose**: Test backend endpoints without browser
- **Run**: `npm run test:api`
- **Features**: Local server, auto-cleanup, fixtures from `tests/api/fixtures/`

#### SSE (Server-Sent Events) Tests

SSE tests require special handling due to long-lived HTTP connections. When tests maintain multiple SSE connections and need to make additional HTTP requests, the Node.js HTTP connection pool can become exhausted.

**HTTP Connection Pool Configuration:**

SSE test files must increase the HTTP agent's max sockets at the top of the file:

```javascript
import http from 'node:http';

// Increase connection pool to handle SSE connections + regular requests
http.globalAgent.maxSockets = 50;
```

**Why This Is Needed:**

- Node.js HTTP agent defaults to ~5-10 concurrent connections per host
- Each SSE connection holds a connection slot for its entire lifetime
- When multiple tests create SSE connections, these accumulate
- Additional HTTP requests (like POST to trigger events) can timeout if no connections are available
- Increasing `maxSockets` to 50 prevents connection pool exhaustion

**Symptoms of Connection Pool Exhaustion:**

- Tests pass in isolation but hang/timeout in full test suite
- HTTP requests timeout after SSE connections are established
- Error: "Broadcast request timed out after 5s" or similar

**Example Pattern:**

```javascript
import { test, describe } from 'node:test';
import http from 'node:http';
import { createEventSource } from 'eventsource-client';

// CRITICAL: Increase connection pool for SSE tests
http.globalAgent.maxSockets = 50;

describe('SSE Tests', () => {
  test('Broadcast to multiple sessions', async () => {
    // Create SSE connections (long-lived)
    const conn1 = createEventSource({ url: '/sse/subscribe' });
    const conn2 = createEventSource({ url: '/sse/subscribe' });

    // Wait for connections to establish
    await new Promise(resolve => setTimeout(resolve, 2000));

    // Make additional HTTP request - would fail without increased maxSockets
    const response = await fetch('/sse/test/broadcast', { method: 'POST' });

    // Clean up
    conn1.close();
    conn2.close();
  });
});
```

**See Also:** [tests/api/v1/sse.test.js](../../tests/api/v1/sse.test.js) for complete SSE testing patterns.

#### Fixture System

API tests use a two-phase fixture loading system:

**Phase 1: Config Loading** (before server starts)

- Copies JSON config files from `tests/api/fixtures/{fixture-name}/config/` to `tests/api/runtime/config/`
- Config files: `users.json`, `groups.json`, `roles.json`, `collections.json`
- Server reads these during initialization to set up RBAC

**Phase 2: File Import** (after server starts)

- Imports PDF/XML files from `tests/api/fixtures/{fixture-name}/files/` using `bin/import_files.py`
- Files are imported into the `default` collection (matches default group access)
- Creates database entries and stores files in content-addressable hash-sharded structure
- TEI files in the fixture directory are detected as gold standard files and linked to their PDFs

**Available Fixtures**:

- `minimal`: Basic config with minimal users/groups (no files)
- `standard`: Standard config + sample PDF/TEI files for testing

**Access Control Considerations**:

- Fixture files are imported into the `default` collection
- Test users need appropriate group membership to access fixture files
- Standard fixture users (`reviewer`, `annotator`, `user`) are in the `default` group with access to `_inbox` and `default` collections
- `admin` user has wildcard access (`*`) to all collections

**CRITICAL:** Do not alter fixtures

- Tests should NOT permanently modify fixture files or delete them in cleanup - they're shared across tests and automatically cleaned up when the runtime directory is wiped.
- Ideally, make a copy of the fixture file or add new files to alter/delete, at a minimun restore altered files after the test.

### E2E Tests

- **Location**: `tests/e2e/tests/`
- **Naming**: `*.spec.js`
- **Purpose**: Test full workflows in browser
- **Run**: `npm run test:e2e`
- **Features**: Playwright, `window.ui` navigation, `testLog()` for state verification

#### Isolated Component Harness Tests

Some specs test individual components in isolation using a standalone HTML harness page rather than the full application. The harness is served by the dev server and loads only the dependencies needed for the component under test (via the importmap from `app/web/importmap.json`, rebased to absolute paths). No login, no fixture loading, no application state required.

Use this approach when:

- The component has browser-engine-specific behavior (e.g. Lezer parser eagerness differences between Chromium and Firefox)
- You need to reproduce or rule out an editor bug without the overhead of the full application
- Fast cross-browser iteration is needed

**Available harnesses:**

| Harness HTML | Spec | Tests |
|---|---|---|
| `tests/e2e/harness/xmleditor-harness.html` | `tests/e2e/tests/xmleditor-cross-browser.spec.js` | `xmlTagSync` CodeMirror extension |

**Running:**

```bash
# All three browsers
npm run test:e2e:xmleditor-browsers

# Single browser for fast iteration
node tests/e2e-runner.js tests/e2e/tests/xmleditor-cross-browser.spec.js --browser firefox

# Headed to observe behaviour
node tests/e2e-runner.js tests/e2e/tests/xmleditor-cross-browser.spec.js --browser chromium --headed
```

**Adding a new harness:**

1. Create `tests/e2e/harness/<name>-harness.html` — use `xmleditor-harness.html` as the template (contains the importmap rebasing logic).
2. Create `tests/e2e/harness/<name>-harness.js` — import the component under test, expose helpers on `window`, and emit `window.testLog('HARNESS_READY', {})` when ready.
3. Create `tests/e2e/tests/<name>-cross-browser.spec.js` — call `setupTestConsoleCapture` + `waitForTestMessage(logs, 'HARNESS_READY')` before each test.
4. Add an npm script in `package.json` following the `test:e2e:xmleditor-browsers` pattern.

#### Container Testing

**Run Tests Inside Container** (`npm run test:container`):

- **Purpose**: Run all tests in an isolated container environment (same as CI)
- **What it does**:
  - Builds the `ci` container image with all dependencies and test code
  - Runs tests inside the container (not against it)
  - Streams output in real-time to your terminal
  - Uses smart test runner with dependency analysis
  - Supports all test runner options (browsers, grep, specific files)
- **Use when**:
  - Testing changes that require a clean environment
  - Validating behavior in production-like container
  - Reproducing CI failures locally
  - Testing Dockerfile or dependency changes

**Examples:**

```bash
# Run all tests with caching (fastest for iterative testing)
npm run test:container

# Force rebuild (use after Dockerfile or dependency changes)
npm run test:container -- --no-cache

# Test specific files
npm run test:container -- app/src/plugins/authentication.js

# Run E2E tests with specific browser
npm run test:container -- --browser firefox

# Test multiple browsers
npm run test:container -- --browser chromium,firefox,webkit

# Run all tests (skip smart selection)
npm run test:container -- --all
```

**Technical Details:**

- Tests run inside container using `docker/entrypoint-ci.sh`
- All fixtures and test code baked into image at build time
- Real-time streaming via `PYTHONUNBUFFERED=1` and `stdio: 'inherit'`
- Playwright browsers installed early for optimal layer caching
- Exit code matches test results (0 = success, non-zero = failure)

## Test Filtering with --grep

**IMPORTANT:** The `--grep` parameter works differently for API vs E2E tests:

### API Tests (backend-test-runner.js)

`--grep` matches **file paths** (test file names):

```bash
# Matches files containing "save" in the path
npm run test:api -- --grep "save"
# → Runs: tests/api/v1/files_save.test.js

# Matches specific test file
npm run test:api -- --grep "files_serve_caching"
# → Runs: tests/api/v1/files_serve_caching.test.js

# Matches multiple files with common pattern
npm run test:api -- --grep "files_"
# → Runs: all files starting with "files_"
```

**Why:** The backend test runner filters test files before passing them to Node.js test runner.

### E2E Tests (e2e-runner.js / Playwright)

`--grep` matches **test names** (test titles in your code):

```bash
# Matches test titles containing "upload"
npm run test:e2e -- --grep "upload"
# → Runs: test('should upload file', ...), test('upload validation', ...), etc.

# Matches specific test case
npm run test:e2e -- --grep "should create new version"
# → Runs: test('should create new version', ...)

# Case-insensitive regex pattern
npm run test:e2e -- --grep "save.*revision"
# → Runs: test('should save current revision', ...), etc.
```

**Alternative: Run specific test files directly**

Instead of using `--grep` with test names, you can pass test file paths as positional arguments:

```bash
# Run single test file
node tests/e2e-runner.js tests/e2e/tests/app-loading.spec.js

# Run multiple test files
node tests/e2e-runner.js tests/e2e/tests/auth-workflow.spec.js tests/e2e/tests/export-workflow.spec.js

# Combine with options
node tests/e2e-runner.js --headed tests/e2e/tests/auth-workflow.spec.js
```

**Why:** Playwright's `--grep` matches `test()` descriptions, not file paths. The smart-test-runner automatically uses file paths when selecting specific tests.

### Rule of Thumb for Code Assistants

- **Debugging API tests**: Use file name patterns → `--grep "files_save"`
- **Debugging E2E tests**: Use test name patterns → `--grep "should upload"` OR pass file paths directly
- **When in doubt**: Check test file extension:
  - `*.test.js` → API test → grep by file path
  - `*.spec.js` → E2E test → grep by test name OR pass file paths

### Smart Test Runner Behavior

The `smart-test-runner.js` automatically uses the correct approach:

- **API tests**: Constructs `--grep` patterns with file paths
- **E2E tests**: Passes test file paths as positional arguments (NOT via --grep)

## Writing Tests

### Always Add Coverage Annotations

```javascript
/**
 * @testCovers fastapi_app/routers/files_save.py
 * @testCovers app/src/plugins/filedata.js
 */
test('should save file', async () => {
  // Test code
});
```

### Suppressing Expected Log Output

When testing code that produces expected warnings or log messages, use `assertLogs` to suppress output and verify the messages:

**Python Tests:**

```python
import logging

def test_expected_warning(self):
    """Test that verifies expected warning is logged."""
    # Suppress and capture expected warnings
    with self.assertLogs('module.name', level='WARNING') as cm:
        result = function_that_warns()

    # Verify expected warning was logged
    self.assertTrue(any('expected message' in msg for msg in cm.output))

    # Continue with assertions
    self.assertEqual(result, expected_value)
```

This pattern:

- Suppresses console output during test runs (keeps test output clean)
- Captures log messages for verification
- Ensures expected warnings/errors are actually being logged
- Prevents test pollution when warnings are intentional

**When to use:**

- Testing error handling that logs warnings
- Testing validation that produces expected errors
- Testing deprecated functionality that warns
- Any scenario where log output is part of the expected behavior

### Ignoring Auto-Generated Files

The smart test runner automatically ignores certain files from change detection:

- `app/src/modules/api-client-v1.js` (auto-generated from OpenAPI schema)

To ignore additional files, configure in the test runner:

```javascript
const runner = new SmartTestRunner({
  ignoreChanges: [
    'app/src/modules/api-client-v1.js',  // Exact file path
    /.*-generated\.js$/                  // Regex pattern
  ]
});
```

This prevents unnecessary test runs when only metadata (like timestamps) changes in auto-generated files.

This enables smart test selection via `npm run test:changed`.

### Plugin Integration Tests

Plugin integration tests verify backend plugin functionality by testing custom routes and plugin endpoints with a live server. These tests are located in the plugin's `tests/` directory.

#### Location and Structure

```
fastapi_app/plugins/
└── my-plugin/
    ├── __init__.py
    ├── plugin.py
    ├── routes.py
    └── tests/
        ├── .env.test          # Plugin-specific environment config
        └── test_*.test.js     # Integration tests
```

#### Running Plugin Tests

```bash
# Run plugin tests with test-specific environment
node tests/backend-test-runner.js \
  --test-dir fastapi_app/plugins/my-plugin/tests \
  --env-file fastapi_app/plugins/my-plugin/tests/.env.test
```

#### Plugin Test Environment Configuration

Plugin tests require a `.env.test` file to configure the test environment. This file must include:

**Required Settings:**

```bash
# Test Paths - runtime data is ephemeral
DATA_ROOT=tests/api/runtime
DB_DIR=tests/api/runtime/db
LOG_DIR=tests/api/runtime/logs

# Plugin Configuration
PLUGIN_MY_PLUGIN_ENABLED=true
PLUGIN_MY_PLUGIN_OPTION=value
```

**Why These Are Required:**

- `DATA_ROOT`, `DB_DIR`, `LOG_DIR` - Tell the server to use test directories instead of production paths
- Plugin config vars - Enable and configure the plugin for testing

Without these paths, the server will try to use production directories (`fastapi_app/db`, etc.) which may not exist in the test environment.

#### Authentication in Plugin Tests

Plugin routes often require authentication. The `login()` helper returns an **object** with `sessionId` and `user` properties:

```javascript
import { login, authenticatedApiCall } from '../../../../tests/api/helpers/test-auth.js';

// CORRECT - Destructure the session ID
const { sessionId } = await login('admin', 'admin', BASE_URL);

// WRONG - login() returns an object, not a string
const sessionId = await login('admin', 'admin', BASE_URL); // sessionId will be an object!
```

Pass session ID in requests via header or query parameter:

```javascript
// Via header (preferred)
const response = await fetch(`${BASE_URL}/api/plugins/my-plugin/route`, {
  headers: { 'X-Session-Id': sessionId }
});

// Via query parameter
const response = await authenticatedApiCall(
  sessionId,
  '/api/plugins/my-plugin/route',
  'GET',
  null,
  BASE_URL
);
```

#### Testing Custom Plugin Routes

Backend plugins typically expose custom routes (not just plugin endpoints). Test these routes directly:

```javascript
import { test, describe } from 'node:test';
import assert from 'node:assert';
import { login, authenticatedApiCall } from '../../../../tests/api/helpers/test-auth.js';
import { logger } from '../../../../tests/api/helpers/test-logger.js';

const BASE_URL = process.env.E2E_BASE_URL || 'http://localhost:8000';

describe('My Plugin API Tests', () => {
  let sessionId = null;

  test('Setup: login to get session ID', async () => {
    const loginResult = await login('admin', 'admin', BASE_URL);
    sessionId = loginResult.sessionId;
    assert.ok(sessionId, 'Should have session ID');
    logger.success('Logged in successfully');
  });

  test('Plugin availability check', async () => {
    const response = await authenticatedApiCall(
      sessionId,
      '/plugins',
      'GET',
      null,
      BASE_URL
    );

    const myPlugin = response.plugins.find(p => p.id === 'my-plugin');
    assert.ok(myPlugin, 'Plugin should be available');
    logger.success('Plugin is available');
  });

  test('Preview route returns HTML', async () => {
    const previewUrl = `/api/plugins/my-plugin/preview?param=value`;

    const response = await fetch(`${BASE_URL}${previewUrl}`, {
      headers: { 'X-Session-Id': sessionId }
    });

    assert.strictEqual(response.status, 200);
    const html = await response.text();
    assert.ok(html.includes('expected content'));
    logger.success('Preview route works');
  });

  test('Execute route (GET request)', async () => {
    // Note: Check route definition - some use GET, others POST
    const executeUrl = `/api/plugins/my-plugin/execute?param=value`;

    const response = await fetch(`${BASE_URL}${executeUrl}`, {
      headers: { 'X-Session-Id': sessionId }
    });

    assert.strictEqual(response.status, 200);
    logger.success('Execute route completes');
  });
});
```

#### Common Patterns

**Check HTTP Method:**

Custom routes may use GET or POST. Check the route definition:

```python
@router.get("/execute")  # GET request
@router.post("/execute") # POST request
```

**Test with Fixtures:**

Plugin tests run after fixture import. Use fixture data:

```javascript
test('Works with fixture data', async () => {
  // Fixture files are already imported at this point
  const response = await authenticatedApiCall(
    sessionId,
    '/files/list',
    'GET',
    null,
    BASE_URL
  );

  // Use fixture document IDs in your tests
  const fixtureDoc = response.files.find(f => f.doc_id === 'known-fixture-id');
  assert.ok(fixtureDoc, 'Fixture document should exist');
});
```

**Temporary Test Files:**

Create temporary files for tests that need filesystem data:

```javascript
import { mkdir, writeFile, rm } from 'fs/promises';
import { join } from 'path';

const testDir = '/tmp/my-plugin-test';

test('Setup: create test files', async () => {
  await mkdir(testDir, { recursive: true });
  await writeFile(join(testDir, 'test.xml'), '<root>test</root>');
});

test('Cleanup: remove test files', async () => {
  await rm(testDir, { recursive: true, force: true });
});
```

#### Debugging Plugin Tests

```bash
# Verbose output
node tests/backend-test-runner.js \
  --test-dir fastapi_app/plugins/my-plugin/tests \
  --env-file fastapi_app/plugins/my-plugin/tests/.env.test \
  --verbose

# Keep server running
node tests/backend-test-runner.js \
  --test-dir fastapi_app/plugins/my-plugin/tests \
  --env-file fastapi_app/plugins/my-plugin/tests/.env.test \
  --no-cleanup

# Keep database for inspection
node tests/backend-test-runner.js \
  --test-dir fastapi_app/plugins/my-plugin/tests \
  --env-file fastapi_app/plugins/my-plugin/tests/.env.test \
  --keep-db
```

#### Complete Example

See `fastapi_app/plugins/local_sync/tests/test_sync_api.test.js` for a complete working example that includes:

- Environment configuration via `.env.test`
- Authentication setup
- Plugin availability verification
- Testing custom routes (preview and execute)
- Temporary file creation and cleanup
- Fixture data integration

### API Test Template

```javascript
import { test, describe } from 'node:test';
import assert from 'node:assert';
import { login, authenticatedApiCall } from '../helpers/test-auth.js';

const BASE_URL = process.env.E2E_BASE_URL || 'http://localhost:8000';

describe('Feature Name', () => {
  let sessionId;

  test('Setup: login', async () => {
    const { sessionId: sid } = await login('reviewer', 'reviewer', BASE_URL);
    sessionId = sid;
  });

  test('should do something', async () => {
    const result = await authenticatedApiCall(
      sessionId,
      '/api/endpoint',
      'POST',
      { data: 'value' },
      BASE_URL
    );
    assert.strictEqual(result.status, 'success');
  });

  test('Cleanup: release locks', async () => {
    // Clean up resources
  });
});
```

### E2E Test Template

```javascript
/** @import { namedElementsTree } from '../../app/src/ui.js' */
import { test, expect } from '../fixtures/debug-on-failure.js';

test('should do something', async ({ page }) => {
  await page.goto('http://localhost:8000');

  // Use window.ui for interactions
  await page.evaluate(() => {
    /** @type {namedElementsTree} */
    const ui = /** @type {any} */(window).ui;
    ui.someButton.click();
  });

  // Use testLog() for state verification (preferred over DOM queries)
  const result = await page.evaluate(() => window.someGlobalState);
  expect(result).toBeTruthy();
});
```

### Using API Client in Browser Context (E2E Tests)

**CRITICAL:** Never use manual `fetch()` calls to the API backend in `page.evaluate()`. Always use the `client` object:

```javascript
/**
 * @import { api as Client } from '../../../app/src/plugins/client.js'
 */

const result = await page.evaluate(async () => {
  /** @type {Client} */
  const client = /** @type {any} */(window).client;

  // Use typed API methods
  return await client.apiClient.sseTestProgress({
    steps: 3,
    delay_ms: 500,
    label_prefix: 'Test step'
  });
});
```

- Check `app/src/modules/api-client-v1.js` for all available API methods (auto-generated from OpenAPI schema)
- See [../development/testing.md](../development/testing.md#using-the-api-client-in-browser-context) for full documentation

## Key Principles

1. **Use `@testCovers` annotations** - Enables smart test selection
2. **Clean up after tests** - Release locks, delete test files
3. **API tests run locally** - Fast iteration with `backend-test-runner.js`
4. **E2E tests use local server by default** - Isolation and speed
5. **ALWAYS use helper functions** - Never reimplement auth or API utilities

## Common Patterns

### Authentication in API Tests

```javascript
import { login, authenticatedApiCall, createAdminSession } from '../helpers/test-auth.js';

// Regular user login
const session = await login('reviewer', 'reviewer', BASE_URL);

// Admin session
const adminSession = await createAdminSession(BASE_URL);

// Authenticated API call
const result = await authenticatedApiCall(
  session.sessionId,
  '/api/endpoint',
  'POST',
  data,
  BASE_URL
);
```

### UI Navigation in E2E Tests

```javascript
await page.evaluate(() => {
  /** @type {namedElementsTree} */
  const ui = /** @type {any} */(window).ui;
  ui.loginDialog.username.value = 'testuser';
  ui.loginDialog.submit.click();
});
```

### Test Logging for State Verification

```javascript
import { setupTestConsoleCapture, waitForTestMessage } from './helpers/test-logging.js';

const consoleLogs = setupTestConsoleCapture(page);
// ... perform action ...
const log = await waitForTestMessage(consoleLogs, 'FILE_SAVED');
expect(log.value.file_id).toBeTruthy();
```

**Enhanced Error Reporting:**

`setupTestConsoleCapture()` now includes automatic network request tracking. When a console error occurs, the error message will include:

- The console error message
- Recent failed network requests (within 5 seconds of the error)
- Recent network activity (last 2 seconds)

Example enhanced error output:

```text
Error: Unexpected console error detected: Failed to load resource: the server responded with a status of 404 (Not Found)

Recent failed network requests:
  GET http://localhost:8010/api/v1/files/nonexistent → 404 Not Found

Recent network activity (last 2 seconds):
  GET http://localhost:8010/api/v1/files/list → 200
  POST http://localhost:8010/api/v1/files/save → 200
  GET http://localhost:8010/api/v1/files/nonexistent → 404
```

This eliminates the need to access server logs or trace files when debugging network-related errors in CI.

### Checking State for Shoelace Components

Playwright's accessibility-based matchers don't work properly with Shoelace web components. Always check attributes directly:

**Disabled State:**

```javascript
// ❌ WRONG - May fail for Shoelace components
await expect(button).toBeDisabled();
await expect(button).toBeEnabled();

// ✅ CORRECT - Check disabled attribute directly
await expect(button).toHaveAttribute('disabled', '');  // Disabled
await expect(button).not.toHaveAttribute('disabled'); // Enabled
```

**Checked State:**

```javascript
// ❌ WRONG - Fails with "Not a checkbox or radio button"
await expect(checkbox).toBeChecked();
await expect(checkbox).not.toBeChecked();

// ✅ CORRECT - Check checked attribute directly
await expect(checkbox).toHaveAttribute('checked', '');  // Checked
await expect(checkbox).not.toHaveAttribute('checked'); // Unchecked
```

**Dialog Button Clicks:**

Shoelace dialogs require a short delay before button clicks to ensure the dialog is fully rendered and interactive:

```javascript
// Wait for dialog to open
await page.waitForSelector('sl-dialog[name="myDialog"][open]', { timeout: 5000 });

// Fill in form fields
await page.evaluate(() => {
  const ui = window.ui;
  ui.myDialog.inputField.value = 'test value';
});

// CRITICAL: Add 500ms delay before clicking submit
await page.waitForTimeout(500);

await page.evaluate(() => {
  const ui = window.ui;
  ui.myDialog.submit.click();
});
```

**Why this is needed:**

- Shoelace dialogs use Shadow DOM and animation
- Clicking too quickly can result in the click being ignored
- 500ms ensures the dialog is fully interactive

This applies to all Shoelace dialog buttons (`<sl-button>` inside `<sl-dialog>`).

## Critical: Always Use Helper Functions

**Never reimplement authentication, API calls, or common test utilities.** This causes maintenance burden and bugs.

❌ **WRONG** - Reimplementing login:

```javascript
async function loginAsAdmin() {
  const response = await fetch(`${API_BASE}/api/v1/auth/login`, {
    method: 'POST',
    body: JSON.stringify({
      username: 'admin',
      password: 'admin'  // Wrong! Should be passwd_hash with hashing
    })
  });
}
```

✅ **CORRECT** - Use existing helper:

```javascript
import { createAdminSession } from '../helpers/test-auth.js';

async function loginAsAdmin() {
  const { sessionId } = await createAdminSession(API_BASE);
  return sessionId;
}
```

### Common Helper Locations

```javascript
// Authentication and API calls
import {
  login,
  logout,
  createAdminSession,
  hashPassword,
  authenticatedApiCall,
  authenticatedRequest
} from '../helpers/test-auth.js';

// Role-based login helpers (use standard fixture users)
import {
  loginAsAdmin,      // Returns {sessionId, user} for admin user
  loginAsReviewer,   // Returns {sessionId, user} for reviewer user
  loginAsAnnotator,  // Returns {sessionId, user} for annotator user
  loginAsBasicUser   // Returns {sessionId, user} for basic user (no special roles)
} from '../helpers/test-auth.js';

// Lock management helpers
import {
  tryAcquireLock,  // Returns Response object (check .status)
  acquireLock,     // Throws on failure
  releaseLock,     // Release a held lock
  checkLock        // Check lock status: {is_locked, locked_by}
} from '../helpers/test-auth.js';

// Test logging and state verification
import {
  setupTestConsoleCapture,
  waitForTestMessage
} from './helpers/test-logging.js';
```

### Lock Management in Tests

When testing file operations that require locks:

```javascript
import { loginAsReviewer, tryAcquireLock, releaseLock } from '../helpers/test-auth.js';

const session = await loginAsReviewer(BASE_URL);

// Try to acquire lock (returns Response for status checking)
const response = await tryAcquireLock(session.sessionId, fileId, BASE_URL);
if (response.status === 200) {
  // Lock acquired - perform operations
  // ...

  // Always release lock in cleanup
  await releaseLock(session.sessionId, fileId, BASE_URL);
} else if (response.status === 403) {
  // Access denied (permission issue)
} else if (response.status === 409) {
  // Conflict (already locked by another user)
}
```

**Role-based login helpers** use standard fixture users:

- `loginAsAdmin()` - User with wildcard (`*`) access
- `loginAsReviewer()` - User with `reviewer` role (can edit gold files)
- `loginAsAnnotator()` - User with `annotator` role (can edit version files)
- `loginAsBasicUser()` - User with no special roles (limited permissions)

## Troubleshooting

### Missing Dependencies

If tests fail with "test failed" or module import errors, ensure all dependencies are installed:

```bash
npm install
```

This is particularly important for SSE tests which require the `eventsource-client` package.

## Debugging

### API Tests

```bash
# Verbose output
npm run test:api -- --verbose --grep "save"

# Keep server running for manual testing
npm run test:api -- --no-cleanup

# Keep database for inspection
npm run test:api -- --keep-db
```

### E2E Tests

```bash
# Show browser
npm run test:e2e:headed

# Step-through debugging (Playwright inspector)
npm run test:e2e:debug

# Post-mortem debugging (capture artifacts on failure, headless)
npm run test:e2e:debug-failure
npm run test:e2e:debug-failure -- --grep "new version"

# Combine with headed mode to watch test execution
npm run test:e2e -- --debug-on-failure --headed

# Add breakpoints in test code
await page.pause();
```

#### Debug-on-Failure Mode

When using `--debug-on-failure`, tests run in **headless mode** (faster) and capture comprehensive debug information on failure:

**Captured Artifacts:**

- Console messages → `console-messages.json`
- Page errors → `page-errors.json`
- **Network traffic → `network-log.json`** (all HTTP requests/responses with bodies)
- Screenshots (automatically via Playwright)
- Video recording of test execution

**Behavior:**

- Stops on first failure (`--max-failures=1`)
- Runs headless by default (use `--headed` flag to watch execution)
- All artifacts saved to `tests/e2e/test-results/<test-name>/`

**Network Log Format:**

```json
[
  {
    "type": "request",
    "timestamp": "2025-01-15T10:30:00.000Z",
    "method": "GET",
    "url": "http://localhost:8000/api/v1/files",
    "headers": {...},
    "postData": null
  },
  {
    "type": "response",
    "timestamp": "2025-01-15T10:30:00.100Z",
    "method": "GET",
    "url": "http://localhost:8000/api/v1/files",
    "status": 200,
    "statusText": "OK",
    "headers": {...},
    "body": "{\"files\": [...]}"
  }
]
```

**Note**: All E2E tests should import from the debug-on-failure fixture by default:

```javascript
import { test, expect } from '../fixtures/debug-on-failure.js';
```

This has no effect during normal test runs - it only activates when `--debug-on-failure` is used.

## JSDOM Limitations for Browser-Targeted Code

When testing JavaScript code designed for the browser using Node.js with JSDOM, be aware of these limitations:

### XPath Namespace Support

JSDOM has incomplete XPath support, particularly for namespace resolvers:

```javascript
// This works in browsers but may fail in JSDOM:
const result = xmlDoc.evaluate(
  "//tei:bibl",
  xmlDoc,
  (prefix) => prefix === "tei" ? "http://www.tei-c.org/ns/1.0" : null,
  XPathResult.FIRST_ORDERED_NODE_TYPE,
  null
);
```

**Workaround:** Export helper functions that accept elements directly, bypassing XPath:

```javascript
// In the enhancement module:
export function execute(xmlDoc, currentState, configMap) {
  const targetNode = xmlDoc.evaluate(xpath, ...);  // Uses XPath
  processElement(targetNode, xmlDoc);
}

export function processElement(element, xmlDoc) {
  // Core logic - testable without XPath
}

// In tests:
import { processElement } from './enhancement.js';
const bibl = xmlDoc.getElementsByTagNameNS(TEI_NS, 'bibl')[0];
processElement(bibl, xmlDoc);  // Direct element reference
```

### Other JSDOM Limitations

- **No `XPathResult` constants** - Must expose via `global.XPathResult = dom.window.XPathResult`
- **Limited `Range` and `Selection` APIs** - May not behave identically to browsers
- **No layout engine** - Properties like `offsetWidth`, `getBoundingClientRect()` return 0
- **Partial Web Component support** - Shadow DOM works but custom elements have limitations

### JSDOM Setup Pattern

```javascript
import { JSDOM } from 'jsdom';

const dom = new JSDOM();
global.window = dom.window;
global.document = dom.window.document;
global.Node = dom.window.Node;
global.Document = dom.window.Document;
global.DOMParser = dom.window.DOMParser;
global.XPathResult = dom.window.XPathResult;  // For XPath-using code
```

### When to Use E2E Tests Instead

For code that heavily depends on browser-specific APIs, consider E2E tests with Playwright instead of Node.js unit tests:

- Complex XPath expressions
- Web Component interactions
- Layout-dependent behavior
- CSS styling logic

## Important Notes

- **API tests run against local server** - No containers needed, faster iteration
- **E2E tests use local server by default** - Containerized mode available but slower
- **Use `--keep-db`** - When debugging to preserve state between runs
- **Clean up locks** - Always release locks in test cleanup
- **Fixtures auto-load** - Defined in `tests/api/fixtures/` and `tests/e2e/fixtures/`
- **Check helpers first** - Look in `tests/*/helpers/` before implementing utilities

## Backend Authentication Requirements

When implementing FastAPI endpoints that require authentication:

```python
from ..lib.dependencies import get_current_user
from fastapi import Depends, HTTPException

def require_admin(current_user: Optional[dict] = Depends(get_current_user)) -> dict:
    if not current_user:
        raise HTTPException(status_code=401, detail="Authentication required")
    # ... check admin role ...
    return current_user

@router.get("/endpoint")
def endpoint(current_user: dict = Depends(require_admin)):
    # Endpoint implementation
```

**Key Points**:

- Use `Depends(get_current_user)` to inject authentication
- FastAPI automatically handles session extraction from `X-Session-Id` header
- Session header is case-sensitive: `X-Session-Id`