docs/code-assistant/testing-guide.md

Testing Guide

Essential testing information for code assistants. For comprehensive testing documentation, see ../development/testing.md.

Test Structure

tests/
├── unit/{js,fastapi}/         # Unit tests
├── api/v1/                     # API integration tests (*.test.js)
├── e2e/tests/                  # E2E Playwright tests (*.spec.js)
├── backend-test-runner.js      # API test runner
├── e2e-runner.js              # E2E test runner
└── smart-test-runner.js       # Intelligent test selection

Quick Commands

# Run tests for changed files only (use this most often)
npm run test:changed

# Check which tests would run without executing them
npm run test:changed -- --dry-run
npm run test:changed -- --names-only  # Output only test file names

# Unit tests
npm run test:unit:js          # JavaScript units
npm run test:unit:fastapi     # Python units

# API tests (local FastAPI server, no containers)
npm run test:api              # All API tests
npm run test:api -- --grep "save"    # Filter by file path pattern
npm run test:api -- --grep "files_serve_caching"  # Run specific test file

# E2E tests (Playwright)
npm run test:e2e              # All E2E tests with local server
npm run test:e2e:headed       # Show browser
npm run test:e2e:debug        # Step-through debugging
npm run test:e2e:debug-failure # Capture debug artifacts on failure (headless)
npm run test:e2e -- --grep "should upload"  # Filter by test name pattern

# Cross-browser E2E tests (real browser engines, no login required)
npm run test:e2e:xmleditor-browsers  # xmlTagSync tests in chromium, firefox, webkit

# Container tests (runs all tests inside container)
npm run test:container                                  # Run with cache
npm run test:container -- --no-cache                    # Rebuild all layers
npm run test:container -- path/to/file.js               # Test specific files
npm run test:container -- --browser firefox             # Use specific browser
npm run test:container -- --browser chromium,firefox,webkit  # Test multiple browsers

Test Types

Unit Tests

• Location: tests/unit/{js,fastapi}/
• Purpose: Test isolated functions/classes
• Run: npm run test:unit

Python Unit Tests with FastAPI Routes

Python unit tests for backend plugins and custom routes use a combination of dependency injection overrides and mocking.

Pattern: Testing FastAPI Routes with Mixed Dependencies

Routes often mix FastAPI dependency injection with direct function calls. Test both using:

1. Dependency Overrides (in setUp) - for consistent mocks across all tests
2. @patch decorators - for functions called inside routes

Example:

import unittest
from unittest.mock import MagicMock, patch
from fastapi import FastAPI
from fastapi.testclient import TestClient

class TestMyRoute(unittest.IsolatedAsyncioTestCase):
    """Test my custom route."""

    def setUp(self):
        """Set up test fixtures."""
        from myapp.routes import router
        from myapp.dependencies import get_auth_manager, get_session_manager

        self.app = FastAPI()
        self.app.include_router(router)

        # Create mocks for injected dependencies
        self.mock_session_manager = MagicMock()
        self.mock_auth_manager = MagicMock()

        # Override dependencies - these work for all tests
        self.app.dependency_overrides[get_session_manager] = lambda: self.mock_session_manager
        self.app.dependency_overrides[get_auth_manager] = lambda: self.mock_auth_manager

        self.client = TestClient(self.app)

    @patch("myapp.config.get_settings")  # Patch functions called inside route
    @patch("myapp.routes.get_db")        # Not dependency-injected, called directly
    def test_my_endpoint(self, mock_get_db, mock_settings):
        """Test endpoint with authentication."""
        # Mock settings (called inside route via import)
        mock_settings_obj = MagicMock()
        mock_settings_obj.session_timeout = 3600
        mock_settings.return_value = mock_settings_obj

        # Configure dependency override mocks (set in setUp)
        self.mock_session_manager.is_session_valid.return_value = True
        mock_user = MagicMock()
        self.mock_auth_manager.get_user_by_session_id.return_value = mock_user

        # Mock direct function calls
        mock_get_db.return_value = MagicMock()

        # Make request
        response = self.client.get(
            "/api/my-endpoint",
            params={"session_id": "valid-session"}
        )

        self.assertEqual(response.status_code, 200)

Key Points:

• Dependency overrides (self.app.dependency_overrides) handle FastAPI Depends() parameters
• @patch decorators handle functions imported and called inside routes (get_settings(), get_db())
• Mock at import location: Patch where the function is used, not where it's defined
  • Route imports from fastapi_app.config import get_settings → patch "myapp.routes.get_settings"
  • Route imports function from another module → patch at that import path
• Order matters: @patch decorators apply bottom-to-top, parameters passed in reverse order

Common Mistakes:

• Trying to patch dependency-injected functions with @patch instead of using dependency_overrides
• Patching at definition location instead of import location
• Forgetting to mock functions called inside routes that aren't dependency-injected
• Calling functions directly instead of using dependency injection - Routes should use Depends(get_db) and Depends(get_file_storage) as parameters, not call db = get_db() inside the function body. This ensures proper mocking via dependency_overrides and prevents database access failures in CI environments where databases may not exist

Best Practice - Always Use Dependency Injection:

Routes should declare all dependencies as parameters using Depends():

# ✅ CORRECT - Use dependency injection
@router.get("/export")
async def export_csv(
    pdf: str = Query(...),
    db=Depends(get_db),
    file_storage=Depends(get_file_storage)
):
    file_repo = FileRepository(db)
    # ... use db and file_storage ...

# ❌ WRONG - Direct function calls
@router.get("/export")
async def export_csv(
    pdf: str = Query(...)
):
    db = get_db()  # Can't be mocked via dependency_overrides
    file_storage = get_file_storage()  # Fails in CI if database doesn't exist

This pattern:

• Enables proper mocking via app.dependency_overrides in tests
• Prevents CI failures when databases/resources don't exist
• Aligns with FastAPI best practices
• Matches patterns in existing routes (edit_history plugin)

Authentication Testing Pattern:

When testing routes that require authentication (via session_id query parameter or X-Session-ID header), set up default authentication in setUp():

def setUp(self):
    """Set up test fixtures."""
    from fastapi import FastAPI
    from myapp.routes import router
    from fastapi_app.lib.core.dependencies import (
        get_auth_manager,
        get_session_manager,
        get_db,
        get_file_storage,
    )

    self.app = FastAPI()
    self.app.include_router(router)

    # Create mocks for dependencies
    self.mock_session_manager = MagicMock()
    self.mock_auth_manager = MagicMock()
    self.mock_db = MagicMock()
    self.mock_storage = MagicMock()

    # Mock valid authentication by default
    self.mock_session_manager.is_session_valid.return_value = True
    self.mock_auth_manager.get_user_by_session_id.return_value = MagicMock(
        username="testuser",
        groups=["*"]  # Wildcard access
    )

    # Override dependencies
    self.app.dependency_overrides[get_session_manager] = lambda: self.mock_session_manager
    self.app.dependency_overrides[get_auth_manager] = lambda: self.mock_auth_manager
    self.app.dependency_overrides[get_db] = lambda: self.mock_db
    self.app.dependency_overrides[get_file_storage] = lambda: self.mock_storage

    self.client = TestClient(self.app)

Then include session_id in test requests:

response = self.client.get(
    "/api/my-endpoint",
    params={"session_id": "test-session", "other_param": "value"}
)

Why This Pattern:

• Routes using Depends(get_session_manager) and Depends(get_auth_manager) require dependency overrides
• Setting up valid authentication in setUp() makes all tests pass authentication by default
• Individual tests can override these defaults to test authentication failures
• Must include session_id parameter in requests to satisfy route authentication requirements

See Also:

• fastapi_app/plugins/edit_history/tests/test_edit_history_export.py - Complete example with authentication, authorization, and data mocking
• fastapi_app/plugins/annotation_history/tests/test_annotation_history.py - Routes requiring authentication via dependency injection

API Integration Tests

• Location: tests/api/v1/
• Naming: *.test.js
• Purpose: Test backend endpoints without browser
• Run: npm run test:api
• Features: Local server, auto-cleanup, fixtures from tests/api/fixtures/

SSE (Server-Sent Events) Tests

SSE tests require special handling due to long-lived HTTP connections. When tests maintain multiple SSE connections and need to make additional HTTP requests, the Node.js HTTP connection pool can become exhausted.

HTTP Connection Pool Configuration:

SSE test files must increase the HTTP agent's max sockets at the top of the file:

import http from 'node:http';

// Increase connection pool to handle SSE connections + regular requests
http.globalAgent.maxSockets = 50;

Why This Is Needed:

• Node.js HTTP agent defaults to ~5-10 concurrent connections per host
• Each SSE connection holds a connection slot for its entire lifetime
• When multiple tests create SSE connections, these accumulate
• Additional HTTP requests (like POST to trigger events) can timeout if no connections are available
• Increasing maxSockets to 50 prevents connection pool exhaustion

Symptoms of Connection Pool Exhaustion:

• Tests pass in isolation but hang/timeout in full test suite
• HTTP requests timeout after SSE connections are established
• Error: "Broadcast request timed out after 5s" or similar

Example Pattern:

import { test, describe } from 'node:test';
import http from 'node:http';
import { createEventSource } from 'eventsource-client';

// CRITICAL: Increase connection pool for SSE tests
http.globalAgent.maxSockets = 50;

describe('SSE Tests', () => {
  test('Broadcast to multiple sessions', async () => {
    // Create SSE connections (long-lived)
    const conn1 = createEventSource({ url: '/sse/subscribe' });
    const conn2 = createEventSource({ url: '/sse/subscribe' });

    // Wait for connections to establish
    await new Promise(resolve => setTimeout(resolve, 2000));

    // Make additional HTTP request - would fail without increased maxSockets
    const response = await fetch('/sse/test/broadcast', { method: 'POST' });

    // Clean up
    conn1.close();
    conn2.close();
  });
});

See Also: tests/api/v1/sse.test.js for complete SSE testing patterns.

Fixture System

API tests use a two-phase fixture loading system:

Phase 1: Config Loading (before server starts)

• Copies JSON config files from tests/api/fixtures/{fixture-name}/config/ to tests/api/runtime/config/
• Config files: users.json, groups.json, roles.json, collections.json
• Server reads these during initialization to set up RBAC

Phase 2: File Import (after server starts)

• Imports PDF/XML files from tests/api/fixtures/{fixture-name}/files/ using bin/import_files.py
• Files are imported into the default collection (matches default group access)
• Creates database entries and stores files in content-addressable hash-sharded structure
• TEI files in the fixture directory are detected as gold standard files and linked to their PDFs

Available Fixtures:

• minimal: Basic config with minimal users/groups (no files)
• standard: Standard config + sample PDF/TEI files for testing

Access Control Considerations:

• Fixture files are imported into the default collection
• Test users need appropriate group membership to access fixture files
• Standard fixture users (reviewer, annotator, user) are in the default group with access to _inbox and default collections
• admin user has wildcard access (*) to all collections

CRITICAL: Do not alter fixtures

• Tests should NOT permanently modify fixture files or delete them in cleanup - they're shared across tests and automatically cleaned up when the runtime directory is wiped.
• Ideally, make a copy of the fixture file or add new files to alter/delete, at a minimun restore altered files after the test.

E2E Tests

• Location: tests/e2e/tests/
• Naming: *.spec.js
• Purpose: Test full workflows in browser
• Run: npm run test:e2e
• Features: Playwright, window.ui navigation, testLog() for state verification

Isolated Component Harness Tests

Some specs test individual components in isolation using a standalone HTML harness page rather than the full application. The harness is served by the dev server and loads only the dependencies needed for the component under test (via the importmap from app/web/importmap.json, rebased to absolute paths). No login, no fixture loading, no application state required.

Use this approach when:

• The component has browser-engine-specific behavior (e.g. Lezer parser eagerness differences between Chromium and Firefox)
• You need to reproduce or rule out an editor bug without the overhead of the full application
• Fast cross-browser iteration is needed

Available harnesses:

Harness HTML	Spec	Tests
tests/e2e/harness/xmleditor-harness.html	tests/e2e/tests/xmleditor-cross-browser.spec.js	xmlTagSync CodeMirror extension

Running:

# All three browsers
npm run test:e2e:xmleditor-browsers

# Single browser for fast iteration
node tests/e2e-runner.js tests/e2e/tests/xmleditor-cross-browser.spec.js --browser firefox

# Headed to observe behaviour
node tests/e2e-runner.js tests/e2e/tests/xmleditor-cross-browser.spec.js --browser chromium --headed

Adding a new harness:

1. Create tests/e2e/harness/<name>-harness.html — use xmleditor-harness.html as the template (contains the importmap rebasing logic).
2. Create tests/e2e/harness/<name>-harness.js — import the component under test, expose helpers on window, and emit window.testLog('HARNESS_READY', {}) when ready.
3. Create tests/e2e/tests/<name>-cross-browser.spec.js — call setupTestConsoleCapture + waitForTestMessage(logs, 'HARNESS_READY') before each test.
4. Add an npm script in package.json following the test:e2e:xmleditor-browsers pattern.

Container Testing

Run Tests Inside Container (npm run test:container):

• Purpose: Run all tests in an isolated container environment (same as CI)
• What it does:
  • Builds the ci container image with all dependencies and test code
  • Runs tests inside the container (not against it)
  • Streams output in real-time to your terminal
  • Uses smart test runner with dependency analysis
  • Supports all test runner options (browsers, grep, specific files)
• Use when:
  • Testing changes that require a clean environment
  • Validating behavior in production-like container
  • Reproducing CI failures locally
  • Testing Dockerfile or dependency changes

Examples:

# Run all tests with caching (fastest for iterative testing)
npm run test:container

# Force rebuild (use after Dockerfile or dependency changes)
npm run test:container -- --no-cache

# Test specific files
npm run test:container -- app/src/plugins/authentication.js

# Run E2E tests with specific browser
npm run test:container -- --browser firefox

# Test multiple browsers
npm run test:container -- --browser chromium,firefox,webkit

# Run all tests (skip smart selection)
npm run test:container -- --all

Technical Details:

• Tests run inside container using docker/entrypoint-ci.sh
• All fixtures and test code baked into image at build time
• Real-time streaming via PYTHONUNBUFFERED=1 and stdio: 'inherit'
• Playwright browsers installed early for optimal layer caching
• Exit code matches test results (0 = success, non-zero = failure)

Test Filtering with --grep

IMPORTANT: The --grep parameter works differently for API vs E2E tests:

API Tests (backend-test-runner.js)

--grep matches file paths (test file names):

# Matches files containing "save" in the path
npm run test:api -- --grep "save"
# → Runs: tests/api/v1/files_save.test.js

# Matches specific test file
npm run test:api -- --grep "files_serve_caching"
# → Runs: tests/api/v1/files_serve_caching.test.js

# Matches multiple files with common pattern
npm run test:api -- --grep "files_"
# → Runs: all files starting with "files_"

Why: The backend test runner filters test files before passing them to Node.js test runner.

E2E Tests (e2e-runner.js / Playwright)

--grep matches test names (test titles in your code):

# Matches test titles containing "upload"
npm run test:e2e -- --grep "upload"
# → Runs: test('should upload file', ...), test('upload validation', ...), etc.

# Matches specific test case
npm run test:e2e -- --grep "should create new version"
# → Runs: test('should create new version', ...)

# Case-insensitive regex pattern
npm run test:e2e -- --grep "save.*revision"
# → Runs: test('should save current revision', ...), etc.

Alternative: Run specific test files directly

Instead of using --grep with test names, you can pass test file paths as positional arguments:

# Run single test file
node tests/e2e-runner.js tests/e2e/tests/app-loading.spec.js

# Run multiple test files
node tests/e2e-runner.js tests/e2e/tests/auth-workflow.spec.js tests/e2e/tests/export-workflow.spec.js

# Combine with options
node tests/e2e-runner.js --headed tests/e2e/tests/auth-workflow.spec.js

Why: Playwright's --grep matches test() descriptions, not file paths. The smart-test-runner automatically uses file paths when selecting specific tests.

Rule of Thumb for Code Assistants

• Debugging API tests: Use file name patterns → --grep "files_save"
• Debugging E2E tests: Use test name patterns → --grep "should upload" OR pass file paths directly
• When in doubt: Check test file extension:
  • *.test.js → API test → grep by file path
  • *.spec.js → E2E test → grep by test name OR pass file paths

Smart Test Runner Behavior

The smart-test-runner.js automatically uses the correct approach:

• API tests: Constructs --grep patterns with file paths
• E2E tests: Passes test file paths as positional arguments (NOT via --grep)

Writing Tests

Always Add Coverage Annotations

/**
 * @testCovers fastapi_app/routers/files_save.py
 * @testCovers app/src/plugins/filedata.js
 */
test('should save file', async () => {
  // Test code
});

Suppressing Expected Log Output

When testing code that produces expected warnings or log messages, use assertLogs to suppress output and verify the messages:

Python Tests:

import logging

def test_expected_warning(self):
    """Test that verifies expected warning is logged."""
    # Suppress and capture expected warnings
    with self.assertLogs('module.name', level='WARNING') as cm:
        result = function_that_warns()

    # Verify expected warning was logged
    self.assertTrue(any('expected message' in msg for msg in cm.output))

    # Continue with assertions
    self.assertEqual(result, expected_value)

This pattern:

• Suppresses console output during test runs (keeps test output clean)
• Captures log messages for verification
• Ensures expected warnings/errors are actually being logged
• Prevents test pollution when warnings are intentional

When to use:

• Testing error handling that logs warnings
• Testing validation that produces expected errors
• Testing deprecated functionality that warns
• Any scenario where log output is part of the expected behavior

Ignoring Auto-Generated Files

The smart test runner automatically ignores certain files from change detection:

• app/src/modules/api-client-v1.js (auto-generated from OpenAPI schema)

To ignore additional files, configure in the test runner:

const runner = new SmartTestRunner({
  ignoreChanges: [
    'app/src/modules/api-client-v1.js',  // Exact file path
    /.*-generated\.js$/                  // Regex pattern
  ]
});

This prevents unnecessary test runs when only metadata (like timestamps) changes in auto-generated files.

This enables smart test selection via npm run test:changed.

Plugin Integration Tests

Plugin integration tests verify backend plugin functionality by testing custom routes and plugin endpoints with a live server. These tests are located in the plugin's tests/ directory.

Location and Structure

fastapi_app/plugins/
└── my-plugin/
    ├── __init__.py
    ├── plugin.py
    ├── routes.py
    └── tests/
        ├── .env.test          # Plugin-specific environment config
        └── test_*.test.js     # Integration tests

Running Plugin Tests

# Run plugin tests with test-specific environment
node tests/backend-test-runner.js \
  --test-dir fastapi_app/plugins/my-plugin/tests \
  --env-file fastapi_app/plugins/my-plugin/tests/.env.test

Plugin Test Environment Configuration

Plugin tests require a .env.test file to configure the test environment. This file must include:

Required Settings:

# Test Paths - runtime data is ephemeral
DATA_ROOT=tests/api/runtime
DB_DIR=tests/api/runtime/db
LOG_DIR=tests/api/runtime/logs

# Plugin Configuration
PLUGIN_MY_PLUGIN_ENABLED=true
PLUGIN_MY_PLUGIN_OPTION=value

Why These Are Required:

• DATA_ROOT, DB_DIR, LOG_DIR - Tell the server to use test directories instead of production paths
• Plugin config vars - Enable and configure the plugin for testing

Without these paths, the server will try to use production directories (fastapi_app/db, etc.) which may not exist in the test environment.

Authentication in Plugin Tests

Plugin routes often require authentication. The login() helper returns an object with sessionId and user properties:

import { login, authenticatedApiCall } from '../../../../tests/api/helpers/test-auth.js';

// CORRECT - Destructure the session ID
const { sessionId } = await login('admin', 'admin', BASE_URL);

// WRONG - login() returns an object, not a string
const sessionId = await login('admin', 'admin', BASE_URL); // sessionId will be an object!

Pass session ID in requests via header or query parameter:

// Via header (preferred)
const response = await fetch(`${BASE_URL}/api/plugins/my-plugin/route`, {
  headers: { 'X-Session-Id': sessionId }
});

// Via query parameter
const response = await authenticatedApiCall(
  sessionId,
  '/api/plugins/my-plugin/route',
  'GET',
  null,
  BASE_URL
);

Testing Custom Plugin Routes

Backend plugins typically expose custom routes (not just plugin endpoints). Test these routes directly:

import { test, describe } from 'node:test';
import assert from 'node:assert';
import { login, authenticatedApiCall } from '../../../../tests/api/helpers/test-auth.js';
import { logger } from '../../../../tests/api/helpers/test-logger.js';

const BASE_URL = process.env.E2E_BASE_URL || 'http://localhost:8000';

describe('My Plugin API Tests', () => {
  let sessionId = null;

  test('Setup: login to get session ID', async () => {
    const loginResult = await login('admin', 'admin', BASE_URL);
    sessionId = loginResult.sessionId;
    assert.ok(sessionId, 'Should have session ID');
    logger.success('Logged in successfully');
  });

  test('Plugin availability check', async () => {
    const response = await authenticatedApiCall(
      sessionId,
      '/plugins',
      'GET',
      null,
      BASE_URL
    );

    const myPlugin = response.plugins.find(p => p.id === 'my-plugin');
    assert.ok(myPlugin, 'Plugin should be available');
    logger.success('Plugin is available');
  });

  test('Preview route returns HTML', async () => {
    const previewUrl = `/api/plugins/my-plugin/preview?param=value`;

    const response = await fetch(`${BASE_URL}${previewUrl}`, {
      headers: { 'X-Session-Id': sessionId }
    });

    assert.strictEqual(response.status, 200);
    const html = await response.text();
    assert.ok(html.includes('expected content'));
    logger.success('Preview route works');
  });

  test('Execute route (GET request)', async () => {
    // Note: Check route definition - some use GET, others POST
    const executeUrl = `/api/plugins/my-plugin/execute?param=value`;

    const response = await fetch(`${BASE_URL}${executeUrl}`, {
      headers: { 'X-Session-Id': sessionId }
    });

    assert.strictEqual(response.status, 200);
    logger.success('Execute route completes');
  });
});

Common Patterns

Check HTTP Method:

Custom routes may use GET or POST. Check the route definition:

@router.get("/execute")  # GET request
@router.post("/execute") # POST request

Test with Fixtures:

Plugin tests run after fixture import. Use fixture data:

test('Works with fixture data', async () => {
  // Fixture files are already imported at this point
  const response = await authenticatedApiCall(
    sessionId,
    '/files/list',
    'GET',
    null,
    BASE_URL
  );

  // Use fixture document IDs in your tests
  const fixtureDoc = response.files.find(f => f.doc_id === 'known-fixture-id');
  assert.ok(fixtureDoc, 'Fixture document should exist');
});

Temporary Test Files:

Create temporary files for tests that need filesystem data:

import { mkdir, writeFile, rm } from 'fs/promises';
import { join } from 'path';

const testDir = '/tmp/my-plugin-test';

test('Setup: create test files', async () => {
  await mkdir(testDir, { recursive: true });
  await writeFile(join(testDir, 'test.xml'), '<root>test</root>');
});

test('Cleanup: remove test files', async () => {
  await rm(testDir, { recursive: true, force: true });
});

Debugging Plugin Tests

# Verbose output
node tests/backend-test-runner.js \
  --test-dir fastapi_app/plugins/my-plugin/tests \
  --env-file fastapi_app/plugins/my-plugin/tests/.env.test \
  --verbose

# Keep server running
node tests/backend-test-runner.js \
  --test-dir fastapi_app/plugins/my-plugin/tests \
  --env-file fastapi_app/plugins/my-plugin/tests/.env.test \
  --no-cleanup

# Keep database for inspection
node tests/backend-test-runner.js \
  --test-dir fastapi_app/plugins/my-plugin/tests \
  --env-file fastapi_app/plugins/my-plugin/tests/.env.test \
  --keep-db

Complete Example

See fastapi_app/plugins/local_sync/tests/test_sync_api.test.js for a complete working example that includes:

• Environment configuration via .env.test
• Authentication setup
• Plugin availability verification
• Testing custom routes (preview and execute)
• Temporary file creation and cleanup
• Fixture data integration

API Test Template

import { test, describe } from 'node:test';
import assert from 'node:assert';
import { login, authenticatedApiCall } from '../helpers/test-auth.js';

const BASE_URL = process.env.E2E_BASE_URL || 'http://localhost:8000';

describe('Feature Name', () => {
  let sessionId;

  test('Setup: login', async () => {
    const { sessionId: sid } = await login('reviewer', 'reviewer', BASE_URL);
    sessionId = sid;
  });

  test('should do something', async () => {
    const result = await authenticatedApiCall(
      sessionId,
      '/api/endpoint',
      'POST',
      { data: 'value' },
      BASE_URL
    );
    assert.strictEqual(result.status, 'success');
  });

  test('Cleanup: release locks', async () => {
    // Clean up resources
  });
});

E2E Test Template

/** @import { namedElementsTree } from '../../app/src/ui.js' */
import { test, expect } from '../fixtures/debug-on-failure.js';

test('should do something', async ({ page }) => {
  await page.goto('http://localhost:8000');

  // Use window.ui for interactions
  await page.evaluate(() => {
    /** @type {namedElementsTree} */
    const ui = /** @type {any} */(window).ui;
    ui.someButton.click();
  });

  // Use testLog() for state verification (preferred over DOM queries)
  const result = await page.evaluate(() => window.someGlobalState);
  expect(result).toBeTruthy();
});

Using API Client in Browser Context (E2E Tests)

CRITICAL: Never use manual fetch() calls to the API backend in page.evaluate(). Always use the client object:

/**
 * @import { api as Client } from '../../../app/src/plugins/client.js'
 */

const result = await page.evaluate(async () => {
  /** @type {Client} */
  const client = /** @type {any} */(window).client;

  // Use typed API methods
  return await client.apiClient.sseTestProgress({
    steps: 3,
    delay_ms: 500,
    label_prefix: 'Test step'
  });
});

• Check app/src/modules/api-client-v1.js for all available API methods (auto-generated from OpenAPI schema)
• See ../development/testing.md for full documentation

Key Principles

1. Use @testCovers annotations - Enables smart test selection
2. Clean up after tests - Release locks, delete test files
3. API tests run locally - Fast iteration with backend-test-runner.js
4. E2E tests use local server by default - Isolation and speed
5. ALWAYS use helper functions - Never reimplement auth or API utilities

Common Patterns

Authentication in API Tests

import { login, authenticatedApiCall, createAdminSession } from '../helpers/test-auth.js';

// Regular user login
const session = await login('reviewer', 'reviewer', BASE_URL);

// Admin session
const adminSession = await createAdminSession(BASE_URL);

// Authenticated API call
const result = await authenticatedApiCall(
  session.sessionId,
  '/api/endpoint',
  'POST',
  data,
  BASE_URL
);

UI Navigation in E2E Tests

await page.evaluate(() => {
  /** @type {namedElementsTree} */
  const ui = /** @type {any} */(window).ui;
  ui.loginDialog.username.value = 'testuser';
  ui.loginDialog.submit.click();
});

Test Logging for State Verification

import { setupTestConsoleCapture, waitForTestMessage } from './helpers/test-logging.js';

const consoleLogs = setupTestConsoleCapture(page);
// ... perform action ...
const log = await waitForTestMessage(consoleLogs, 'FILE_SAVED');
expect(log.value.file_id).toBeTruthy();

Enhanced Error Reporting:

setupTestConsoleCapture() now includes automatic network request tracking. When a console error occurs, the error message will include:

• The console error message
• Recent failed network requests (within 5 seconds of the error)
• Recent network activity (last 2 seconds)

Example enhanced error output:

Error: Unexpected console error detected: Failed to load resource: the server responded with a status of 404 (Not Found)

Recent failed network requests:
  GET http://localhost:8010/api/v1/files/nonexistent → 404 Not Found

Recent network activity (last 2 seconds):
  GET http://localhost:8010/api/v1/files/list → 200
  POST http://localhost:8010/api/v1/files/save → 200
  GET http://localhost:8010/api/v1/files/nonexistent → 404

This eliminates the need to access server logs or trace files when debugging network-related errors in CI.

Checking State for Shoelace Components

Playwright's accessibility-based matchers don't work properly with Shoelace web components. Always check attributes directly:

Disabled State:

// ❌ WRONG - May fail for Shoelace components
await expect(button).toBeDisabled();
await expect(button).toBeEnabled();

// ✅ CORRECT - Check disabled attribute directly
await expect(button).toHaveAttribute('disabled', '');  // Disabled
await expect(button).not.toHaveAttribute('disabled'); // Enabled

Checked State:

// ❌ WRONG - Fails with "Not a checkbox or radio button"
await expect(checkbox).toBeChecked();
await expect(checkbox).not.toBeChecked();

// ✅ CORRECT - Check checked attribute directly
await expect(checkbox).toHaveAttribute('checked', '');  // Checked
await expect(checkbox).not.toHaveAttribute('checked'); // Unchecked

Dialog Button Clicks:

Shoelace dialogs require a short delay before button clicks to ensure the dialog is fully rendered and interactive:

// Wait for dialog to open
await page.waitForSelector('sl-dialog[name="myDialog"][open]', { timeout: 5000 });

// Fill in form fields
await page.evaluate(() => {
  const ui = window.ui;
  ui.myDialog.inputField.value = 'test value';
});

// CRITICAL: Add 500ms delay before clicking submit
await page.waitForTimeout(500);

await page.evaluate(() => {
  const ui = window.ui;
  ui.myDialog.submit.click();
});

Why this is needed:

• Shoelace dialogs use Shadow DOM and animation
• Clicking too quickly can result in the click being ignored
• 500ms ensures the dialog is fully interactive

This applies to all Shoelace dialog buttons (<sl-button> inside <sl-dialog>).

Critical: Always Use Helper Functions

Never reimplement authentication, API calls, or common test utilities. This causes maintenance burden and bugs.

❌ WRONG - Reimplementing login:

async function loginAsAdmin() {
  const response = await fetch(`${API_BASE}/api/v1/auth/login`, {
    method: 'POST',
    body: JSON.stringify({
      username: 'admin',
      password: 'admin'  // Wrong! Should be passwd_hash with hashing
    })
  });
}

✅ CORRECT - Use existing helper:

import { createAdminSession } from '../helpers/test-auth.js';

async function loginAsAdmin() {
  const { sessionId } = await createAdminSession(API_BASE);
  return sessionId;
}

Common Helper Locations

// Authentication and API calls
import {
  login,
  logout,
  createAdminSession,
  hashPassword,
  authenticatedApiCall,
  authenticatedRequest
} from '../helpers/test-auth.js';

// Role-based login helpers (use standard fixture users)
import {
  loginAsAdmin,      // Returns {sessionId, user} for admin user
  loginAsReviewer,   // Returns {sessionId, user} for reviewer user
  loginAsAnnotator,  // Returns {sessionId, user} for annotator user
  loginAsBasicUser   // Returns {sessionId, user} for basic user (no special roles)
} from '../helpers/test-auth.js';

// Lock management helpers
import {
  tryAcquireLock,  // Returns Response object (check .status)
  acquireLock,     // Throws on failure
  releaseLock,     // Release a held lock
  checkLock        // Check lock status: {is_locked, locked_by}
} from '../helpers/test-auth.js';

// Test logging and state verification
import {
  setupTestConsoleCapture,
  waitForTestMessage
} from './helpers/test-logging.js';

Lock Management in Tests

When testing file operations that require locks:

import { loginAsReviewer, tryAcquireLock, releaseLock } from '../helpers/test-auth.js';

const session = await loginAsReviewer(BASE_URL);

// Try to acquire lock (returns Response for status checking)
const response = await tryAcquireLock(session.sessionId, fileId, BASE_URL);
if (response.status === 200) {
  // Lock acquired - perform operations
  // ...

  // Always release lock in cleanup
  await releaseLock(session.sessionId, fileId, BASE_URL);
} else if (response.status === 403) {
  // Access denied (permission issue)
} else if (response.status === 409) {
  // Conflict (already locked by another user)
}

Role-based login helpers use standard fixture users:

• loginAsAdmin() - User with wildcard (*) access
• loginAsReviewer() - User with reviewer role (can edit gold files)
• loginAsAnnotator() - User with annotator role (can edit version files)
• loginAsBasicUser() - User with no special roles (limited permissions)

Troubleshooting

Missing Dependencies

If tests fail with "test failed" or module import errors, ensure all dependencies are installed:

npm install

This is particularly important for SSE tests which require the eventsource-client package.

Debugging

API Tests

# Verbose output
npm run test:api -- --verbose --grep "save"

# Keep server running for manual testing
npm run test:api -- --no-cleanup

# Keep database for inspection
npm run test:api -- --keep-db

E2E Tests

# Show browser
npm run test:e2e:headed

# Step-through debugging (Playwright inspector)
npm run test:e2e:debug

# Post-mortem debugging (capture artifacts on failure, headless)
npm run test:e2e:debug-failure
npm run test:e2e:debug-failure -- --grep "new version"

# Combine with headed mode to watch test execution
npm run test:e2e -- --debug-on-failure --headed

# Add breakpoints in test code
await page.pause();

Debug-on-Failure Mode

When using --debug-on-failure, tests run in headless mode (faster) and capture comprehensive debug information on failure:

Captured Artifacts:

• Console messages → console-messages.json
• Page errors → page-errors.json
• Network traffic → network-log.json (all HTTP requests/responses with bodies)
• Screenshots (automatically via Playwright)
• Video recording of test execution

Behavior:

• Stops on first failure (--max-failures=1)
• Runs headless by default (use --headed flag to watch execution)
• All artifacts saved to tests/e2e/test-results/<test-name>/

Network Log Format:

[
  {
    "type": "request",
    "timestamp": "2025-01-15T10:30:00.000Z",
    "method": "GET",
    "url": "http://localhost:8000/api/v1/files",
    "headers": {...},
    "postData": null
  },
  {
    "type": "response",
    "timestamp": "2025-01-15T10:30:00.100Z",
    "method": "GET",
    "url": "http://localhost:8000/api/v1/files",
    "status": 200,
    "statusText": "OK",
    "headers": {...},
    "body": "{\"files\": [...]}"
  }
]

Note: All E2E tests should import from the debug-on-failure fixture by default:

import { test, expect } from '../fixtures/debug-on-failure.js';

This has no effect during normal test runs - it only activates when --debug-on-failure is used.

JSDOM Limitations for Browser-Targeted Code

When testing JavaScript code designed for the browser using Node.js with JSDOM, be aware of these limitations:

XPath Namespace Support

JSDOM has incomplete XPath support, particularly for namespace resolvers:

// This works in browsers but may fail in JSDOM:
const result = xmlDoc.evaluate(
  "//tei:bibl",
  xmlDoc,
  (prefix) => prefix === "tei" ? "http://www.tei-c.org/ns/1.0" : null,
  XPathResult.FIRST_ORDERED_NODE_TYPE,
  null
);

Workaround: Export helper functions that accept elements directly, bypassing XPath:

// In the enhancement module:
export function execute(xmlDoc, currentState, configMap) {
  const targetNode = xmlDoc.evaluate(xpath, ...);  // Uses XPath
  processElement(targetNode, xmlDoc);
}

export function processElement(element, xmlDoc) {
  // Core logic - testable without XPath
}

// In tests:
import { processElement } from './enhancement.js';
const bibl = xmlDoc.getElementsByTagNameNS(TEI_NS, 'bibl')[0];
processElement(bibl, xmlDoc);  // Direct element reference

Other JSDOM Limitations

• No XPathResult constants - Must expose via global.XPathResult = dom.window.XPathResult
• Limited Range and Selection APIs - May not behave identically to browsers
• No layout engine - Properties like offsetWidth, getBoundingClientRect() return 0
• Partial Web Component support - Shadow DOM works but custom elements have limitations

JSDOM Setup Pattern

import { JSDOM } from 'jsdom';

const dom = new JSDOM();
global.window = dom.window;
global.document = dom.window.document;
global.Node = dom.window.Node;
global.Document = dom.window.Document;
global.DOMParser = dom.window.DOMParser;
global.XPathResult = dom.window.XPathResult;  // For XPath-using code

When to Use E2E Tests Instead

For code that heavily depends on browser-specific APIs, consider E2E tests with Playwright instead of Node.js unit tests:

• Complex XPath expressions
• Web Component interactions
• Layout-dependent behavior
• CSS styling logic

Important Notes

• API tests run against local server - No containers needed, faster iteration
• E2E tests use local server by default - Containerized mode available but slower
• Use --keep-db - When debugging to preserve state between runs
• Clean up locks - Always release locks in test cleanup
• Fixtures auto-load - Defined in tests/api/fixtures/ and tests/e2e/fixtures/
• Check helpers first - Look in tests/*/helpers/ before implementing utilities

Backend Authentication Requirements

When implementing FastAPI endpoints that require authentication:

from ..lib.dependencies import get_current_user
from fastapi import Depends, HTTPException

def require_admin(current_user: Optional[dict] = Depends(get_current_user)) -> dict:
    if not current_user:
        raise HTTPException(status_code=401, detail="Authentication required")
    # ... check admin role ...
    return current_user

@router.get("/endpoint")
def endpoint(current_user: dict = Depends(require_admin)):
    # Endpoint implementation

Key Points:

• Use Depends(get_current_user) to inject authentication
• FastAPI automatically handles session extraction from X-Session-Id header
• Session header is case-sensitive: X-Session-Id