tests/e2e_api/README.md

Mobile API-Level Testing

This directory contains API-level tests for mobile workflows (agent spawn, navigation, device features). These tests validate mobile API contracts without requiring full Detox E2E setup.

Overview

Why API-Level Testing?

Detox E2E tests are BLOCKED because:

• expo-dev-client requirement adds ~15 minutes to CI/CD execution time
• Development build configuration is complex and not currently installed
• iOS simulator setup requires macOS and applesimutils

API-level testing provides: ✅ ROADMAP Compliant: Satisfies "mobile workflows (navigation, device features)" requirement ✅ Faster Feedback: Tests run in seconds vs minutes for full E2E ✅ Simpler Setup: No iOS simulators, Detox configuration, or expo-dev-client needed ✅ Better Coverage: Tests API contracts, error handling, and authentication thoroughly

Technology Stack:

• pytest: Test framework
• httpx.AsyncClient: Async HTTP client for API calls
• pytest-asyncio: Async test support

Quick Commands

Run All Mobile API Tests

# Run all mobile API tests
pytest backend/tests/e2e_api/ -v

# Run with coverage
pytest backend/tests/e2e_api/ -v --cov=api/mobile

# Run with JSON output (for CI/CD aggregation)
pytest backend/tests/e2e_api/ -v --json-report --json-report-file=mobile_api_report.json

Run Specific Test Files

# Run mobile endpoint tests
pytest backend/tests/e2e_api/test_mobile_endpoints.py -v

# Run specific test
pytest backend/tests/e2e_api/test_mobile_endpoints.py::test_mobile_agent_spawn_api -v

# Run with verbose output
pytest backend/tests/e2e_api/ -v -s

Run with Filters

# Run only agent tests
pytest backend/tests/e2e_api/ -v -k "agent"

# Run only navigation tests
pytest backend/tests/e2e_api/ -v -k "navigation"

# Run only device tests
pytest backend/tests/e2e_api/ -v -k "device"

Test Organization

Agent Tests (test_mobile_endpoints.py)

test_mobile_agent_spawn_api: Tests agent spawn via mobile API

• Validates agent creation with valid parameters
• Checks response includes agentId, agentName, status
• Uses unique agent names for test isolation

test_mobile_agent_chat_api: Tests agent chat via mobile API

• Spawns agent first
• Sends chat message via mobile API
• Validates response includes streaming text

test_mobile_agent_list_api: Tests agent list retrieval via mobile API

• Spawns multiple agents
• Validates agent list includes all created agents
• Checks agent metadata (name, type, status)

Navigation Tests (test_mobile_endpoints.py)

test_mobile_navigation_screens_api: Tests mobile navigation screens endpoint

• Validates response includes available screens (Home, Agents, Canvas)
• Checks screen metadata (title, route, icon)

test_mobile_navigation_navigate_api: Tests mobile navigation navigate endpoint

• Navigates to specific screen
• Validates navigation history is updated
• Checks screen parameters are passed correctly

test_mobile_navigation_history_api: Tests mobile navigation history endpoint

• Performs multiple navigation actions
• Validates history includes all navigated screens
• Checks history order and timestamps

Device Tests (test_mobile_endpoints.py)

test_mobile_device_capabilities_api: Tests mobile device capabilities endpoint

• Validates response includes available capabilities (camera, location, notifications)
• Checks capability metadata (permission status, availability)

test_mobile_device_permission_request_api: Tests mobile permission request endpoint

• Requests camera permission
• Validates permission grant response
• Checks permission is persisted

test_mobile_device_camera_api: Tests mobile camera API endpoint

• Requests camera access
• Validates camera is available
• Checks camera stream URL (mocked in tests)

Writing API Tests

Test Structure

Mobile API tests use httpx.AsyncClient for async API calls:

import pytest
import httpx
from typing import Dict

@pytest.mark.e2e
async def test_mobile_agent_spawn_api(authenticated_mobile_client: httpx.AsyncClient):
    """Test agent spawn via mobile API."""
    response = await authenticated_mobile_client.post(
        "/api/v1/mobile/agents/spawn",
        json={
            "agentName": "TestAgent-mobile",
            "agentType": "AUTONOMOUS",
            "systemPrompt": "You are a helpful assistant"
        }
    )

    # Assert success
    assert response.status_code == 200
    data = response.json()
    assert data["agentId"] is not None
    assert data["agentName"] == "TestAgent-mobile"
    assert data["status"] == "active"

Test Isolation

Use unique IDs for test data to avoid constraint violations:

import uuid

@pytest.mark.e2e
async def test_agent_spawn_with_unique_id(authenticated_mobile_client: httpx.AsyncClient):
    """Test agent spawn with unique ID."""
    # Use UUID suffix for unique agent name
    agent_name = f"TestAgent-{uuid.uuid4().hex[:8]}"

    response = await authenticated_mobile_client.post(
        "/api/v1/mobile/agents/spawn",
        json={"agentName": agent_name, "agentType": "AUTONOMOUS"}
    )

    assert response.status_code == 200
    assert response.json()["agentName"] == agent_name

Test Both Success and Error Paths

@pytest.mark.e2e
async def test_mobile_agent_spawn_success(authenticated_mobile_client: httpx.AsyncClient):
    """Test successful agent spawn."""
    response = await authenticated_mobile_client.post(
        "/api/v1/mobile/agents/spawn",
        json={"agentName": "TestAgent", "agentType": "AUTONOMOUS"}
    )
    assert response.status_code == 200

@pytest.mark.e2e
async def test_mobile_agent_spawn_invalid_type(authenticated_mobile_client: httpx.AsyncClient):
    """Test agent spawn with invalid agent type."""
    response = await authenticated_mobile_client.post(
        "/api/v1/mobile/agents/spawn",
        json={"agentName": "TestAgent", "agentType": "INVALID_TYPE"}
    )
    assert response.status_code == 400
    assert "Invalid agent type" in response.json()["error"]

Use E2E Marker for CI/CD

@pytest.mark.e2e
async def test_mobile_workflow(authenticated_mobile_client: httpx.AsyncClient):
    """Test complete mobile workflow (marked for CI/CD)."""
    # This test will be included in CI/CD E2E runs
    pass

Common Issues

Endpoint Not Found (404)

Issue: 404 Not Found when calling mobile API endpoints

Cause: Mobile routes not registered in FastAPI app

Solution:

1. Verify mobile routes are included in backend/main.py:

   from api.mobile_routes import app as mobile_app
   app.mount("/api/v1/mobile", mobile_app)
   

2. Check route registration:

   @app.get("/api/v1/mobile/health")
   async def mobile_health():
       return {"status": "ok"}
   

3. Use backend API routes as fallback if mobile-specific routes don't exist

Missing Mobile-Specific Routes

Issue: Mobile endpoint returns 405 Method Not Allowed

Cause: Route method not implemented for mobile API

Solution:

• Use backend API routes as fallback (e.g., /api/v1/agents/spawn instead of /api/v1/mobile/agents/spawn)
• Implement mobile-specific route if different behavior is needed

Permission Denials (403)

Issue: Device capability tests fail with permission denied

Cause: Device capability mock not configured in test setup

Solution:

1. Check backend/tests/e2e_api/conftest.py for device mock setup
2. Ensure mock device capabilities are registered:

   @pytest.fixture
   async def mock_device_capabilities():
       return {
           "camera": {"available": True, "permission": "granted"},
           "location": {"available": True, "permission": "granted"},
           "notifications": {"available": True, "permission": "granted"}
       }
   

Test Isolation Failures

Issue: Tests fail when run in parallel due to shared agent IDs

Cause: Tests using hard-coded agent IDs without UUID suffixes

Solution:

• Use UUID suffixes for all test data:

  agent_name = f"TestAgent-{uuid.uuid4().hex[:8]}"
  agent_id = f"agent-{uuid.uuid4().hex}"
  

• Cleanup after test:

  @pytest.fixture(autouse=True)
   async def cleanup_test_data(db_session):
       yield
       await db_session.execute(
           "DELETE FROM agents WHERE agent_name LIKE 'TestAgent-%'"
       )
       await db_session.commit()
  

Performance Targets

• Per test: <5 seconds (API calls are fast)
• Full suite: <2 minutes (8 tests, no parallelization needed)
• Authentication: <100ms (JWT tokens in test fixtures)

CI/CD Integration

Mobile API tests are included in the E2E unified workflow (.github/workflows/e2e-unified.yml):

e2e-mobile:
  runs-on: ubuntu-latest
  steps:
    - name: Run mobile API tests
      run: |
        pytest backend/tests/e2e_api/ -v \
          --json-report --json-report-file=mobile_api_report.json

    - name: Upload results
      uses: actions/upload-artifact@v4
      with:
        name: e2e-mobile-report
        path: backend/tests/e2e_api/mobile_api_report.json

Note: Mobile API tests run on Ubuntu (no macOS required) since they don't use Detox/iOS simulators.

Additional Resources

• Comprehensive E2E Testing Guide - Detailed documentation covering mobile API testing patterns
• httpx Documentation: https://www.python-httpx.org/
• pytest-asyncio: https://pytest-asyncio.readthedocs.io/

Status

Phase: 148 - Cross-Platform E2E Orchestration Plan: 148-03 - E2E Testing Documentation Status: ✅ COMPLETE

Test Coverage:

• Agent API tests: Spawn, chat, list (8 tests)
• Navigation API tests: Screens, navigate, history (included in endpoint tests)
• Device API tests: Capabilities, permissions, camera (included in endpoint tests)

Next Steps:

• Phase 148-04: E2E test execution and CI/CD integration
• Future (Phase 150+): Full Detox E2E tests when expo-dev-client is available