docs/TESTING.md

VoiceForge Testing Guide

Overview

VoiceForge uses a comprehensive, multi-layered testing strategy to ensure code quality, security, and performance.

Test Structure

backend/tests/
├── unit/                    # Service-level unit tests
│   ├── test_stt_service.py
│   ├── test_tts_service.py
│   ├── test_translation_service.py
│   ├── test_emotion_meeting_service.py
│   ├── test_audio.py
│   ├── test_cloning.py
│   ├── test_export.py
│   ├── test_nlp.py
│   └── test_sign.py
├── integration/             # API & E2E tests
│   ├── test_auth.py
│   ├── test_api_integration.py
│   ├── test_e2e_full_flow.py
│   ├── test_diarization.py
│   └── test_project_health.py
├── performance/             # Load & benchmark tests
│   ├── locustfile.py
│   ├── benchmark_*.py
│   └── run_benchmarks.py
├── quality/                 # Code quality tools
│   ├── analyze_codebase.py
│   ├── check_syntax.py
│   ├── check_dependencies.py
│   ├── check_pipeline.py
│   ├── coverage_tracker.py
│   ├── lighthouse_audit.py
│   └── project_audit.py
├── security/                # Security audits
│   └── run_audit.py
├── conftest.py              # Shared fixtures
├── pytest.ini               # Pytest configuration
└── run_all_tests.py         # Master test runner

Running Tests

Quick Start

cd backend
python tests/run_all_tests.py

Individual Test Categories

# Unit tests
pytest tests/unit -v

# Integration tests
pytest tests/integration -v

# Performance benchmarks
python tests/performance/run_benchmarks.py

# Security audit
python tests/security/run_audit.py

Docker (Recommended)

# Run all tests in clean Docker environment
docker-compose run --rm backend python tests/run_all_tests.py

# Run specific category
docker-compose run --rm backend pytest tests/unit -v

Quality Tools

Code Quality Analysis

python tests/quality/analyze_codebase.py --path app

Checks:

• File sizes (flags >500 lines)
• Cyclomatic complexity (Radon)
• Maintainability index
• Long functions (>50 lines)
• Import dependencies

Syntax & Import Check

python tests/quality/check_syntax.py --path app

Checks:

• Python syntax errors
• Circular imports
• Missing __init__.py files

Dependency Health

python tests/quality/check_dependencies.py

Checks:

• Local pip compatibility (pip check)
• PyPI availability (online check)
• Outdated packages
• Security vulnerabilities (via pip-audit)

Pipeline Validation

python tests/quality/check_pipeline.py --root ..

Checks:

• GitHub Actions workflow syntax
• Dockerfile validation
• docker-compose.yml structure
• Environment file presence

Coverage Tracking

python tests/quality/coverage_tracker.py --app app --tests tests

Provides:

• Module coverage matrix
• Untested function identification
• Coverage percentage by component

Frontend Performance (Lighthouse)

python tests/quality/lighthouse_audit.py --url http://localhost:8501

Audits:

• Performance score
• Accessibility score
• Best practices
• SEO metrics

Test Fixtures

Database Setup

@pytest.fixture(scope="module")
def setup_db():
    Base.metadata.create_all(bind=engine)
    yield
    # Cleanup optional

Test Client (Sync)

@pytest.fixture
def client():
    return TestClient(app)

Async Client

@pytest_asyncio.fixture
async def async_client():
    transport = ASGITransport(app=app)
    async with AsyncClient(transport=transport, base_url="http://test") as client:
        yield client

Best Practices

Writing Unit Tests

• Test one function per test case
• Use mocks for external dependencies
• Follow AAA pattern (Arrange, Act, Assert)

@pytest.mark.asyncio
async def test_synthesize_returns_audio_bytes():
    # Arrange
    with patch('app.services.edge_tts_service.EdgeTTSService') as MockService:
        mock_service = MockService.return_value
        mock_service.synthesize = AsyncMock(return_value=b'AUDIO_DATA')
        
        # Act
        audio = await mock_service.synthesize("Hello", voice="en-US-Neural2-F")
        
        # Assert
        assert isinstance(audio, bytes)
        assert len(audio) > 0

Writing Integration Tests

• Test realistic user flows
• Use actual database (with cleanup)
• Test authentication & authorization

def test_full_user_journey(client):
    # Register
    response = client.post("/api/v1/auth/register", json={...})
    assert response.status_code == 200
    
    # Login
    response = client.post("/api/v1/auth/login", data={...})
    token = response.json()["access_token"]
    
    # Use protected endpoint
    response = client.get("/api/v1/auth/me", headers={"Authorization": f"Bearer {token}"})
    assert response.status_code == 200

CI/CD Integration

GitHub Actions Workflow

name: Test Suite
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Run tests
        run: |
          docker-compose run --rm backend python tests/run_all_tests.py

Coverage Goals

• Unit Tests: >80% line coverage
• Integration Tests: All critical user flows
• Performance: All endpoints benchmarked
• Security: Zero high/critical vulnerabilities

Current Status

✅ 74+ tests across all categories
✅ 100% module coverage (all services have tests)
✅ 7 quality tools for automated analysis
✅ Master runner for one-command execution