API_TESTING/README.md

API Test Suite Documentation

Overview

The Secure AI Agents Suite includes a comprehensive API testing framework that validates all external service integrations, ensuring your system works correctly with OpenAI, Google ML, ElevenLabs, and Modal platforms.

Quick Start

1. Setup Configuration

# Copy the configuration template
cp api_test_config.yaml my_config.yaml

# Edit with your API keys
nano my_config.yaml

2. Run All Tests

# Run all API tests
python test_runner.py

# Run specific test
python test_runner.py --test openai

# Use custom config
python test_runner.py --config my_config.yaml

3. View Results

# View summary
python test_runner.py --show-summary

# Validate config only
python test_runner.py --validate-only

Configuration

API Keys Required

Edit api_test_config.yaml with your actual API keys:

# OpenAI Configuration
openai:
  api_key: "sk-your-openai-api-key-here"
  base_url: "https://api.openai.com/v1"
  timeout: 30
  rate_limit: 100

# Google ML Configuration  
google:
  api_key: "your-google-api-key-here"
  base_url: "https://generativelanguage.googleapis.com/v1"
  timeout: 30
  rate_limit: 100

# ElevenLabs Configuration
elevenlabs:
  api_key: "your-elevenlabs-api-key-here"
  base_url: "https://api.elevenlabs.io/v1"
  timeout: 60
  rate_limit: 100

# Modal Configuration
modal:
  api_key: "your-modal-api-key-here"
  base_url: "https://modal.com/api"
  timeout: 30
  rate_limit: 100

Test Configuration

Customize test behavior:

test_config:
  # Minimum success rate to consider tests passing (percentage)
  success_rate_threshold: 80
  
  # Timeout for individual requests (seconds)
  request_timeout: 30
  
  # Maximum retries for failed requests
  max_retries: 3
  
  # Performance thresholds (in milliseconds)
  performance_thresholds:
    openai_text_generation: 5000
    google_text_generation: 5000
    elevenlabs_tts: 10000
    modal_execution: 30000

Test Categories

OpenAI Tests

• Connection Test: Validates API connectivity and authentication
• Text Generation Test: Tests GPT models with sample prompts
• Batch Processing Test: Validates multiple concurrent requests

Google ML Tests

• Connection Test: Validates Google Generative AI API access
• Text Generation Test: Tests various Google AI models

ElevenLabs Tests

• Connection Test: Validates voice API connectivity
• Text-to-Speech Test: Tests audio generation capabilities
• Voice Cloning Test: Validates custom voice functionality

Modal Tests

• Connection Test: Validates Modal platform access
• Function Deployment Test: Tests serverless function deployment

Usage Examples

Basic Usage

# Run all tests with default settings
python test_runner.py

# Run only OpenAI tests
python test_runner.py --test openai

# Run with custom config and output file
python test_runner.py --config production_config.yaml --output results_production.json

CI/CD Integration

# Use in automated pipelines
python test_runner.py --config ci_config.yaml --output ci_results.json

# Check for configuration errors
python test_runner.py --validate-only

Performance Monitoring

# Run all tests and save detailed results
python test_runner.py --output performance_report_$(date +%Y%m%d).json

# Run specific performance tests
python test_runner.py --test elevenlabs

Results Format

Test results are saved in JSON format with the following structure:

{
  "summary": {
    "timestamp": "2025-11-30T09:49:17.197Z",
    "total_tests": 12,
    "passed_tests": 11,
    "failed_tests": 1,
    "success_rate": 91.7,
    "total_duration": 45.23,
    "api_availability": {
      "openai": true,
      "google": true,
      "elevenlabs": true,
      "modal": true
    }
  },
  "test_results": {
    "openai_connection": {
      "success": true,
      "duration": 1.23,
      "message": "Connection successful"
    },
    "openai_text_generation": {
      "success": true,
      "duration": 2.45,
      "message": "Text generation successful",
      "performance_score": 85
    }
  },
  "performance_metrics": {
    "average_response_time": 2.1,
    "success_rate": 91.7,
    "api_uptime": 100.0
  }
}

Interpreting Results

Success Indicators

• Success Rate: Aim for >80% for production readiness
• Response Time: Should be within configured thresholds
• API Availability: All APIs should be accessible

Performance Thresholds

• OpenAI Text Generation: <5 seconds
• Google Text Generation: <5 seconds
• ElevenLabs TTS: <10 seconds
• Modal Execution: <30 seconds

Troubleshooting

Common Issues

1. API Key Errors

# Check configuration
python test_runner.py --validate-only

# Verify keys in configuration file
cat api_test_config.yaml | grep api_key

2. Network Connectivity

# Test individual API connectivity
python test_runner.py --test openai

# Check internet connection
ping api.openai.com

3. Rate Limiting

• Reduce rate_limit in configuration
• Implement exponential backoff
• Monitor API usage quotas

4. Performance Issues

• Check network latency
• Verify API service status
• Review configuration timeouts

Best Practices

Security

• Never commit API keys to version control
• Use environment variables for sensitive data
• Regularly rotate API keys
• Monitor API usage for anomalies

Monitoring

• Run tests regularly (daily/weekly)
• Track performance trends over time
• Set up alerts for test failures
• Monitor API rate limits and quotas

CI/CD Integration

# In your CI pipeline
- name: API Tests
  run: |
    python test_runner.py --config ci_config.yaml --output test_results.json
    if [ $(jq '.summary.success_rate' test_results.json) -lt 80 ]; then
      echo "API tests failed"
      exit 1
    fi

Advanced Usage

Custom Test Cases

Extend the test suite by adding custom test methods in api_test_suite.py:

def test_custom_functionality(self):
    """Test custom API functionality"""
    try:
        # Your custom test logic here
        result = self.make_request("your_custom_endpoint")
        return {
            'success': True,
            'duration': result['duration'],
            'message': 'Custom test passed'
        }
    except Exception as e:
        return {
            'success': False,
            'error': str(e),
            'duration': 0
        }

Performance Monitoring

Track API performance over time:

# Add to your monitoring script
import json
from datetime import datetime

def track_performance(results_file):
    with open(results_file, 'r') as f:
        results = json.load(f)
    
    metrics = {
        'timestamp': datetime.now().isoformat(),
        'success_rate': results['summary']['success_rate'],
        'avg_response_time': results['performance_metrics']['average_response_time']
    }
    
    # Save to performance database
    save_metrics(metrics)

Support

For issues or questions:

1. Check the troubleshooting section above
2. Review API service status pages
3. Validate your configuration using --validate-only
4. Run individual tests to isolate problems

---

Last Updated: November 30, 2025
Version: 1.0.0
Compatibility: Python 3.8+