test/README.md

# NL→SQL Leaderboard Tests

This directory contains all test files for the NL→SQL Leaderboard project.

## Test Structure

```
test/
├── __init__.py              # Test package initialization
├── conftest.py              # Pytest configuration and fixtures
├── test_config.py           # Configuration loading tests
├── test_evaluation.py       # Evaluation pipeline tests
├── test_models.py           # Model testing utilities
├── test_system.py           # System integration tests
└── README.md                # This file
```

## Running Tests

### Quick Test Run
```bash
python run_tests.py
```

### Using pytest directly
```bash
# Run all tests
pytest test/

# Run specific test file
pytest test/test_config.py

# Run with coverage
pytest test/ --cov=src --cov-report=html

# Run only fast tests
pytest test/ -m "not slow"
```

### Test Categories

- **Unit Tests**: Fast, isolated tests for individual components
- **Integration Tests**: Tests that verify component interactions
- **System Tests**: End-to-end tests of the complete system

## Test Configuration

Tests are configured to run in mock mode by default:
- `MOCK_MODE=true` - Uses mock SQL generation
- `HF_TOKEN=""` - Prevents real API calls
- All external dependencies are mocked

## Test Fixtures

- `mock_mode`: Ensures mock mode is enabled
- `test_data_dir`: Path to test data directory
- `config_dir`: Path to configuration directory

## Writing New Tests

1. Create test files with `test_*.py` naming
2. Use descriptive test function names starting with `test_`
3. Use fixtures from `conftest.py` when needed
4. Mark slow tests with `@pytest.mark.slow`
5. Use proper assertions and error messages

## Test Coverage

The test suite aims for comprehensive coverage:
- Configuration loading and validation
- Model registry functionality
- Evaluation pipeline
- Scoring and metrics
- UI components
- Error handling

## Continuous Integration

Tests are designed to run in CI/CD environments:
- No external dependencies required
- Mock mode prevents API calls
- Fast execution for quick feedback
- Comprehensive coverage reporting