security/ci_security.md

# AegisLM CI/CD Security

## Overview

This document defines the security requirements and controls for the AegisLM CI/CD pipeline. All pipeline changes must comply with these security standards.

## CI/CD Security Architecture

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                        CI/CD Pipeline Security Flow                          │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌──────────────┐                                                           │
│  │    GitHub    │                                                           │
│  │   Push/PR    │                                                           │
│  └──────┬───────┘                                                           │
│         │                                                                    │
│         ▼                                                                    │
│  ┌─────────────────────────────────────────────────────────────┐            │
│  │                    GitHub Actions Workflow                   │            │
│  └─────────────────────────────────────────────────────────────┘            │
│         │                                                                    │
│         ├──► Lint & Type Check                                             │
│         │         └── Ruff, MyPy                                           │
│         │                                                                    │
│         ├──► Schema Validation                                             │
│         │         └── Pydantic validation                                   │
│         │                                                                    │
│         ├──► Unit Tests                                                    │
│         │         └── pytest with coverage                                  │
│         │                                                                    │
│         ├──► Docker Build                                                  │
│         │         └── Multi-stage build                                    │
│         │                                                                    │
│         ├──► Security Scan ◄────────────────────────────────────┐          │
│         │         ├── Dependency scan (safety)                 │          │
│         │         ├── Secret detection (trufflehog)           │          │
│         │         ├── Container scan (trivy)                  │          │
│         │         └── SAST (bandit)                          │          │
│         │                                                            │          │
│         └──► Artifact Signing ◄───────────────────────────────────┘          │
│                   └── Cosign, GPG                                         │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────────┘
```

---

## Security Controls

### ✅ Required Security Checks

| Check | Tool | Severity | Status |
|-------|------|----------|--------|
| Dependency vulnerability scan | `safety`, `pip-audit` | Critical | ✅ Implemented |
| Secret detection | `trufflehog`, `gitleaks` | Critical | 🔄 To Implement |
| Container image scan | `trivy`, `clair` | High | 🔄 To Implement |
| SAST (Static Analysis) | `bandit`, `semgrep` | High | 🔄 To Implement |
| Code linting | `ruff` | Medium | ✅ Implemented |
| Type checking | `mypy` | Medium | ✅ Implemented |
| Artifact signing | `cosign`, `GPG` | High | 🔄 To Implement |

---

## Enhanced GitHub Actions Workflow

Update `.github/workflows/ci.yml` with enhanced security:

```
yaml
# AegisLM CI/CD Pipeline - Enhanced Security Version
# GitHub Actions workflow with comprehensive security scanning

name: CI

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

env:
  # Security settings
  TRUFFLEHOG_VERSION: "3.63.1"
  TRIVY_VERSION: "0.49.1"
  SEMGREP_VERSION: "1.64.0"

jobs:
  # ===========================================================================
  # Secret Detection - Fail on secrets
  # ===========================================================================
  secret-detection:
    name: Secret Detection
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for trufflehog
      
      - name: Run Trufflehog
        uses: trufflesecurity/trufflehog@main
        with:
          path: ./
          base: ${{ github.event.repository.default_branch }}
          head: HEAD
          extra_args: --fail --verified-only
      
      - name: Run Gitleaks
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GITLEAKS_ENABLE_UPDATES: "false"

  # ===========================================================================
  # SAST - Static Application Security Testing
  # ===========================================================================
  sast:
    name: SAST Analysis
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Run Semgrep
        uses: returntocorp/semgrep-action@v1
        with:
          config: auto
          generateSarif: true
      
      - name: Upload Semgrep results
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: semgrep.sarif
      
      - name: Run Bandit
        run: |
          pip install bandit
          bandit -r backend/ -f sarif -o bandit.sarif || true
        continue-on-error: true

  # ===========================================================================
  # Lint and Type Check
  # ===========================================================================
  lint:
    name: Lint & Type Check
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'
      
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          pip install ruff mypy
      
      - name: Run Ruff linter
        run: ruff check . --output-format=github
      
      - name: Run MyPy type checker
        run: mypy backend --ignore-missing-imports --no-error-summary
        continue-on-error: true

  # ===========================================================================
  # Validate Pydantic Schemas
  # ===========================================================================
  schema-validation:
    name: Validate Pydantic Schemas
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'
      
      - name: Install dependencies
        run: pip install -r requirements.txt
      
      - name: Validate Pydantic schemas
        run: |
          python -c "
            from backend.benchmarking.schemas import BenchmarkConfig, BenchmarkWeights
            from backend.core.config import settings
            from backend.db.models import EvaluationRun
            print('All Pydantic schemas imported successfully')
          "
      
      - name: Validate weights sum to 1
        run: |
          python -c "
            from backend.benchmarking.schemas import BenchmarkWeights
            weights = BenchmarkWeights()
            total = weights.hallucination + weights.toxicity + weights.bias + weights.confidence
            if abs(total - 1.0) > 1e-6:
              raise ValueError(f'Weights must sum to 1.0, got {total}')
            print(f'Weights validation passed: sum = {total}')
          "

  # ===========================================================================
  # Unit Tests
  # ===========================================================================
  tests:
    name: Unit Tests
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'
      
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          pip install pytest pytest-asyncio pytest-cov
      
      - name: Run unit tests
        run: pytest tests/ -v --cov=backend --cov-report=xml --cov-report=term-missing
        continue-on-error: true
      
      - name: Upload coverage
        uses: actions/upload-artifact@v4
        with:
          name: coverage-report
          path: coverage.xml

  # ===========================================================================
  # Docker Build with Security Scanning
  # ===========================================================================
  docker:
    name: Docker Build & Scan
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      
      - name: Build Docker image
        uses: docker/build-push-action@v5
        with:
          context: .
          push: false
          tags: aegislm:test
          load: true
          cache-from: type=gha
          cache-to: type=gha,mode=max
      
      - name: Run Trivy scanner
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: 'aegislm:test'
          format: sarif
          output: 'trivy-results.sarif'
          severity: 'CRITICAL,HIGH'
          exit-code: '1'
          ignore-unfixed: true
      
      - name: Upload Trivy results
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: 'trivy-results.sarif'

  # ===========================================================================
  # Dependency Security Scan
  # ===========================================================================
  dependency-security:
    name: Dependency Security
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'
      
      - name: Install dependencies
        run: pip install -r requirements.txt
      
      - name: Run pip-audit
        uses: pypa/pip-audit@main
        with:
          pip-audit-args: --exclude-editable --require-hashes
      
      - name: Run safety
        run: |
          pip install safety
          safety check --json > safety-results.json || true
        continue-on-error: true
      
      - name: Upload security results
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: security-results
          path: |
            safety-results.json
            trivy-results.sarif

  # ===========================================================================
  # Build Summary
  # ===========================================================================
  summary:
    name: Build Summary
    needs: [secret-detection, sast, lint, schema-validation, tests, docker, dependency-security]
    runs-on: ubuntu-latest
    if: always()
    
    steps:
      - name: Summary
        run: |
          echo "## Build Summary" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "| Job | Status |" >> $GITHUB_STEP_SUMMARY
          echo "|-----|--------|" >> $GITHUB_STEP_SUMMARY
          echo "| Secret Detection | ${{ needs.secret-detection.result }} |" >> $GITHUB_STEP_SUMMARY
          echo "| SAST Analysis | ${{ needs.sast.result }} |" >> $GITHUB_STEP_SUMMARY
          echo "| Lint & Type Check | ${{ needs.lint.result }} |" >> $GITHUB_STEP_SUMMARY
          echo "| Schema Validation | ${{ needs.schema-validation.result }} |" >> $GITHUB_STEP_SUMMARY
          echo "| Unit Tests | ${{ needs.tests.result }} |" >> $GITHUB_STEP_SUMMARY
          echo "| Docker Build & Scan | ${{ needs.docker.result }} |" >> $GITHUB_STEP_SUMMARY
          echo "| Dependency Security | ${{ needs.dependency-security.result }} |" >> $GITHUB_STEP_SUMMARY
          
          # Fail if critical security issues found
          if [ "${{ needs.secret-detection.result }}" == "failure" ]; then
            echo "" >> $GITHUB_STEP_SUMMARY
            echo "❌ **SECURITY FAILURE**: Secrets detected in code" >> $GITHUB_STEP_SUMMARY
            exit 1
          fi

  # ===========================================================================
  # Artifact Signing (for releases)
  # ===========================================================================
  sign:
    name: Sign Artifacts
    needs: [lint, tests, docker]
    runs-on: ubuntu-latest
    if: github.event_name == 'release'
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Cosign
        uses: sigstore/cosign-installer@v3
      
      - name: Sign Docker image
        run: |
          cosign sign --yes aegislm/test:latest
      
      - name: Sign Python packages
        run: |
          # Generate key pair
          cosign generate-key-pair
          
          # Sign wheel files
          for wheel in dist/*.whl; do
            cosign sign-file --key cosign.key "$wheel"
          done
      
      - name: Upload signatures
        uses: actions/upload-artifact@v4
        with:
          name: signatures
          path: |
            *.sig
            cosign.pub
```

---

## Security Scanning Configuration

### TruffleHog Configuration

Create `.trufflehog.yaml`:

```
yaml
# TruffleHog configuration
directories:
  - .
exclude:
  - .git
  - node_modules
  - venv
  - __pycache__

rules:
  - name: AWS Access Key
    pattern: 'AKIA[0-9A-Z]{16}'
    entropy: true
  
  - name: GitHub Token
    pattern: 'gh[pousr]_[A-Za-z0-9_]{36,255}'
    entropy: true
  
  - name: JWT Token
    pattern: 'eyJ[A-Za-z0-9-_=]+\.eyJ[A-Za-z0-9-_=]+\.[A-Za-z0-9-_.+/=]*'
    entropy: true
  
  - name: Private Key
    pattern: '-----BEGIN (RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----'
    entropy: false
  
  - name: Generic API Key
    pattern: '(?i)(api[_-]?key|apikey|secret[_-]?key)[\s:=]+["\']?[a-zA-Z0-9]{16,}'
    entropy: true
```

### Semgrep Configuration

Create `.semgrep.yaml`:

```
yaml
rules:
  - id: python-secrets
    pattern: |
      os.environ["$KEY"]
    message: Hardcoded secrets in environment variables
    languages:
      - python
    severity: ERROR
    metadata:
      cwe: "CWE-798: Use of Hard-coded Credentials"
      owasp: "A2:2017-Broken Authentication"
  
  - id: python-sql-injection
    pattern: |
      cursor.execute("SELECT ..." + $USER_INPUT)
    message: Potential SQL injection
    languages:
      - python
    severity: ERROR
    metadata:
      cwe: "CWE-89: SQL Injection"
      owasp: "A1:2017-Injection"
  
  - id: python-path-traversal
    pattern: |
      open($FILE, ...)
    message: Potential path traversal
    languages:
      - python
    severity: WARNING
    metadata:
      cwe: "CWE-22: Path Traversal"
```

---

## Dependency Vulnerability Policy

### Severity Thresholds

| Severity | Action | Pipeline Result |
|----------|--------|-----------------|
| Critical | Fail build | ❌ Fail |
| High | Fail build | ❌ Fail |
| Medium | Warning only | ⚠️ Warn |
| Low | Info only | ℹ️ Info |

### Allowed Vulnerabilities

For legitimate exceptions, create `.vuln-allowlist.json`:

```
json
{
  "allowed_vulnerabilities": [
    {
      "cve": "CVE-2024-12345",
      "package": "some-library",
      "reason": "False positive - not used in our code path",
      "expires": "2024-12-31"
    }
  ]
}
```

---

## Artifact Signing

### Cosign Setup

```
bash
# Install cosign
brew install cosign

# Generate key pair
cosign generate-key-pair

# Sign a container image
cosign sign aegislm/api:v1.0.0

# Verify signature
cosign verify aegislm/api:v1.0.0 --key cosign.pub
```

### Python Package Signing

```
yaml
# In release workflow
- name: Build Python package
  run: |
    python -m build

- name: Sign packages
  run: |
    for file in dist/*.whl; do
      cosign sign-file --key cosign.key "$file"
    done
```

---

## Pipeline Security Best Practices

### 1. Secrets Management

```
yaml
# Use GitHub Secrets for sensitive values
env:
  # Don't use secrets directly in env!
  # Instead, use the secrets context
  API_TOKEN: ${{ secrets.API_TOKEN }}
```

### 2. Permission Minimization

```
yaml
# Use minimal permissions
permissions:
  contents: read
  security-events: write
  actions: read
  checks: write
```

### 3. Cache Security

```
yaml
# Use secure cache
- name: Setup Python
  uses: actions/setup-python@v5
  with:
    python-version: '3.11'
    cache: 'pip'  # Uses hash of requirements.txt
```

---

## Compliance Mapping

| Control | Implementation | Standard |
|---------|---------------|----------|
| Secret scanning | TruffleHog, Gitleaks | PCI DSS 6.5.10 |
| SAST | Semgrep, Bandit | OWASP Top 10 |
| Dependency scanning | pip-audit, Safety | NIST 800-53 |
| Container scanning | Trivy | CIS Docker |
| Code signing | Cosign | NIST 800-53 |

---

## Emergency Procedures

### Handling Security Vulnerabilities

1. **Immediate**: Block vulnerable images in registry
2. **Short-term**: Patch vulnerable dependencies
3. **Long-term**: Update to secure alternatives

### Pipeline Failure Response

```
┌─────────────────────────────────────────────────────────────────┐
│                 Security Failure Response                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Security Scan Fails                                            │
│         │                                                        │
│         ▼                                                        │
│  ┌─────────────┐                                                 │
│  │ Identify    │  → Review vulnerability details                 │
│  │ Issue       │  → Check if applicable to our code             │
│  └──────┬──────┘                                                 │
│         │                                                        │
│         ▼                                                        │
│  ┌─────────────┐                                                 │
│  │ Decide      │  → Patch available? Use it                     │
│  │ Action      │  → No patch? Add to allowlist (temp)          │
│  └──────┬──────┘  → Critical? Block deployment                │
│         │                                                        │
│         ▼                                                        │
│  ┌─────────────┐                                                 │
│  │ Document    │  → Create security issue                       │
│  │ & Notify    │  → Notify security team                        │
│  └─────────────┘                                                 │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

---

## References

- [GitHub Actions Security](https://docs.github.com/en/actions/security-guides)
- [OWASP Top 10](https://owasp.org/www-project-top-ten/)
- [NIST 800-53 Security Controls](https://csrc.nist.gov/publications/detail/sp/800-53/rev-5/final)
- [Cosign Documentation](https://docs.sigstore.dev/cosign/signing/overview/)
- [TruffleHog](https://trufflesecurity.com/trufflehog/)