AegisLM CI/CD Security
Overview
This document defines the security requirements and controls for the AegisLM CI/CD pipeline. All pipeline changes must comply with these security standards.
CI/CD Security Architecture
┌─────────────────────────────────────────────────────────────────────────────┐
│ CI/CD Pipeline Security Flow │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ │
│ │ GitHub │ │
│ │ Push/PR │ │
│ └──────┬───────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ GitHub Actions Workflow │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │ │
│ ├──► Lint & Type Check │
│ │ └── Ruff, MyPy │
│ │ │
│ ├──► Schema Validation │
│ │ └── Pydantic validation │
│ │ │
│ ├──► Unit Tests │
│ │ └── pytest with coverage │
│ │ │
│ ├──► Docker Build │
│ │ └── Multi-stage build │
│ │ │
│ ├──► Security Scan ◄────────────────────────────────────┐ │
│ │ ├── Dependency scan (safety) │ │
│ │ ├── Secret detection (trufflehog) │ │
│ │ ├── Container scan (trivy) │ │
│ │ └── SAST (bandit) │ │
│ │ │ │
│ └──► Artifact Signing ◄───────────────────────────────────┘ │
│ └── Cosign, GPG │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Security Controls
✅ Required Security Checks
| Check | Tool | Severity | Status |
|---|---|---|---|
| Dependency vulnerability scan | safety, pip-audit |
Critical | ✅ Implemented |
| Secret detection | trufflehog, gitleaks |
Critical | 🔄 To Implement |
| Container image scan | trivy, clair |
High | 🔄 To Implement |
| SAST (Static Analysis) | bandit, semgrep |
High | 🔄 To Implement |
| Code linting | ruff |
Medium | ✅ Implemented |
| Type checking | mypy |
Medium | ✅ Implemented |
| Artifact signing | cosign, GPG |
High | 🔄 To Implement |
Enhanced GitHub Actions Workflow
Update .github/workflows/ci.yml with enhanced security:
yaml
# AegisLM CI/CD Pipeline - Enhanced Security Version
# GitHub Actions workflow with comprehensive security scanning
name: CI
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
env:
# Security settings
TRUFFLEHOG_VERSION: "3.63.1"
TRIVY_VERSION: "0.49.1"
SEMGREP_VERSION: "1.64.0"
jobs:
# ===========================================================================
# Secret Detection - Fail on secrets
# ===========================================================================
secret-detection:
name: Secret Detection
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0 # Full history for trufflehog
- name: Run Trufflehog
uses: trufflesecurity/trufflehog@main
with:
path: ./
base: ${{ github.event.repository.default_branch }}
head: HEAD
extra_args: --fail --verified-only
- name: Run Gitleaks
uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITLEAKS_ENABLE_UPDATES: "false"
# ===========================================================================
# SAST - Static Application Security Testing
# ===========================================================================
sast:
name: SAST Analysis
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Run Semgrep
uses: returntocorp/semgrep-action@v1
with:
config: auto
generateSarif: true
- name: Upload Semgrep results
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: semgrep.sarif
- name: Run Bandit
run: |
pip install bandit
bandit -r backend/ -f sarif -o bandit.sarif || true
continue-on-error: true
# ===========================================================================
# Lint and Type Check
# ===========================================================================
lint:
name: Lint & Type Check
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install ruff mypy
- name: Run Ruff linter
run: ruff check . --output-format=github
- name: Run MyPy type checker
run: mypy backend --ignore-missing-imports --no-error-summary
continue-on-error: true
# ===========================================================================
# Validate Pydantic Schemas
# ===========================================================================
schema-validation:
name: Validate Pydantic Schemas
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
- name: Install dependencies
run: pip install -r requirements.txt
- name: Validate Pydantic schemas
run: |
python -c "
from backend.benchmarking.schemas import BenchmarkConfig, BenchmarkWeights
from backend.core.config import settings
from backend.db.models import EvaluationRun
print('All Pydantic schemas imported successfully')
"
- name: Validate weights sum to 1
run: |
python -c "
from backend.benchmarking.schemas import BenchmarkWeights
weights = BenchmarkWeights()
total = weights.hallucination + weights.toxicity + weights.bias + weights.confidence
if abs(total - 1.0) > 1e-6:
raise ValueError(f'Weights must sum to 1.0, got {total}')
print(f'Weights validation passed: sum = {total}')
"
# ===========================================================================
# Unit Tests
# ===========================================================================
tests:
name: Unit Tests
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install pytest pytest-asyncio pytest-cov
- name: Run unit tests
run: pytest tests/ -v --cov=backend --cov-report=xml --cov-report=term-missing
continue-on-error: true
- name: Upload coverage
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage.xml
# ===========================================================================
# Docker Build with Security Scanning
# ===========================================================================
docker:
name: Docker Build & Scan
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build Docker image
uses: docker/build-push-action@v5
with:
context: .
push: false
tags: aegislm:test
load: true
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Run Trivy scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: 'aegislm:test'
format: sarif
output: 'trivy-results.sarif'
severity: 'CRITICAL,HIGH'
exit-code: '1'
ignore-unfixed: true
- name: Upload Trivy results
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: 'trivy-results.sarif'
# ===========================================================================
# Dependency Security Scan
# ===========================================================================
dependency-security:
name: Dependency Security
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
- name: Install dependencies
run: pip install -r requirements.txt
- name: Run pip-audit
uses: pypa/pip-audit@main
with:
pip-audit-args: --exclude-editable --require-hashes
- name: Run safety
run: |
pip install safety
safety check --json > safety-results.json || true
continue-on-error: true
- name: Upload security results
uses: actions/upload-artifact@v4
if: always()
with:
name: security-results
path: |
safety-results.json
trivy-results.sarif
# ===========================================================================
# Build Summary
# ===========================================================================
summary:
name: Build Summary
needs: [secret-detection, sast, lint, schema-validation, tests, docker, dependency-security]
runs-on: ubuntu-latest
if: always()
steps:
- name: Summary
run: |
echo "## Build Summary" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "| Job | Status |" >> $GITHUB_STEP_SUMMARY
echo "|-----|--------|" >> $GITHUB_STEP_SUMMARY
echo "| Secret Detection | ${{ needs.secret-detection.result }} |" >> $GITHUB_STEP_SUMMARY
echo "| SAST Analysis | ${{ needs.sast.result }} |" >> $GITHUB_STEP_SUMMARY
echo "| Lint & Type Check | ${{ needs.lint.result }} |" >> $GITHUB_STEP_SUMMARY
echo "| Schema Validation | ${{ needs.schema-validation.result }} |" >> $GITHUB_STEP_SUMMARY
echo "| Unit Tests | ${{ needs.tests.result }} |" >> $GITHUB_STEP_SUMMARY
echo "| Docker Build & Scan | ${{ needs.docker.result }} |" >> $GITHUB_STEP_SUMMARY
echo "| Dependency Security | ${{ needs.dependency-security.result }} |" >> $GITHUB_STEP_SUMMARY
# Fail if critical security issues found
if [ "${{ needs.secret-detection.result }}" == "failure" ]; then
echo "" >> $GITHUB_STEP_SUMMARY
echo "❌ **SECURITY FAILURE**: Secrets detected in code" >> $GITHUB_STEP_SUMMARY
exit 1
fi
# ===========================================================================
# Artifact Signing (for releases)
# ===========================================================================
sign:
name: Sign Artifacts
needs: [lint, tests, docker]
runs-on: ubuntu-latest
if: github.event_name == 'release'
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Cosign
uses: sigstore/cosign-installer@v3
- name: Sign Docker image
run: |
cosign sign --yes aegislm/test:latest
- name: Sign Python packages
run: |
# Generate key pair
cosign generate-key-pair
# Sign wheel files
for wheel in dist/*.whl; do
cosign sign-file --key cosign.key "$wheel"
done
- name: Upload signatures
uses: actions/upload-artifact@v4
with:
name: signatures
path: |
*.sig
cosign.pub
Security Scanning Configuration
TruffleHog Configuration
Create .trufflehog.yaml:
yaml
# TruffleHog configuration
directories:
- .
exclude:
- .git
- node_modules
- venv
- __pycache__
rules:
- name: AWS Access Key
pattern: 'AKIA[0-9A-Z]{16}'
entropy: true
- name: GitHub Token
pattern: 'gh[pousr]_[A-Za-z0-9_]{36,255}'
entropy: true
- name: JWT Token
pattern: 'eyJ[A-Za-z0-9-_=]+\.eyJ[A-Za-z0-9-_=]+\.[A-Za-z0-9-_.+/=]*'
entropy: true
- name: Private Key
pattern: '-----BEGIN (RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----'
entropy: false
- name: Generic API Key
pattern: '(?i)(api[_-]?key|apikey|secret[_-]?key)[\s:=]+["\']?[a-zA-Z0-9]{16,}'
entropy: true
Semgrep Configuration
Create .semgrep.yaml:
yaml
rules:
- id: python-secrets
pattern: |
os.environ["$KEY"]
message: Hardcoded secrets in environment variables
languages:
- python
severity: ERROR
metadata:
cwe: "CWE-798: Use of Hard-coded Credentials"
owasp: "A2:2017-Broken Authentication"
- id: python-sql-injection
pattern: |
cursor.execute("SELECT ..." + $USER_INPUT)
message: Potential SQL injection
languages:
- python
severity: ERROR
metadata:
cwe: "CWE-89: SQL Injection"
owasp: "A1:2017-Injection"
- id: python-path-traversal
pattern: |
open($FILE, ...)
message: Potential path traversal
languages:
- python
severity: WARNING
metadata:
cwe: "CWE-22: Path Traversal"
Dependency Vulnerability Policy
Severity Thresholds
| Severity | Action | Pipeline Result |
|---|---|---|
| Critical | Fail build | ❌ Fail |
| High | Fail build | ❌ Fail |
| Medium | Warning only | ⚠️ Warn |
| Low | Info only | ℹ️ Info |
Allowed Vulnerabilities
For legitimate exceptions, create .vuln-allowlist.json:
json
{
"allowed_vulnerabilities": [
{
"cve": "CVE-2024-12345",
"package": "some-library",
"reason": "False positive - not used in our code path",
"expires": "2024-12-31"
}
]
}
Artifact Signing
Cosign Setup
bash
# Install cosign
brew install cosign
# Generate key pair
cosign generate-key-pair
# Sign a container image
cosign sign aegislm/api:v1.0.0
# Verify signature
cosign verify aegislm/api:v1.0.0 --key cosign.pub
Python Package Signing
yaml
# In release workflow
- name: Build Python package
run: |
python -m build
- name: Sign packages
run: |
for file in dist/*.whl; do
cosign sign-file --key cosign.key "$file"
done
Pipeline Security Best Practices
1. Secrets Management
yaml
# Use GitHub Secrets for sensitive values
env:
# Don't use secrets directly in env!
# Instead, use the secrets context
API_TOKEN: ${{ secrets.API_TOKEN }}
2. Permission Minimization
yaml
# Use minimal permissions
permissions:
contents: read
security-events: write
actions: read
checks: write
3. Cache Security
yaml
# Use secure cache
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip' # Uses hash of requirements.txt
Compliance Mapping
| Control | Implementation | Standard |
|---|---|---|
| Secret scanning | TruffleHog, Gitleaks | PCI DSS 6.5.10 |
| SAST | Semgrep, Bandit | OWASP Top 10 |
| Dependency scanning | pip-audit, Safety | NIST 800-53 |
| Container scanning | Trivy | CIS Docker |
| Code signing | Cosign | NIST 800-53 |
Emergency Procedures
Handling Security Vulnerabilities
- Immediate: Block vulnerable images in registry
- Short-term: Patch vulnerable dependencies
- Long-term: Update to secure alternatives
Pipeline Failure Response
┌─────────────────────────────────────────────────────────────────┐
│ Security Failure Response │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Security Scan Fails │
│ │ │
│ ▼ │
│ ┌─────────────┐ │
│ │ Identify │ → Review vulnerability details │
│ │ Issue │ → Check if applicable to our code │
│ └──────┬──────┘ │
│ │ │
│ ▼ │
│ ┌─────────────┐ │
│ │ Decide │ → Patch available? Use it │
│ │ Action │ → No patch? Add to allowlist (temp) │
│ └──────┬──────┘ → Critical? Block deployment │
│ │ │
│ ▼ │
│ ┌─────────────┐ │
│ │ Document │ → Create security issue │
│ │ & Notify │ → Notify security team │
│ └─────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘