# CI/CD Pipeline Monitoring Guide

## 🔍 **Where to Monitor Pipeline Execution**

### **Primary Dashboard**
**GitHub Actions**: https://github.com/sethmcknight/msse-ai-engineering/actions

### **Real-Time Monitoring Locations**

#### **1. GitHub Actions Tab**
- Navigate to your repository
- Click the **"Actions"** tab
- See all workflow runs with real-time status

#### **2. Pull Request Checks**
- Open any Pull Request
- Scroll to bottom to see **"All checks have passed"** or **"Some checks were not successful"**
- Click **"Details"** to see full logs

#### **3. Commit Status**
- Each commit shows ✅ or ❌ status
- Click the status icon to see detailed results

## 🛡️ **Pipeline Safeguards - Tests MUST Pass Before Deployment**

```yaml
Pipeline Flow:
┌─────────────────────────────────────────────────────────────┐
│                    SAFETY GATES                            │
└─────────────────────────────────────────────────────────────┘

1. 🔍 Code Quality Checks (ALL MUST PASS)
   ├── Black formatting validation
   ├── isort import ordering
   ├── flake8 linting
   └── Pre-commit hooks (PR only)

2. 🧪 Comprehensive Testing (ALL MUST PASS)
   ├── Python 3.10 & 3.11 matrix testing
   ├── 27+ unit tests with coverage
   ├── HF embedding service tests (12 tests)
   ├── Citation validation tests
   ├── LLM component tests
   └── End-to-end pipeline validation

3. 🚀 Deployment (ONLY IF TESTS PASS)
   ├── needs: test-hybrid-architecture ← CRITICAL DEPENDENCY
   ├── Deploy to Render
   ├── Deploy to HuggingFace Spaces
   └── Post-deployment validation

4. ✅ Final Validation
   └── Health checks across all platforms
```

## 🔒 **Critical Safety Configuration**

### **Deployment Dependency Chain**
```yaml
deploy-to-render:
  needs: test-hybrid-architecture  # ← BLOCKS deployment if tests fail

deploy-to-huggingface:
  needs: test-hybrid-architecture  # ← BLOCKS deployment if tests fail

post-deployment-validation:
  needs: [deploy-to-render, deploy-to-huggingface]  # ← Waits for both deployments
```

### **Test Requirements**
- **All 27+ tests** must pass
- **Code formatting** must be compliant
- **Citation validation** must succeed
- **Service integration** must work

## 📊 **How to Monitor a Pipeline Run**

### **Step 1: Trigger Pipeline**
```bash
# Push to main or hf-main-local triggers pipeline
git push origin main
```

### **Step 2: Monitor in Real-Time**
1. **Go to Actions**: https://github.com/sethmcknight/msse-ai-engineering/actions
2. **Click latest workflow run** (shows "Enhanced CI/CD - HuggingFace + Hybrid Architecture")
3. **Watch job progress**:
   - 🟡 **Yellow**: Job running
   - ✅ **Green**: Job passed
   - ❌ **Red**: Job failed (BLOCKS deployment)

### **Step 3: Review Test Results**
- Click **"test-hybrid-architecture"** job
- Expand test steps to see detailed results
- Check coverage reports
- Verify all 27+ tests passed

### **Step 4: Deployment Monitoring**
- **Only runs if tests pass**
- Monitor Render deployment
- Monitor HuggingFace deployment
- Watch health check validation

## 🚨 **What Happens if Tests Fail**

### **Immediate Blocking**
```
❌ test-hybrid-architecture job fails
    ↓
🛑 deploy-to-render: SKIPPED (dependency failed)
🛑 deploy-to-huggingface: SKIPPED (dependency failed)
🛑 post-deployment-validation: SKIPPED (dependencies failed)
```

### **Deployment Prevention**
- **No code reaches production** if any test fails
- **Email notification** sent about failure
- **GitHub status** shows ❌ on commit
- **Must fix tests** before deployment can proceed

## 🎯 **Emergency Override**
If you need to skip deployment for testing:
```bash
git commit -m "Fix critical bug [skip-deploy]"
```
This will run tests but skip deployment.

## 📈 **Pipeline Success Indicators**

### **All Green Status**
- ✅ Pre-commit checks (PR only)
- ✅ Code quality validation
- ✅ 27+ comprehensive tests
- ✅ Service integration tests
- ✅ Citation validation
- ✅ Render deployment + health check
- ✅ HuggingFace deployment + health check
- ✅ Post-deployment validation

### **Deployment URLs** (only available after successful deployment)
- **Render**: Available in workflow logs
- **HF Team**: https://huggingface.co/spaces/msse-team-3/ai-engineering-project
- **HF Personal**: https://huggingface.co/spaces/sethmcknight/msse-ai-engineering

## 🔧 **Test Locally Before Push**
```bash
# Run the same tests locally
./scripts/hf_test_runner.sh

# Or run specific test categories
pytest tests/ -v
pytest -k "citation" -v
python scripts/test_e2e_pipeline.py
```

---

**Bottom Line**: The pipeline is configured with **mandatory test gates** that PREVENT deployment if any tests fail. Monitor at: https://github.com/sethmcknight/msse-ai-engineering/actions