tests/docs/COVERAGE_GUIDE.md

# Coverage Report Interpretation Guide

**Purpose**: Comprehensive guide for interpreting coverage reports and improving test coverage.

**Last Updated**: 2026-02-11

---

## Overview

Code coverage measures how much of your codebase is executed by tests. Coverage reports help identify untested code, track testing progress, and enforce quality standards.

**Key Concepts**:
- **Line Coverage**: Percentage of executable lines executed by tests
- **Branch Coverage**: Percentage of code branches (if/else, try/except) executed by tests
- **Coverage Paradox**: High coverage numbers don't guarantee high quality

**Project Coverage Targets**:
- Overall backend: 80%
- Governance domain: 80%
- Security domain: 80%
- Episodes domain: 80%

---

## Coverage Metrics Explained

### Line Coverage vs Branch Coverage

**Line Coverage** measures which lines of code were executed:
```

def check_permission(user, action):

    if user.is_admin:           # Line 1

        return True              # Line 2

    return False                 # Line 3

```

If tests only call `check_permission(admin_user, "read")`:
- Line coverage: 67% (lines 1-2 executed, line 3 not executed)
- Branch coverage: 50% (only "if" branch tested, "else" branch not tested)

**Branch Coverage** is more accurate because it measures all possible execution paths:
- If statements: 2 branches (true, false)
- Try/except: 2 branches (success, exception)
- Short-circuit operators: 2 branches (or, and)

**Why Branch Coverage Matters**:
```python

# Line coverage: 100%, Branch coverage: 50%

def divide(a, b):

    if b != 0 and a / b > 10:   # Both lines executed

        return "large"

    return "small"



# Test: divide(20, 2) → lines 1-3 executed

# Missing: divide(10, 0) → exception path not tested

```

**Recommendation**: Always enable branch coverage with `--cov-branch` flag (already in pytest.ini).

---

### Coverage Percentage Interpretation

| Coverage Range | Interpretation | Action |
|----------------|----------------|--------|
| 0-20% | Untested codebase | Start with critical paths |
| 20-50% | Early testing | Focus on core functionality |
| 50-80% | Good coverage | Target for most projects |
| 80-90% | Excellent coverage | Diminishing returns above 80% |
| 90-100% | Near-perfect | Risk of over-testing |

**Diminishing Returns Above 90%**:
- Last 10% costs 50% of testing effort
- Focus on critical paths first
- 100% coverage is rarely practical

**When 80% is Sufficient**:
- Error handling is tested
- Critical paths covered
- Property tests validate invariants
- Integration tests cover workflows

---

### Coverage Paradox

**High Coverage, Low Quality**: Tests cover lines but not scenarios.

**Example**:
```python

# Line coverage: 100%, Branch coverage: 100%

# Quality: LOW (no assertions verify correctness)



def calculate_discount(price, user):

    if user.is_vip:

        return price * 0.9

    return price



def test_calculate_discount():

    calculate_discount(100, vip_user)    # Line covered

    calculate_discount(100, regular_user) # Line covered

    # No assertions! Bugs would pass silently.

```

**Detection Strategies**:
1. **Property-based tests**: Validate invariants across all inputs (Hypothesis)
2. **Mutation testing**: Detect weak tests (mutmut)
3. **Assertion density**: Track assertions per line (conftest.py)

**See**: `property_tests/INVARIANTS.md` for property test patterns.

---

### Missing Coverage

**0% Coverage Means**:
- No tests execute this code
- Untested bugs may exist
- Refactoring is risky

**Common Causes**:
1. **Dead code**: Unused functions (consider removing)
2. **Error paths**: Exception handlers not tested
3. **Edge cases**: Boundary conditions not covered
4. **New code**: Written without tests

**Priority for Improving Coverage**:
1. **Critical paths**: Authentication, authorization, payments
2. **Security**: Input validation, encryption, access control
3. **Data integrity**: Database operations, transactions
4. **User-facing**: API endpoints, business logic

---

## Reading Coverage Reports

### HTML Report Navigation

**Location**: `backend/tests/coverage_reports/html/index.html`

**Usage**:
```bash

# Generate HTML report

pytest tests/ --cov=core --cov=api --cov=tools --cov-report=html



# Open in browser

open tests/coverage_reports/html/index.html

```

**Interpretation**:
- **Green**: >80% coverage (good)
- **Yellow**: 50-80% coverage (needs improvement)
- **Red**: <50% coverage (critical gaps)

**HTML Report Features**:
- Click directory to see files
- Click file to see line-by-line coverage
- Red lines = not executed
- Yellow lines = partially executed (branch coverage)

---

### JSON Report Structure

**Location**: `backend/tests/coverage_reports/metrics/coverage.json`

**Format**:
```json

{

  "meta": {

    "timestamp": "2026-02-11T10:30:00",

    "branch_coverage": true,

    "show_contexts": true

  },

  "totals": {

    "covered_lines": 2340,

    "num_statements": 15000,

    "percent_covered": 15.6,

    "covered_branches": 1200,

    "num_branches": 3000,

    "percent_covered_branch": 40.0

  },

  "files": {

    "core/agent_governance_service.py": {

      "summary": {

        "percent_covered": 45.2,

        "percent_covered_branch": 30.0

      },

      "executed_lines": [10, 11, 15, 20],

      "missing_lines": [25, 30, 35],

      "excluded_lines": [1, 2, 3]

    }

  }

}

```

**Key Fields**:
- `percent_covered`: Line coverage percentage
- `percent_covered_branch`: Branch coverage percentage
- `missing_lines`: Line numbers not executed
- `executed_lines`: Line numbers executed

**Usage in CI/CD**:
```bash

# Extract coverage percentage

COVERAGE=$(jq '.totals.percent_covered' tests/coverage_reports/metrics/coverage.json)



# Fail if below threshold

if (( $(echo "$COVERAGE < 80" | bc -l) )); then

    echo "Coverage $COVERAGE% below 80% threshold"

    exit 1

fi

```

---

### Terminal Report Interpretation

**Generate Terminal Report**:
```bash

pytest tests/ --cov-report=term-missing

```

**Output Format**:
```

Name                                         Stmts   Miss  Cover   Missing

------------------------------------------------------------------------

core/agent_governance_service.py               150     75    50%   23-45, 89-102

core/models.py                                 300    120    60%   156-189, 234-267

api/auth_routes.py                              50     40    20%   12-35, 67-89

------------------------------------------------------------------------

TOTAL                                          500    235    53%

```

**Columns**:
- `Stmts`: Total executable statements
- `Miss`: Statements not executed
- `Cover`: Coverage percentage
- `Missing`: Line ranges not covered

**Color Coding**:
- Green: >80%
- Yellow: 50-80%
- Red: <50%

---

## Coverage by Domain

### Governance Domain

**Target**: 80%
**Current**: 13.37% (as of 2026-02-11)

**Files**:
- `agent_governance_service.py` - Agent lifecycle and permissions
- `agent_context_resolver.py` - Agent resolution
- `governance_cache.py` - High-performance caching
- `trigger_interceptor.py` - Maturity-based trigger routing

**Priority Tests**:
1. Maturity threshold enforcement (STUDENT agents block high-complexity actions)
2. Confidence-based routing (low confidence → supervised)
3. Permission checks (all 4x4 maturity/complexity combinations)

**Impact**: Governance failures cause unauthorized actions or security violations.

---

### Security Domain

**Target**: 80%
**Current**: 22.40% (as of 2026-02-11)

**Files**:
- `auth_routes.py` - Authentication endpoints
- `encryption.py` - Data encryption/decryption
- `validation.py` - Input validation

**Priority Tests**:
1. JWT token validation (expired, invalid, malformed)
2. Password hashing (bcrypt, salt rounds)
3. Input validation (OWASP Top 10 payloads)
4. SQL injection prevention (parameterized queries)

**Impact**: Security failures cause unauthorized access or data breaches.

---

### Episodes Domain

**Target**: 80%
**Current**: 15.52% (as of 2026-02-11)

**Files**:
- `episode_segmentation_service.py` - Episode creation
- `episode_retrieval_service.py` - Episode retrieval
- `episode_lifecycle_service.py` - Episode lifecycle
- `agent_graduation_service.py` - Graduation validation

**Priority Tests**:
1. Time gap detection (boundaries, thresholds)
2. Semantic retrieval (similarity, ranking)
3. Graduation criteria (episode count, intervention rate)
4. Constitutional compliance (intervention tracking)

**Impact**: Episode failures cause incorrect agent learning or premature promotion.

---

### Backend Overall

**Target**: 80%
**Current**: 15.57% (as of 2026-02-11)

**Files Tracked**: 401 files

**Breakdown**:
- Governance: 13.37%
- Security: 22.40%
- Episodes: 15.52%
- Other domains: Variable

**Focus Areas**:
1. Critical paths: Auth, governance, payments
2. Error handling: Exception paths, edge cases
3. Integration points: API endpoints, database operations

---

## Coverage Trending

### Tracking Coverage Over Time

**Purpose**: Detect coverage regressions and measure progress.

**Location**: `backend/tests/coverage_reports/trends/coverage_trend.json`

**Format**:
```json

{

  "coverage_history": [

    {

      "date": "2026-02-11",

      "commit": "abc123",

      "overall_percent": 15.57,

      "governance_percent": 13.37,

      "security_percent": 22.40,

      "episodes_percent": 15.52

    }

  ],

  "targets": {

    "overall": 80,

    "governance": 80,

    "security": 80,

    "episodes": 80

  }

}

```

**Usage**:
```bash

# View trend

jq '.coverage_history[] | .date + ": " + (.overall_percent | tostring) + "%"' \

  tests/coverage_reports/trends/coverage_trend.json



# Detect regression

LATEST=$(jq '.coverage_history[0].overall_percent' coverage_trend.json)

PREVIOUS=$(jq '.coverage_history[1].overall_percent' coverage_trend.json)



if (( $(echo "$LATEST < $PREVIOUS" | bc -l) )); then

    echo "Coverage regression detected: $PREVIOUS% → $LATEST%"

fi

```

---

### Detecting Coverage Regressions

**Regression**: Coverage decreases between commits.

**CI/CD Check**:
```yaml

# .github/workflows/coverage-report.yml

- name: Check coverage regression

  run: |

    LATEST=$(jq '.coverage_history[0].overall_percent' coverage_trend.json)

    PREVIOUS=$(jq '.coverage_history[1].overall_percent' coverage_trend.json)

    THRESHOLD=1.0  # 1% tolerance



    if (( $(echo "$PREVIOUS - $LATEST > $THRESHOLD" | bc -l) )); then

      echo "Coverage regression: $PREVIOUS% → $LATEST%"

      exit 1

    fi

```

**Common Causes**:
1. Deleted tests without replacement
2. Refactored code without updating tests
3. New code written without tests

---

### Coverage Thresholds and Quality Gates

**Quality Gate**: Minimum coverage requirement for PR approval.

**Configuration** (pytest.ini):
```ini

[pytest]

cov_fail_under = 80

```

**CI/CD Enforcement**:
```yaml

- name: Run tests with coverage

  run: pytest tests/ --cov=core --cov=api --cov=tools --cov-fail-under=80

```

**Per-Domain Thresholds**:
```python

# conftest.py

def pytest_terminal_summary(terminalreporter):

    coverage_data = json.load(open("coverage.json"))



    governance_cov = coverage_data['files']['core/agent_governance_service.py']['summary']['percent_covered']

    if governance_cov < 80:

        terminalreporter.write_line(f"ERROR: Governance coverage {governance_cov}% below 80%", red=True)

```

**Recommendation**: Start with 60% threshold, increase to 80% as coverage improves.

---

## Improving Coverage

### Identifying Uncovered Lines

**Step 1: Generate HTML Report**
```bash

pytest tests/ --cov=core --cov=api --cov=tools --cov-report=html

open tests/coverage_reports/html/index.html

```

**Step 2: Find Red Files**
- Click through directories
- Look for files with <80% coverage
- Click file to see line-by-line breakdown

**Step 3: Analyze Missing Lines**
- Red lines = not executed
- Yellow lines = partially executed (branch coverage)
- Right-click line → "show context" to see why it's not covered

---

### Prioritizing Coverage Gaps

**Priority Matrix**:

| Impact | High Coverage | Low Coverage | Action |
|--------|---------------|--------------|--------|
| **Critical** | Error in test | **IMMEDIATE** | Fix tests |
| **High** | Monitor | **HIGH** | Add tests |
| **Medium** | Defer | **MEDIUM** | Add tests |
| **Low** | Accept | **LOW** | Document |

**Critical Paths** (test first):
1. Authentication: Login, logout, token refresh
2. Authorization: Permission checks, maturity enforcement
3. Data integrity: Database transactions, validation
4. Security: Encryption, input validation, access control

**Example Priority**:
```python

# CRITICAL: Auth bypass

if user.is_admin:  # Branch coverage: 50% → Fix immediately

    return sensitive_data



# HIGH: Business logic

if discount > 0.9:  # Branch coverage: 75% → Add test

    apply_discount()



# MEDIUM: Edge case

if user.preferences.color == "blue":  # Branch coverage: 0% → Defer

    set_theme("blue")

```

---

### Writing Tests for Uncovered Code

**Step 1: Understand What's Missing**
```python

# Original code (lines 25-30 not covered)

def delete_agent(agent_id, user):

    if not user.is_admin:          # Line 25

        raise PermissionError()     # Line 26

    agent = db.get(agent_id)        # Line 27

    if agent.status == "ACTIVE":    # Line 28

        raise ValueError()          # Line 29

    db.delete(agent)                # Line 30

```

**Step 2: Write Test for Missing Path**
```python

def test_delete_agent_requires_admin(unique_resource_name):

    """Test non-admin cannot delete agents (line 25-26)."""

    user = UserFactory.create(role="member")  # Not admin

    agent = AgentFactory.create(id=unique_resource_name)



    with pytest.raises(PermissionError):

        delete_agent(agent.id, user)



def test_delete_active_agent_fails(unique_resource_name):

    """Test deleting active agent raises error (line 28-29)."""

    admin = UserFactory.create(role="admin")

    agent = AgentFactory.create(id=unique_resource_name, status="ACTIVE")



    with pytest.raises(ValueError):

        delete_agent(agent.id, admin)



def test_delete_inactive_agent_succeeds(unique_resource_name):

    """Test deleting inactive agent works (line 30)."""

    admin = UserFactory.create(role="admin")

    agent = AgentFactory.create(id=unique_resource_name, status="INACTIVE")



    delete_agent(agent.id, admin)

    assert db.get(agent.id) is None

```

**Step 3: Run Tests**
```bash

pytest tests/test_agent_service.py -v --cov=core/agent_service --cov-report=term-missing

```

**Step 4: Verify Coverage Improved**
- Check HTML report
- Lines 25-30 should now be green

---

### When NOT to Chase 100% Coverage

**Exclusions from Coverage Targets**:

1. **Generated Code**:
   - Protobuf-generated files
   - OpenAPI-generated models
   - Database migration scripts

2. **Configuration**:
   - Constants, enums
   - Environment variable loading
   - Logging configuration

3. **Third-Party Wrappers**:
   - Thin wrappers around libraries
   - Adapter classes (1-2 lines)

4. **Impossible Paths**:
   - OS-specific code (Windows tests on Mac)
   - Hardware-dependent code
   - GUI code in CI/CD

**Example .coveragerc**:
```ini

[report]

exclude_lines =

    pragma: no cover

    def __repr__

    raise AssertionError

    raise NotImplementedError

    if __name__ == .__main__.:

    if TYPE_CHECKING:

    @abstractmethod

```

**Guideline**: 80% coverage is practical. 100% coverage is rarely worth the cost.

---

## Coverage Tools Reference

### pytest-cov Command Reference

**Basic Usage**:
```bash

# Run tests with coverage

pytest tests/ --cov=core



# Multiple modules

pytest tests/ --cov=core --cov=api --cov=tools



# Generate reports

pytest tests/ --cov=core --cov-report=html --cov-report=json --cov-report=term

```

**Report Formats**:
- `--cov-report=html`: HTML report (coverage_reports/html/index.html)

- `--cov-report=json`: JSON report (coverage.json)

- `--cov-report=term`: Terminal output

- `--cov-report=term-missing`: Terminal with missing lines



**Branch Coverage**:

```bash

pytest tests/ --cov=core --cov-branch

```



**Coverage Fail Threshold**:

```bash

pytest tests/ --cov=core --cov-fail-under=80

```



**Source Configuration**:

```bash

pytest tests/ --cov=core --cov-context=test

```



---



### Coverage.py CLI Reference



**Combine Coverage Data**:

```bash

# Run tests in parallel

pytest tests/ -n auto --cov=core --cov-parallel



# Combine .coverage files

coverage combine

```



**Generate Report After Tests**:

```bash

pytest tests/ --cov=core

coverage html    # Generate HTML

coverage report  # Generate terminal

coverage json    # Generate JSON

```



**Debug Coverage**:

```bash

coverage debug sys

coverage debug config

```



**Erase Coverage Data**:

```bash

coverage erase  # Delete .coverage file

```



---



### Codecov/Coveralls Integration



**Codecov Upload** (via GitHub Actions):

```yaml

- name: Upload to Codecov

  uses: codecov/codecov-action@v4

  with:

    files: ./tests/coverage_reports/metrics/coverage.json
    flags: backend

    name: backend-coverage

    fail_ci_if_error: false  # Don't fail CI if Codecov is down

```


**Coveralls Upload** (alternative):
```yaml

- name: Upload to Coveralls

  uses: coverallsapp/github-action@v2

  with:

    github-token: ${{ secrets.GITHUB_TOKEN }}

    path-to-lcov: ./coverage.lcov

```

**Benefits**:
- Historical trending dashboard
- PR comments with coverage diff
- File-by-file coverage breakdown
- Coverage badges for README

**Alternative**: Git-tracked JSON (no external service needed)

---

## Related Documentation

- **[TEST_ISOLATION_PATTERNS.md](./TEST_ISOLATION_PATTERNS.md)** - Test isolation patterns and examples
- **[FLAKY_TEST_GUIDE.md](./FLAKY_TEST_GUIDE.md)** - Flaky test prevention and fixing
- **[../property_tests/INVARIANTS.md](../property_tests/INVARIANTS.md)** - Property test invariants
- **[../factories/README.md](../factories/README.md)** - Test data factory usage
- **[../TESTING_GUIDE.md](../TESTING_GUIDE.md)** - Comprehensive testing guide

---

## Summary

**Key Takeaways**:
1. **Branch coverage > Line coverage**: Always use `--cov-branch`
2. **80% is practical**: Diminishing returns above 90%
3. **Coverage paradox**: High coverage ≠ high quality (use property tests)
4. **Trending matters**: Track coverage over time to detect regressions
5. **Prioritize critical paths**: Auth, governance, security first

**Quick Reference**:
```bash

# Generate all reports

pytest tests/ --cov=core --cov=api --cov=tools \

  --cov-branch \

  --cov-report=html \

  --cov-report=json \

  --cov-report=term-missing



# Check coverage trend

jq '.coverage_history[0]' tests/coverage_reports/trends/coverage_trend.json



# View HTML report

open tests/coverage_reports/html/index.html

```

**Next Steps**:
1. Generate coverage report for your domain
2. Identify files <80% coverage
3. Write tests for critical paths first
4. Track coverage in CI/CD
5. Review coverage trends weekly

## See Also

### Platform-Specific Testing Guides

- [Frontend Testing Guide](../../docs/FRONTEND_TESTING_GUIDE.md) - Jest, React Testing Library, MSW patterns
- [Mobile Testing Guide](../../docs/MOBILE_TESTING_GUIDE.md) - jest-expo, React Native Testing Library
- [Desktop Testing Guide](../../docs/DESKTOP_TESTING_GUIDE.md) - cargo test, proptest, tarpaulin

### Related Documentation

- [Testing Documentation Index](../../docs/TESTING_INDEX.md) - Central hub for all testing documentation
- [Testing Onboarding Guide](../../docs/TESTING_ONBOARDING.md) - 15-minute quick start for all platforms
- [Coverage Trending Guide](COVERAGE_TRENDING_GUIDE.md) - 30-day trending and regression detection
- [Flaky Test Quarantine](FLAKY_TEST_QUARANTINE.md) - Multi-run flaky detection and tracking