agent/src/knowledge/effect_models.md

Effect Models in Meta-Analysis

Overview

Effect models determine how individual study effects are combined to produce a pooled estimate.

Fixed Effect Model

Concept

Assumes all studies estimate the same underlying true effect. Differences between studies are due only to sampling error.

Mathematical Framework

θ_i = θ + ε_i

Where:
- θ_i = observed effect in study i
- θ = true common effect (unknown)
- ε_i ~ N(0, SE_i²) sampling error

Weighting

Studies weighted by inverse variance:

w_i = 1 / SE_i²

θ_pooled = Σ(w_i × θ_i) / Σw_i
Var(θ_pooled) = 1 / Σw_i

When to Use

• Studies are very similar (identical protocol)
• Heterogeneity is negligible (I² < 25%)
• Inference limited to included studies only
• Combining results from single large study

TSA Java Code Reference

// From MetaAnalysis.java
public static final int FIXED = 10;
// Uses standard inverse variance weighting

Random Effects Model

Concept

Assumes each study estimates a different true effect, and these effects come from a distribution.

Mathematical Framework

θ_i = μ + u_i + ε_i

Where:
- μ = mean of effect distribution (what we estimate)
- u_i ~ N(0, τ²) between-study variance
- ε_i ~ N(0, SE_i²) within-study variance

Weighting

Modified weights incorporating τ²:

w_i* = 1 / (SE_i² + τ²)

θ_pooled = Σ(w_i* × θ_i) / Σw_i*
Var(θ_pooled) = 1 / Σw_i*

Common Estimators for τ²

DerSimonian-Laird (DL)

• Most widely used
• Method of moments estimator
• Can underestimate τ² with few studies

// From MetaAnalysis.java
public static final int RANDOM_DL = 11;

Sidik-Jonkman (SJ)

• Alternative estimator
• Better performance with few studies
• Less biased than DL

public static final int RANDOM_SJ = 16;

Bayesian (BT)

• Uses prior distribution for τ
• Incorporates uncertainty in τ² estimate
• Better coverage properties

public static final int RANDOM_BT = 12;

When to Use

• Clinical/methodological diversity expected
• Heterogeneity observed (I² > 25%)
• Generalizing beyond included studies
• Most real-world meta-analyses

Hybrid Models

Concept

Combines features of fixed and random effects based on heterogeneity.

public static final int HYBRID_DL = 14;
public static final int HYBRID_BT = 15;

Implementation

• If I² = 0: uses fixed effect (τ² = 0)
• If I² > 0: uses random effects with estimated τ²
• Provides adaptive weighting

Effect Measures

Odds Ratio (OR)

public static final int ODDS_RATIO = 3;

Properties:

• Range: 0 to ∞ (1 = no effect)
• Symmetrical when log-transformed
• Works in case-control studies
• Overestimates RR for common events

Relative Risk (RR)

public static final int RELATIVE_RISK = 1;

Properties:

• Range: 0 to ∞ (1 = no effect)
• More intuitive than OR
• Cannot use in case-control studies
• Can calculate NNT directly

Risk Difference (RD)

public static final int RISK_DIFFERENCE = 2;

Properties:

• Range: -1 to +1 (0 = no effect)
• Absolute measure
• Directly shows clinical impact
• Varies with baseline risk

Mean Difference (MD)

public static final int MEAN_DIFFERENCE = 4;

Properties:

• For continuous outcomes
• Natural units
• Requires same measurement scale
• Can use SMD if scales differ

Peto Odds Ratio

public static final int PETO_ODDS_RATIO = 6;

Properties:

• For rare events
• No continuity correction needed
• Handles zero cells well
• Biased if OR ≠ 1 or groups unbalanced

Model Selection Guidelines

Use Fixed Effect When:

1. Studies are truly identical
2. I² < 25% and Q-test non-significant
3. Only inferring about these specific studies
4. Very few studies (2-3) with similar precision

Use Random Effects When:

1. Any clinical/methodological diversity
2. I² ≥ 25% or Q-test significant
3. Generalizing to broader population
4. Default choice for most analyses

Model Comparison

Aspect	Fixed Effect	Random Effects
Assumption	Single true effect	Distribution of effects
CI width	Narrower	Wider
Small study weight	Less	More
Generalizability	Limited	Broader
With heterogeneity	Inappropriate	Appropriate

TSA Implications

• Fixed effect: OIS based on assumed common effect
• Random effects: OIS adjusted for heterogeneity
• Model choice affects boundary calculations
• High heterogeneity increases required sample size