Causal Inference

Correlation vs Causation

A predictive ML model answers: "Given X, what is the likely value of Y?"

A causal model answers: "If I change X, what will happen to Y?"

Why the Distinction Matters

Setting	Wrong (Correlation)	Right (Causal)
Ad targeting	"Users who saw the ad converted more"	"Showing the ad caused conversions to rise"
Medicine	"Patients who took aspirin had better outcomes"	"Aspirin reduced cardiovascular events"
Policy	"Cities with more police have more crime"	"Adding police changes crime rates"
Product	"Users with premium features churn less"	"Giving users premium features reduces churn"

Spurious correlation: ice cream sales and drowning deaths are correlated (both driven by summer temperature). Restricting ice cream would not reduce drownings.

Confounding: a hidden variable $Z$ causes both $X$ and $Y$, creating an observed correlation with no direct causal link.

$X \leftarrow Z \rightarrow Y \quad \text{(not } X \rightarrow Y\text{)}$

Potential Outcomes Framework (Rubin Causal Model)

Define for each unit $i$:

• $Y_i(1)$: potential outcome if treated ($T_i = 1$)
• $Y_i(0)$: potential outcome if untreated ($T_i = 0$)

Individual Treatment Effect (ITE): $\tau_i = Y_i(1) - Y_i(0)$

The Fundamental Problem of Causal Inference: we only ever observe one of the two potential outcomes for each unit. The other is the counterfactual — it cannot be directly observed.

Population-Level Estimands

Estimand	Formula	When to Use
ATE (Average Treatment Effect)	$\mathbb{E}[Y(1) - Y(0)]$	Population-wide policy
ATT (Avg Treatment Effect on Treated)	$\mathbb{E}[Y(1) - Y(0) \mid T=1]$	Assess effect on those who opted in
ATC (Avg Treatment Effect on Control)	$\mathbb{E}[Y(1) - Y(0) \mid T=0]$	Assess effect if untreated had been treated
CATE (Conditional ATE)	$\mathbb{E}[Y(1) - Y(0) \mid X=x]$	Heterogeneous effects by subgroup

Key Assumptions

1. SUTVA (Stable Unit Treatment Value Assumption): no interference between units; treatment is well-defined
2. Ignorability / Unconfoundedness: $Y(0), Y(1) \perp T \mid X$ — no hidden confounders after conditioning on $X$
3. Overlap / Positivity: $0 < P(T=1 \mid X) < 1$ — every unit has a nonzero probability of either treatment

A/B Testing (Randomized Controlled Trials)

Randomization is the gold standard because it ensures treatment assignment is independent of all confounders — both observed and hidden.

$T_i \perp (Y_i(0), Y_i(1)) \quad \Rightarrow \quad \hat{\text{ATE}} = \bar{Y}_\text{treatment} - \bar{Y}_\text{control}$

Designing a Valid A/B Test

1. Define hypothesis: $H_0: \Delta = 0$ vs $H_1: \Delta \neq 0$
2. Choose metric: primary (conversion rate) + guardrail metrics (revenue per user)
3. Compute sample size: based on baseline rate, MDE (minimum detectable effect), $\alpha$, power
4. Randomize: at the user/session level, avoiding network effects
5. Run for full business cycles: capture weekly seasonality
6. Analyze: two-sample t-test, z-test for proportions, or regression with covariates

$n = \frac{2(z_{\alpha/2} + z_\beta)^2 \sigma^2}{\delta^2}$

where $\delta$ = MDE, $\sigma^2$ = outcome variance, $z_{\alpha/2}$ and $z_\beta$ = critical values.

Common Pitfalls

• Peeking: checking significance before experiment ends inflates Type I error
• Multiple testing: running 20 A/B tests simultaneously → 1 false positive at $\alpha=0.05$
• SUTVA violations: spillover (users in both groups interact), novelty effects, carryover
• SRM (Sample Ratio Mismatch): treatment/control split differs from intended → bug

import numpy as np
from scipy import stats
from statsmodels.stats.power import NormalIndPower

# ---- Sample size calculation ----
baseline_rate = 0.10    # 10% conversion
mde = 0.02              # detect a 2pp lift (10% → 12%)
alpha = 0.05
power = 0.80

effect_size = mde / np.sqrt(baseline_rate * (1 - baseline_rate))
analysis = NormalIndPower()
n_per_group = analysis.solve_power(effect_size=effect_size, alpha=alpha, power=power, alternative='two-sided')
print(f"Required sample size per group: {int(np.ceil(n_per_group))}")

# ---- Simulate experiment results ----
np.random.seed(42)
n = int(np.ceil(n_per_group))
control = np.random.binomial(1, baseline_rate, n)
treatment = np.random.binomial(1, baseline_rate + mde, n)

p_control = control.mean()
p_treatment = treatment.mean()
print(f"
Control conversion:   {p_control:.4f}")
print(f"Treatment conversion: {p_treatment:.4f}")
print(f"Observed lift:        {p_treatment - p_control:.4f}")

# ---- Two-proportion z-test ----
from statsmodels.stats.proportion import proportions_ztest
count = np.array([treatment.sum(), control.sum()])
nobs  = np.array([n, n])
z_stat, p_value = proportions_ztest(count, nobs)

print(f"
Z-statistic: {z_stat:.3f}")
print(f"P-value:     {p_value:.4f}")
print(f"Significant (α=0.05): {p_value < alpha}")

# ---- Confidence interval on lift ----
se = np.sqrt(p_control*(1-p_control)/n + p_treatment*(1-p_treatment)/n)
z_crit = stats.norm.ppf(1 - alpha/2)
lift = p_treatment - p_control
ci_lo, ci_hi = lift - z_crit*se, lift + z_crit*se
print(f"
95% CI on lift: [{ci_lo:.4f}, {ci_hi:.4f}]")

Observational Studies: When You Can't Randomize

Many important decisions cannot be randomized (ethical, legal, or practical constraints). Causal inference methods recover causal effects from observational data under assumptions.

Propensity Score Matching (PSM)

The propensity score is the probability of treatment given observed covariates: $e(X_i) = P(T_i = 1 \mid X_i)$

Propensity score theorem (Rosenbaum & Rubin, 1983): if ignorability holds given $X$, it also holds given $e(X)$ — reducing matching to one dimension regardless of covariate count.

Steps:

1. Estimate $e(X)$ via logistic regression (or gradient boosting)
2. Match treated units to control units with similar $e(X)$
3. Estimate ATT on matched sample

Variants: matching, stratification, inverse probability weighting (IPW), doubly-robust estimators.

Difference-in-Differences (DiD)

Compares the change in outcomes over time between treated and control groups: $\hat{\tau}^{DiD} = (\bar{Y}_{\text{treat, post}} - \bar{Y}_{\text{treat, pre}}) - (\bar{Y}_{\text{ctrl, post}} - \bar{Y}_{\text{ctrl, pre}})$

Key assumption: Parallel trends — without treatment, treated and control groups would have followed the same trend.

Used widely in economics and policy evaluation (e.g., effect of minimum wage laws).

Instrumental Variables (IV)

When hidden confounders remain, use an instrument $Z$ that:

• Affects treatment $T$ (relevance)
• Affects outcome $Y$ only through $T$ (exclusion restriction)
• Is independent of confounders (exogeneity)

$\hat{\tau}^{IV} = \frac{\text{Cov}(Y, Z)}{\text{Cov}(T, Z)}$

Classic example: draft lottery as IV for military service → effect of service on earnings.

import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler

# ---- Simulate observational data with confounding ----
np.random.seed(42)
n = 2000

age = np.random.normal(40, 10, n)
health_score = np.random.normal(70, 15, n)

# Treatment: older, healthier people more likely to exercise (confounder)
log_odds = -3 + 0.05 * age + 0.02 * health_score
p_treat = 1 / (1 + np.exp(-log_odds))
treatment = np.random.binomial(1, p_treat, n)

# Outcome: exercise (treatment) improves health, but older/healthier start higher
noise = np.random.normal(0, 5, n)
outcome = 10 * treatment + 0.1 * health_score - 0.05 * age + noise

df = pd.DataFrame({'age': age, 'health_score': health_score,
                   'treatment': treatment, 'outcome': outcome})

# ---- Naive estimate (biased by confounding) ----
naive_ate = df[df.treatment==1].outcome.mean() - df[df.treatment==0].outcome.mean()
print(f"Naive ATE (biased):  {naive_ate:.2f}  (true = 10.00)")

# ---- Estimate propensity scores ----
X = StandardScaler().fit_transform(df[['age', 'health_score']])
ps_model = LogisticRegression()
ps_model.fit(X, df['treatment'])
df['propensity'] = ps_model.predict_proba(X)[:, 1]

# ---- Nearest-neighbor matching (manual) ----
treated = df[df.treatment == 1].copy()
control = df[df.treatment == 0].copy()

matched_controls = []
for _, t_row in treated.iterrows():
    distances = (control['propensity'] - t_row['propensity']).abs()
    matched_controls.append(control.loc[distances.idxmin(), 'outcome'])

att_psm = treated['outcome'].values.mean() - np.mean(matched_controls)
print(f"PSM ATT (debiased):  {att_psm:.2f}  (true = 10.00)")

# ---- Inverse Probability Weighting (IPW) ----
eps = 1e-6
df['weight'] = np.where(
    df.treatment == 1,
    1 / (df.propensity + eps),
    1 / (1 - df.propensity + eps),
)
ipw_ate = (
    (df[df.treatment==1]['outcome'] * df[df.treatment==1]['weight']).sum() / df[df.treatment==1]['weight'].sum()
  - (df[df.treatment==0]['outcome'] * df[df.treatment==0]['weight']).sum() / df[df.treatment==0]['weight'].sum()
)
print(f"IPW ATE (debiased):  {ipw_ate:.2f}  (true = 10.00)")

Summary

• Correlation ≠ causation — predictive ML answers "what is likely?" not "what would happen if?"
• Pearl's causal ladder: association → intervention (do-calculus) → counterfactuals
• Potential outcomes: $Y(1) - Y(0)$ is the ITE; the unobserved outcome is the counterfactual
• Randomized A/B tests are the gold standard — randomization eliminates confounders
• Observational methods: propensity score matching, IPW, DiD, and IV recover causal effects when RCTs are impossible
• Causal inference is essential for responsible AI: understanding what your model's recommendations actually cause

Next: Bayesian Machine Learning — probabilistic reasoning, uncertainty quantification, and Gaussian Processes.