Interactive notebook

This tutorial is a Jupyter notebook. You can view it on GitHub or download it to run locally.

Tutorial 8: Triple Difference (DDD) Estimation

This tutorial covers the Triple Difference (DDD) estimator, which extends standard Difference-in-Differences to settings where treatment requires satisfying two criteria.

When to Use Triple Difference

Triple Difference is appropriate when:

1. Treatment requires two criteria: Units must satisfy BOTH conditions to be treated:

   • Belonging to a treated group (e.g., states that enacted a policy)

   • Being in an eligible partition (e.g., women, low-income individuals)

2. You want to relax parallel trends: DDD allows for group-specific AND partition-specific violations of parallel trends, as long as the differential trend is parallel.

Classic Example: Maternity Benefits

Gruber (1994) studied state mandates requiring employers to provide maternity benefits:

• Group: States that enacted mandates vs. states that didn’t

• Partition: Women of childbearing age vs. other workers

• Outcome: Wages

Only women in mandate states were “treated” - the policy affected their labor costs.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from diff_diff import TripleDifference, triple_difference

Generating Synthetic DDD Data

Let’s create synthetic data that mimics a DDD setting. We’ll simulate a policy that:

• Was enacted in some states (group=1) but not others (group=0)

• Affects only eligible individuals (partition=1) but not others (partition=0)

• Has a true treatment effect of 2.0

# Generate DDD data using the library function
from diff_diff import generate_ddd_data

# Generate synthetic DDD data that mimics a policy setting:
# - Enacted in some states (group=1) but not others (group=0)
# - Affects only eligible individuals (partition=1) but not others (partition=0)
# - Has a true treatment effect of 2.0
data = generate_ddd_data(
    n_per_cell=200,
    treatment_effect=2.0,
    group_effect=5.0,      # Main effect of being in treated group
    partition_effect=3.0,  # Main effect of being in eligible partition
    time_effect=2.0,       # Main effect of post-treatment period
    noise_sd=3.0,
    add_covariates=True,   # Include age and education covariates
    seed=42
)

print(f"Dataset shape: {data.shape}")
print(f"\nSample composition:")
print(data.groupby(['group', 'partition', 'time']).size().unstack(fill_value=0))

Basic DDD Estimation

Let’s estimate the treatment effect using the TripleDifference class:

# Create and fit the DDD estimator
ddd = TripleDifference(estimation_method='dr')  # doubly robust (recommended)

results = ddd.fit(
    data,
    outcome='outcome',
    group='group',
    partition='partition',
    time='time'
)

# Print results
results.print_summary()

Understanding the DDD Estimand

The Triple Difference can be written as:

DDD = [Y(G=1,P=1,T=1) - Y(G=1,P=1,T=0)]   # Change for treated, eligible
    - [Y(G=1,P=0,T=1) - Y(G=1,P=0,T=0)]   # Change for treated, ineligible
    - [Y(G=0,P=1,T=1) - Y(G=0,P=1,T=0)]   # Change for control, eligible
    + [Y(G=0,P=0,T=1) - Y(G=0,P=0,T=0)]   # Change for control, ineligible

Let’s verify this manually:

# Compute cell means
cell_means = data.groupby(['group', 'partition', 'time'])['outcome'].mean().unstack()
print("Cell Means:")
print(cell_means)
print()

# Manual DDD calculation
y_111 = data[(data['group']==1) & (data['partition']==1) & (data['time']==1)]['outcome'].mean()
y_110 = data[(data['group']==1) & (data['partition']==1) & (data['time']==0)]['outcome'].mean()
y_101 = data[(data['group']==1) & (data['partition']==0) & (data['time']==1)]['outcome'].mean()
y_100 = data[(data['group']==1) & (data['partition']==0) & (data['time']==0)]['outcome'].mean()
y_011 = data[(data['group']==0) & (data['partition']==1) & (data['time']==1)]['outcome'].mean()
y_010 = data[(data['group']==0) & (data['partition']==1) & (data['time']==0)]['outcome'].mean()
y_001 = data[(data['group']==0) & (data['partition']==0) & (data['time']==1)]['outcome'].mean()
y_000 = data[(data['group']==0) & (data['partition']==0) & (data['time']==0)]['outcome'].mean()

manual_ddd = (y_111 - y_110) - (y_101 - y_100) - (y_011 - y_010) + (y_001 - y_000)
print(f"Manual DDD calculation: {manual_ddd:.4f}")
print(f"Estimator DDD result:   {results.att:.4f}")

Estimation Methods

The TripleDifference class supports three estimation methods:

1. Regression Adjustment (``reg``): Uses outcome regression with full interactions

2. Inverse Probability Weighting (``ipw``): Uses propensity scores to reweight observations

3. Doubly Robust (``dr``): Combines both methods for robustness

The doubly robust estimator is recommended as it’s consistent if either the outcome model or the propensity score model is correctly specified.

# Compare estimation methods
methods = ['reg', 'ipw', 'dr']
results_comparison = {}

for method in methods:
    est = TripleDifference(estimation_method=method)
    res = est.fit(
        data,
        outcome='outcome',
        group='group',
        partition='partition',
        time='time'
    )
    results_comparison[method] = res
    print(f"{method.upper():4s}: ATT = {res.att:7.4f} (SE = {res.se:.4f}, p = {res.p_value:.4f})")

Adding Covariates

A key insight from Ortiz-Villavicencio & Sant’Anna (2025) is that naive DDD implementations are invalid when covariates are needed for identification. The TripleDifference class properly incorporates covariates:

# Estimate with covariates
ddd_with_cov = TripleDifference(estimation_method='dr')

results_cov = ddd_with_cov.fit(
    data,
    outcome='outcome',
    group='group',
    partition='partition',
    time='time',
    covariates=['age', 'education']
)

results_cov.print_summary()

Convenience Function

For quick estimation, you can use the triple_difference() convenience function:

# One-liner estimation
quick_results = triple_difference(
    data,
    outcome='outcome',
    group='group',
    partition='partition',
    time='time',
    covariates=['age', 'education'],
    estimation_method='dr'
)

print(f"ATT: {quick_results.att:.4f} (95% CI: [{quick_results.conf_int[0]:.4f}, {quick_results.conf_int[1]:.4f}])")

Visualizing Cell Means

It’s often helpful to visualize the data structure to understand the DDD:

# Plot cell means over time
fig, axes = plt.subplots(1, 2, figsize=(12, 5))

cell_means = data.groupby(['group', 'partition', 'time'])['outcome'].mean().reset_index()

# Plot by group
for g, group_name in [(0, 'Control States'), (1, 'Treated States')]:
    ax = axes[g]
    for p, (style, label) in enumerate([('--', 'Ineligible'), ('-', 'Eligible')]):
        subset = cell_means[(cell_means['group']==g) & (cell_means['partition']==p)]
        ax.plot(subset['time'], subset['outcome'], style, marker='o',
                linewidth=2, markersize=8, label=label)

    ax.set_xlabel('Time Period (0=Pre, 1=Post)', fontsize=12)
    ax.set_ylabel('Mean Outcome', fontsize=12)
    ax.set_title(group_name, fontsize=14)
    ax.set_xticks([0, 1])
    ax.set_xticklabels(['Pre', 'Post'])
    ax.legend()
    ax.grid(True, alpha=0.3)

plt.suptitle('DDD Structure: Comparing Trends Across Groups and Partitions', fontsize=14, y=1.02)
plt.tight_layout()
plt.show()

The DDD Parallel Trends Assumption

The key identifying assumption for DDD is:

In the absence of treatment, the differential trend between eligible and ineligible units would have been the same across treated and control groups.

This is weaker than requiring two separate DiD parallel trends assumptions. Even if:

• Eligible units have different trends than ineligible units

• Treated states have different trends than control states

…the DDD is valid as long as the difference in differences is constant.

When DDD Helps

DDD is particularly useful when you suspect:

1. Group-specific shocks (e.g., economic conditions in treatment states)

2. Partition-specific shocks (e.g., trends affecting the eligible population everywhere)

As long as these biases are additive, DDD differences them out.

Accessing Results

The TripleDifferenceResults object provides easy access to all estimation details:

# Access individual results
print(f"ATT estimate: {results.att:.4f}")
print(f"Standard error: {results.se:.4f}")
print(f"t-statistic: {results.t_stat:.4f}")
print(f"p-value: {results.p_value:.4f}")
print(f"95% CI: ({results.conf_int[0]:.4f}, {results.conf_int[1]:.4f})")
print(f"\nStatistically significant at 5% level: {results.is_significant}")
print(f"Significance stars: {results.significance_stars}")

# Convert to DataFrame for further analysis
results_df = results.to_dataframe()
print(results_df.T)

# View cell means
print("Cell Means from Estimation:")
for cell, mean in results.group_means.items():
    print(f"  {cell}: {mean:.4f}")

Summary

Key takeaways:

1. Use DDD when treatment requires two criteria (group membership AND partition eligibility)

2. DDD relaxes parallel trends by allowing group-specific and partition-specific violations

3. Use doubly robust estimation (estimation_method='dr') for robustness to model misspecification

4. Properly handle covariates - the TripleDifference class correctly incorporates them, unlike naive implementations

References

• Ortiz-Villavicencio, M., & Sant’Anna, P. H. C. (2025). Better Understanding Triple Differences Estimators. arXiv:2505.09942.

• Gruber, J. (1994). The incidence of mandated maternity benefits. American Economic Review, 84(3), 622-641.

• Olden, A., & Møen, J. (2022). The triple difference estimator. The Econometrics Journal, 25(3), 531-553.