Pre-Trends Power Analysis

Power analysis for pre-trends tests (Roth 2022).

Overview

Passing a pre-trends test does not guarantee that parallel trends holds. Roth (2022) shows that pre-trends tests are often underpowered to detect economically meaningful violations of parallel trends. This module provides tools to assess:

Power: The probability of rejecting the null (no pre-trends) given a violation
Minimum Detectable Violation (MDV): The smallest violation detectable at target power

Key insights from Roth (2022):

Pre-trends tests are joint tests that pre-period coefficients equal zero
Standard pre-trends tests often have low power against linear trends
A “passed” pre-trends test may simply reflect lack of statistical power
MDV provides the minimum violation the test could have detected

PreTrendsPower

Main class for pre-trends power analysis.

class diff_diff.PreTrendsPower[source]

Bases: object

Pre-trends power analysis (Roth 2022).

Computes the power of pre-trends tests to detect violations of parallel trends, and the minimum detectable violation (MDV).

Parameters:

alpha (float, default=0.05) – Significance level for the pre-trends test.
power (float, default=0.80) – Target power level for MDV calculation.
violation_type (str, default='linear') – Type of violation pattern to consider: - ‘linear’: Violations follow a linear trend (most common) - ‘constant’: Same violation in all pre-periods - ‘last_period’: Violation only in the last pre-period - ‘custom’: User-specified violation pattern (via violation_weights)
violation_weights (array-like, optional) – Custom weights for violation pattern. Length must equal number of pre-periods. Only used when violation_type=’custom’.

Examples

Basic usage with MultiPeriodDiD results:

>>> from diff_diff import MultiPeriodDiD
>>> from diff_diff.pretrends import PreTrendsPower
>>>
>>> # Fit event study
>>> mp_did = MultiPeriodDiD()
>>> results = mp_did.fit(data, outcome='y', treatment='treated',
...                      time='period', post_periods=[4, 5, 6, 7])
>>>
>>> # Analyze pre-trends power
>>> pt = PreTrendsPower(alpha=0.05, power=0.80)
>>> power_results = pt.fit(results)
>>> print(power_results.summary())
>>>
>>> # Get power curve
>>> curve = pt.power_curve(results)
>>> curve.plot()

Notes

The pre-trends test is typically a joint test that all pre-period coefficients are zero. This test has limited power to detect small violations, especially when:

There are few pre-periods
Standard errors are large
The violation pattern is smooth (e.g., linear trend)

Passing a pre-trends test does NOT mean parallel trends holds. It means violations smaller than the MDV cannot be ruled out. For robust inference, combine with HonestDiD sensitivity analysis.

References

Roth, J. (2022). Pretest with Caution: Event-Study Estimates after Testing: for Parallel Trends. American Economic Review: Insights, 4(3), 305-322.

Methods

`fit`(results[, M, pre_periods])	Compute pre-trends power analysis.
`power_curve`(results[, M_grid, n_points, ...])	Compute power across a range of violation magnitudes.
`sensitivity_to_honest_did`(results[, pre_periods])	Compare pre-trends power analysis with HonestDiD sensitivity.

__init__(alpha=0.05, power=0.8, violation_type='linear', violation_weights=None)[source]

Parameters:

alpha (float)
power (float)
violation_type (Literal['linear', 'constant', 'last_period', 'custom'])
violation_weights (ndarray | None)

get_params()[source]

Get parameters for this estimator.

Return type:: Dict[str, Any]

set_params(**params)[source]

Set parameters for this estimator.

Return type:: PreTrendsPower

fit(results, M=None, pre_periods=None)[source]

Compute pre-trends power analysis.

Parameters:

results (MultiPeriodDiDResults, CallawaySantAnnaResults, or SunAbrahamResults) – Results from an event study estimation.
M (float, optional) – Specific violation magnitude to evaluate. If None, evaluates at a default magnitude based on the data.
pre_periods (list of int, optional) – Explicit list of pre-treatment periods to use for power analysis. If None, attempts to infer from results.pre_periods. Use this when you’ve estimated an event study with all periods in post_periods and need to specify which are actually pre-treatment.

Returns:

Power analysis results including power and MDV.

Return type:

PreTrendsPowerResults

power_at(results, M, pre_periods=None)[source]

Compute power to detect a specific violation magnitude.

Parameters:

results (results object) – Event study results.
M (float) – Violation magnitude.
pre_periods (list of int, optional) – Explicit list of pre-treatment periods. See fit() for details.

Returns:

Power to detect violation of magnitude M.

Return type:

float

power_curve(results, M_grid=None, n_points=50, pre_periods=None)[source]

Compute power across a range of violation magnitudes.

Parameters:

results (results object) – Event study results.
M_grid (list of float, optional) – Specific violation magnitudes to evaluate. If None, creates automatic grid from 0 to 2.5 * MDV.
n_points (int, default=50) – Number of points in automatic grid.
pre_periods (list of int, optional) – Explicit list of pre-treatment periods. See fit() for details.

Returns:

Power curve data with plot method.

Return type:

PreTrendsPowerCurve

sensitivity_to_honest_did(results, pre_periods=None)[source]

Compare pre-trends power analysis with HonestDiD sensitivity.

This method helps interpret how informative a passing pre-trends test is in the context of HonestDiD’s relative magnitudes restriction.

Parameters:

results (results object) – Event study results.
pre_periods (list of int, optional) – Explicit list of pre-treatment periods. See fit() for details.

Returns:

Dictionary with: - mdv: Minimum detectable violation from pre-trends test - honest_M_at_mdv: Corresponding M value for HonestDiD - interpretation: Text explaining the relationship

Return type:

dict

Example

from diff_diff import MultiPeriodDiD, PreTrendsPower

# First fit an event study
model = MultiPeriodDiD(reference_period=-1)
results = model.fit(data, outcome='y', treated='treated',
                    time='period', unit='unit_id', treatment_start=5)

# Compute pre-trends power for linear violations
pt = PreTrendsPower(alpha=0.05, power=0.80, violation_type='linear')
pt_results = pt.fit(results)

print(f"MDV: {pt_results.mdv:.3f}")
print(f"Power: {pt_results.power:.2%}")

PreTrendsPowerResults

Results from pre-trends power analysis.

class diff_diff.PreTrendsPowerResults[source]

Bases: object

Results from pre-trends power analysis.

power

Power to detect the specified violation pattern at given alpha.

Type:: float

mdv

Minimum detectable violation (smallest M detectable at target power).

Type:: float

violation_magnitude

The magnitude of violation tested (M parameter).

Type:: float

violation_type

Type of violation pattern (‘linear’, ‘constant’, ‘last_period’, ‘custom’).

Type:: str

alpha

Significance level for the pre-trends test.

Type:: float

target_power

Target power level used for MDV calculation.

Type:: float

n_pre_periods

Number of pre-treatment periods in the event study.

Type:: int

test_statistic

Expected test statistic under the specified violation.

Type:: float

critical_value

Critical value for the pre-trends test.

Type:: float

noncentrality

Non-centrality parameter under the alternative hypothesis.

Type:: float

pre_period_effects

Estimated pre-period effects from the event study.

Type:: np.ndarray

pre_period_ses

Standard errors of pre-period effects.

Type:: np.ndarray

vcov

Variance-covariance matrix of pre-period effects.

Type:: np.ndarray

power: float

mdv: float

violation_magnitude: float

violation_type: str

alpha: float

target_power: float

n_pre_periods: int

test_statistic: float

critical_value: float

noncentrality: float

pre_period_effects: ndarray

pre_period_ses: ndarray

vcov: ndarray

original_results: Any | None = None

property is_informative: bool

Check if the pre-trends test is informative.

A pre-trends test is considered informative if the MDV is reasonably small relative to typical effect sizes. This is a heuristic check; see the summary for interpretation guidance.

property power_adequate: bool: Check if power meets the target threshold.

summary()[source]

Generate formatted summary of pre-trends power analysis.

Returns:: Formatted summary.
Return type:: str

print_summary()[source]

Print summary to stdout.

Return type:: None

to_dict()[source]

Convert results to dictionary.

Return type:: Dict[str, Any]

to_dataframe()[source]

Convert results to DataFrame.

Return type:: DataFrame

power_at(M)[source]

Compute power to detect a specific violation magnitude.

This method allows computing power at different M values without re-fitting the model, using the stored variance-covariance matrix.

Parameters:: M (float) – Violation magnitude to evaluate.
Returns:: Power to detect violation of magnitude M.
Return type:: float

__init__(power, mdv, violation_magnitude, violation_type, alpha, target_power, n_pre_periods, test_statistic, critical_value, noncentrality, pre_period_effects, pre_period_ses, vcov, original_results=None)

Parameters:

power (float)
mdv (float)
violation_magnitude (float)
violation_type (str)
alpha (float)
target_power (float)
n_pre_periods (int)
test_statistic (float)
critical_value (float)
noncentrality (float)
pre_period_effects (ndarray)
pre_period_ses (ndarray)
vcov (ndarray)
original_results (Any | None)

Return type:

None

PreTrendsPowerCurve

Power curve across violation magnitudes.

class diff_diff.PreTrendsPowerCurve[source]

Bases: object

Power curve across violation magnitudes.

M_values

Grid of violation magnitudes tested.

Type:: np.ndarray

powers

Power at each violation magnitude.

Type:: np.ndarray

mdv

Minimum detectable violation.

Type:: float

alpha

Significance level.

Type:: float

target_power

Target power level.

Type:: float

violation_type

Type of violation pattern.

Type:: str

M_values: ndarray

powers: ndarray

mdv: float

alpha: float

target_power: float

violation_type: str

to_dataframe()[source]

Convert to DataFrame with M and power columns.

Return type:: DataFrame

plot(ax=None, show_mdv=True, show_target=True, color='#2563eb', mdv_color='#dc2626', target_color='#22c55e', **kwargs)[source]

Plot the power curve.

Parameters:

ax (matplotlib.axes.Axes, optional) – Axes to plot on. If None, creates new figure.
show_mdv (bool, default=True) – Whether to show vertical line at MDV.
show_target (bool, default=True) – Whether to show horizontal line at target power.
color (str) – Color for power curve line.
mdv_color (str) – Color for MDV vertical line.
target_color (str) – Color for target power horizontal line.
**kwargs – Additional arguments passed to plt.plot().

Returns:

ax – The axes with the plot.

Return type:

matplotlib.axes.Axes

__init__(M_values, powers, mdv, alpha, target_power, violation_type)

Parameters:

M_values (ndarray)
powers (ndarray)
mdv (float)
alpha (float)
target_power (float)
violation_type (str)

Return type:

None

Convenience Functions

compute_pretrends_power

Quick computation of pre-trends power.

diff_diff.compute_pretrends_power(results, M=None, alpha=0.05, target_power=0.8, violation_type='linear', pre_periods=None)[source]

Convenience function for pre-trends power analysis.

Parameters:

results (results object) – Event study results.
M (float, optional) – Violation magnitude to evaluate.
alpha (float, default=0.05) – Significance level.
target_power (float, default=0.80) – Target power for MDV calculation.
violation_type (str, default='linear') – Type of violation pattern.
pre_periods (list of int, optional) – Explicit list of pre-treatment periods. If None, attempts to infer from results. Use when you’ve estimated all periods as post_periods.

Returns:

Power analysis results.

Return type:

PreTrendsPowerResults

Examples

>>> from diff_diff import MultiPeriodDiD
>>> from diff_diff.pretrends import compute_pretrends_power
>>>
>>> results = MultiPeriodDiD().fit(data, ...)
>>> power_results = compute_pretrends_power(results, pre_periods=[0, 1, 2, 3])
>>> print(f"MDV: {power_results.mdv:.3f}")
>>> print(f"Power: {power_results.power:.1%}")

compute_mdv

Compute minimum detectable violation.

diff_diff.compute_mdv(results, alpha=0.05, target_power=0.8, violation_type='linear', pre_periods=None)[source]

Compute minimum detectable violation.

Parameters:

results (results object) – Event study results.
alpha (float, default=0.05) – Significance level.
target_power (float, default=0.80) – Target power for MDV calculation.
violation_type (str, default='linear') – Type of violation pattern.
pre_periods (list of int, optional) – Explicit list of pre-treatment periods. If None, attempts to infer from results. Use when you’ve estimated all periods as post_periods.

Returns:

Minimum detectable violation.

Return type:

float

Violation Types

The module supports several types of pre-trends violations:

linear: Linear trend violations where each pre-period differs from the reference by an amount proportional to distance. delta[t] = M * t for pre-periods.
constant: Constant violations where all pre-periods have the same deviation. delta[t] = M for all pre-periods.
last_period: Only the period immediately before treatment is violated. delta[-1] = M, all other pre-periods are zero.
custom: User-specified violation pattern via the custom_delta parameter.

Complete Example

import numpy as np
from diff_diff import (
    MultiPeriodDiD,
    PreTrendsPower,
    compute_mdv,
    plot_pretrends_power,
)

# Fit event study
model = MultiPeriodDiD(reference_period=-1)
results = model.fit(data, outcome='y', treated='treated',
                    time='period', unit='unit_id', treatment_start=5)

# Compute MDV
mdv = compute_mdv(results, alpha=0.05, power=0.80)
print(f"Minimum Detectable Violation: {mdv:.3f}")

# Power curve analysis
pt = PreTrendsPower(alpha=0.05, violation_type='linear')
curve = pt.power_curve(results, n_points=50)

# Plot power curve
fig = plot_pretrends_power(curve, show_mdv=True, target_power=0.80)
fig.savefig('pretrends_power.png')

# Integration with HonestDiD
sensitivity = pt.sensitivity_to_honest_did(
    results,
    honest_method='smoothness',
    M_grid=np.linspace(0, mdv, 21)
)

References

Roth, J. (2022). Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends. American Economic Review: Insights, 4(3), 305-322.
R package: pretrends

Pre-Trends Power Analysis

Overview

PreTrendsPower

Example

PreTrendsPowerResults

PreTrendsPowerCurve

Convenience Functions

compute_pretrends_power

compute_mdv

Violation Types

Complete Example

References

See Also