Pre-Trends Power Analysis

Power analysis for pre-trends tests (Roth 2022).

Overview

Passing a pre-trends test does not guarantee that parallel trends holds. Roth (2022) shows that pre-trends tests are often underpowered to detect economically meaningful violations of parallel trends. This module provides tools to assess:

1. Power: The probability of rejecting the null (no pre-trends) given a violation

2. Minimum Detectable Violation (MDV): The smallest violation detectable at target power

Key insights from Roth (2022):

• Pre-trends tests are joint tests that pre-period coefficients equal zero

• Standard pre-trends tests often have low power against linear trends

• A “passed” pre-trends test may simply reflect lack of statistical power

• MDV provides the minimum violation the test could have detected

PreTrendsPower

Main class for pre-trends power analysis.

class diff_diff.PreTrendsPower

Bases: object

Pre-trends power analysis (Roth 2022).

Computes the power of pre-trends tests to detect violations of parallel trends, and the minimum detectable violation (MDV).

Parameters:

• alpha (float, default=0.05) – Significance level for the pre-trends test.

• power (float, default=0.80) – Target power level for MDV calculation.

• violation_type (str, default='linear') – Type of violation pattern to consider: - ‘linear’: Violations follow a linear trend (most common) - ‘constant’: Same violation in all pre-periods - ‘last_period’: Violation only in the last pre-period - ‘custom’: User-specified violation pattern (via violation_weights)

• violation_weights (array-like, optional) – Custom weights for violation pattern. Length must equal number of pre-periods. Only used when violation_type=’custom’.

• pretest_form ({'nis', 'wald'}, default='nis') –

  Pre-trends test acceptance-region form:

  • 'nis': Roth (2022) no-individually-significant pretest (Section II.A-B). Acceptance region is B_NIS(Σ) = { b : |b_t| <= z_{1-α/2} σ_t for all t }. Power computed via multivariate normal box probability. This is the new default (PR-B 2026-05-17), matching both the paper’s primary analysis and the R pretrends package.

  • 'wald': Noncentral chi-squared on the quadratic form δ' Σ_22^{-1} δ (the shipped behavior prior to PR-B 2026-05-17). Retained as a paper-supported alternative under Propositions 1+3+4 (Wald acceptance region is a convex ellipsoid, so all four propositions apply). Use this for backwards-compat with shipped numerical baselines.

Examples

Basic usage with MultiPeriodDiD results:

>>> from diff_diff import MultiPeriodDiD
>>> from diff_diff.pretrends import PreTrendsPower
>>>
>>> # Fit event study
>>> mp_did = MultiPeriodDiD()
>>> results = mp_did.fit(data, outcome='y', treatment='treated',
...                      time='period', post_periods=[4, 5, 6, 7])
>>>
>>> # Analyze pre-trends power
>>> pt = PreTrendsPower(alpha=0.05, power=0.80)
>>> power_results = pt.fit(results)
>>> print(power_results.summary())
>>>
>>> # Get power curve
>>> curve = pt.power_curve(results)
>>> curve.plot()

Notes

The pre-trends test is typically a joint test that all pre-period coefficients are zero. This test has limited power to detect small violations, especially when:

1. There are few pre-periods

2. Standard errors are large

3. The violation pattern is smooth (e.g., linear trend)

Passing a pre-trends test does NOT mean parallel trends holds. It means violations smaller than the MDV cannot be ruled out. For robust inference, combine with HonestDiD sensitivity analysis.

References

Roth, J. (2022). Pretest with Caution: Event-Study Estimates after Testing

for Parallel Trends. American Economic Review: Insights, 4(3), 305-322.

Methods

fit(results[, M, pre_periods])	Compute pre-trends power analysis.
power_curve(results[, M_grid, n_points, ...])	Compute power across a range of violation magnitudes.
sensitivity_to_honest_did(results[, pre_periods])	Compare pre-trends power analysis with HonestDiD sensitivity.

__init__(alpha=0.05, power=0.8, violation_type='linear', violation_weights=None, pretest_form='nis')

Parameters:

• alpha (float)

• power (float)

• violation_type (Literal['linear', 'constant', 'last_period', 'custom'])

• violation_weights (ndarray | None)

• pretest_form (Literal['nis', 'wald'])

get_params()

Get parameters for this estimator.

Return type:

Dict[str, Any]

set_params(**params)

Set parameters for this estimator.

Return type:

PreTrendsPower

fit(results, M=None, pre_periods=None)

Compute pre-trends power analysis.

Parameters:

• results (MultiPeriodDiDResults, CallawaySantAnnaResults, or SunAbrahamResults) – Results from an event study estimation.

• M (float, optional) – Specific violation magnitude to evaluate. If None, evaluates at a default magnitude based on the data.

• pre_periods (list of int, optional) – Explicit list of pre-treatment periods to use for power analysis. If None, attempts to infer from results.pre_periods. Use this when you’ve estimated an event study with all periods in post_periods and need to specify which are actually pre-treatment.

Returns:

Power analysis results including power and MDV.

Return type:

PreTrendsPowerResults

power_at(results, M, pre_periods=None)

Compute power to detect a specific violation magnitude.

Parameters:

• results (results object) – Event study results.

• M (float) – Violation magnitude.

• pre_periods (list of int, optional) – Explicit list of pre-treatment periods. See fit() for details.

Returns:

Power to detect violation of magnitude M.

Return type:

float

power_curve(results, M_grid=None, n_points=50, pre_periods=None)

Compute power across a range of violation magnitudes.

Parameters:

• results (results object) – Event study results.

• M_grid (list of float, optional) – Specific violation magnitudes to evaluate. If None, creates automatic grid from 0 to 2.5 * MDV.

• n_points (int, default=50) – Number of points in automatic grid.

• pre_periods (list of int, optional) – Explicit list of pre-treatment periods. See fit() for details.

Returns:

Power curve data with plot method.

Return type:

PreTrendsPowerCurve

sensitivity_to_honest_did(results, pre_periods=None)

Compare pre-trends power analysis with HonestDiD sensitivity.

This method helps interpret how informative a passing pre-trends test is in the context of HonestDiD’s relative magnitudes restriction.

Parameters:

• results (results object) – Event study results.

• pre_periods (list of int, optional) – Explicit list of pre-treatment periods. See fit() for details.

Returns:

Dictionary with: - mdv: Minimum detectable violation from pre-trends test - honest_M_at_mdv: Corresponding M value for HonestDiD - interpretation: Text explaining the relationship

Return type:

dict

Example

from diff_diff import MultiPeriodDiD, PreTrendsPower

# First fit an event study
model = MultiPeriodDiD()
results = model.fit(data, outcome='y', treatment='treated',
                    time='period', unit='unit_id',
                    post_periods=[5, 6, 7], reference_period=4)

# Compute pre-trends power for linear violations.
# Default acceptance region is the Roth (2022) NIS box probability.
pt = PreTrendsPower(alpha=0.05, power=0.80, violation_type='linear')
pt_results = pt.fit(results)

print(f"MDV: {pt_results.mdv:.3f}")
print(f"Power: {pt_results.power:.2%}")
print(f"NIS box probability (accept H0): {pt_results.nis_box_probability:.4f}")

# Select the Wald (noncentral-χ²) acceptance-region form instead of the
# default NIS box probability. Wald preserves the pre-PR-B acceptance-
# region math byte-identically; numerical-output bit-identity to pre-PR-B
# fitted results only holds on regular pre-period grids and on the
# legacy `relative_times=None` path. PR-B Step 4's `relative_times`
# threading applies to BOTH NIS and Wald, so on irregular grids the
# Wald MDV is also in Roth's γ units (see REGISTRY linear-pattern Note).
pt_wald = PreTrendsPower(
    alpha=0.05, power=0.80, violation_type='linear', pretest_form='wald'
)

PreTrendsPowerResults

Results from pre-trends power analysis.

class diff_diff.PreTrendsPowerResults

Bases: object

Results from pre-trends power analysis.

power

Power to detect the specified violation pattern at given alpha.

Type:

float

mdv

Minimum detectable violation (smallest M detectable at target power).

Type:

float

violation_magnitude

The magnitude of violation tested (M parameter).

Type:

float

violation_type

Type of violation pattern (‘linear’, ‘constant’, ‘last_period’, ‘custom’).

Type:

str

alpha

Significance level for the pre-trends test.

Type:

float

target_power

Target power level used for MDV calculation.

Type:

float

n_pre_periods

Number of pre-treatment periods in the event study.

Type:

int

test_statistic

Expected test statistic under the specified violation (Wald only; NaN for NIS fits).

Type:

float

critical_value

Critical value for the pre-trends test.

Type:

float

noncentrality

Non-centrality parameter under the alternative hypothesis (Wald only; NaN for NIS fits).

Type:

float

pre_period_effects

Estimated pre-period effects from the event study.

Type:

np.ndarray

pre_period_ses

Standard errors of pre-period effects.

Type:

np.ndarray

vcov

Variance-covariance matrix of pre-period effects.

Type:

np.ndarray

pretest_form

Pretest acceptance-region form used: 'nis' (no-individually- significant box probability — Roth 2022 Section II.A-B, default for new fits) or 'wald' (noncentral-chi-squared on the quadratic form delta' Sigma_22^{-1} delta — paper-supported alternative, retained for backwards compatibility with shipped numerical baselines).

Type:

str

nis_box_probability

Acceptance probability P(beta_hat_pre in B_NIS(Sigma)) under the alternative M * weights. NIS-only; NaN for Wald fits.

Type:

float

violation_weights

The violation-direction vector used at fit time. Populated for all violation types on fresh fits. Normalization depends on the type so that M always matches the documented per-pattern contract:

• linear threaded with relative_times (post PR-B Step 4): |t| directly, NOT L2-normalized, so δ_t = M·|t| and the reported MDV equals Roth’s γ exactly.

• linear without relative_times (legacy): [n_pre-1, ..., 0] L2-normalized.

• constant (post PR-B R13): [1, ..., 1] directly, NOT L2-normalized, so δ_t = M is a true per-period level shift.

• last_period: [0, ..., 0, 1] (already unit-norm).

• custom: user vector L2-normalized to unit norm.

Old serialized results may have None here; power_at() falls back to reconstruction in that case (with the PR-A NotImplementedError guard retained only for violation_type='custom' with violation_weights=None).

Type:

np.ndarray, optional

power: float

mdv: float

violation_magnitude: float

violation_type: str

alpha: float

target_power: float

n_pre_periods: int

test_statistic: float

critical_value: float

noncentrality: float

pre_period_effects: ndarray

pre_period_ses: ndarray

vcov: ndarray

original_results: Any | None = None

pretest_form: Literal['nis', 'wald'] = 'wald'

nis_box_probability: float = nan

violation_weights: ndarray | None = None

covariance_source: str = 'unknown'

property is_informative: bool

Check if the pre-trends test is informative.

A pre-trends test is considered informative if the MAX level-scale pre-period violation under the MDV is reasonably small relative to the per-period standard errors. Post PR-B Step 4 the linear MDV is in Roth’s γ units (a slope), so comparing the raw mdv scalar to the level-scale max(pre_period_ses) would mix units on irregular pre-period grids. The comparable level-scale scalar is mdv * max(|violation_weights|) (the largest pre-period deviation under the MDV — see max_abs_pre_violation).

property max_abs_pre_violation: float

Largest level-scale pre-period deviation under the MDV.

Returns mdv * max(|violation_weights|) — the maximum absolute pre-period violation δ_t when the violation magnitude equals the MDV. This is the right level-scale scalar for comparing pre-trends sensitivity against coefficient-scale quantities (post-treatment ATT, per-period SEs, HonestDiD’s M bound).

Why this matters: PR-B Step 4 made the linear mdv report Roth’s γ units (a slope on relative time). On a regular grid [-3, -2, -1] the max deviation is γ * 3; on an irregular grid [-5, -3, -1] it is γ * 5. Raw mdv alone cannot be compared to level effects without applying the weight scale.

For non-linear violation types under the PR-B R13 level-shift convention: constant weights [1, ..., 1] (unnormalized) yield max_abs_pre_violation = mdv * 1 = mdv — raw mdv IS the per-period level shift, so level- and γ-scales coincide. Last_period [0, ..., 0, 1] yields max_abs_pre_violation = mdv for the same reason. Custom uses the L2-normalized user-supplied weight vector, so max_abs_pre_violation depends on the user’s direction.

Backwards-compat: legacy serialized results without violation_weights (pre-PR-B) fall back to the raw mdv (which under the pre-PR-B count-based L2-normalized linear convention already had a roughly level-scale magnitude).

property power_adequate: bool

Check if power meets the target threshold.

summary()

Generate formatted summary of pre-trends power analysis.

Returns:

Formatted summary.

Return type:

str

print_summary()

Print summary to stdout.

Return type:

None

to_dict()

Convert results to JSON-serializable dictionary.

Includes the post-PR-B provenance fields (violation_weights, covariance_source) so callers that round-trip the result through to_dict/to_dataframe (e.g., for serialization or downstream transport) preserve the same information the reporting layer reads off the dataclass directly.

violation_weights is emitted as list[float] (or None) so json.dumps(result.to_dict()) works out of the box. Use self.violation_weights directly on the dataclass when an ndarray is needed.

Return type:

Dict[str, Any]

to_dataframe()

Convert results to DataFrame.

violation_weights is stored as a Python list in the single row (pandas-friendly); covariance_source is a plain string. Mirrors to_dict.

Return type:

DataFrame

power_at(M)

Compute power to detect a specific violation magnitude.

Uses the stored fitted violation_weights and the stored pretest_form to dispatch to the NIS or Wald power computation without re-fitting.

Parameters:

M (float) – Violation magnitude to evaluate.

Returns:

Power to detect violation of magnitude M.

Return type:

float

Raises:

NotImplementedError – If the result was produced by an older library version (before the violation_weights field was added to PreTrendsPowerResults) AND violation_type='custom'. The reconstruction fallback can handle linear/constant/last_period from stored metadata, but custom weights cannot be reconstructed; refit PreTrendsPower(violation_type='custom', violation_weights=...) with the new M instead.

__init__(power, mdv, violation_magnitude, violation_type, alpha, target_power, n_pre_periods, test_statistic, critical_value, noncentrality, pre_period_effects, pre_period_ses, vcov, original_results=None, pretest_form='wald', nis_box_probability=nan, violation_weights=None, covariance_source='unknown')

Parameters:

• power (float)

• mdv (float)

• violation_magnitude (float)

• violation_type (str)

• alpha (float)

• target_power (float)

• n_pre_periods (int)

• test_statistic (float)

• critical_value (float)

• noncentrality (float)

• pre_period_effects (ndarray)

• pre_period_ses (ndarray)

• vcov (ndarray)

• original_results (Any | None)

• pretest_form (Literal['nis', 'wald'])

• nis_box_probability (float)

• violation_weights (ndarray | None)

• covariance_source (str)

Return type:

None

PreTrendsPowerCurve

Power curve across violation magnitudes.

class diff_diff.PreTrendsPowerCurve

Bases: object

Power curve across violation magnitudes.

M_values

Grid of violation magnitudes tested.

Type:

np.ndarray

powers

Power at each violation magnitude.

Type:

np.ndarray

mdv

Minimum detectable violation.

Type:

float

alpha

Significance level.

Type:

float

target_power

Target power level.

Type:

float

violation_type

Type of violation pattern.

Type:

str

pretest_form

Pretest acceptance-region form ('nis' or 'wald') used to compute the curve. NIS and Wald curves can differ materially under correlated Σ_22; persisting the form prevents callers from misinterpreting a serialized/plotted curve.

Type:

str

M_values: ndarray

powers: ndarray

mdv: float

alpha: float

target_power: float

violation_type: str

pretest_form: Literal['nis', 'wald'] = 'wald'

to_dataframe()

Convert to DataFrame with M, power, and pretest_form columns.

Return type:

DataFrame

plot(ax=None, show_mdv=True, show_target=True, color='#2563eb', mdv_color='#dc2626', target_color='#22c55e', **kwargs)

Plot the power curve.

Parameters:

• ax (matplotlib.axes.Axes, optional) – Axes to plot on. If None, creates new figure.

• show_mdv (bool, default=True) – Whether to show vertical line at MDV.

• show_target (bool, default=True) – Whether to show horizontal line at target power.

• color (str) – Color for power curve line.

• mdv_color (str) – Color for MDV vertical line.

• target_color (str) – Color for target power horizontal line.

• **kwargs – Additional arguments passed to plt.plot().

Returns:

ax – The axes with the plot.

Return type:

matplotlib.axes.Axes

__init__(M_values, powers, mdv, alpha, target_power, violation_type, pretest_form='wald')

Parameters:

• M_values (ndarray)

• powers (ndarray)

• mdv (float)

• alpha (float)

• target_power (float)

• violation_type (str)

• pretest_form (Literal['nis', 'wald'])

Return type:

None

Convenience Functions

compute_pretrends_power

Quick computation of pre-trends power.

diff_diff.compute_pretrends_power(results, M=None, alpha=0.05, target_power=0.8, violation_type='linear', pre_periods=None, violation_weights=None, pretest_form='nis')

Convenience function for pre-trends power analysis.

Parameters:

• results (results object) – Event study results.

• M (float, optional) – Violation magnitude to evaluate.

• alpha (float, default=0.05) – Significance level.

• target_power (float, default=0.80) – Target power for MDV calculation.

• violation_type (str, default='linear') – Type of violation pattern: linear / constant / last_period / custom. For custom, also pass violation_weights.

• pre_periods (list of int, optional) – Explicit list of pre-treatment periods. If None, attempts to infer from results. Use when you’ve estimated all periods as post_periods.

• violation_weights (np.ndarray, optional) – Custom violation pattern weights. Required when violation_type='custom'; ignored for other violation types.

• pretest_form ({'nis', 'wald'}, default='nis') – Pretest acceptance-region form. 'nis' (default) implements Roth (2022) Section II.A-B no-individually-significant box probability via scipy.stats.multivariate_normal.cdf; 'wald' is the noncentral-chi-squared form retained for backwards compatibility with the pre-PR-B shipped numerical output (also a paper-supported alternative under Propositions 1+3+4).

Returns:

Power analysis results.

Return type:

PreTrendsPowerResults

Examples

>>> from diff_diff import MultiPeriodDiD
>>> from diff_diff.pretrends import compute_pretrends_power
>>>
>>> results = MultiPeriodDiD().fit(data, ...)
>>> power_results = compute_pretrends_power(results, pre_periods=[0, 1, 2, 3])
>>> print(f"MDV: {power_results.mdv:.3f}")
>>> print(f"Power: {power_results.power:.1%}")

compute_mdv

Compute minimum detectable violation.

diff_diff.compute_mdv(results, alpha=0.05, target_power=0.8, violation_type='linear', pre_periods=None, violation_weights=None, pretest_form='nis')

Compute minimum detectable violation.

Parameters:

• results (results object) – Event study results.

• alpha (float, default=0.05) – Significance level.

• target_power (float, default=0.80) – Target power for MDV calculation.

• violation_type (str, default='linear') – Type of violation pattern: linear / constant / last_period / custom. For custom, also pass violation_weights.

• pre_periods (list of int, optional) – Explicit list of pre-treatment periods. If None, attempts to infer from results. Use when you’ve estimated all periods as post_periods.

• violation_weights (np.ndarray, optional) – Custom violation pattern weights. Required when violation_type='custom'; ignored for other violation types.

• pretest_form ({'nis', 'wald'}, default='nis') – Pretest acceptance-region form. See compute_pretrends_power and PreTrendsPower for the NIS-vs-Wald discussion.

Returns:

Minimum detectable violation.

Return type:

float

plot_pretrends_power

Plot a pre-trends test power curve.

diff_diff.plot_pretrends_power(results=None, *, M_values=None, powers=None, mdv=None, target_power=0.8, figsize=(10, 6), title='Pre-Trends Test Power Curve', xlabel='Violation Magnitude (M)', ylabel='Power', color='#2563eb', mdv_color='#dc2626', target_color='#22c55e', linewidth=2.0, show_mdv_line=True, show_target_line=True, show_grid=True, ax=None, show=True, backend='matplotlib')

Plot pre-trends test power curve.

Visualizes how the power to detect parallel trends violations changes with the violation magnitude (M). This helps understand what violations your pre-trends test is capable of detecting.

Parameters:

• results (PreTrendsPowerResults, PreTrendsPowerCurve, or DataFrame, optional) – Results from PreTrendsPower.fit() or power_curve(), or a DataFrame with columns ‘M’ and ‘power’. If None, must provide M_values and powers.

• M_values (list of float, optional) – Violation magnitudes (x-axis). Required if results is None.

• powers (list of float, optional) – Power values (y-axis). Required if results is None.

• mdv (float, optional) – Minimum detectable violation to mark on the plot.

• target_power (float, default=0.80) – Target power level to show as horizontal line.

• figsize (tuple, default=(10, 6)) – Figure size (width, height) in inches.

• title (str) – Plot title.

• xlabel (str) – X-axis label.

• ylabel (str) – Y-axis label.

• color (str, default="#2563eb") – Color for the power curve line.

• mdv_color (str, default="#dc2626") – Color for the MDV vertical line.

• target_color (str, default="#22c55e") – Color for the target power horizontal line.

• linewidth (float, default=2.0) – Line width for the power curve.

• show_mdv_line (bool, default=True) – Whether to show vertical line at MDV.

• show_target_line (bool, default=True) – Whether to show horizontal line at target power.

• show_grid (bool, default=True) – Whether to show grid lines.

• ax (matplotlib.axes.Axes, optional) – Axes to plot on. If None, creates new figure.

• show (bool, default=True) – Whether to call plt.show() at the end.

• backend (str, default="matplotlib") – Plotting backend: "matplotlib" or "plotly".

Returns:

The axes object (matplotlib) or figure (plotly).

Return type:

matplotlib.axes.Axes or plotly.graph_objects.Figure

Examples

From PreTrendsPower results:

>>> from diff_diff import MultiPeriodDiD
>>> from diff_diff.pretrends import PreTrendsPower
>>> from diff_diff.visualization import plot_pretrends_power
>>>
>>> mp_did = MultiPeriodDiD()
>>> event_results = mp_did.fit(data, outcome='y', treatment='treated',
...                            time='period', post_periods=[4, 5, 6, 7])
>>>
>>> pt = PreTrendsPower()
>>> curve = pt.power_curve(event_results)
>>> plot_pretrends_power(curve)

Notes

The power curve shows how likely you are to reject the null hypothesis of parallel trends given a true violation of magnitude M.

See also

PreTrendsPower

Main class for pre-trends power analysis

plot_sensitivity

Plot HonestDiD sensitivity analysis

Violation Types

The module supports several types of pre-trends violations:

linear

Linear trend violations where each pre-period differs from the reference by an amount proportional to distance. delta[t] = M * t for pre-periods.

constant

Constant violations where all pre-periods have the same deviation. delta[t] = M for all pre-periods.

last_period

Only the period immediately before treatment is violated. delta[-1] = M, all other pre-periods are zero.

custom

User-specified violation pattern via the violation_weights parameter. Accepted by both PreTrendsPower (constructor kwarg) and the convenience helpers compute_pretrends_power / compute_mdv (forwarded kwarg).

Complete Example

import numpy as np
from diff_diff import (
    MultiPeriodDiD,
    PreTrendsPower,
    compute_mdv,
    plot_pretrends_power,
)

# Fit event study
model = MultiPeriodDiD()
results = model.fit(data, outcome='y', treatment='treated',
                    time='period', unit='unit_id',
                    post_periods=[5, 6, 7], reference_period=4)

# Compute MDV
mdv = compute_mdv(results, alpha=0.05, target_power=0.80)
print(f"Minimum Detectable Violation: {mdv:.3f}")

# Power curve analysis
pt = PreTrendsPower(alpha=0.05, violation_type='linear')
curve = pt.power_curve(results, n_points=50)

# Plot power curve
ax = plot_pretrends_power(curve, target_power=0.80)
ax.figure.savefig('pretrends_power.png')

# Integration with HonestDiD
sensitivity = pt.sensitivity_to_honest_did(results)

References

• Roth, J. (2022). Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends. American Economic Review: Insights, 4(3), 305-322.

• R package: pretrends

See Also

• Honest DiD - Sensitivity analysis under parallel trends violations

• Utilities - Standard parallel trends tests