Honest DiD#

Sensitivity analysis for violations of parallel trends (Rambachan & Roth 2023).

Overview#

The Honest DiD approach provides inference that is robust to violations of the parallel trends assumption. Instead of assuming parallel trends holds exactly, it bounds the violation and provides confidence intervals that account for potential deviations.

Two main restriction types are supported:

Relative Magnitudes (ΔRM): Post-treatment violations are bounded by M̄ times the maximum pre-treatment violation.
Smoothness (ΔSD): Bounds on the second differences of the trend violations, restricting how much the violation can change between periods.

HonestDiD#

Main class for computing honest bounds and confidence intervals.

class diff_diff.HonestDiD[source]

Bases: object

Honest DiD sensitivity analysis (Rambachan & Roth 2023).

Computes robust inference for difference-in-differences allowing for bounded violations of parallel trends.

Parameters:

method ({"smoothness", "relative_magnitude", "combined"}) – Type of restriction on trend violations: - “smoothness”: Bounds on second differences of trend violations (Delta^SD) - “relative_magnitude”: Post first differences <= M * max pre first difference (Delta^RM) - “combined”: Both restrictions (Delta^SDRM)
M (float, optional) – Restriction parameter. Interpretation depends on method: - smoothness: Max second difference - relative_magnitude: Scaling factor for max pre-period first difference Default is 1.0 for relative_magnitude, 0.0 for smoothness.
alpha (float) – Significance level for confidence intervals.
l_vec (array-like or None) – Weighting vector for scalar parameter (length = num_post_periods). If None, uses uniform weights (average effect).

Examples

>>> from diff_diff import MultiPeriodDiD
>>> from diff_diff.honest_did import HonestDiD
>>>
>>> # Fit event study
>>> mp_did = MultiPeriodDiD()
>>> results = mp_did.fit(data, outcome='y', treatment='treated',
...                      time='period', post_periods=[4,5,6,7])
>>>
>>> # Sensitivity analysis with relative magnitudes
>>> honest = HonestDiD(method='relative_magnitude', M=1.0)
>>> bounds = honest.fit(results)
>>> print(bounds.summary())
>>>
>>> # Sensitivity curve over M values
>>> sensitivity = honest.sensitivity_analysis(results, M_grid=[0, 0.5, 1, 1.5, 2])
>>> sensitivity.plot()

Methods

`fit`(results[, M])	Compute bounds and robust confidence intervals.
`sensitivity_analysis`(results[, M_grid])	Perform sensitivity analysis over a grid of M values.
`breakdown_value`(results[, tol])	Find the breakdown value directly using binary search.

__init__(method='relative_magnitude', M=None, alpha=0.05, l_vec=None)[source]

Parameters:

method (Literal['smoothness', 'relative_magnitude', 'combined'])
M (float | None)
alpha (float)
l_vec (ndarray | None)

get_params()[source]

Get parameters for this estimator.

Return type:: Dict[str, Any]

set_params(**params)[source]

Set parameters for this estimator.

Return type:: HonestDiD

fit(results, M=None)[source]

Compute bounds and robust confidence intervals.

Parameters:

results (MultiPeriodDiDResults, CallawaySantAnnaResults, or ChaisemartinDHaultfoeuilleResults) – Results from event study estimation.
M (float, optional) – Override the M parameter for this fit.

Returns:

Results containing bounds and robust confidence intervals.

Return type:

HonestDiDResults

sensitivity_analysis(results, M_grid=None)[source]

Perform sensitivity analysis over a grid of M values.

Parameters:

results (MultiPeriodDiDResults, CallawaySantAnnaResults, or ChaisemartinDHaultfoeuilleResults) – Results from event study estimation.
M_grid (list of float, optional) – Grid of M values to evaluate. If None, uses default grid based on method.

Returns:

Results containing bounds and CIs for each M value.

Return type:

SensitivityResults

breakdown_value(results, tol=0.01)[source]

Find the breakdown value directly using binary search.

The breakdown value is the smallest M where the robust confidence interval includes zero.

Parameters:

results (MultiPeriodDiDResults, CallawaySantAnnaResults, or ChaisemartinDHaultfoeuilleResults) – Results from event study estimation.
tol (float) – Tolerance for binary search.

Returns:

Breakdown value, or None if effect is always significant.

Return type:

float or None

Example#

from diff_diff import MultiPeriodDiD, HonestDiD

# First fit an event study
model = MultiPeriodDiD()
results = model.fit(data, outcome='y', treatment='treated',
                    time='period', unit='unit_id',
                    post_periods=[5, 6, 7], reference_period=4)

# Compute bounds under relative magnitudes restriction
honest = HonestDiD(method='relative_magnitude', M=1.0)
bounds = honest.fit(results)

print(f"Original estimate: {bounds.original_estimate:.3f}")
print(f"Robust CI: [{bounds.ci_lb:.3f}, {bounds.ci_ub:.3f}]")

HonestDiDResults#

Results from HonestDiD estimation.

class diff_diff.HonestDiDResults[source]

Bases: object

Results from Honest DiD sensitivity analysis.

Contains bounds on the treatment effect under the specified restrictions on violations of parallel trends.

lb

Lower bound of identified set.

Type:: float

ub

Upper bound of identified set.

Type:: float

ci_lb

Lower bound of robust confidence interval.

Type:: float

ci_ub

Upper bound of robust confidence interval.

Type:: float

M

The restriction parameter value used.

Type:: float

method

The type of restriction (“smoothness”, “relative_magnitude”, or “combined”).

Type:: str

original_estimate

The original point estimate (under parallel trends).

Type:: float

original_se

The original standard error.

Type:: float

alpha

Significance level for confidence interval.

Type:: float

ci_method

Method used for CI construction (“FLCI” or “C-LF”).

Type:: str

original_results

The original estimation results object.

Type:: Any

lb: float

ub: float

ci_lb: float

ci_ub: float

M: float

method: str

original_estimate: float

original_se: float

alpha: float = 0.05

ci_method: str = 'FLCI'

target_label: str = 'Equal-weight avg over post horizons'

pre_periods_used: List[Any] | None = None

post_periods_used: List[Any] | None = None

original_results: Any | None = None

event_study_bounds: Dict[Any, Dict[str, float]] | None = None

survey_metadata: Any | None = None

df_survey: int | None = None

property is_significant: bool

Check if CI excludes zero (effect is robust to violations).

Returns False for undefined (NaN) CIs.

property significance_stars: str

Return significance indicator if robust CI excludes zero.

Note: Unlike point estimation, partial identification does not yield a single p-value. This returns “*” if the robust CI excludes zero at the specified alpha level, indicating the effect is robust to the assumed violations of parallel trends.

property identified_set_width: float: Width of the identified set.

property ci_width: float: Width of the confidence interval.

summary()[source]

Generate formatted summary of sensitivity analysis results.

Returns:: Formatted summary.
Return type:: str

print_summary()[source]

Print summary to stdout.

Return type:: None

to_dict()[source]

Convert results to dictionary.

Return type:: Dict[str, Any]

to_dataframe()[source]

Convert results to DataFrame.

Return type:: DataFrame

__init__(lb, ub, ci_lb, ci_ub, M, method, original_estimate, original_se, alpha=0.05, ci_method='FLCI', target_label='Equal-weight avg over post horizons', pre_periods_used=None, post_periods_used=None, original_results=None, event_study_bounds=None, survey_metadata=None, df_survey=None)

Parameters:

lb (float)
ub (float)
ci_lb (float)
ci_ub (float)
M (float)
method (str)
original_estimate (float)
original_se (float)
alpha (float)
ci_method (str)
target_label (str)
pre_periods_used (List[Any] | None)
post_periods_used (List[Any] | None)
original_results (Any | None)
event_study_bounds (Dict[Any, Dict[str, float]] | None)
survey_metadata (Any | None)
df_survey (int | None)

Return type:

None

SensitivityResults#

Results from sensitivity analysis over a grid of M values.

class diff_diff.SensitivityResults[source]

Bases: object

Results from sensitivity analysis over a grid of M values.

Contains bounds and confidence intervals for each M value, plus the breakdown value.

M_values

Grid of M parameter values.

Type:: np.ndarray

bounds

List of (lb, ub) identified set bounds for each M.

Type:: List[Tuple[float, float]]

robust_cis

List of (ci_lb, ci_ub) robust CIs for each M.

Type:: List[Tuple[float, float]]

breakdown_M

Smallest M where robust CI includes zero.

Type:: float

method

Type of restriction used.

Type:: str

original_estimate

Original point estimate.

Type:: float

original_se

Original standard error.

Type:: float

alpha

Significance level.

Type:: float

M_values: ndarray

bounds: List[Tuple[float, float]]

robust_cis: List[Tuple[float, float]]

breakdown_M: float | None

method: str

original_estimate: float

original_se: float

alpha: float = 0.05

property has_breakdown: bool: Check if there is a finite breakdown value.

summary()[source]

Generate formatted summary.

Return type:: str

print_summary()[source]

Print summary to stdout.

Return type:: None

to_dataframe()[source]

Convert to DataFrame with one row per M value.

Return type:: DataFrame

plot(ax=None, show_bounds=True, show_ci=True, breakdown_line=True, **kwargs)[source]

Plot sensitivity analysis results.

Parameters:

ax (matplotlib.axes.Axes, optional) – Axes to plot on. If None, creates new figure.
show_bounds (bool) – Whether to show identified set bounds.
show_ci (bool) – Whether to show confidence intervals.
breakdown_line (bool) – Whether to show vertical line at breakdown value.
**kwargs – Additional arguments passed to plotting functions.

Returns:

ax – The axes with the plot.

Return type:

matplotlib.axes.Axes

__init__(M_values, bounds, robust_cis, breakdown_M, method, original_estimate, original_se, alpha=0.05)

Parameters:

M_values (ndarray)
bounds (List[Tuple[float, float]])
robust_cis (List[Tuple[float, float]])
breakdown_M (float | None)
method (str)
original_estimate (float)
original_se (float)
alpha (float)

Return type:

None

Restriction Classes#

DeltaSD#

Smoothness restriction class.

class diff_diff.DeltaSD[source]

Bases: object

Smoothness restriction on trend violations (Delta^{SD}).

Restricts the second differences of the trend violations:

\[|\delta_{t+1} - 2\delta_t + \delta_{t-1}| \le M\]

When M=0, this enforces that violations follow a linear trend (linear extrapolation of pre-trends). Larger M allows more curvature in the violation path.

Parameters:: M (float) – Maximum allowed second difference. M=0 means linear trends only.

Examples

>>> delta = DeltaSD(M=0.5)
>>> delta.M
0.5

M: float = 0.0

__init__(M=0.0)

Parameters:: M (float)
Return type:: None

DeltaRM#

Relative magnitudes restriction class.

class diff_diff.DeltaRM[source]

Bases: object

Relative magnitudes restriction on trend violations (Delta^{RM}).

Post-treatment consecutive first differences are bounded by Mbar times the maximum pre-treatment first difference:

\[|\delta_{t+1} - \delta_t| \le \overline{M} \cdot \max_{s<0} |\delta_{s+1} - \delta_s|\]

When Mbar=0, this enforces zero post-treatment first differences. Mbar=1 means post-period first differences can be as large as the worst observed pre-period first difference.

Parameters:: Mbar (float) – Scaling factor for maximum pre-period first difference.

Examples

>>> delta = DeltaRM(Mbar=1.0)
>>> delta.Mbar
1.0

Mbar: float = 1.0

__init__(Mbar=1.0)

Parameters:: Mbar (float)
Return type:: None

DeltaSDRM#

Combined smoothness and relative magnitudes restriction.

class diff_diff.DeltaSDRM[source]

Bases: object

Combined smoothness and relative magnitudes restriction.

Imposes both:

Smoothness: \(|\delta_{t+1} - 2\delta_t + \delta_{t-1}| \le M\)
Relative magnitudes: \(|\delta_{t+1} - \delta_t| \le \overline{M} \cdot \max_{s<0} |\delta_{s+1} - \delta_s|\)

This is more restrictive than either constraint alone.

Parameters:

M (float) – Maximum allowed second difference (smoothness).
Mbar (float) – Scaling factor for maximum pre-period first difference (relative magnitudes).

Examples

>>> delta = DeltaSDRM(M=0.5, Mbar=1.0)

M: float = 0.0

Mbar: float = 1.0

__init__(M=0.0, Mbar=1.0)

Parameters:

M (float)
Mbar (float)

Return type:

None

Convenience Functions#

compute_honest_did#

Quick computation of honest bounds.

diff_diff.compute_honest_did(results, method='relative_magnitude', M=1.0, alpha=0.05, l_vec=None)[source]#

Convenience function for computing Honest DiD bounds.

Parameters:

results (MultiPeriodDiDResults, CallawaySantAnnaResults, or ChaisemartinDHaultfoeuilleResults) – Results from event study estimation.
method (str) – Type of restriction (“smoothness”, “relative_magnitude”, “combined”).
M (float) – Restriction parameter.
alpha (float) – Significance level.
l_vec (np.ndarray, optional) – Weight vector defining the scalar target theta = l_vec' tau over post-treatment horizons. Length must equal the number of post-treatment periods. None (default) uses equal weights (uniform average). To target the on-impact effect only (R’s default), pass np.array([1, 0, ..., 0]).

Returns:

Bounds and robust confidence intervals.

Return type:

HonestDiDResults

Examples

>>> bounds = compute_honest_did(event_study_results, method='relative_magnitude', M=1.0)
>>> print(f"Robust CI: [{bounds.ci_lb:.3f}, {bounds.ci_ub:.3f}]")

sensitivity_plot#

Convenience function for sensitivity visualization.

diff_diff.sensitivity_plot(results, method='relative_magnitude', M_grid=None, alpha=0.05, ax=None, **kwargs)[source]#

Create a sensitivity analysis plot.

Parameters:

results (MultiPeriodDiDResults, CallawaySantAnnaResults, or ChaisemartinDHaultfoeuilleResults) – Results from event study estimation.
method (str) – Type of restriction.
M_grid (list of float, optional) – Grid of M values.
alpha (float) – Significance level.
ax (matplotlib.axes.Axes, optional) – Axes to plot on.
**kwargs – Additional arguments passed to plot method.

Returns:

ax – The axes with the plot.

Return type:

matplotlib.axes.Axes

Complete Example#

import numpy as np
from diff_diff import (
    MultiPeriodDiD,
    HonestDiD,
    plot_sensitivity,
    plot_honest_event_study,
)

# Fit event study
model = MultiPeriodDiD()
results = model.fit(data, outcome='y', treatment='treated',
                    time='period', unit='unit_id',
                    post_periods=[5, 6, 7], reference_period=4)

# Sensitivity analysis under relative magnitudes
honest_rm = HonestDiD(method='relative_magnitude', M=1.0)
sensitivity_rm = honest_rm.sensitivity_analysis(
    results,
    M_grid=np.linspace(0, 2, 21).tolist()
)

# Find breakdown value
breakdown = honest_rm.breakdown_value(results)
print(f"Breakdown M̄: {breakdown}")

# Plot sensitivity
ax1 = plot_sensitivity(sensitivity_rm)
ax1.figure.savefig('sensitivity_rm.png')

# Event study with honest CIs
bounds = honest_rm.fit(results)
ax2 = plot_honest_event_study(bounds)
ax2.figure.savefig('honest_event_study.png')

References#

Rambachan, A., & Roth, J. (2023). A More Credible Approach to Parallel Trends. Review of Economic Studies, 90(5), 2555-2591.
R package: HonestDiD