Honest DiD

Sensitivity analysis for violations of parallel trends (Rambachan & Roth 2023).

Overview

The Honest DiD approach provides inference that is robust to violations of the parallel trends assumption. Instead of assuming parallel trends holds exactly, it bounds the violation and provides confidence intervals that account for potential deviations.

Two main restriction types are supported:

Relative Magnitudes (ΔRM): Post-treatment violations are bounded by M̄ times the maximum pre-treatment violation.
Smoothness (ΔSD): Bounds on the second differences of the trend violations, restricting how much the violation can change between periods.

HonestDiD

Main class for computing honest bounds and confidence intervals.

class diff_diff.HonestDiD[source]

Bases: object

Honest DiD sensitivity analysis (Rambachan & Roth 2023).

Computes robust inference for difference-in-differences allowing for bounded violations of parallel trends.

Parameters:

method ({"smoothness", "relative_magnitude", "combined"}) – Type of restriction on trend violations: - “smoothness”: Bounds on second differences (Delta^SD) - “relative_magnitude”: Post violations <= M * max pre violation (Delta^RM) - “combined”: Both restrictions (Delta^SDRM)
M (float, optional) – Restriction parameter. Interpretation depends on method: - smoothness: Max second difference - relative_magnitude: Scaling factor for max pre-period violation Default is 1.0 for relative_magnitude, 0.0 for smoothness.
alpha (float) – Significance level for confidence intervals.
l_vec (array-like or None) – Weighting vector for scalar parameter (length = num_post_periods). If None, uses uniform weights (average effect).

Examples

>>> from diff_diff import MultiPeriodDiD
>>> from diff_diff.honest_did import HonestDiD
>>>
>>> # Fit event study
>>> mp_did = MultiPeriodDiD()
>>> results = mp_did.fit(data, outcome='y', treatment='treated',
...                      time='period', post_periods=[4,5,6,7])
>>>
>>> # Sensitivity analysis with relative magnitudes
>>> honest = HonestDiD(method='relative_magnitude', M=1.0)
>>> bounds = honest.fit(results)
>>> print(bounds.summary())
>>>
>>> # Sensitivity curve over M values
>>> sensitivity = honest.sensitivity_analysis(results, M_grid=[0, 0.5, 1, 1.5, 2])
>>> sensitivity.plot()

Methods

`fit`(results[, M])	Compute bounds and robust confidence intervals.
`sensitivity_analysis`(results[, M_grid])	Perform sensitivity analysis over a grid of M values.
`breakdown_value`(results[, tol])	Find the breakdown value directly using binary search.

__init__(method='relative_magnitude', M=None, alpha=0.05, l_vec=None)[source]

Parameters:

method (Literal['smoothness', 'relative_magnitude', 'combined'])
M (float | None)
alpha (float)
l_vec (ndarray | None)

get_params()[source]

Get parameters for this estimator.

Return type:: Dict[str, Any]

set_params(**params)[source]

Set parameters for this estimator.

Return type:: HonestDiD

fit(results, M=None)[source]

Compute bounds and robust confidence intervals.

Parameters:

results (MultiPeriodDiDResults or CallawaySantAnnaResults) – Results from event study estimation.
M (float, optional) – Override the M parameter for this fit.

Returns:

Results containing bounds and robust confidence intervals.

Return type:

HonestDiDResults

sensitivity_analysis(results, M_grid=None)[source]

Perform sensitivity analysis over a grid of M values.

Parameters:

results (MultiPeriodDiDResults or CallawaySantAnnaResults) – Results from event study estimation.
M_grid (list of float, optional) – Grid of M values to evaluate. If None, uses default grid based on method.

Returns:

Results containing bounds and CIs for each M value.

Return type:

SensitivityResults

breakdown_value(results, tol=0.01)[source]

Find the breakdown value directly using binary search.

The breakdown value is the smallest M where the robust confidence interval includes zero.

Parameters:

results (MultiPeriodDiDResults or CallawaySantAnnaResults) – Results from event study estimation.
tol (float) – Tolerance for binary search.

Returns:

Breakdown value, or None if effect is always significant.

Return type:

float or None

Example

from diff_diff import MultiPeriodDiD, HonestDiD, DeltaRM

# First fit an event study
model = MultiPeriodDiD(reference_period=-1)
results = model.fit(data, outcome='y', treated='treated',
                    time='period', unit='unit_id', treatment_start=5)

# Compute bounds under relative magnitudes restriction
honest = HonestDiD(delta=DeltaRM(M_bar=1.0))
bounds = honest.fit(results)

print(f"Original CI: [{results.att - 1.96*results.se:.3f}, "
      f"{results.att + 1.96*results.se:.3f}]")
print(f"Robust CI: [{bounds.robust_ci[0]:.3f}, {bounds.robust_ci[1]:.3f}]")

HonestDiDResults

Results from HonestDiD estimation.

class diff_diff.HonestDiDResults[source]

Bases: object

Results from Honest DiD sensitivity analysis.

Contains bounds on the treatment effect under the specified restrictions on violations of parallel trends.

lb

Lower bound of identified set.

Type:: float

ub

Upper bound of identified set.

Type:: float

ci_lb

Lower bound of robust confidence interval.

Type:: float

ci_ub

Upper bound of robust confidence interval.

Type:: float

M

The restriction parameter value used.

Type:: float

method

The type of restriction (“smoothness”, “relative_magnitude”, or “combined”).

Type:: str

original_estimate

The original point estimate (under parallel trends).

Type:: float

original_se

The original standard error.

Type:: float

alpha

Significance level for confidence interval.

Type:: float

ci_method

Method used for CI construction (“FLCI” or “C-LF”).

Type:: str

original_results

The original estimation results object.

Type:: Any

lb: float

ub: float

ci_lb: float

ci_ub: float

M: float

method: str

original_estimate: float

original_se: float

alpha: float = 0.05

ci_method: str = 'FLCI'

original_results: Any | None = None

event_study_bounds: Dict[Any, Dict[str, float]] | None = None

property is_significant: bool: Check if CI excludes zero (effect is robust to violations).

property significance_stars: str

Return significance indicator if robust CI excludes zero.

Note: Unlike point estimation, partial identification does not yield a single p-value. This returns “*” if the robust CI excludes zero at the specified alpha level, indicating the effect is robust to the assumed violations of parallel trends.

property identified_set_width: float: Width of the identified set.

property ci_width: float: Width of the confidence interval.

summary()[source]

Generate formatted summary of sensitivity analysis results.

Returns:: Formatted summary.
Return type:: str

print_summary()[source]

Print summary to stdout.

Return type:: None

to_dict()[source]

Convert results to dictionary.

Return type:: Dict[str, Any]

to_dataframe()[source]

Convert results to DataFrame.

Return type:: DataFrame

__init__(lb, ub, ci_lb, ci_ub, M, method, original_estimate, original_se, alpha=0.05, ci_method='FLCI', original_results=None, event_study_bounds=None)

Parameters:

lb (float)
ub (float)
ci_lb (float)
ci_ub (float)
M (float)
method (str)
original_estimate (float)
original_se (float)
alpha (float)
ci_method (str)
original_results (Any | None)
event_study_bounds (Dict[Any, Dict[str, float]] | None)

Return type:

None

SensitivityResults

Results from sensitivity analysis over a grid of M values.

class diff_diff.SensitivityResults[source]

Bases: object

Results from sensitivity analysis over a grid of M values.

Contains bounds and confidence intervals for each M value, plus the breakdown value.

M_values

Grid of M parameter values.

Type:: np.ndarray

bounds

List of (lb, ub) identified set bounds for each M.

Type:: List[Tuple[float, float]]

robust_cis

List of (ci_lb, ci_ub) robust CIs for each M.

Type:: List[Tuple[float, float]]

breakdown_M

Smallest M where robust CI includes zero.

Type:: float

method

Type of restriction used.

Type:: str

original_estimate

Original point estimate.

Type:: float

original_se

Original standard error.

Type:: float

alpha

Significance level.

Type:: float

M_values: ndarray

bounds: List[Tuple[float, float]]

robust_cis: List[Tuple[float, float]]

breakdown_M: float | None

method: str

original_estimate: float

original_se: float

alpha: float = 0.05

property has_breakdown: bool: Check if there is a finite breakdown value.

summary()[source]

Generate formatted summary.

Return type:: str

print_summary()[source]

Print summary to stdout.

Return type:: None

to_dataframe()[source]

Convert to DataFrame with one row per M value.

Return type:: DataFrame

plot(ax=None, show_bounds=True, show_ci=True, breakdown_line=True, **kwargs)[source]

Plot sensitivity analysis results.

Parameters:

ax (matplotlib.axes.Axes, optional) – Axes to plot on. If None, creates new figure.
show_bounds (bool) – Whether to show identified set bounds.
show_ci (bool) – Whether to show confidence intervals.
breakdown_line (bool) – Whether to show vertical line at breakdown value.
**kwargs – Additional arguments passed to plotting functions.

Returns:

ax – The axes with the plot.

Return type:

matplotlib.axes.Axes

__init__(M_values, bounds, robust_cis, breakdown_M, method, original_estimate, original_se, alpha=0.05)

Parameters:

M_values (ndarray)
bounds (List[Tuple[float, float]])
robust_cis (List[Tuple[float, float]])
breakdown_M (float | None)
method (str)
original_estimate (float)
original_se (float)
alpha (float)

Return type:

None

Restriction Classes

DeltaSD

Smoothness restriction class.

class diff_diff.DeltaSD[source]

Bases: object

Smoothness restriction on trend violations (Delta^{SD}).

Restricts the second differences of the trend violations:: |delta_{t+1} - 2*delta_t + delta_{t-1}| <= M

When M=0, this enforces that violations follow a linear trend (linear extrapolation of pre-trends). Larger M allows more curvature in the violation path.

Parameters:: M (float) – Maximum allowed second difference. M=0 means linear trends only.

Examples

>>> delta = DeltaSD(M=0.5)
>>> delta.M
0.5

M: float = 0.0

__init__(M=0.0)

Parameters:: M (float)
Return type:: None

DeltaRM

Relative magnitudes restriction class.

class diff_diff.DeltaRM[source]

Bases: object

Relative magnitudes restriction on trend violations (Delta^{RM}).

Post-treatment violations are bounded by Mbar times the maximum absolute pre-treatment violation:

|delta_post| <= Mbar * max(|delta_pre|)

When Mbar=0, this enforces exact parallel trends post-treatment. Mbar=1 means post-period violations can be as large as the worst observed pre-period violation.

Parameters:: Mbar (float) – Scaling factor for maximum pre-period violation.

Examples

>>> delta = DeltaRM(Mbar=1.0)
>>> delta.Mbar
1.0

Mbar: float = 1.0

__init__(Mbar=1.0)

Parameters:: Mbar (float)
Return type:: None

DeltaSDRM

Combined smoothness and relative magnitudes restriction.

class diff_diff.DeltaSDRM[source]

Bases: object

Combined smoothness and relative magnitudes restriction.

Imposes both: 1. Smoothness: |delta_{t+1} - 2*delta_t + delta_{t-1}| <= M 2. Relative magnitudes: |delta_post| <= Mbar * max(|delta_pre|)

This is more restrictive than either constraint alone.

Parameters:

M (float) – Maximum allowed second difference (smoothness).
Mbar (float) – Scaling factor for maximum pre-period violation (relative magnitudes).

Examples

>>> delta = DeltaSDRM(M=0.5, Mbar=1.0)

M: float = 0.0

Mbar: float = 1.0

__init__(M=0.0, Mbar=1.0)

Parameters:

M (float)
Mbar (float)

Return type:

None

Convenience Functions

compute_honest_did

Quick computation of honest bounds.

diff_diff.compute_honest_did(results, method='relative_magnitude', M=1.0, alpha=0.05)[source]

Convenience function for computing Honest DiD bounds.

Parameters:

results (MultiPeriodDiDResults or CallawaySantAnnaResults) – Results from event study estimation.
method (str) – Type of restriction (“smoothness”, “relative_magnitude”, “combined”).
M (float) – Restriction parameter.
alpha (float) – Significance level.

Returns:

Bounds and robust confidence intervals.

Return type:

HonestDiDResults

Examples

>>> bounds = compute_honest_did(event_study_results, method='relative_magnitude', M=1.0)
>>> print(f"Robust CI: [{bounds.ci_lb:.3f}, {bounds.ci_ub:.3f}]")

sensitivity_plot

Convenience function for sensitivity visualization.

diff_diff.sensitivity_plot(results, method='relative_magnitude', M_grid=None, alpha=0.05, ax=None, **kwargs)[source]

Create a sensitivity analysis plot.

Parameters:

results (MultiPeriodDiDResults or CallawaySantAnnaResults) – Results from event study estimation.
method (str) – Type of restriction.
M_grid (list of float, optional) – Grid of M values.
alpha (float) – Significance level.
ax (matplotlib.axes.Axes, optional) – Axes to plot on.
**kwargs – Additional arguments passed to plot method.

Returns:

ax – The axes with the plot.

Return type:

matplotlib.axes.Axes

Complete Example

import numpy as np
from diff_diff import (
    MultiPeriodDiD,
    HonestDiD,
    DeltaRM,
    DeltaSD,
    plot_sensitivity,
    plot_honest_event_study,
)

# Fit event study
model = MultiPeriodDiD(reference_period=-1)
results = model.fit(data, outcome='y', treated='treated',
                    time='period', unit='unit_id', treatment_start=5)

# Sensitivity analysis under relative magnitudes
honest_rm = HonestDiD(delta=DeltaRM(M_bar=1.0))
sensitivity_rm = honest_rm.sensitivity_analysis(
    results,
    M_grid=np.linspace(0, 2, 21)
)

# Find breakdown value
breakdown = honest_rm.breakdown_value(results)
print(f"Breakdown M̄: {breakdown:.3f}")

# Plot sensitivity
fig1 = plot_sensitivity(sensitivity_rm)
fig1.savefig('sensitivity_rm.png')

# Event study with honest CIs
bounds = honest_rm.fit(results)
fig2 = plot_honest_event_study(results, bounds)
fig2.savefig('honest_event_study.png')

References

Rambachan, A., & Roth, J. (2023). A More Credible Approach to Parallel Trends. Review of Economic Studies, 90(5), 2555-2591.
R package: HonestDiD