Changes-in-Changes (CiC) and Quantile DiD#

Distributional difference-in-differences for the canonical 2x2 design (two groups, two periods) with continuous outcomes, from Athey & Imbens (2006). Where standard DiD delivers a mean effect, these estimators recover the entire counterfactual outcome distribution of the treated group, yielding quantile treatment effects on the treated alongside the ATT.

ChangesInChanges (alias CiC) builds the counterfactual distribution F_10(F_00^{-1}(F_01(y))): each treated pre-period outcome is ranked in the control pre-period distribution and pushed through the control post-period quantile function. The model is invariant to monotone transformations of the outcome (levels vs logs give consistent answers - a property the estimator inherits exactly in unconditional fits; the covariate branch’s linear quantile regressions are not equivariant to nonlinear monotone transforms) and places no testable restrictions on continuous data.

QDiD is the quantile-by-quantile DiD comparison estimator the same paper formalizes: it adds the control group’s over-time quantile change to the treated pre-period quantile. Athey & Imbens recommend CiC over QDiD - QDiD’s justifying model is not scale-invariant and imposes testable restrictions (in unconditional fits, a warning fires when they appear violated).

Note

Point estimation matches the R qte package (v1.3.1) exactly - CiC via R type-1 (ceiling-order-statistic) quantiles, the paper’s empirical-inverse convention, and QDiD via qte’s additive type-7 form (see the labeled Note in docs/methodology/REGISTRY.md on its finite-sample deviation from the paper’s transformation form). Inference is bootstrap-only in this release (n_bootstrap=200 default, seedable): panel mode resamples units with both periods together, repeated cross-section mode draws a pooled row resample; SEs are replicate SDs with symmetric normal-approximation intervals, plus qte’s sup-t critical value for uniform bands at a fixed 95% level (independent of alpha).

Covariates are supported via qte’s xformla-parity route (covariates=[...] or trailing formula terms): per-cell linear quantile regressions on qte’s fixed internal 0.01-0.99 tau grid impute each treated pre-period observation’s conditional counterfactual (the Melly-Santangelo 2015 pipeline in qte’s simplified form). Covariates must be numeric (dummy-encode categoricals), and every bootstrap replicate refits the quantile regressions - a covariate fit at the default n_bootstrap=200 solves ~40k small linear programs (typically tens of seconds; the same cost profile as qte::CiC). Discrete-outcome bounds, analytical standard errors, staggered designs, and the full Melly-Santangelo covariate estimator remain deferred - see docs/methodology/REGISTRY.md for the documented scope.

When to use ChangesInChanges:

  • Exactly two groups and two periods, continuous outcome, and you care about effect heterogeneity across the outcome distribution (which quantiles moved, not just the mean)

  • You want results invariant to monotone rescaling of the outcome (unconditional fits; the covariate branch’s linear quantile regressions are not equivariant to nonlinear monotone transforms)

  • CiC quantile effects are point-identified on the interior range where the treated pre-period distribution overlaps the control pre-period support; effects outside it keep their point estimates but report NaN inference (unconditional fits only - with covariates a conditional-envelope support diagnostic applies instead and q_lower/q_upper are NaN)

Reference: Athey, S., & Imbens, G. W. (2006). Identification and Inference in Nonlinear Difference-in-Differences Models. Econometrica, 74(2), 431-497.

ChangesInChanges#

Main changes-in-changes estimator class.

class diff_diff.ChangesInChanges[source]

Bases: object

Changes-in-Changes estimator (Athey & Imbens 2006) for the 2x2 design.

Estimates the ATT (eq. 36) and quantile treatment effects on the treated (eq. 18) by building the treated group’s counterfactual untreated outcome distribution F_10(F_00^{-1}(F_01(y))) (Theorem 3.1). Point estimates match qte::CiC() (R, v1.3.1) exactly; the empirical inverse CDF is the paper’s eq. (35)/(A.1) ceiling-order-statistic convention (R type-1).

Inference is bootstrap-only (n_bootstrap=0 disables it and reports NaN inference fields): panel mode resamples units with both periods together; repeated cross-section mode draws a pooled row resample. Standard errors are replicate SDs with symmetric normal-approximation intervals; results_.sup_t_crit carries qte’s sup-t critical value for uniform bands at a fixed 95% level (independent of alpha, matching qte).

Parameters:

  • quantiles (array-like, optional) – Quantile grid strictly inside (0, 1). Default: 0.05 to 0.95 in steps of 0.05 (matches qte’s probs).

  • n_bootstrap (int, default=200) – Bootstrap replicates. 0 disables inference (NaN se/t/p/CI).

  • alpha (float, default=0.05) – Significance level for pointwise confidence intervals.

  • panel (bool, default=False) – True when the same units are observed in both periods (requires unit= at fit time; enforces a balanced panel). Affects only the bootstrap resampling scheme - the point estimator uses the four marginal cell distributions either way.

  • seed (int, optional) – Seed for the bootstrap RNG (numpy.random.default_rng).

Notes

Quantile effects are point-identified only on the eq. (17) interior range (q_lower, q_upper); effects outside it keep their point estimates (qte parity) but report NaN inference with a warning. This guard applies to unconditional (no-covariate) fits only: with covariates the eq. (17) bounds are not the relevant objects (q_lower/q_upper are NaN) and a conditional support diagnostic replaces the unconditional one.

Covariates (covariates= at fit time, or trailing formula terms) port qte’s xformla branch exactly: linear quantile regressions of the outcome on the covariates within the control pre- and post-period cells on qte’s fixed internal 0.01-0.99 tau grid (99 points, not user-configurable), conditional-rank imputation per treated pre-period observation, and the same bootstrap schemes with every quantile regression refit inside each replicate. Covariates must be numeric (dummy-encode categoricals). Runtime note: a covariate fit solves roughly 2 x 99 x (1 + n_bootstrap) small linear programs (~40k at the default n_bootstrap=200) - typically tens of seconds at moderate cell sizes, the same cost profile as qte::CiC.

Additive random group-time shocks (random effects at the group x period level) BIAS the CiC estimator - unlike linear DiD, where they only complicate inference - and are not detectable in a 2x2 design (Athey & Imbens 2006, p. 476). With more than two groups/periods they are testable (Theorem 6.4), but that extension is deferred.

The full Melly-Santangelo (2015) covariate estimator (monotonized integrated-indicator CDFs, treated-post covariate integration, exchangeable bootstrap), discrete-outcome bounds, and analytical standard errors remain deferred and documented in docs/methodology/REGISTRY.md.

Methods

fit(data[, outcome, treatment, time, ...])

Fit the CiC estimator on a 2x2 dataset.

get_params([deep])

Return constructor hyperparameters (raw values, round-trips __init__).

set_params(**params)

Set hyperparameters transactionally (a failing call mutates nothing).
__init__(quantiles=None, n_bootstrap=200, alpha=0.05, panel=False, seed=None)[source]
Parameters:
get_params(deep=True)[source]

Return constructor hyperparameters (raw values, round-trips __init__).

deep is accepted for sklearn compatibility (sklearn.base.clone calls get_params(deep=False)) and is ignored - there are no nested estimators.

Parameters:

deep (bool)

Return type:

Dict[str, Any]

set_params(**params)[source]

Set hyperparameters transactionally (a failing call mutates nothing).

Parameters:

params (Any)

Return type:

ChangesInChanges

fit(data, outcome=None, treatment=None, time=None, formula=None, covariates=None, unit=None)[source]

Fit the CiC estimator on a 2x2 dataset.

Parameters:

  • data (pd.DataFrame) – Long-format data with one row per observation.

  • outcome (str, optional) – Column names: continuous outcome, binary group indicator (1 = treated group in BOTH periods), binary post-period indicator. Required unless formula is given.

  • treatment (str, optional) – Column names: continuous outcome, binary group indicator (1 = treated group in BOTH periods), binary post-period indicator. Required unless formula is given.

  • time (str, optional) – Column names: continuous outcome, binary group indicator (1 = treated group in BOTH periods), binary post-period indicator. Required unless formula is given.

  • formula (str, optional) – R-style 2x2 formula, e.g. "y ~ treated * post" or "y ~ treated * post + x1 + x2" (trailing terms are covariates). Mixing formula with any explicit column argument raises - deliberately stricter than DifferenceInDifferences, which silently lets the formula win.

  • covariates (list of str, optional) – Numeric covariate columns for the conditional (quantile- regression) CiC - qte’s xformla. Fit-time argument only (not a hyperparameter; absent from get_params(), like unit). Dummy-encode categorical covariates first. In panel mode, covariates may be time-varying: each (group, period) cell uses its own rows’ covariate values, exactly like qte.

  • unit (str, optional) – Unit identifier column. Required when panel=True; ignored (documented) when panel=False, matching qte’s idname.

Return type:

ChangesInChangesResults

QDiD#

Quantile difference-in-differences comparison estimator.

class diff_diff.QDiD[source]

Bases: object

Quantile Difference-in-Differences comparison estimator (2x2 design).

Applies DiD quantile-by-quantile: qte(tau) = Q(y11, tau) - [Q(y10, tau) + Q(y01, tau) - Q(y00, tau)] with R type-7 (linear-interpolation) quantiles, matching qte::QDiD() (v1.3.1) exactly - including its ATT formula, which evaluates the control-group quantile functions at the treated pre-period’s own-sample ranks. This finite-sample form deviates from the Athey-Imbens k^QDID transformation mean (they are population-equivalent; see the labeled Note in the methodology registry).

Athey & Imbens recommend ChangesInChanges over QDiD (2006, p. 447): QDiD’s justifying model is not invariant to monotone transformations of the outcome, forces the unobservable distribution to be identical across all four cells, and places testable restrictions on the data (in unconditional fits, a warning fires when the implied counterfactual quantile function is non-monotone; with covariates the check is moot - the imputed counterfactual’s quantile curve is monotone by construction). QDiD’s mean effect matches standard DiD’s ATT in population; the paper provides no asymptotic theory for QDiD, so inference is a bootstrap convention shared with the qte package.

Covariates port qte’s xformla branch exactly: quantile regressions in THREE cells (both control cells plus treated-pre), conditional ranks from the treated pre-period cell’s own conditional distribution, and an additive imputation. qte’s covariate QDiD mixes quantile types - type-7 for the treated post-period quantiles, type-1 for the imputed counterfactual - and that asymmetry is ported verbatim (REGISTRY Note).

Constructor parameters, fit signature, bootstrap behavior, and the results container are identical to ChangesInChanges (no interior-range guard: eq. 17 has no QDiD analogue).

Methods

fit(data[, outcome, treatment, time, ...])

Fit the QDiD estimator on a 2x2 dataset (see ChangesInChanges.fit).

get_params([deep])

Return constructor hyperparameters (raw values, round-trips __init__).

set_params(**params)

Set hyperparameters transactionally (a failing call mutates nothing).
__init__(quantiles=None, n_bootstrap=200, alpha=0.05, panel=False, seed=None)[source]
Parameters:
get_params(deep=True)[source]

Return constructor hyperparameters (raw values, round-trips __init__).

deep is accepted for sklearn compatibility (sklearn.base.clone calls get_params(deep=False)) and is ignored - there are no nested estimators.

Parameters:

deep (bool)

Return type:

Dict[str, Any]

set_params(**params)[source]

Set hyperparameters transactionally (a failing call mutates nothing).

Parameters:

params (Any)

Return type:

QDiD

fit(data, outcome=None, treatment=None, time=None, formula=None, covariates=None, unit=None)[source]

Fit the QDiD estimator on a 2x2 dataset (see ChangesInChanges.fit).

Parameters:
Return type:

ChangesInChangesResults

ChangesInChangesResults#

Results container shared by both estimators (the estimator field records which produced it; QDiDResults is an alias of this class).

class diff_diff.changes_in_changes_results.ChangesInChangesResults[source]

Bases: BaseResults

Results for ChangesInChanges and QDiD.

The headline att/se/t_stat/p_value/conf_int fields carry the mean effect; quantile_effects is a DataFrame with one row per requested quantile (columns quantile, qte, se, t_stat, p_value, conf_low, conf_high). All inference derives from the bootstrap (replicate-SD standard errors, symmetric normal-approximation intervals at level alpha); with n_bootstrap=0 every inference field is NaN.

q_lower/q_upper bound the point-identified interior quantile range for unconditional CiC fits (Athey-Imbens eq. 17; NaN for QDiD and for covariate fits, where the unconditional bounds are not the relevant objects). sup_t_crit is the qte package’s sup-t critical value for uniform bands - computed at a FIXED 95% level regardless of alpha (qte parity); see uniform_bands(). covariates records the covariate columns used by the conditional (quantile-regression) fit, or None for unconditional fits.

Methods

summary()

Fixed-width text summary: headline ATT block plus the quantile-effects table.

print_summary()

Print summary() to stdout.

to_dataframe([level])

Return the quantile-effects table or the one-row ATT summary.

to_dict()

Flat headline dictionary (ATT inference, ranges, sizes, bootstrap metadata).

uniform_bands()

Simultaneous (sup-t) confidence bands over the quantile grid.
att: float
se: float
t_stat: float
p_value: float
conf_int: Tuple[float, float]
quantile_effects: DataFrame
q_lower: float
q_upper: float
sup_t_crit: float
n_obs: int
cell_sizes: Dict[str, int]
n_bootstrap: int
n_bootstrap_valid: int
panel: bool
estimator: str
quantiles: ndarray
alpha: float = 0.05
covariates: List[str] | None = None
property is_significant: bool

Whether the headline ATT is significant at level alpha (False on NaN).

property significance_stars: str

Significance stars for the headline ATT p-value (’’ when NaN).

uniform_bands()[source]

Simultaneous (sup-t) confidence bands over the quantile grid.

qte +/- sup_t_crit * se per quantile, using the qte package’s IQR-scaled sup-t critical value at its hard-coded 95% level - the band level does NOT follow alpha (qte parity). Rows whose se is NaN (no bootstrap, failed replicate gate, or outside the interior range in an unconditional CiC fit) get NaN bands.

Return type:

DataFrame

to_dict()[source]

Flat headline dictionary (ATT inference, ranges, sizes, bootstrap metadata).

Return type:

Dict[str, Any]

to_dataframe(level='quantiles')[source]

Return the quantile-effects table or the one-row ATT summary.

Parameters:

level ({"quantiles", "att"}) – "quantiles" (default) returns a copy of quantile_effects; "att" returns a single-row frame with the headline fields.

Return type:

DataFrame

summary()[source]

Fixed-width text summary: headline ATT block plus the quantile-effects table.

Return type:

str

print_summary()[source]

Print summary() to stdout.

Return type:

None

__init__(att, se, t_stat, p_value, conf_int, quantile_effects, q_lower, q_upper, sup_t_crit, n_obs, cell_sizes, n_bootstrap, n_bootstrap_valid, panel, estimator, quantiles, alpha=0.05, covariates=None)
Parameters:
Return type:

None

Example Usage#

Estimate quantile treatment effects in a 2x2 design:

import numpy as np
import pandas as pd
from diff_diff import ChangesInChanges

rng = np.random.default_rng(0)
n = 400
treated = np.repeat([1, 0], n // 2)
u = rng.normal(0, 1, n)
y_pre = u + rng.normal(0, 0.3, n)
y_post = u + 0.5 + rng.normal(0, 0.3, n) + treated * (0.8 + 0.4 * (u > 0))
data = pd.DataFrame({
    "unit": np.tile(np.arange(n), 2),
    "post": np.repeat([0, 1], n),
    "treated": np.tile(treated, 2),
    "y": np.concatenate([y_pre, y_post]),
})

cic = ChangesInChanges(n_bootstrap=200, seed=42)
results = cic.fit(data, outcome="y", treatment="treated", time="post")
results.print_summary()

# Quantile effects table and simultaneous bands
qte = results.to_dataframe("quantiles")
bands = results.uniform_bands()

Panel mode (same units in both periods) changes only the bootstrap:

import numpy as np
import pandas as pd
from diff_diff import ChangesInChanges

rng = np.random.default_rng(0)
n = 400
treated = np.repeat([1, 0], n // 2)
u = rng.normal(0, 1, n)
data = pd.DataFrame({
    "unit": np.tile(np.arange(n), 2),
    "post": np.repeat([0, 1], n),
    "treated": np.tile(treated, 2),
    "y": np.concatenate(
        [u + rng.normal(0, 0.3, n), u + 0.5 + rng.normal(0, 0.3, n) + treated]
    ),
})

cic_panel = ChangesInChanges(n_bootstrap=200, seed=42, panel=True)
results_panel = cic_panel.fit(
    data, outcome="y", treatment="treated", time="post", unit="unit"
)

QDiD as a comparison estimator:

from diff_diff import QDiD

qdid = QDiD(n_bootstrap=200, seed=42)
results_qdid = qdid.fit(data, outcome="y", treatment="treated", time="post")

Covariates (conditional CiC via per-cell quantile regression, matching qte’s xformla; a small n_bootstrap keeps the example fast - every replicate refits the quantile regressions):

import numpy as np
import pandas as pd
from diff_diff import ChangesInChanges

rng = np.random.default_rng(1)
n = 120
treated = np.repeat([1, 0], n // 2)
x1 = rng.uniform(0, 2, n) + 0.3 * treated
y_pre = 0.5 + 0.8 * x1 + rng.normal(0, 0.4, n)
y_post = 0.8 + 1.1 * x1 + rng.normal(0, 0.4, n) + treated * 0.7
data = pd.DataFrame({
    "post": np.repeat([0, 1], n),
    "treated": np.tile(treated, 2),
    "x1": np.tile(x1, 2),
    "y": np.concatenate([y_pre, y_post]),
})

cic_cov = ChangesInChanges(n_bootstrap=10, seed=42)
results_cov = cic_cov.fit(
    data, outcome="y", treatment="treated", time="post", covariates=["x1"]
)
# Equivalent formula route: "y ~ treated * post + x1"
print(results_cov.covariates)  # ['x1']