Changes-in-Changes (CiC) and Quantile DiD#

Distributional difference-in-differences for the canonical 2x2 design (two groups, two periods) with continuous outcomes, from Athey & Imbens (2006). Where standard DiD delivers a mean effect, these estimators recover the entire counterfactual outcome distribution of the treated group, yielding quantile treatment effects on the treated alongside the ATT.

ChangesInChanges (alias CiC) builds the counterfactual distribution F_10(F_00^{-1}(F_01(y))): each treated pre-period outcome is ranked in the control pre-period distribution and pushed through the control post-period quantile function. The model is invariant to monotone transformations of the outcome (levels vs logs give consistent answers - a property the estimator inherits exactly in unconditional fits; the covariate branch’s linear quantile regressions are not equivariant to nonlinear monotone transforms) and places no testable restrictions on continuous data.

QDiD is the quantile-by-quantile DiD comparison estimator the same paper formalizes: it adds the control group’s over-time quantile change to the treated pre-period quantile. Athey & Imbens recommend CiC over QDiD - QDiD’s justifying model is not scale-invariant and imposes testable restrictions (in unconditional fits, a warning fires when they appear violated).

Note

Point estimation matches the R qte package (v1.3.1) exactly - CiC via R type-1 (ceiling-order-statistic) quantiles, the paper’s empirical-inverse convention, and QDiD via qte’s additive type-7 form (see the labeled Note in docs/methodology/REGISTRY.md on its finite-sample deviation from the paper’s transformation form). Inference is bootstrap-only in this release (n_bootstrap=200 default, seedable): panel mode resamples units with both periods together, repeated cross-section mode draws a pooled row resample; SEs are replicate SDs with symmetric normal-approximation intervals, plus qte’s sup-t critical value for uniform bands at a fixed 95% level (independent of alpha).

Covariates are supported via qte’s xformla-parity route (covariates=[...] or trailing formula terms): per-cell linear quantile regressions on qte’s fixed internal 0.01-0.99 tau grid impute each treated pre-period observation’s conditional counterfactual (the Melly-Santangelo 2015 pipeline in qte’s simplified form). Covariates must be numeric (dummy-encode categoricals), and every bootstrap replicate refits the quantile regressions - a covariate fit at the default n_bootstrap=200 solves ~40k small linear programs (typically tens of seconds; the same cost profile as qte::CiC). Discrete-outcome bounds, analytical standard errors, staggered designs, and the full Melly-Santangelo covariate estimator remain deferred - see docs/methodology/REGISTRY.md for the documented scope.

When to use ChangesInChanges:

  • Exactly two groups and two periods, continuous outcome, and you care about effect heterogeneity across the outcome distribution (which quantiles moved, not just the mean)

  • You want results invariant to monotone rescaling of the outcome (unconditional fits; the covariate branch’s linear quantile regressions are not equivariant to nonlinear monotone transforms)

  • CiC quantile effects are point-identified on the interior range where the treated pre-period distribution overlaps the control pre-period support; effects outside it keep their point estimates but report NaN inference (unconditional fits only - with covariates a conditional-envelope support diagnostic applies instead and q_lower/q_upper are NaN)

Reference: Athey, S., & Imbens, G. W. (2006). Identification and Inference in Nonlinear Difference-in-Differences Models. Econometrica, 74(2), 431-497.

ChangesInChanges#

Main changes-in-changes estimator class.

class diff_diff.ChangesInChanges[source]

Bases: BaseEstimator

Changes-in-Changes estimator (Athey & Imbens 2006) for the 2x2 design.

Estimates the ATT (eq. 36) and quantile treatment effects on the treated (eq. 18) by building the treated group’s counterfactual untreated outcome distribution F_10(F_00^{-1}(F_01(y))) (Theorem 3.1). Point estimates match qte::CiC() (R, v1.3.1) exactly; the empirical inverse CDF is the paper’s eq. (35)/(A.1) ceiling-order-statistic convention (R type-1).

Inference is bootstrap-only (n_bootstrap=0 disables it and reports NaN inference fields): panel mode resamples units with both periods together; repeated cross-section mode draws a pooled row resample. Standard errors are replicate SDs with symmetric normal-approximation intervals; results_.sup_t_crit carries qte’s sup-t critical value for uniform bands at a fixed 95% level (independent of alpha, matching qte).

Parameters:

  • quantiles (array-like, optional) – Quantile grid strictly inside (0, 1). Default: 0.05 to 0.95 in steps of 0.05 (matches qte’s probs).

  • n_bootstrap (int, default=200) – Bootstrap replicates. 0 disables inference (NaN se/t/p/CI).

  • alpha (float, default=0.05) – Significance level for pointwise confidence intervals.

  • panel (bool, default=False) – True when the same units are observed in both periods (requires unit= at fit time; enforces a balanced panel). Affects only the bootstrap resampling scheme - the point estimator uses the four marginal cell distributions either way.

  • seed (int, optional) – Seed for the bootstrap RNG (numpy.random.default_rng).

  • method (str, default="cic") – Which 2x2 distributional estimator to fit. "cic" is Changes-in-Changes (Athey & Imbens 2006); "qdid" is the quantile-DiD comparison estimator, matching qte::QDiD(). The default encodes Athey & Imbens’ recommendation of CiC over QDiD (2006, p. 447), but "qdid" is a fully supported mode and selecting it emits no warning. Every other constructor parameter, the fit signature, the bootstrap machinery and the results container are shared; the value is echoed back as results.method.

Notes

CiC only (``method=”cic”``). Quantile effects are point-identified only on the eq. (17) interior range (q_lower, q_upper); effects outside it keep their point estimates (qte parity) but report NaN inference with a warning. This guard applies to unconditional (no-covariate) fits only: with covariates the eq. (17) bounds are not the relevant objects (q_lower/q_upper are NaN) and a conditional support diagnostic replaces the unconditional one. QDiD has no eq. (17) analogue, so q_lower/q_upper are NaN for every method="qdid" fit; its own diagnostic is a non-monotonicity warning on the implied counterfactual quantile function (footnote 21), unconditional fits only.

Covariates (covariates= at fit time, or trailing formula terms) port qte’s xformla branch exactly: linear quantile regressions of the outcome on the covariates, on qte’s fixed internal 0.01-0.99 tau grid (99 points, not user-configurable), conditional-rank imputation per treated pre-period observation, and the same bootstrap schemes with every quantile regression refit inside each replicate. The cells regressed differ by method: CiC fits the two CONTROL cells, QDiD fits three (both control cells plus treated-pre, with the type-7/type-1 quantile asymmetry qte itself carries). Covariates must be numeric (dummy-encode categoricals). Runtime note: a covariate fit solves roughly k x 99 x (1 + n_bootstrap) small linear programs for k in {2, 3} (~40-60k at the default n_bootstrap=200) - typically tens of seconds at moderate cell sizes, the same cost profile as qte::CiC / qte::QDiD.

Additive random group-time shocks (random effects at the group x period level) BIAS the CiC estimator - unlike linear DiD, where they only complicate inference - and are not detectable in a 2x2 design (Athey & Imbens 2006, p. 476). With more than two groups/periods they are testable (Theorem 6.4), but that extension is deferred.

The full Melly-Santangelo (2015) covariate estimator (monotonized integrated-indicator CDFs, treated-post covariate integration, exchangeable bootstrap), discrete-outcome bounds, and analytical standard errors remain deferred and documented in docs/methodology/REGISTRY.md.

Methods

fit(data[, outcome, treatment, time, ...])

Fit the selected estimator (method=) on a 2x2 dataset.

get_params([deep])

Get estimator parameters (sklearn-compatible).

set_params(**params)

Set estimator parameters (sklearn-compatible, transactional).
__init__(quantiles=None, n_bootstrap=200, alpha=0.05, panel=False, seed=None, *, method='cic')[source]
Parameters:
fit(data, outcome=None, treatment=None, time=None, formula=None, covariates=None, unit=None)[source]

Fit the selected estimator (method=) on a 2x2 dataset.

Parameters:

  • data (pd.DataFrame) – Long-format data with one row per observation.

  • outcome (str, optional) – Column names: continuous outcome, binary group indicator (1 = treated group in BOTH periods), binary post-period indicator. Required unless formula is given.

  • treatment (str, optional) – Column names: continuous outcome, binary group indicator (1 = treated group in BOTH periods), binary post-period indicator. Required unless formula is given.

  • time (str, optional) – Column names: continuous outcome, binary group indicator (1 = treated group in BOTH periods), binary post-period indicator. Required unless formula is given.

  • formula (str, optional) – R-style 2x2 formula, e.g. "y ~ treated * post" or "y ~ treated * post + x1 + x2" (trailing terms are covariates). Mixing formula with any explicit column argument raises - deliberately stricter than DifferenceInDifferences, which silently lets the formula win.

  • covariates (list of str, optional) – Numeric covariate columns selecting the conditional (quantile-regression) branch of whichever estimator method names - qte’s xformla, supported for BOTH "cic" and "qdid". The cells regressed differ: CiC fits the two control cells, QDiD fits three (both control cells plus treated-pre). Fit-time argument only (not a hyperparameter; absent from get_params(), like unit). Dummy-encode categorical covariates first. In panel mode, covariates may be time-varying: each (group, period) cell uses its own rows’ covariate values, exactly like qte.

  • unit (str, optional) – Unit identifier column. Required when panel=True; ignored (documented) when panel=False, matching qte’s idname.

Return type:

ChangesInChangesResults

get_params(deep=True)

Get estimator parameters (sklearn-compatible).

Parameters:

deep (bool, default True) – Accepted for sklearn compatibility (sklearn.base.clone calls get_params(deep=False)) and ignored: no diff-diff estimator nests another estimator.

Returns:

Estimator parameters suitable for passing to __init__. Keys follow the __init__ signature order. Where a class stores a parameter’s raw value under a different attribute (_PARAM_ATTR_ALIASES), the raw value is returned so that type(est)(**est.get_params()) reconstructs the same configuration.

Return type:

Dict[str, Any]

set_params(**params)

Set estimator parameters (sklearn-compatible, transactional).

Validates the merged configuration by constructing a throwaway instance of type(self) - so a rejected call raises the same error __init__ would and leaves this estimator unchanged - then adopts the probe’s parameter attributes and the class’s declared derived-config attributes. Fitted state is never touched.

Parameters:

  • **params (Any) – Estimator parameters. Unknown names raise ValueError before any validation or mutation.

  • self (TSelf)

  • **params

Return type:

self

QDiD#

Quantile difference-in-differences comparison estimator.

Deprecated in 3.9, removed in 4.0 - use ChangesInChanges(method="qdid") instead. The same engine runs either way, so the numbers are unchanged; only the class spelling is deprecated, not the estimator.

class diff_diff.QDiD[source]

Bases: BaseEstimator

Quantile Difference-in-Differences comparison estimator (2x2 design).

Deprecated since version 3.9: Use ChangesInChanges(method="qdid") instead; this class is removed in 4.0 (ledger row M-015). The ESTIMATOR is not deprecated - only this class spelling. The merged surface runs the same engine, so the numbers are unchanged, and method="qdid" emits no warning of its own.

Applies DiD quantile-by-quantile: qte(tau) = Q(y11, tau) - [Q(y10, tau) + Q(y01, tau) - Q(y00, tau)] with R type-7 (linear-interpolation) quantiles, matching qte::QDiD() (v1.3.1) exactly - including its ATT formula, which evaluates the control-group quantile functions at the treated pre-period’s own-sample ranks. This finite-sample form deviates from the Athey-Imbens k^QDID transformation mean (they are population-equivalent; see the labeled Note in the methodology registry).

Athey & Imbens recommend ChangesInChanges over QDiD (2006, p. 447): QDiD’s justifying model is not invariant to monotone transformations of the outcome, forces the unobservable distribution to be identical across all four cells, and places testable restrictions on the data (in unconditional fits, a warning fires when the implied counterfactual quantile function is non-monotone; with covariates the check is moot - the imputed counterfactual’s quantile curve is monotone by construction). QDiD’s mean effect matches standard DiD’s ATT in population; the paper provides no asymptotic theory for QDiD, so inference is a bootstrap convention shared with the qte package.

Covariates port qte’s xformla branch exactly: quantile regressions in THREE cells (both control cells plus treated-pre), conditional ranks from the treated pre-period cell’s own conditional distribution, and an additive imputation. qte’s covariate QDiD mixes quantile types - type-7 for the treated post-period quantiles, type-1 for the imputed counterfactual - and that asymmetry is ported verbatim (REGISTRY Note).

Fit signature, bootstrap behavior, and the results container are identical to ChangesInChanges (no interior-range guard: eq. 17 has no QDiD analogue). The constructor parameters are identical EXCEPT that the merged class additionally carries method=, which is what selects this estimator there; this class deliberately keeps its frozen five-parameter signature through removal, so QDiD(method=...) is not accepted.

Methods

fit(data[, outcome, treatment, time, ...])

Fit the QDiD estimator on a 2x2 dataset (see ChangesInChanges.fit).

get_params([deep])

Get estimator parameters (sklearn-compatible).

set_params(**params)

Set estimator parameters (sklearn-compatible, transactional).
__init__(quantiles=None, n_bootstrap=200, alpha=0.05, panel=False, seed=None)[source]
Parameters:
fit(data, outcome=None, treatment=None, time=None, formula=None, covariates=None, unit=None)[source]

Fit the QDiD estimator on a 2x2 dataset (see ChangesInChanges.fit).

Parameters:
Return type:

ChangesInChangesResults

get_params(deep=True)

Get estimator parameters (sklearn-compatible).

Parameters:

deep (bool, default True) – Accepted for sklearn compatibility (sklearn.base.clone calls get_params(deep=False)) and ignored: no diff-diff estimator nests another estimator.

Returns:

Estimator parameters suitable for passing to __init__. Keys follow the __init__ signature order. Where a class stores a parameter’s raw value under a different attribute (_PARAM_ATTR_ALIASES), the raw value is returned so that type(est)(**est.get_params()) reconstructs the same configuration.

Return type:

Dict[str, Any]

set_params(**params)

Set estimator parameters (sklearn-compatible, transactional).

Validates the merged configuration by constructing a throwaway instance of type(self) - so a rejected call raises the same error __init__ would and leaves this estimator unchanged - then adopts the probe’s parameter attributes and the class’s declared derived-config attributes. Fitted state is never touched.

Parameters:

  • **params (Any) – Estimator parameters. Unknown names raise ValueError before any validation or mutation.

  • self (TSelf)

  • **params

Return type:

self

ChangesInChangesResults#

Results container shared by both estimators (the method field records which produced it - the pre-3.9 name estimator still reads it with a FutureWarning until 4.0; QDiDResults is an alias of this class).

class diff_diff.changes_in_changes_results.ChangesInChangesResults[source]

Bases: BaseResults

Results for ChangesInChanges and QDiD.

The headline att/se/t_stat/p_value/conf_int fields carry the mean effect; quantile_effects is a DataFrame with one row per requested quantile (columns quantile, qte, se, t_stat, p_value, conf_low, conf_high). All inference derives from the bootstrap (replicate-SD standard errors, symmetric normal-approximation intervals at level alpha); with n_bootstrap=0 every inference field is NaN.

q_lower/q_upper bound the point-identified interior quantile range for unconditional CiC fits (Athey-Imbens eq. 17; NaN for QDiD and for covariate fits, where the unconditional bounds are not the relevant objects). sup_t_crit is the qte package’s sup-t critical value for uniform bands - computed at a FIXED 95% level regardless of alpha (qte parity); see uniform_bands(). covariates records the covariate columns used by the conditional (quantile-regression) fit, or None for unconditional fits.

Methods

summary()

Fixed-width text summary: headline ATT block plus the quantile-effects table.

print_summary()

Print summary() to stdout.

to_dataframe([level])

Return the quantile-effects table or the one-row ATT summary.

to_dict()

Flat headline dictionary (ATT inference, ranges, sizes, bootstrap metadata).

uniform_bands()

Simultaneous (sup-t) confidence bands over the quantile grid.
att: float
se: float
t_stat: float
p_value: float
conf_int: Tuple[float, float]
quantile_effects: DataFrame
q_lower: float
q_upper: float
sup_t_crit: float
n_obs: int
cell_sizes: Dict[str, int]
n_bootstrap: int
n_bootstrap_valid: int
panel: bool
method: str
quantiles: ndarray
alpha: float = 0.05
covariates: List[str] | None = None
property is_significant: bool

Whether the headline ATT is significant at level alpha (False on NaN).

property significance_stars: str

Significance stars for the headline ATT p-value (’’ when NaN).

uniform_bands()[source]

Simultaneous (sup-t) confidence bands over the quantile grid.

qte +/- sup_t_crit * se per quantile, using the qte package’s IQR-scaled sup-t critical value at its hard-coded 95% level - the band level does NOT follow alpha (qte parity). Rows whose se is NaN (no bootstrap, failed replicate gate, or outside the interior range in an unconditional CiC fit) get NaN bands.

Return type:

DataFrame

to_dict()[source]

Flat headline dictionary (ATT inference, ranges, sizes, bootstrap metadata).

Return type:

Dict[str, Any]

to_dataframe(level='quantiles')[source]

Return the quantile-effects table or the one-row ATT summary.

Parameters:

level ({"quantiles", "att"}) – "quantiles" (default) returns a copy of quantile_effects; "att" returns a single-row frame with the headline fields.

Return type:

DataFrame

summary()[source]

Fixed-width text summary: headline ATT block plus the quantile-effects table.

Return type:

str

print_summary()[source]

Print summary() to stdout.

Return type:

None

property estimator: Any

Deprecated alias for method (removed in 4.0).

__setstate__(state)[source]

Migrate pickles created before the estimator -> method rename.

Results pickled before row M-143 stored the tag under estimator; rewriting the key on load keeps both method and the deprecated estimator alias working on old pickles. Dataclass unpickling routes through here for CURRENT objects too, so this must stay a pass-through in the common case.

Parameters:

state (Dict[str, Any])

Return type:

None

__init__(att, se, t_stat, p_value, conf_int, quantile_effects, q_lower, q_upper, sup_t_crit, n_obs, cell_sizes, n_bootstrap, n_bootstrap_valid, panel, method, quantiles, alpha=0.05, covariates=None)
Parameters:
Return type:

None

Example Usage#

Estimate quantile treatment effects in a 2x2 design:

import numpy as np
import pandas as pd
from diff_diff import ChangesInChanges

rng = np.random.default_rng(0)
n = 400
treated = np.repeat([1, 0], n // 2)
u = rng.normal(0, 1, n)
y_pre = u + rng.normal(0, 0.3, n)
y_post = u + 0.5 + rng.normal(0, 0.3, n) + treated * (0.8 + 0.4 * (u > 0))
data = pd.DataFrame({
    "unit": np.tile(np.arange(n), 2),
    "post": np.repeat([0, 1], n),
    "treated": np.tile(treated, 2),
    "y": np.concatenate([y_pre, y_post]),
})

cic = ChangesInChanges(n_bootstrap=200, seed=42)
results = cic.fit(data, outcome="y", treatment="treated", time="post")
results.print_summary()

# Quantile effects table and simultaneous bands
qte = results.to_dataframe("quantiles")
bands = results.uniform_bands()

Panel mode (same units in both periods) changes only the bootstrap:

import numpy as np
import pandas as pd
from diff_diff import ChangesInChanges

rng = np.random.default_rng(0)
n = 400
treated = np.repeat([1, 0], n // 2)
u = rng.normal(0, 1, n)
data = pd.DataFrame({
    "unit": np.tile(np.arange(n), 2),
    "post": np.repeat([0, 1], n),
    "treated": np.tile(treated, 2),
    "y": np.concatenate(
        [u + rng.normal(0, 0.3, n), u + 0.5 + rng.normal(0, 0.3, n) + treated]
    ),
})

cic_panel = ChangesInChanges(n_bootstrap=200, seed=42, panel=True)
results_panel = cic_panel.fit(
    data, outcome="y", treatment="treated", time="post", unit="unit"
)

QDiD as a comparison estimator (selected with method= on the merged class):

from diff_diff import ChangesInChanges

qdid = ChangesInChanges(method="qdid", n_bootstrap=200, seed=42)
results_qdid = qdid.fit(data, outcome="y", treatment="treated", time="post")

Covariates (conditional CiC via per-cell quantile regression, matching qte’s xformla; a small n_bootstrap keeps the example fast - every replicate refits the quantile regressions):

import numpy as np
import pandas as pd
from diff_diff import ChangesInChanges

rng = np.random.default_rng(1)
n = 120
treated = np.repeat([1, 0], n // 2)
x1 = rng.uniform(0, 2, n) + 0.3 * treated
y_pre = 0.5 + 0.8 * x1 + rng.normal(0, 0.4, n)
y_post = 0.8 + 1.1 * x1 + rng.normal(0, 0.4, n) + treated * 0.7
data = pd.DataFrame({
    "post": np.repeat([0, 1], n),
    "treated": np.tile(treated, 2),
    "x1": np.tile(x1, 2),
    "y": np.concatenate([y_pre, y_post]),
})

cic_cov = ChangesInChanges(n_bootstrap=10, seed=42)
results_cov = cic_cov.fit(
    data, outcome="y", treatment="treated", time="post", covariates=["x1"]
)
# Equivalent formula route: "y ~ treated * post + x1"
print(results_cov.covariates)  # ['x1']