{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cell-00",
   "metadata": {},
   "source": "# When Who Answers Changes: Survey Calibration for Causal DiD\n\nSurvey samples change composition over time: response rates decline, and they decline unevenly across demographic groups. This tutorial shows when that drift breaks a difference-in-differences estimate itself - not just descriptive statistics - and how to fix it by pairing diff-diff with Meta's [balance](https://import-balance.org/) calibration package.\n\n**A matched pair.** This notebook is the diff-diff-side companion to balance's [`balance_diff_diff_brfss` tutorial](https://github.com/facebookresearch/balance/blob/main/tutorials/balance_diff_diff_brfss.ipynb). Their lesson is reassurance: when non-response drift is *common* to treated and control units, it badly biases descriptive trends (which raking repairs) but largely **differences out** of the DiD. This notebook is the warning that completes the pair: when the drift is *differential* - correlated with treatment timing - the DiD is biased too, pre-trend tests won't catch it, and calibration becomes essential for the **causal** estimand. Along the way we hit a subtlety their setting never triggers: raking *granularity*.\n\n**This tutorial covers:**\n\n1. A BRFSS-style staggered smoking-ban dataset with no arm-specific trends by construction - population parallel trends hold in expectation (planted ATT -3.0pp, realized -2.98pp after a rarely-binding probability floor) - but differentially drifting non-response\n2. Why treatment-correlated composition drift does **not** difference out of a DiD - with clean pre-trends the whole way\n3. A per-wave *national* rake as a false fix, and raking at the granularity of your comparison units (state-year, as BRFSS itself does) as the real one\n4. The diff-diff/balance seam both ways: the native `SurveyDesign` + `aggregate_survey` three-liner, and the `balance.interop.diff_diff` adapter (`to_panel_for_did` / `fit_did`) - with an exact-parity check\n5. An estimator sweep (CallawaySantAnna / SunAbraham / ImputationDiD): composition bias is a data problem, not an estimator problem\n6. Cross-package diagnostics: `survey_metadata` design effects and `as_balance_diagnostic`\n7. A practical checklist: when calibration matters for causal vs descriptive estimands\n\n**Prerequisites:** [Tutorial 01: Basic DiD](01_basic_did.ipynb), [Tutorial 02: Staggered DiD](02_staggered_did.ipynb), [Tutorial 16: Survey DiD](16_survey_did.ipynb); ideally the [balance quickstart](https://import-balance.org/docs/tutorials/quickstart/).\n\n**Requirements:** `pip install diff-diff \"balance>=0.21\" matplotlib`. balance is needed **only for this tutorial** - it is not a diff-diff dependency. (`pip install \"balance[did]\"` installs both packages at once.)\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-01",
   "metadata": {},
   "source": "## 1. Two regimes of non-response drift\n\n\"Do the survey weights matter?\" is really two questions - one per estimand:\n\n| Non-response drift | Descriptive estimand (a prevalence trend) | Causal DiD estimand (an ATT) |\n|---|---|---|\n| **Common** - same in treated and control units | **Biased** - raking to population margins repairs it | **Robust** - composition changes difference out ([balance's tutorial](https://github.com/facebookresearch/balance/blob/main/tutorials/balance_diff_diff_brfss.ipynb)) |\n| **Differential** - correlated with treatment x time | Biased | **Biased** - the drift *is* treatment x post shaped, so the DiD reads it as treatment effect (**this tutorial**) |\n\nThe dangerous cell is the bottom-right one, because nothing in the standard DiD toolkit flags it: the fit converges, standard errors look fine, and - as we will see - the pre-treatment event-study coefficients stay clean.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-02",
   "metadata": {},
   "outputs": [],
   "source": "import logging\nimport warnings\n\nimport numpy as np\nimport pandas as pd\n\nimport balance  # >= 0.21 for balance.interop.diff_diff\nfrom balance import Sample\nfrom balance.interop import diff_diff as bd\n\n# Quiet balance's per-call INFO logging (must run AFTER `import balance` -\n# its __init__ re-arms the logger) and pandas FutureWarnings.\nlogging.getLogger(\"balance\").setLevel(logging.ERROR)\nwarnings.filterwarnings(\"ignore\", category=FutureWarning)\n# diff-diff normalizes pweights to mean 1 and warns to say so; expected here.\nwarnings.filterwarnings(\n    \"ignore\", message=\".*weights normalized.*\", category=UserWarning\n)\n\nimport diff_diff\nfrom diff_diff import CallawaySantAnna, SurveyDesign, aggregate_survey\n\ntry:\n    import matplotlib.pyplot as plt\n\n    plt.style.use(\"seaborn-v0_8-whitegrid\")\n    HAS_MATPLOTLIB = True\nexcept ImportError:\n    HAS_MATPLOTLIB = False\n\nprint(f\"diff-diff {diff_diff.__version__} | balance {balance.__version__}\")\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-03",
   "metadata": {},
   "source": "## 2. The data: a BRFSS-shaped survey around staggered smoking bans\n\nWe simulate respondent-level microdata mirroring the public-use [BRFSS](https://www.cdc.gov/brfss/annual_data/annual_2024.html) file: 50 states x 7 years (2018-2024), with 1,200 adults invited per state-year, of whom roughly 580-860 respond (~265,000 respondent rows in total). Comprehensive indoor-smoking bans take effect in 10 states in 2020 and 10 more in 2022; 30 states never adopt. The outcome is **current smoking** (binary `smoker`). Each respondent carries raking demographics (`age_band`, `educ_cat`), a complex design (`stratum`, `psu`, `fpc`), and a demographic-blind `design_weight` standing in for BRFSS `_LLCPWT`. A separate 20,000-row \"ACS\" frame supplies the population margins (and per-state adult population counts) we will rake to.\n\nThe generator is built so every headline claim is checkable against ground truth:\n\n- **No arm-specific trends by construction.** Smoking prevalence is additive and linear in education, age, and time; demographic margins are identical in every state and year. Trends differ *by demographic* (smoking falls fastest among the college-educated) but identically across treatment arms, so population parallel trends hold **in expectation**; mean-zero PSU-year shocks (kept in deliberately - they create the design effects the diagnostics section reads) add seed-specific noise but no systematic drift.\n- **The planted treatment effect is -3.0pp** in every treated state-year. One honest wrinkle: probabilities are floored at 1%, and that floor binds for ~2% of treated-post person-years (elderly college-educated respondents in low-smoking states), so the **realized population ATT is -2.98pp**. The generator computes it from the clipped potential outcomes and returns it as `truth[\"realized_att_pp\"]` - that is the truth line we hold every estimate against below.\n- **Non-response depends only on the raking observables** (`age_band`, `educ_cat`) - never on smoking itself. Response rates decline over time everywhere, concentrated among younger and less-educated adults (the real BRFSS pattern). That is the *common* drift.\n- **The differential twist** (`differential=True`): after a state's ban takes effect, response among `hs_or_less` adults falls an extra 7pp per event year - enforcement publicity raises refusals in the low-SES communities where smoking is concentrated, and the implementation consumes the follow-up-call budget. Scenario A (`differential=False`) and scenario B share the identical invited population and outcomes; they differ **only in who answers the phone**.\n\nBecause non-response is driven entirely by variables inside the raking margins (missing-at-random given the margins), calibration *can* fix everything - the interesting question is what kind of calibration. To run this on real data, replace this one cell with `pyreadstat.read_xport(...)` calls per BRFSS year.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-04",
   "metadata": {},
   "outputs": [],
   "source": "N_STATES = 50\nYEARS = np.arange(2018, 2025)\nN_INVITED = 1200\nN_STRATA = 5\nPSUS_PER_STATE = 8\nFPC_PSUS_PER_STRATUM = 200.0\n\nAGE_BANDS = [\"18-34\", \"35-49\", \"50-64\", \"65+\"]\nAGE_SHARES = np.array([0.30, 0.25, 0.25, 0.20])\nEDUC_CATS = [\"hs_or_less\", \"some_college\", \"college_plus\"]\nEDUC_SHARES = np.array([0.35, 0.30, 0.35])\n\nBASE_EDUC_PP = np.array([22.0, 15.0, 9.0])\nAGE_ADJ_PP = np.array([2.0, 3.0, 1.0, -2.0])\nTREND_COMMON_PP = 0.25\nTREND_EDUC_PP = np.array([-0.15, 0.0, 0.10])\nTRUE_ATT_PP = -3.0\nSTATE_RE_SD_PP = 1.5\nPSU_SHOCK_SD_PP = 0.8\nP_CLIP_PP = (1.0, 60.0)\n\nR_BASE = 0.70\nR_AGE_SHIFT = np.array([-0.10, -0.02, 0.03, 0.08])\nR_EDUC_SHIFT = np.array([-0.09, 0.00, 0.06])\nR_COMMON_DRIFT_EDUC = np.array([0.015, 0.0075, 0.0])\nR_COMMON_DRIFT_YOUNG = 0.010\nR_DIFF_DRIFT_PER_EVENT_YEAR = 0.07\nR_CLIP = (0.10, 0.95)\n\nTARGET_N = 20_000\nSEED = 20260704\n\n\ndef simulate_brfss_smoking(differential, seed=SEED, drift_start_offset=0):\n    \"\"\"BRFSS-style microdata around staggered smoking bans.\n\n    No arm-specific trends by construction: population parallel trends\n    hold in expectation (mean-zero PSU-year shocks add noise, not drift),\n    and the planted effect is -3.0pp (realized population ATT ~-2.98pp\n    after the probability floor). All SYSTEMATIC estimator bias in this\n    notebook comes from sample composition. Scenarios A/B share invited\n    respondents and outcomes for the same seed - they differ only in who\n    responds.\n    \"\"\"\n    rng = np.random.default_rng(seed)\n\n    perm = rng.permutation(N_STATES)\n    g_of_state = np.zeros(N_STATES, dtype=int)\n    g_of_state[perm[:10]] = 2020\n    g_of_state[perm[10:20]] = 2022\n    stratum_of_state = rng.integers(0, N_STRATA, size=N_STATES)\n    state_pop = rng.lognormal(mean=np.log(4e6), sigma=0.6, size=N_STATES)\n    state_re = np.clip(rng.normal(0.0, STATE_RE_SD_PP, size=N_STATES), -3.0, 3.0)\n    psu_shock = rng.normal(\n        0.0, PSU_SHOCK_SD_PP, size=(N_STATES, PSUS_PER_STATE, len(YEARS))\n    )\n\n    n_inv = N_STATES * len(YEARS) * N_INVITED\n    state = np.repeat(np.arange(N_STATES), len(YEARS) * N_INVITED)\n    year = np.tile(np.repeat(YEARS, N_INVITED), N_STATES)\n    age_idx = rng.choice(len(AGE_BANDS), size=n_inv, p=AGE_SHARES)\n    educ_idx = rng.choice(len(EDUC_CATS), size=n_inv, p=EDUC_SHARES)\n    psu_idx = rng.integers(0, PSUS_PER_STATE, size=n_inv)\n    u_respond = rng.uniform(size=n_inv)\n    u_smoker = rng.uniform(size=n_inv)\n    weight_jitter = rng.uniform(0.85, 1.15, size=n_inv)\n\n    k = year - YEARS[0]\n    year_idx = year - YEARS[0]\n    g = g_of_state[state]\n    treated_post = (g > 0) & (year >= g)\n\n    base_pp = (\n        BASE_EDUC_PP[educ_idx]\n        + AGE_ADJ_PP[age_idx]\n        + state_re[state]\n        + psu_shock[state, psu_idx, year_idx]\n        - (TREND_COMMON_PP + TREND_EDUC_PP[educ_idx]) * k\n    )\n    p_pp = np.clip(base_pp + TRUE_ATT_PP * treated_post, *P_CLIP_PP)\n    smoker = (u_smoker < p_pp / 100.0).astype(int)\n\n    r = (\n        R_BASE\n        + R_AGE_SHIFT[age_idx]\n        + R_EDUC_SHIFT[educ_idx]\n        - R_COMMON_DRIFT_EDUC[educ_idx] * k\n        - R_COMMON_DRIFT_YOUNG * k * (age_idx == 0)\n    )\n    if differential:\n        event_time = year - g - drift_start_offset\n        hit = (g > 0) & (event_time >= 0) & (educ_idx == 0)\n        r = r - R_DIFF_DRIFT_PER_EVENT_YEAR * (event_time + 1) * hit\n    r = np.clip(r, *R_CLIP)\n    responded = u_respond < r\n\n    micro = pd.DataFrame(\n        {\n            \"id\": np.arange(n_inv)[responded],\n            \"state\": state[responded],\n            \"year\": year[responded],\n            \"g\": g[responded],\n            \"smoker\": smoker[responded],\n            \"age_band\": np.array(AGE_BANDS)[age_idx[responded]],\n            \"educ_cat\": np.array(EDUC_CATS)[educ_idx[responded]],\n            \"stratum\": stratum_of_state[state[responded]],\n            \"psu\": state[responded] * 100 + psu_idx[responded],\n            \"fpc\": FPC_PSUS_PER_STRATUM,\n            \"design_weight\": (state_pop[state] / N_INVITED * weight_jitter)[\n                responded\n            ],\n        }\n    )\n\n    rng_t = np.random.default_rng(seed + 1)\n    target_df = pd.DataFrame(\n        {\n            \"id\": np.arange(TARGET_N),\n            \"age_band\": np.array(AGE_BANDS)[\n                rng_t.choice(len(AGE_BANDS), size=TARGET_N, p=AGE_SHARES)\n            ],\n            \"educ_cat\": np.array(EDUC_CATS)[\n                rng_t.choice(len(EDUC_CATS), size=TARGET_N, p=EDUC_SHARES)\n            ],\n        }\n    )\n\n    kk = YEARS - YEARS[0]\n    cell = (\n        BASE_EDUC_PP[None, :, None]\n        + AGE_ADJ_PP[None, None, :]\n        - (TREND_COMMON_PP + TREND_EDUC_PP[None, :, None]) * kk[:, None, None]\n    )\n    tp = (g_of_state[None, :] > 0) & (YEARS[:, None] >= g_of_state[None, :])\n    base_prev = np.einsum(\"tea,e,a->t\", cell, EDUC_SHARES, AGE_SHARES)\n    w_s = state_pop / state_pop.sum()\n    pop_prev = base_prev + TRUE_ATT_PP * (tp * w_s[None, :]).sum(axis=1)\n    # Realized population ATT: the probability floor P_CLIP_PP[0] binds for\n    # ~2% of treated-post person-years, attenuating the planted -3.0pp.\n    y1 = np.clip(base_pp + TRUE_ATT_PP, *P_CLIP_PP)\n    y0 = np.clip(base_pp, *P_CLIP_PP)\n    w_pop = state_pop[state]\n    realized_att_pp = ((y1 - y0) * w_pop)[treated_post].sum() / w_pop[\n        treated_post\n    ].sum()\n    truth = {\n        \"true_att_pp\": TRUE_ATT_PP,\n        \"realized_att_pp\": float(realized_att_pp),\n        \"floor_bind_share\": float(\n            ((y1 - y0) > TRUE_ATT_PP + 1e-12)[treated_post].mean()\n        ),\n        \"pop_prevalence_by_year\": dict(zip(YEARS.tolist(), pop_prev / 100.0)),\n        \"g_of_state\": g_of_state,\n        \"state_pop\": state_pop,\n    }\n    return micro, target_df, truth\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-05",
   "metadata": {},
   "outputs": [],
   "source": "micro_common, target, truth = simulate_brfss_smoking(differential=False)\nmicro_diff, _, _ = simulate_brfss_smoking(differential=True)\n\nby_cell = micro_diff.groupby([\"state\", \"year\"]).size()\nprint(f\"scenario A respondents: {len(micro_common):,}\")\nprint(f\"scenario B respondents: {len(micro_diff):,}\")\nprint(f\"respondents per state-year (B): {by_cell.min()}-{by_cell.max()}\")\nTRUE_ATT = truth[\"realized_att_pp\"]\nprint(f\"planted ATT: {truth['true_att_pp']}pp | realized population ATT: \"\n      f\"{TRUE_ATT:+.2f}pp (probability floor binds for \"\n      f\"{truth['floor_bind_share']:.1%} of treated-post person-years)\")\ntarget.head(3)\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-06",
   "metadata": {},
   "source": "## 3. The native seam: `SurveyDesign` + `aggregate_survey` + `CallawaySantAnna`\n\ndiff-diff's survey path consumes calibrated weights as *just a column*. The handoff from microdata to a modern staggered estimator is three calls:\n\n1. **`SurveyDesign(weights=..., strata=..., psu=..., fpc=...)`** - declare the complex design on the microdata (column *names*, no copies).\n2. **`aggregate_survey(micro, by=[\"state\", \"year\"], outcomes=\"smoker\", ...)`** - collapse respondents to a state-year panel of design-weighted prevalences with full Taylor-linearized precision tracking, returning the panel *and* a pre-configured second-stage `SurveyDesign` (population weights + state-level clustering, taken from the first `by` column).\n3. **`CallawaySantAnna(...).fit(panel, ..., survey_design=second_stage)`** - the staggered DiD with design-based variance.\n\nOne convention worth noticing now: we named the design columns `stratum` / `psu` / `fpc`. Those are exactly the defaults in `balance.interop.conventions.DEFAULT_DESIGN_COLUMNS`, so when we reach the balance adapter in section 8 it will wire the same design automatically.\n\nWe wrap the three calls in a helper because we are about to fit the same model under four different weight columns - that swap being a one-argument change is the point of the design.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-07",
   "metadata": {},
   "outputs": [],
   "source": "def fit_survey_cs(micro, weights_col):\n    \"\"\"Native seam: microdata + a weight column -> survey-weighted CS fit.\"\"\"\n    design = SurveyDesign(\n        weights=weights_col, strata=\"stratum\", psu=\"psu\", fpc=\"fpc\"\n    )\n    panel, second_stage = aggregate_survey(\n        micro, by=[\"state\", \"year\"], outcomes=\"smoker\", survey_design=design\n    )\n    panel = panel.merge(\n        micro[[\"state\", \"g\"]].drop_duplicates(), on=\"state\", how=\"left\"\n    )\n    cs = CallawaySantAnna(\n        estimation_method=\"reg\",\n        control_group=\"not_yet_treated\",\n        base_period=\"universal\",\n    )\n    return cs.fit(\n        panel,\n        outcome=\"smoker_mean\",\n        unit=\"state\",\n        time=\"year\",\n        first_treat=\"g\",\n        survey_design=second_stage,\n        aggregate=\"all\",\n    )\n\n\ndef show_att(res, label):\n    lo, hi = res.overall_conf_int\n    print(\n        f\"{label}: ATT = {res.overall_att * 100:+.2f}pp \"\n        f\"(SE {res.overall_se * 100:.2f}, 95% CI [{lo * 100:+.2f}, {hi * 100:+.2f}])\"\n    )\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-08",
   "metadata": {},
   "source": "## 4. Scenario A - common drift: the descriptive trend breaks, the DiD does not\n\nFirst the regime the balance tutorial covers. Response rates decline over time, concentrated among younger and less-educated adults, but *identically* in treated and control states. The design-weighted national prevalence drifts away from the truth (the sample is increasingly educated, and educated adults smoke less - so the naive trend overstates the decline). Yet the DiD contrast is untouched, because both arms mis-measure levels the same way.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-09",
   "metadata": {},
   "outputs": [],
   "source": "res_a = fit_survey_cs(micro_common, \"design_weight\")\nshow_att(res_a, \"Scenario A, design weights\")\n\npop_series = pd.Series(truth[\"pop_prevalence_by_year\"])\ndesign_series = micro_common.groupby(\"year\")[[\"smoker\", \"design_weight\"]].apply(\n    lambda d: (d.smoker * d.design_weight).sum() / d.design_weight.sum()\n)\ngap_pp = (design_series - pop_series) * 100\nprint(f\"descriptive gap, design-weighted vs population: \"\n      f\"{gap_pp.loc[2018]:+.2f}pp (2018) -> {gap_pp.loc[2024]:+.2f}pp (2024)\")\n\nif HAS_MATPLOTLIB:\n    fig, ax = plt.subplots(figsize=(8, 4.5))\n    ax.plot(pop_series.index, pop_series * 100, \"k-\", lw=2, label=\"population (truth)\")\n    ax.plot(design_series.index, design_series * 100, \"C1--\", lw=2, marker=\"o\",\n            label=\"design-weighted sample\")\n    ax.set_xlabel(\"year\")\n    ax.set_ylabel(\"smoking prevalence (%)\")\n    ax.set_title(\"Scenario A: common drift biases the descriptive trend\")\n    ax.legend()\n    plt.tight_layout()\n    plt.show()\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-10",
   "metadata": {},
   "source": "The ATT comes back at about **-3.1pp against the -2.98pp realized truth** - the causal estimate is robust even though the descriptive series visibly drifts below the truth. This is exactly the reassurance half of the story; for the full descriptive-repair workflow (per-wave raking, Love plots, effective sample sizes) see the [balance tutorial](https://github.com/facebookresearch/balance/blob/main/tutorials/balance_diff_diff_brfss.ipynb) - we will not repeat it here.\n\nNow we break it.\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-11",
   "metadata": {},
   "source": "## 5. Scenario B - differential drift: the causal estimand breaks\n\nSame states, same bans, same outcomes, same common drift - plus one mechanism: once a ban takes effect, response among `hs_or_less` adults in that state falls an extra 7pp per event year. The composition damage is easy to see if you know to look at it:\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-12",
   "metadata": {},
   "outputs": [],
   "source": "hs_share = (\n    micro_diff.assign(hs=lambda d: (d.educ_cat == \"hs_or_less\").astype(float))\n    .groupby([\"year\", \"g\"])[[\"hs\", \"design_weight\"]]\n    .apply(lambda d: (d.hs * d.design_weight).sum() / d.design_weight.sum())\n    .unstack()\n)\nprint(\"design-weighted hs_or_less share by cohort (population share: 0.350):\")\nprint(hs_share.round(3).to_string())\n\nif HAS_MATPLOTLIB:\n    fig, ax = plt.subplots(figsize=(8, 4.5))\n    styles = {0: (\"C0\", \"never treated (30 states)\"),\n              2020: (\"C3\", \"ban in 2020 (10 states)\"),\n              2022: (\"C1\", \"ban in 2022 (10 states)\")}\n    for gval, (color, label) in styles.items():\n        ax.plot(hs_share.index, hs_share[gval], color=color, marker=\"o\", label=label)\n    ax.axhline(0.35, color=\"gray\", ls=\":\", label=\"population share (0.35)\")\n    for gval, color in [(2020, \"C3\"), (2022, \"C1\")]:\n        ax.axvline(gval, color=color, ls=\"--\", alpha=0.4)\n    ax.set_xlabel(\"year\")\n    ax.set_ylabel(\"hs_or_less share of design-weighted sample\")\n    ax.set_title(\"Scenario B: who answers changes exactly when treatment starts\")\n    ax.legend()\n    plt.tight_layout()\n    plt.show()\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-13",
   "metadata": {},
   "source": "Each cohort's low-education share peels away from ~0.30 **exactly at its adoption year**, sliding to ~0.11 (2020 cohort) and ~0.19 (2022 cohort) by 2024 while never-treated states stay flat.\n\nWhy this bias cannot difference out: the design weights are demographic-blind, so a state-year's measured prevalence is $m(s,t) = \\sum_e \\sigma_e(s,t)\\, p_e(t)$, where $\\sigma_e$ is the *realized sample* share of education group $e$ and $p_e$ its true prevalence. A DiD passes any additive term that is common across arms - that killed the composition term in scenario A. But here $\\sigma_{hs}$ falls **only in treated states, only after adoption**: the composition artifact $\\Delta\\sigma_{hs} \\times (p_{hs} - \\bar{p}_{rest})$ sits precisely in the treatment x post cell of the design. To the estimator, it *is* an ATT. With a ~12pp gap in smoking prevalence between `hs_or_less` and the (share-weighted) rest, a 17pp share collapse manufactures roughly an extra -2pp of \"effect\" at long horizons, ramping up from ~-0.3pp at adoption - which masquerades as a treatment effect that *grows over time*.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-14",
   "metadata": {},
   "outputs": [],
   "source": "res_design = fit_survey_cs(micro_diff, \"design_weight\")\nshow_att(res_design, \"Scenario B, design weights\")\nprint(f\"true ATT (realized): {TRUE_ATT:+.2f}pp\")\n\nes_design = res_design.event_study_effects\nif HAS_MATPLOTLIB:\n    fig, ax = plt.subplots(figsize=(8, 4.5))\n    ev = sorted(es_design)\n    eff = np.array([es_design[e][\"effect\"] for e in ev]) * 100\n    se = np.array([es_design[e][\"se\"] for e in ev]) * 100\n    ax.errorbar(ev, eff, yerr=1.96 * np.nan_to_num(se), fmt=\"o-\", color=\"C1\",\n                capsize=3, label=\"design weights\")\n    ax.axhline(0, color=\"gray\", lw=0.8)\n    ax.axhline(TRUE_ATT, color=\"k\", ls=\":\", label=f\"true ATT ({TRUE_ATT:.2f}pp)\")\n    ax.axvline(-0.5, color=\"gray\", ls=\"--\", alpha=0.5)\n    ax.set_xlabel(\"event time (years since ban)\")\n    ax.set_ylabel(\"effect on smoking prevalence (pp)\")\n    ax.set_title(\"Scenario B, design weights: clean pre-trends, biased ATT\")\n    ax.legend()\n    plt.tight_layout()\n    plt.show()\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-15",
   "metadata": {},
   "source": "Three things just happened, and together they are the core lesson of this tutorial:\n\n1. **The ATT is badly overstated**: about **-4.1pp against a realized truth of -2.98pp** (roughly +38%), and the 95% CI (~[-4.9, -3.4]) *excludes* the truth. You would publish \"bans cut smoking by 4 points\" with confidence.\n2. **The dynamics are fake**: the event study \"builds\" to about -5pp at event year 4. That growth is the response-rate ramp, not policy.\n3. **The pre-trends are clean** (every pre-treatment coefficient is well inside its confidence band), because the drift starts *at* adoption. A pre-trend test cannot catch at-adoption composition drift - it certifies parallel trends of *who you measured*, not of the population.\n\nThe standard toolkit passes this regression with flying colors. The composition plot above is the only thing that flagged it.\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-16",
   "metadata": {},
   "source": "## 6. Fix attempt 1: per-wave national raking (the natural first move)\n\nThe textbook response - and the balance quickstart recipe - is to rake each survey wave to population margins: reweight so the sample's `age_band` x `educ_cat` distribution matches the ACS frame, year by year. We wrap balance's `Sample -> set_target -> adjust(method=\"rake\")` in a helper that (a) rakes within each cell of a chosen `granularity`, and (b) rescales each cell's raked weights to its population **count** - in real practice raking targets are population counts, so weight totals carry the state scale rather than balance's internal per-call normalization.\n\nOne mechanical detail: balance casts ids to `str`, so we align the returned weights back to the original rows by string id and assert nothing went missing.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-17",
   "metadata": {},
   "outputs": [],
   "source": "RAKE_VARS = [\"age_band\", \"educ_cat\"]\n\n\ndef rake_to_population(micro, target_df, granularity, weight_name, cell_totals):\n    \"\"\"Rake design weights to population margins within each granularity cell.\n\n    cell_totals maps each groupby key to the population COUNT the cell's\n    raked weights must sum to.\n    Returns (micro + weight_name column, {cell_key: adjusted balance Sample}).\n    \"\"\"\n    target_sample = Sample.from_frame(target_df, id_column=\"id\")\n    cols = [\"id\", *RAKE_VARS, \"smoker\", \"design_weight\"]\n    w_new = pd.Series(np.nan, index=micro.index)\n    adjusted = {}\n    for key, cell in micro.groupby(granularity):\n        if isinstance(key, tuple) and len(key) == 1:\n            key = key[0]\n        s = Sample.from_frame(\n            cell[cols].copy(),\n            id_column=\"id\",\n            weight_column=\"design_weight\",\n            outcome_columns=[\"smoker\"],\n        )\n        adj = s.set_target(target_sample).adjust(method=\"rake\", variables=RAKE_VARS)\n        w = adj.df.set_index(\"id\")[adj.weight_column]\n        aligned = w.reindex(cell[\"id\"].astype(str).values).to_numpy()\n        assert not np.isnan(aligned).any(), f\"NaN raked weights in cell {key}\"\n        aligned = aligned * (cell_totals[key] / aligned.sum())\n        w_new.loc[cell.index] = aligned\n        adjusted[key] = adj\n    out = micro.copy()\n    out[weight_name] = w_new\n    return out, adjusted\n\n\nstate_pop = truth[\"state_pop\"]\n\nmicro_diff, adj_national = rake_to_population(\n    micro_diff, target, [\"year\"], \"w_national\",\n    cell_totals={int(y): state_pop.sum() for y in YEARS},\n)\nres_national = fit_survey_cs(micro_diff, \"w_national\")\nshow_att(res_national, \"Scenario B, per-wave NATIONAL rake\")\n\nm24 = micro_diff[micro_diff.year == 2024].assign(\n    hs=lambda d: (d.educ_cat == \"hs_or_less\").astype(float)\n)\nfor gval, label in [(0, \"never treated\"), (2020, \"2020 cohort\")]:\n    sub = m24[m24.g == gval]\n    share = (sub.hs * sub.w_national).sum() / sub.w_national.sum()\n    print(f\"2024 hs_or_less share after national rake, {label}: \"\n          f\"{share:.3f} (population: 0.350)\")\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-18",
   "metadata": {},
   "source": "**It got *worse*** - about **-4.4pp**. And the composition printout says why: after the national rake, never-treated states sit at ~0.42 low-education share while the 2020 cohort sits at ~0.18. The rake satisfied the national margin *in aggregate* - by pushing extra weight onto low-education respondents everywhere, over-correcting the states that still had them and leaving the treated states (which barely have any left) still far below the margin.\n\nRaking equalizes composition **at the level you rake at**. A DiD compares *states*; a national rake constrains only the national mixture, so the treated-vs-control composition gap - the thing biasing the DiD - survives, and the redistributed weight can even amplify it.\n\n**The rule: rake at (or below) the granularity of the units your design compares.** This is not exotic - it is what BRFSS itself does: `_LLCPWT` is raked *within each state* to state-level demographic control totals. (The balance tutorial's per-wave national rake was the right call *there*: under common drift, every state needs the same correction, so the national rake fixes each state too. Under differential drift, that shortcut collapses.)\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-19",
   "metadata": {},
   "source": "## 7. Fix attempt 2: rake each state-year to the margins\n\nSame helper, `granularity=[\"state\", \"year\"]` - 350 small rakes (~750 respondents each, a couple of seconds total), each rescaled to its state's adult-population count from the ACS frame. State totals are then constant across years by construction, so the second-stage population weights recover the true state scale, uncontaminated by response-rate levels.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-20",
   "metadata": {},
   "outputs": [],
   "source": "micro_diff, adj_cell = rake_to_population(\n    micro_diff, target, [\"state\", \"year\"], \"w_raked\",\n    cell_totals={\n        (st, int(y)): state_pop[st] for st in range(N_STATES) for y in YEARS\n    },\n)\nres_raked = fit_survey_cs(micro_diff, \"w_raked\")\n\nshow_att(res_design, \"design weights       \")\nshow_att(res_national, \"national rake        \")\nshow_att(res_raked, \"state-year rake      \")\nprint(f\"true ATT (realized): {TRUE_ATT:+.2f}pp\")\n\nm24 = micro_diff[micro_diff.year == 2024].assign(\n    hs=lambda d: (d.educ_cat == \"hs_or_less\").astype(float)\n)\nfor gval, label in [(0, \"never treated\"), (2020, \"2020 cohort\"), (2022, \"2022 cohort\")]:\n    sub = m24[m24.g == gval]\n    share = (sub.hs * sub.w_raked).sum() / sub.w_raked.sum()\n    print(f\"2024 hs_or_less share after state-year rake, {label}: {share:.3f}\")\n\nes_raked = res_raked.event_study_effects\nif HAS_MATPLOTLIB:\n    fig, ax = plt.subplots(figsize=(8.5, 5))\n    for es, color, label in [(es_design, \"C1\", \"design weights\"),\n                             (es_raked, \"C0\", \"state-year raked\")]:\n        ev = sorted(es)\n        eff = np.array([es[e][\"effect\"] for e in ev]) * 100\n        se = np.array([es[e][\"se\"] for e in ev]) * 100\n        ax.errorbar(ev, eff, yerr=1.96 * np.nan_to_num(se), fmt=\"o-\", color=color,\n                    capsize=3, label=label, alpha=0.9)\n    ax.axhline(0, color=\"gray\", lw=0.8)\n    ax.axhline(TRUE_ATT, color=\"k\", ls=\":\", lw=1.5,\n               label=f\"true ATT ({TRUE_ATT:.2f}pp)\")\n    ax.axvline(-0.5, color=\"gray\", ls=\"--\", alpha=0.5)\n    ax.set_xlabel(\"event time (years since ban)\")\n    ax.set_ylabel(\"effect on smoking prevalence (pp)\")\n    ax.set_title(\"Composition drift manufactures dynamics; state-level raking removes them\")\n    ax.legend()\n    plt.tight_layout()\n    plt.show()\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-21",
   "metadata": {},
   "source": "Recovered: the state-year rake lands at about **-3.2pp (SE 0.5)** - within half a standard error of the realized -2.98pp truth - the fake build-up flattens onto the truth line at every horizon, and all three cohorts' low-education shares sit at 0.354 against the 0.350 population margin. The scoreboard:\n\n| weights | ATT (pp) | vs realized truth (-2.98) |\n|---|---|---|\n| design weights | ~ -4.1 | ~+38% overstated, CI excludes truth |\n| per-wave national rake | ~ -4.4 | worse - margins met in aggregate only |\n| **state-year rake, population-count totals** | **~ -3.2** | recovered (within 0.5 SE) |\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-22",
   "metadata": {},
   "source": "## 8. The adapter: `balance.interop.diff_diff`\n\nEverything above used the native seam - calibrated weights as a column. balance 0.21 ships an adapter that packages the same handoff when your object in hand is a balance `Sample`:\n\n- **`bd.to_panel_for_did(sample, by=, outcomes=)`** builds the first-stage `SurveyDesign` from the Sample's *active* weight column (auto-wiring our `stratum`/`psu`/`fpc` convention columns), strips balance's bookkeeping columns, and calls `diff_diff.aggregate_survey` - returning the same `(panel, second_stage_design)` pair.\n- **`bd.fit_did(sample, estimator=, ...)`** resolves any diff-diff estimator by name (or short alias: `\"CS\"`, `\"DiD\"`, `\"BJS\"`, `\"HAD\"`), splits your kwargs between `__init__` and `fit()` by signature, forwards `survey_design=`, and attaches the source Sample to the result as `_balance_adjustment` for provenance.\n\nBoth paths must agree exactly - and we assert it:\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-23",
   "metadata": {},
   "outputs": [],
   "source": "keep = [\"id\", \"state\", \"year\", \"smoker\", \"age_band\", \"educ_cat\",\n        \"stratum\", \"psu\", \"fpc\", \"w_raked\"]\nsample = Sample.from_frame(\n    micro_diff[keep].copy(),\n    id_column=\"id\",\n    weight_column=\"w_raked\",\n    outcome_columns=[\"smoker\"],\n)\n\npanel_df, second_stage = bd.to_panel_for_did(\n    sample, by=[\"state\", \"year\"], outcomes=\"smoker\"\n)\nprint(f\"panel: {panel_df.shape[0]} state-years, \"\n      f\"second-stage weights={second_stage.weights!r}, psu={second_stage.psu!r}\")\n\n# g is unit-constant metadata: re-join it at the panel level\npanel_df = panel_df.merge(\n    micro_diff[[\"state\", \"g\"]].drop_duplicates(), on=\"state\", how=\"left\"\n)\npanel_df[\"panel_id\"] = np.arange(len(panel_df))\npanel_sample = Sample.from_frame(\n    panel_df,\n    id_column=\"panel_id\",\n    weight_column=second_stage.weights,\n    outcome_columns=[\"smoker_mean\"],\n)\n\nres_adapter = bd.fit_did(\n    panel_sample,\n    estimator=\"CallawaySantAnna\",\n    outcome=\"smoker_mean\",\n    time=\"year\",\n    unit=\"state\",\n    treatment_first=\"g\",\n    design_columns={\"psu\": \"state\"},  # match the native second-stage design\n    estimation_method=\"reg\",\n    control_group=\"not_yet_treated\",\n    base_period=\"universal\",\n    aggregate=\"all\",\n)\n\nassert np.isclose(res_adapter.overall_att, res_raked.overall_att, rtol=1e-12)\nprint(f\"native  ATT: {res_raked.overall_att * 100:+.4f}pp\")\nprint(f\"adapter ATT: {res_adapter.overall_att * 100:+.4f}pp  (exact match)\")\nprint(f\"provenance attached: {hasattr(res_adapter, '_balance_adjustment')}\")\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-24",
   "metadata": {},
   "source": "Use whichever side of the seam matches your pipeline: if you live in balance (Samples, adjustment lineage, per-wave diagnostics), the adapter keeps that lineage attached to the diff-diff result; if you live in DataFrames, the native three-liner needs no extra dependency. The contract between the packages - `aggregate_survey`'s signature, `SurveyDesign`'s fields, estimator names, the provenance side-channel - is pinned on the diff-diff side by `tests/test_balance_interop_contract.py`, so both routes stay stable.\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-25",
   "metadata": {},
   "source": "## 9. Does the fix depend on the estimator?\n\nA tempting escape: \"maybe a more robust estimator handles it.\" It cannot - the composition artifact lives in the *data*, in the treatment x post cell, upstream of any identification strategy. `fit_did`'s one-string estimator swap makes the sweep trivial:\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-26",
   "metadata": {},
   "outputs": [],
   "source": "ESTIMATOR_KWARGS = {\n    \"CallawaySantAnna\": dict(estimation_method=\"reg\",\n                             control_group=\"not_yet_treated\",\n                             base_period=\"universal\", aggregate=\"all\"),\n    \"SunAbraham\": dict(control_group=\"never_treated\"),\n    \"ImputationDiD\": {},\n}\n\nrows = []\nfor wcol, wlabel in [(\"design_weight\", \"design\"), (\"w_raked\", \"state-raked\")]:\n    s = Sample.from_frame(\n        micro_diff[keep[:-1] + [wcol]].copy(),\n        id_column=\"id\", weight_column=wcol, outcome_columns=[\"smoker\"],\n    )\n    pdf, ss = bd.to_panel_for_did(s, by=[\"state\", \"year\"], outcomes=\"smoker\")\n    pdf = pdf.merge(micro_diff[[\"state\", \"g\"]].drop_duplicates(),\n                    on=\"state\", how=\"left\")\n    pdf[\"panel_id\"] = np.arange(len(pdf))\n    ps = Sample.from_frame(pdf, id_column=\"panel_id\", weight_column=ss.weights,\n                           outcome_columns=[\"smoker_mean\"])\n    for name, extra in ESTIMATOR_KWARGS.items():\n        r = bd.fit_did(\n            ps, estimator=name, outcome=\"smoker_mean\", time=\"year\",\n            unit=\"state\", treatment_first=\"g\",\n            design_columns={\"psu\": \"state\"}, **extra,\n        )\n        rows.append({\"weights\": wlabel, \"estimator\": name,\n                     \"att_pp\": round(r.overall_att * 100, 2),\n                     \"se_pp\": round(r.overall_se * 100, 2)})\n\nsweep = pd.DataFrame(rows).pivot(index=\"estimator\", columns=\"weights\",\n                                 values=\"att_pp\")\nprint(f\"overall ATT (pp) by estimator and weighting (realized truth: {TRUE_ATT:.2f}):\")\nprint(sweep.round(2).to_string())\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-27",
   "metadata": {},
   "source": "All three estimators tell the same story: roughly -3.8 to -4.2pp under design weights, roughly -2.9 to -3.2pp once the composition is fixed. Callaway-Sant'Anna's doubly-robust machinery, Sun-Abraham's interaction weighting, and Borusyak-Jaravel-Spiess's imputation efficiency all faithfully estimate the effect *in the data they are given*. Composition bias is a data problem; fix it in the data.\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-28",
   "metadata": {},
   "source": "## 10. Diagnostics across the seam\n\nEach package owns half the diagnostic picture. diff-diff's `survey_metadata` describes the *second stage* - how much precision the population weighting costs across states. balance's diagnostics describe each *raking cell* - how hard the calibration had to work, which is itself an early-warning signal: the design effect of the rake blows up exactly where composition was most distorted. `bd.as_balance_diagnostic` joins the two into one flat dict.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-29",
   "metadata": {},
   "outputs": [],
   "source": "sm = res_raked.survey_metadata\nprint(f\"second stage: design_effect={sm.design_effect:.2f}, \"\n      f\"effective_n={sm.effective_n:.1f} of {sm.n_psu} states, \"\n      f\"df_survey={sm.df_survey}\")\n\n# per-cell raking cost: never-treated vs treated state, 2024\nnever_state = int(micro_diff.loc[micro_diff.g == 0, \"state\"].iloc[0])\ntreated_state = int(micro_diff.loc[micro_diff.g == 2020, \"state\"].iloc[0])\nfor label, st in [(\"never-treated\", never_state), (\"2020-cohort\", treated_state)]:\n    diag = bd.as_balance_diagnostic(adj_cell[(st, 2024)], res_adapter)\n    print(f\"\\n{label} state {st}, 2024 raking cell:\")\n    print(f\"  balance_design_effect: {diag['balance_design_effect']:.2f}\")\n    print(f\"  balance_kish_ess:      {diag['balance_kish_ess']:.0f}\")\n    print(f\"  balance_asmd_max_post: {diag['balance_asmd_max_post']:.4f}\")\n    print(f\"  att (full panel fit):  {diag['att'] * 100:+.2f}pp\")\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-30",
   "metadata": {},
   "source": "Read the contrast: in a never-treated state the 2024 rake barely works (design effect near 1), while in a 2020-cohort state it pays a large design effect to rebuild the missing low-education mass - and post-adjustment imbalance (`asmd_max_post`) is ~0 in both, confirming the margins were hit. **Rising per-cell raking design effects concentrated in treated-post cells are the operational fingerprint of differential drift.**\n\nThe practical monitoring rule this tutorial suggests: alongside your pre-trend plot, always plot **design-weighted demographic shares by cohort over time** (the section 5 plot) and the **per-cell raking design effects**. Both are one groupby away, and they catch what the pre-trend test structurally cannot.\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-31",
   "metadata": {},
   "source": "## 11. When do you need calibration? A checklist\n\n| Your estimand | Drift type | What you need |\n|---|---|---|\n| Descriptive (levels, trends) | any | Calibrated weights, per wave ([balance tutorial](https://github.com/facebookresearch/balance/blob/main/tutorials/balance_diff_diff_brfss.ipynb)) |\n| Causal DiD | common across arms | Design weights suffice for the point estimate; calibration still improves descriptives and SEs |\n| Causal DiD | **correlated with treatment timing** | **Calibration at the granularity of your comparison units** (this tutorial) |\n\nThree caveats that decide whether raking is *enough*:\n\n- **Margins must come from a policy-unaffected source.** We raked to ACS-style demographics. If the policy itself changed the margins you rake to, calibration bakes the effect away.\n- **Raking fixes drift on observables inside your margins.** Here non-response depended only on `age_band` x `educ_cat` - missing-at-random given the margins - so raking was exact. If response depends on the *outcome itself within* demographic cells (heavier smokers refusing regardless of education), no reweighting on demographics can fix it. That regime is the subject of Sant'Anna & Xu (2023), \"Difference-in-Differences with Compositional Changes\" ([arXiv:2304.13925](https://arxiv.org/abs/2304.13925)) - a model-based route diff-diff tracks on its roadmap.\n- **Drift can pre-date adoption.** Our mechanism switched on at the effective date, which is why the pre-trends stayed clean. As an exercise, rerun with `simulate_brfss_smoking(differential=True, drift_start_offset=-2)` - drift beginning with the legislative campaign two years early - and watch the design-weight pre-treatment coefficients light up. When your event study *does* flag pre-trends in survey data, composition drift belongs on the suspect list right next to genuine trend violations.\n"
  },
  {
   "cell_type": "markdown",
   "id": "cell-32",
   "metadata": {},
   "source": "## Summary\n\n**Key takeaways:**\n\n1. Survey weights matter for causal estimands, not just descriptive ones - but only under the right failure mode. Common non-response drift differences out of a DiD; drift **correlated with treatment timing does not** (it is treatment x post shaped, so the estimator reads it as effect).\n2. In our BRFSS-style smoking-ban setting, differential drift inflated a -2.98pp (realized) true ATT to ~-4.1pp with a fake growing dynamic profile - **while every pre-trend test stayed clean**. Pre-trend tests certify who you measured, not the population.\n3. **Raking granularity must match your comparison units.** A per-wave national rake made things *worse* (margins met in aggregate, arm-level composition untouched); raking each state-year to population-count totals - BRFSS's own practice - recovered ~-3.2pp.\n4. The diff-diff/balance seam works both ways and agrees exactly: native (`SurveyDesign` + `aggregate_survey` + `fit(survey_design=...)`) or the `balance.interop.diff_diff` adapter (`to_panel_for_did` / `fit_did`, with `_balance_adjustment` provenance).\n5. No estimator choice rescues distorted data: CS, Sun-Abraham, and ImputationDiD were all equally biased before raking and all recovered after.\n6. Monitor **demographic shares by cohort** and **per-cell raking design effects** alongside pre-trends; they are the early-warning system for composition drift.\n\n**Quick reference** - the whole workflow:\n\n```python\n# 1. rake each comparison cell to population margins (balance)\nadj = (Sample.from_frame(cell_df, id_column=\"id\", weight_column=\"design_weight\")\n       .set_target(acs_sample).adjust(method=\"rake\", variables=[\"age_band\", \"educ_cat\"]))\n\n# 2. hand the calibrated weights to diff-diff (native seam)\ndesign = SurveyDesign(weights=\"w_raked\", strata=\"stratum\", psu=\"psu\", fpc=\"fpc\")\npanel, stage2 = aggregate_survey(micro, by=[\"state\", \"year\"], outcomes=\"smoker\",\n                                 survey_design=design)\nresult = CallawaySantAnna().fit(panel, outcome=\"smoker_mean\", unit=\"state\",\n                                time=\"year\", first_treat=\"g\", survey_design=stage2)\n```\n\n**Related tutorials:** [Tutorial 16: Survey DiD](16_survey_did.ipynb) (SurveyDesign in depth: replicate weights, subpopulations, DEFF) - [Tutorial 17: Brand Awareness Survey](17_brand_awareness_survey.ipynb) - [Tutorial 22: HAD Survey-Weighted Workflow](22_had_survey_design.ipynb) - [balance quickstart](https://import-balance.org/docs/tutorials/quickstart/) and the [balance x diff-diff BRFSS tutorial](https://github.com/facebookresearch/balance/blob/main/tutorials/balance_diff_diff_brfss.ipynb).\n\n**References:**\n\n- Callaway, B. & Sant'Anna, P. H. C. (2021). \"Difference-in-Differences with Multiple Time Periods.\" *Journal of Econometrics*, 225(2), 200-230.\n- Sant'Anna, P. H. C. & Xu, Q. (2023). \"Difference-in-Differences with Compositional Changes.\" [arXiv:2304.13925](https://arxiv.org/abs/2304.13925).\n- Deville, J.-C. & Särndal, C.-E. (1992). \"Calibration Estimators in Survey Sampling.\" *JASA*, 87(418), 376-382.\n- Deming, W. E. & Stephan, F. F. (1940). \"On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Totals are Known.\" *Annals of Mathematical Statistics*, 11(4), 427-444.\n- Groves, R. M. & Peytcheva, E. (2008). \"The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis.\" *Public Opinion Quarterly*, 72(2), 167-189.\n- Solon, G., Haider, S. J. & Wooldridge, J. M. (2015). \"What Are We Weighting For?\" *Journal of Human Resources*, 50(2), 301-316.\n- Lumley, T. (2004). \"Analysis of Complex Survey Samples.\" *Journal of Statistical Software*, 9(8).\n- Sarig, T., Galili, T. & Eilat, R. (2023). \"balance - a Python package for balancing biased data samples.\" [arXiv:2307.06024](https://arxiv.org/abs/2307.06024).\n"
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}