From 3cfaaf71845c7c258d9702d5417c7338f53b1b93 Mon Sep 17 00:00:00 2001 From: igerber Date: Thu, 29 Oct 2026 20:09:23 -0400 Subject: [PATCH] feat(v4): item 7c of the 4.0 cut - SpilloverDiD df_convention knob (M-154) SpilloverDiD gains the shared three-value df_convention knob with the 4.0 default "cluster": p-values and confidence intervals on its clustered lanes - the unit auto-cluster that 7b introduced and an explicit cluster= - now use the Stata/fixest t(G-1) reference like every other auto-clustered default, with G the effective cluster count reported as n_clusters. Point estimates, standard errors and t-statistics never move. "residual" restores the 3.x / 7b residual-df inference (n_obs_eff - k_effective) bit-for-bit; "normal" is deliberate z. The ladder is the pure helper spillover._resolve_inference_df, called once per fit and feeding every safe_inference site (aggregate tau_total, per-ring delta_j, event-study per-cell tau_k / delta_jk and the scalar lincom att): a survey_design df keeps precedence under every knob value; a saturated stage-2 sample keeps the warn + df=0 all-NaN sentinel under every value (resolve_tail_df is never consulted there); otherwise utils.resolve_tail_df with G forced to None (residual df) on EVERY Conley fit - an explicit product-kernel cluster= included, the SunAbraham precedent - and on the cluster=False HC1 lane. The knob is validated at __init__, through set_params and again at fit(); SpilloverDiDResults carries df_convention and inference_df (None under z and under the NaN sentinel); the hc2/hc2_bm rejection text no longer claims a "generic residual df". Tests: new tests/test_v4_spillover_df_convention.py (M-154 test_ref) pins the knob surface, the default-lane t(G-1) on the headline and the ring rows, an explicit coarser cluster column (G=10), the residual and normal values, the inert cluster=False / Conley lanes (explicit cluster included), survey-df precedence on weights-only and PSU+strata designs, event-study mode under all three values and the df ladder's saturation / survey / knob branches directly on the helper. The variance-conventions matrix gains spillover_cluster_unit (explicit residual, tail df 358) and its _default_df twin (tail df 59) on a deterministic coordinate overlay of the shared panel; the committed table is regenerated. The t23 drift suite pins the tutorial's default fit at t(199) and the HC1 / Conley fits on the residual df; test_spillover's exact get_params key set gains df_convention. Tutorial 23 section 5 names the reference distribution (no code cell changes - the notebook prints estimates and SEs only). Ledger: M-154 -> done (M-080's note updated); spec section 7 / item 7 complete; REGISTRY knob roster gains SpilloverDiD (out-of-scope list drops it) and the SpilloverDiD df Note describes the ladder; the variance-conventions roster, L3 entry, shared-DGP paragraph and tail-df landscape, the migration guide, llms-full.txt, docs/api/spillover.rst, docs/doc-deps.yaml and TODO.md updated. --- TODO.md | 4 +- ...29-v4-pr7c-m154-spillover-df-convention.md | 7 + diff_diff/guides/llms-full.txt | 1 + diff_diff/spillover.py | 203 ++++++-- diff_diff/utils.py | 3 +- docs/api/spillover.rst | 12 +- docs/doc-deps.yaml | 5 + docs/methodology/REGISTRY.md | 44 +- docs/methodology/variance-conventions.md | 34 +- docs/migration-4.0.md | 19 +- docs/tutorials/23_spillover_tva.ipynb | 5 +- docs/v4-deprecations.yaml | 7 +- docs/v4-design.md | 25 +- tests/test_spillover.py | 1 + tests/test_t23_spillover_tva_drift.py | 26 ++ tests/test_v4_cluster_policy.py | 2 +- tests/test_v4_matrix.py | 2 +- tests/test_v4_spillover_df_convention.py | 435 ++++++++++++++++++ tests/test_variance_conventions.py | 47 ++ 19 files changed, 783 insertions(+), 99 deletions(-) create mode 100644 changelog.d/20261029-v4-pr7c-m154-spillover-df-convention.md create mode 100644 tests/test_v4_spillover_df_convention.py diff --git a/TODO.md b/TODO.md index cec98268d..e0a95e517 100644 --- a/TODO.md +++ b/TODO.md @@ -86,7 +86,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m | Issue | Location | Origin | Effort | Priority | |-------|----------|--------|--------|----------| -| Re-execute the nine tutorials whose committed outputs still show the 3.x residual-df p-values/CIs after the `df_convention` default moved to `"cluster"` (t(G-1)): `01_basic_did`, `02_staggered_did`, `04_parallel_trends`, `05_honest_did`, `07_pretrends_power`, `09_real_world_examples`, `11_imputation_did`, `13_stacked_did`, `16_wooldridge_etwfe` (not `16_survey_did`: the survey df wins). Run under `DIFF_DIFF_BACKEND=rust` with the per-worktree kernel, reconcile any prose that restates a moved number, and `--nbmake` each; scheduled for the 4.0 release PR (`docs/v4-design.md` section 9 item 10) together with any item-7c/8/9 movers (24 was refreshed in #879; 23 in 7b and again in 7c). 7b's label changes also drift any committed analytical DiD/TWFE summary (", G=" on the variance line) and every SunAbraham summary (the new "Std. errors:" line) - covered by the same re-execution. | `docs/tutorials/*.ipynb` | #879 | Mid | Medium | +| Re-execute the nine tutorials whose committed outputs still show the 3.x residual-df p-values/CIs after the `df_convention` default moved to `"cluster"` (t(G-1)): `01_basic_did`, `02_staggered_did`, `04_parallel_trends`, `05_honest_did`, `07_pretrends_power`, `09_real_world_examples`, `11_imputation_did`, `13_stacked_did`, `16_wooldridge_etwfe` (not `16_survey_did`: the survey df wins). Run under `DIFF_DIFF_BACKEND=rust` with the per-worktree kernel, reconcile any prose that restates a moved number, and `--nbmake` each; scheduled for the 4.0 release PR (`docs/v4-design.md` section 9 item 10) together with any item-8/9 movers (24 was refreshed in #879; 23 in 7b, re-validated in 7c - it prints no p-values or CIs). 7b's label changes also drift any committed analytical DiD/TWFE summary (", G=" on the variance line) and every SunAbraham summary (the new "Std. errors:" line) - covered by the same re-execution. | `docs/tutorials/*.ipynb` | #879 | Mid | Medium | | TWFE event-study one-way HC2-BM absorbed-FE R parity against the `mpd_absorbed_fe_did` golden (`clubSandwich::vcovCR(cluster=1:n, type="CR2")`): since 4.0 (item 7b) `cluster=False` + hc2_bm reaches the one-way lane directly (no singleton-cluster column needed; `tests/test_v4_cluster_policy.py` pins finiteness and the static-surface LinearRegression parity only), so the open item is the R pin itself; the hc2 lane is pinned by `TestTWFEEventStudyAbsorbedFERParity` and the clustered hc2_bm per-period lane by `test_twfe_event_study_cluster_hc2_bm_per_period_df_matches_clubsandwich`; the MultiPeriodDiD lanes died at the 4.0 cut (PR 4c-ii) | `tests/test_estimators_vcov_type.py` | #v4-4c-i | Mid | Medium | | TWFE event-study power profile: `simulate_power`/`simulate_mde`/`simulate_sample_size` have no `_EstimatorProfile` for `TwoWayFixedEffects.fit(event_study=True)` since the MultiPeriodDiD registry entry went with the class (PR 4c-ii); `EventStudyResults` carries no scalar headline, so the profile needs a design (post-period average via the `tests/helpers/event_study_average.py` arithmetic, or a named period) plus its `_SURVEY_FIT_BUILDERS` twin | `diff_diff/power.py` | #v4-4c-ii | Mid | Low | | Clustered hc2_bm Bell-McCaffrey DOF guard is batch-relative (`linalg._cr2_bm_dof_inner`: a contrast is NaN'd when its scaled max\|B\| falls below 1e-10 of the batch maximum), so the event-study core keeps the otherwise-unused post-period-average contrast column as the anchor to hold the 3.x per-period NaN masks bit-identical (PR 4c-ii, user decision 2026-10-25). Reproduction: `x_nc = treated * 1[period >= 4] + 3e-6 * default_rng(seed).normal(size=n)` as a covariate on the oracle panel flips the mask between `[eye(k), avg]` and `eye(k)` on 20/20 seeds (cond(X) ~ 9e5; 1e-7 never flips, <= 3e-8 rank-drops) - too BLAS-sensitive for a committed oracle, hence the structural pin `test_hc2_bm_contrast_batch_keeps_post_average_anchor`. Decide whether the guard should use a batch-independent anchor (the treatment contrast, or the absolute floor only) and then drop the column | `diff_diff/estimators.py`, `diff_diff/linalg.py` | #v4-4c-ii | Mid | Medium | @@ -99,7 +99,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m | Evaluate adding the `BaseEstimator` param surface (get_params/set_params) to the exported classes that never had it - `PowerAnalysis`, `LinearRegression`, `BusinessReport`, `DiagnosticReport`, `TWFEWeightsResult` (a NEW public surface, deliberately out of the 2(c)-i pure-refactor scope; `LinearRegression` is the one `fit`-bearing class excluded from the contract suite's roster-completeness test). | `diff_diff/linalg.py`, `diff_diff/power.py` | mixin PR | Mid | Low | | Tighten the mypy suppressions that back the enforced-zero posture: burn down `prep_dgp`'s per-module `[index]` override (needs a None-vs-array restructure that preserves the seeded RNG stream), and evaluate re-enabling the globally disabled codes (`arg-type`, `return-value`, `var-annotated`, `assignment`) one at a time — `assignment` alone hid several real annotation drifts found during the 2026-07 triage. | `pyproject.toml` `[tool.mypy]`, `diff_diff/prep_dgp.py` | lint-CI | Mid | Low | | Route LWDiD into the practitioner code surfaces: `practitioner_next_steps` (`diff_diff/practitioner.py`), `agent_workflow.py`, `DiagnosticReport`, and `BusinessReport` never dispatch to or mention LWDiD — the AI guides now cover it, but the code surfaces need contract decisions (when to recommend it, what diagnostics to run) plus tests | `diff_diff/practitioner.py`, `diff_diff/agent_workflow.py`, `diff_diff/diagnostic_report.py`, `diff_diff/business_report.py` | LWDiD release audit | Mid | Low | -| df-provenance completion follow-up (PR C descoping): a Wooldridge public per-row df channel (a per-key dict that can also carry the hc2_bm BM contrast dofs the scalar `_df_analytic_fallback` cannot represent — the unified surface's Wooldridge df column stays NaN until then); ImputationDiD NON-survey per-lead df (the survey-fit scalar `event_study_df` + its bootstrap clearing shipped with the M-092 completion; what remains is surfacing the `resolve_tail_df`-derived per-lead dfs on non-survey fits, which needs the `_compute_lead_coefficients` return-path change); and the `inference_df` (DiD/SA/StackedDiD) vs `df_inference` (CS/dCDH) adapter-naming unification. NOTE (M-024): StackedDiD's `df_survey=None` on analytical fits is DELIBERATE - the container resolver's `df_inference` fallback not matching `inference_df` is load-bearing (normal-theory FLCI; see the REGISTRY StackedDiD M-024 Note and the pin in `tests/test_event_study_consumers.py`) - the unification must preserve that outcome or renegotiate it explicitly. | `diff_diff/wooldridge_results.py`, `diff_diff/imputation.py`, `diff_diff/results_base.py` | #variance-PR-C | Mid | Low | +| df-provenance completion follow-up (PR C descoping): a Wooldridge public per-row df channel (a per-key dict that can also carry the hc2_bm BM contrast dofs the scalar `_df_analytic_fallback` cannot represent — the unified surface's Wooldridge df column stays NaN until then); ImputationDiD NON-survey per-lead df (the survey-fit scalar `event_study_df` + its bootstrap clearing shipped with the M-092 completion; what remains is surfacing the `resolve_tail_df`-derived per-lead dfs on non-survey fits, which needs the `_compute_lead_coefficients` return-path change); and the `inference_df` (DiD/SA/StackedDiD/SpilloverDiD) vs `df_inference` (CS/dCDH) adapter-naming unification. NOTE (M-024): StackedDiD's `df_survey=None` on analytical fits is DELIBERATE - the container resolver's `df_inference` fallback not matching `inference_df` is load-bearing (normal-theory FLCI; see the REGISTRY StackedDiD M-024 Note and the pin in `tests/test_event_study_consumers.py`) - the unification must preserve that outcome or renegotiate it explicitly. | `diff_diff/wooldridge_results.py`, `diff_diff/imputation.py`, `diff_diff/results_base.py` | #variance-PR-C | Mid | Low | | `worktree-rm` safety via a tested argv helper: the prose rewrite (ask-before-remove confirmation gate, detached-HEAD reachability/rescue, tip-identity force-delete guard) was reverted to the main version because editing the prose repeatedly reintroduced shell-injection (last: sourcing a state file built from a git-derived branch name). Restore those guards in a `worktree_rm.py` that takes the name via file ingress, invokes git through argv arrays, and has a metacharacter-branch/path injection regression test — the pattern that worked for `pr_prepare.py`/`premerge_scan.py`. | `.claude/commands/worktree-rm.md`, `.claude/scripts/` | skill-audit | Heavy | Medium | | `premerge_scan.py` should scan the staged blob (`git show :path`) for staged methodology files, not the working-tree copy — a stage-then-revert-working-copy edit currently reads the safe working version and misses the staged violation. Union staged-index findings with unstaged/untracked filesystem findings. | `.claude/scripts/premerge_scan.py` | skill-audit | Mid | Low | | Carousel PDF drift guard: the committed deck PDFs are build artifacts of their `generate_*.py` scripts (README rule: regenerate in the same commit), but nothing verifies a committed PDF matches its generator — a source edit without regeneration, or a stale raster chart, passes every claims test. Applies to ALL decks, not just the MMM one. fpdf embeds creation-date metadata, so a byte-compare needs fixed metadata or a per-page rendered-text manifest. Origin: mmm-carousel local review R1. | `carousel/`, `tests/` | mmm-carousel | Mid | Low | diff --git a/changelog.d/20261029-v4-pr7c-m154-spillover-df-convention.md b/changelog.d/20261029-v4-pr7c-m154-spillover-df-convention.md new file mode 100644 index 000000000..661b634ef --- /dev/null +++ b/changelog.d/20261029-v4-pr7c-m154-spillover-df-convention.md @@ -0,0 +1,7 @@ +### Breaking Changes + +- `SpilloverDiD` gains the three-value `df_convention` knob with the 4.0 default `"cluster"` ([M-154], item 7c of the 4.0 cut): p-values and confidence intervals on its clustered lanes - the unit auto-cluster and an explicit `cluster=` - now use the Stata/fixest `t(G - 1)` reference like every other auto-clustered default, with `G` the effective cluster count reported as `n_clusters`. Point estimates, standard errors and t-statistics never move. `df_convention="residual"` restores the 3.x residual-df inference bit-for-bit; `"normal"` is deliberate z. `"cluster"` falls back to the residual df on `cluster=False` and on every `vcov_type="conley"` fit, an explicit product-kernel `cluster=` included (`"normal"` still selects z there); a `survey_design` df keeps precedence under every value; a saturated stage-2 sample stays all-NaN under every value. `SpilloverDiDResults` reports `df_convention` and `inference_df` (also in `to_dict()`). + +### Internal + +- Ledger: M-154 -> `done`, pinned by `tests/test_v4_spillover_df_convention.py`; the df ladder is the pure helper `spillover._resolve_inference_df` (survey df -> saturation -> `utils.resolve_tail_df`). The variance-conventions matrix gains the `spillover_cluster_unit` / `spillover_cluster_unit_default_df` rows; the REGISTRY `df_convention` knob roster, `docs/methodology/variance-conventions.md`, the migration guide, `llms-full.txt` and `docs/api/spillover.rst` describe the knob; tutorial 23 re-validated (it prints no p-values or CIs). diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index b3b41c946..734b97525 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -520,6 +520,7 @@ SpilloverDiD( conley_cutoff_km: float | None = None, conley_lag_cutoff: int | None = None, cluster: str | None | False = None, # None = auto-cluster at unit (4.0; dropped on Conley, never a survey PSU); False = observation-level HC1 (the 3.x default); str = that column + df_convention: str = "cluster", # Analytical t/p/CI df (4.0, M-154): "cluster" (G-1 on the unit auto-cluster / explicit cluster=; default), "residual" (n-k; the 3.x inference), "normal" (z); "cluster" falls back to n-k on Conley and cluster=False ("normal" still z there); survey df keeps precedence under every value; results carry df_convention / inference_df alpha: float = 0.05, anticipation: int = 0, # non-negative integer; bool rejected event_study: bool = False, # Wave C: per-event-time × ring decomposition (Butts Table 2) diff --git a/diff_diff/spillover.py b/diff_diff/spillover.py index 087d6b36f..b40913e90 100644 --- a/diff_diff/spillover.py +++ b/diff_diff/spillover.py @@ -45,9 +45,11 @@ from diff_diff.utils import ( ClusterArg, _iterative_fe_solve, + resolve_tail_df, safe_inference, validate_anticipation, validate_cluster, + validate_df_convention, ) if TYPE_CHECKING: @@ -827,7 +829,7 @@ def _extract_event_study_results( rectangular_grid: List[Tuple[str, Optional[str], int]], n_obs_per_col: np.ndarray, ref_period: int, - df_resid: int, + df_resid: Optional[float], alpha: float, ring_labels: List[str], weight_sum_per_col: Optional[np.ndarray] = None, @@ -1080,6 +1082,80 @@ def _extract_event_study_results( ) +def _resolve_inference_df( + *, + df_convention: str, + n_obs_eff: int, + k_effective: int, + n_clusters: Optional[int], + vcov_mode: str, + survey_df: Optional[int], + has_survey: bool, +) -> Optional[float]: + """Resolve the ONE tail df every SpilloverDiD ``safe_inference`` call uses. + + Precedence ladder (4.0, [M-154]): + + 1. ``survey_design`` present: ``ResolvedSurveyDesign.df_survey`` (the + standard survey DOF; ``None`` -> the ``0`` fail-closed sentinel). The + survey df beats every ``df_convention`` value, ``"normal"`` included. + 2. Saturated stage-2 sample (``n_obs_eff - k_effective <= 0``): the + residual df itself, which the guard below turns into the ``0.0`` + all-NaN sentinel under every knob value (``resolve_tail_df`` would + fall through to normal theory here - misleading on a degenerate + sample, so it is never consulted). + 3. Otherwise ``diff_diff.utils.resolve_tail_df``: ``"cluster"`` -> + ``G - 1`` with ``G`` the effective cluster count of the Wave D + ``"cluster"`` variance mode (the 4.0 unit auto-cluster or an explicit + ``cluster=``); ``G`` is forced to ``None`` (residual df) on EVERY + Conley fit - including one with an explicit product-kernel + ``cluster=`` - and on the observation-level HC1 lane + (``cluster=False``), the SunAbraham precedent; ``"residual"`` -> + ``n_obs_eff - k_effective``; ``"normal"`` -> ``None`` (z). + + Returns the value handed to ``safe_inference``: a positive float, + ``None`` (normal theory) or ``0.0`` (all-NaN inference). A non-positive + df warns with the saturation message (``stacklevel=3``: the helper is + called from ``fit``, so the warning points at the user's call site). + """ + df_resid = n_obs_eff - k_effective + df: Optional[float] + if has_survey: + df = float(survey_df) if survey_df is not None else 0.0 + elif df_resid <= 0: + df = float(df_resid) + else: + df = resolve_tail_df( + df_convention, + residual_df=float(df_resid), + n_clusters=(n_clusters if vcov_mode == "cluster" else None), + ) + if df is not None and df <= 0: + # Saturated. Either OLS-saturated (n - k <= 0) or + # survey-saturated (df_survey = 0; lonely_psu='remove' may + # have removed all strata). Force NaN inference by setting + # df = 0 (safe_inference treats df = 0 as no usable degrees + # of freedom). Distinct from df = None which would fall + # through to a normal-distribution approximation - + # misleading on a degenerate sample. + survey_note = ( + " (survey-saturated: df_survey = " + f"{int(survey_df) if survey_df is not None else 0}; " + "lonely_psu='remove' may have removed all strata)" + if has_survey + else "" + ) + warnings.warn( + f"SpilloverDiD inference df = {int(df)} (n_obs=" + f"{n_obs_eff}, effective_rank={k_effective}{survey_note}). " + "Inference (t-stat, p-value, CI) will be NaN.", + UserWarning, + stacklevel=3, + ) + return 0.0 + return df + + def _build_ring_indicators( d_values: np.ndarray, rings: List[float], @@ -1701,9 +1777,25 @@ class SpilloverDiD(BaseEstimator): on every lane: CR1, the combined Conley cluster product kernel when paired with ``vcov_type="conley"``, or the effective PSU under a ``survey_design`` without its own PSU. Column labels must be ``str``. - Inference uses the residual df on every non-survey lane, the - default CR1 lane included; under a ``survey_design`` the design df - takes precedence (``df_convention`` arrives with [M-154]). + Inference df: see ``df_convention``. + df_convention : {"residual", "cluster", "normal"}, default "cluster" + Degrees-of-freedom convention for the analytical t-statistics, + p-values and CIs (4.0, row M-154). ``"cluster"`` (default) uses the + Stata/fixest cluster df ``G - 1`` on the clustered lanes - the 4.0 + ``unit`` auto-cluster or an explicit ``cluster=`` column, with ``G`` + the effective cluster count reported as ``n_clusters``; it is inert + (residual df) on ``cluster=False`` and on EVERY ``vcov_type="conley"`` + fit, including one with an explicit ``cluster=`` (the combined + Conley+cluster product kernel has no documented ``G - 1`` df + reference). ``"residual"`` uses the stage-2 residual df + ``n_obs_eff - k_effective`` (the 3.x inference, and the 7b inference + on the default lane); ``"normal"`` deliberately uses normal-theory z + inference at the fallback level on every fit. Applies only at the + fallback level: a ``survey_design`` df always takes precedence, and + a saturated stage-2 sample is all-NaN under every value. Point + estimates, SEs and t-statistics never move - only the reference + distribution does. Results report the configuration as + ``df_convention`` and the df actually used as ``inference_df``. alpha : float, default=0.05 Significance level for confidence intervals. anticipation : int, default=0 @@ -1764,6 +1856,7 @@ def __init__( conley_cutoff_km: Optional[float] = None, conley_lag_cutoff: Optional[int] = None, cluster: ClusterArg = None, + df_convention: str = "cluster", alpha: float = 0.05, anticipation: int = 0, event_study: bool = False, @@ -1782,8 +1875,9 @@ def __init__( if vcov_type in ("hc2", "hc2_bm"): raise NotImplementedError( f"SpilloverDiD does not yet support vcov_type='{vcov_type}'. " - "The current stage-2 inference uses a generic residual df " - "(n - effective_rank) for t-distribution lookups, but " + "The stage-2 inference uses one fallback df per fit (the " + "df_convention-resolved df: t(G-1) on clustered lanes by " + "default, the residual df otherwise), but " "hc2 / hc2_bm require per-coefficient Bell-McCaffrey / CR2 " "degrees of freedom for correct p-values and CIs. Routing " "stage 2 through LinearRegression (which supplies the " @@ -1798,6 +1892,7 @@ def __init__( "vcov_type='hc1' or 'conley', or leave default; combine " "with cluster= for CR1." ) + validate_df_convention(df_convention) self.rings = rings self.d_bar = d_bar self.vcov_type = vcov_type @@ -1806,6 +1901,7 @@ def __init__( self.conley_cutoff_km = conley_cutoff_km self.conley_lag_cutoff = conley_lag_cutoff self.cluster = validate_cluster(cluster) + self.df_convention = df_convention self.alpha = alpha self.anticipation = validate_anticipation(anticipation) self.event_study = event_study @@ -2237,6 +2333,8 @@ def fit( # Same direct-mutation defense for the shared cluster contract # (str | None | False, [M-155]); the lane resolution lives below. validate_cluster(self.cluster) + # ... and for the df_convention knob ([M-154]); resolved in Step 16a. + validate_df_convention(self.df_convention) # Wave C: event-study path is now supported. Validate horizon_max # up front (fail-fast before any stage-1 work). if self.horizon_max is not None: @@ -2312,8 +2410,9 @@ def fit( if self.vcov_type in ("hc2", "hc2_bm"): raise NotImplementedError( f"SpilloverDiD does not yet support vcov_type='{self.vcov_type}'. " - "The current stage-2 inference uses a generic residual df " - "(n - effective_rank) for t-distribution lookups, but " + "The stage-2 inference uses one fallback df per fit (the " + "df_convention-resolved df: t(G-1) on clustered lanes by " + "default, the residual df otherwise), but " "hc2 / hc2_bm require per-coefficient Bell-McCaffrey / CR2 " "degrees of freedom for correct p-values and CIs. Routing " "stage 2 through LinearRegression (which supplies the " @@ -3455,16 +3554,28 @@ def fit( else: vcov = vcov_kept - # Step 16a: shared df_for_inference computation. + # Step 16a: shared df_for_inference computation - ONE df for every + # safe_inference call below (aggregate tau_total, per-ring delta_j, + # event-study per-cell tau_k / delta_jk and the scalar lincom att). + # + # Precedence ladder in `_resolve_inference_df` (4.0, [M-154]): + # 1. Wave E.1 (survey): df = resolved_survey_fit.df_survey, the + # standard survey-statistics DOF (PSU + strata -> n_PSU - n_strata; + # PSU only -> n_PSU - 1; strata only -> n_obs - n_strata; neither + # -> n_obs - 1; see `ResolvedSurveyDesign.df_survey`). The Binder + # TSL meat is design-consistent; the OLS residual df is no longer + # the right t-distribution DOF - and the survey df beats every + # df_convention value. + # 2. Saturated stage-2 sample (n_obs_eff - k_effective <= 0): warn + + # the df = 0 all-NaN sentinel under every knob value. + # 3. Otherwise the shared `df_convention` knob via + # `utils.resolve_tail_df`: "cluster" (the 4.0 default) -> G - 1 on + # the Wave D "cluster" variance mode (the unit auto-cluster or an + # explicit cluster=), residual df on the hc1 lane (cluster=False) + # and on EVERY Conley fit (G forced to None, the SunAbraham + # precedent); "residual" -> n_obs_eff - k_effective (the 3.x / 7b + # inference); "normal" -> None (z). # - # Wave D (non-survey): df = n_obs - effective_rank (OLS residual df). - # Wave E.1 (survey): df = resolved_survey_fit.df_survey, which - # encodes the standard survey-statistics DOF - # (PSU + strata → n_PSU - n_strata; PSU only → n_PSU - 1; - # strata only → n_obs - n_strata; neither → n_obs - 1; see - # `ResolvedSurveyDesign.df_survey` at survey.py:619-627). The - # Binder TSL meat is design-consistent; the OLS residual df is - # no longer the right t-distribution DOF. # Wave E.3 (codex R8 P2 fix): under the survey path, the effective # estimation sample EXCLUDES zero-weight subpop rows because they # are filtered out of the gamma_hat / Psi construction sample by @@ -3476,35 +3587,29 @@ def fit( count_mask = survey_finite_mask if resolved_survey_fit is not None else finite_mask n_obs_eff = int(count_mask.sum()) k_effective = int(np.isfinite(coef).sum()) - df_resid = n_obs_eff - k_effective - if resolved_survey_fit is not None: - df_survey_val = resolved_survey_fit.df_survey - df_for_inference: int = int(df_survey_val) if df_survey_val is not None else 0 - else: - df_for_inference = df_resid - if df_for_inference <= 0: - # Saturated. Either OLS-saturated (n - k <= 0) or - # survey-saturated (df_survey = 0; lonely_psu='remove' may - # have removed all strata). Force NaN inference by setting - # df = 0 (safe_inference treats df = 0 as no usable degrees - # of freedom). Distinct from df = None which would fall - # through to a normal-distribution approximation — - # misleading on a degenerate sample. - survey_note = ( - " (survey-saturated: df_survey = " - f"{int(df_survey_val) if df_survey_val is not None else 0}; " - "lonely_psu='remove' may have removed all strata)" - if resolved_survey_fit is not None - else "" - ) - warnings.warn( - f"SpilloverDiD inference df = {df_for_inference} (n_obs=" - f"{n_obs_eff}, effective_rank={k_effective}{survey_note}). " - "Inference (t-stat, p-value, CI) will be NaN.", - UserWarning, - stacklevel=2, - ) - df_for_inference = 0 + # Effective cluster count of the ids the Wave D meat received (G on + # the default lane, the PSU count under a PSU survey design, None on + # Conley-auto / cluster=False); feeds both the df ladder and the + # results' `n_clusters`. + _n_clusters_fit: Optional[int] = ( + int(len(np.unique(cluster_ids_fit))) if cluster_ids_fit is not None else None + ) + df_for_inference = _resolve_inference_df( + df_convention=self.df_convention, + n_obs_eff=n_obs_eff, + k_effective=k_effective, + n_clusters=_n_clusters_fit, + vcov_mode=_wave_d_vcov_mode, + survey_df=(resolved_survey_fit.df_survey if resolved_survey_fit is not None else None), + has_survey=resolved_survey_fit is not None, + ) + # Results metadata: the df the reported t/p/CI actually used - None + # under normal theory ("normal") and under the all-NaN sentinel. + _inference_df_used: Optional[float] = ( + float(df_for_inference) + if df_for_inference is not None and df_for_inference > 0 + else None + ) # Step 16b: branch on event_study mode for result extraction. att_dynamic_df: Optional[pd.DataFrame] = None @@ -3628,10 +3733,10 @@ def fit( r_squared=None, inference_method="analytical", n_bootstrap=None, - n_clusters=( - int(len(np.unique(cluster_ids_fit))) if cluster_ids_fit is not None else None - ), + n_clusters=_n_clusters_fit, vcov_type=self.vcov_type, + df_convention=self.df_convention, + inference_df=_inference_df_used, # Wave E.1: when survey.psu wins the warn-and-use-PSU override # (`_resolve_effective_cluster`), the EFFECTIVE clustering label # is `survey_design.psu`, not `self.cluster`. Report that so diff --git a/diff_diff/utils.py b/diff_diff/utils.py index 366accd81..1a7b8d91e 100644 --- a/diff_diff/utils.py +++ b/diff_diff/utils.py @@ -679,7 +679,8 @@ def resolve_tail_df( The single implementation of the ``df_convention`` knob's fallback slot for the standalone estimators (SunAbraham aggregates, WooldridgeDiD OLS, - StackedDiD, ImputationDiD pretrends leads, LPDiD). Survey/replicate df + StackedDiD, ImputationDiD pretrends leads, LPDiD, SpilloverDiD (4.0, + [M-154])). Survey/replicate df and per-coefficient Bell-McCaffrey DOF keep precedence at the CALL SITE under every value — callers consult this helper only when no higher-precedence df applies. ``LinearRegression.get_inference`` and diff --git a/docs/api/spillover.rst b/docs/api/spillover.rst index 60f6eeb6c..6ea09ff67 100644 --- a/docs/api/spillover.rst +++ b/docs/api/spillover.rst @@ -57,6 +57,11 @@ SpilloverDiDResults Results container with per-ring spillover-effect table. +Inference df: ``df_convention`` echoes the estimator's knob and ``inference_df`` +the df the reported t / p / CI used (``G - 1`` on the clustered lanes under the +4.0 default, the residual df under ``"residual"`` and on ``cluster=False`` / +Conley fits, ``None`` under ``"normal"``; the survey design df takes precedence). + .. autoclass:: diff_diff.SpilloverDiDResults :no-index: :members: @@ -493,9 +498,10 @@ and planned follow-up enhancements: - **Data-driven d_bar selection** — Butts (2021b) / Butts (2023) JUE Insight propose cross-validation under stronger parallel-trends assumptions. Not in this PR. -- **HC2 / HC2_BM (Bell-McCaffrey / CR2) variance** — current stage-2 - inference uses a generic residual-df (n - effective rank) for - t-distribution lookups. ``vcov_type="hc2"`` / ``"hc2_bm"`` require +- **HC2 / HC2_BM (Bell-McCaffrey / CR2) variance** — stage-2 inference + uses one fallback df per fit (the ``df_convention``-resolved df: t(G-1) + on clustered lanes by default, the residual df otherwise) rather than + per-coefficient DOF. ``vcov_type="hc2"`` / ``"hc2_bm"`` require per-coefficient BM / CR2 DOF and raise ``NotImplementedError``. Routing stage 2 through ``LinearRegression`` (which supplies the per-coefficient DOF metadata) is queued. diff --git a/docs/doc-deps.yaml b/docs/doc-deps.yaml index 6c3d4570d..62f8b745d 100644 --- a/docs/doc-deps.yaml +++ b/docs/doc-deps.yaml @@ -414,6 +414,11 @@ sources: - path: diff_diff/guides/llms.txt section: "Estimators" type: user_guide + - path: docs/methodology/variance-conventions.md + type: methodology + note: "Clustered-variance conventions inventory (CR1 k + tail-df map, D/L classification); table generated by tests/test_variance_conventions.py" + - path: docs/tutorials/23_spillover_tva.ipynb + type: tutorial # ── EfficientDiD (efficient_did group) ────────────────────────────── diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 5d74aea0e..af81c42f8 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -644,8 +644,12 @@ This matches the behavior of R's `fixest::feols()` with absorbed FE. (2026-07, widened 3.9 / M-127):** `df_convention ∈ {"residual", "cluster", "normal"}` on the library-wide knob roster — `DifferenceInDifferences` / `TwoWayFixedEffects` / `LinearRegression` (the TWFE event-study mode included) plus (since 3.9) `SunAbraham`, `WooldridgeDiD` (OLS - arms), `StackedDiD`, `ImputationDiD` (pretrends lead regression), and `LPDiD` (default - `"cluster"`, its Stata-anchored convention). `"cluster"` switches clustered analytical + arms), `StackedDiD`, `ImputationDiD` (pretrends lead regression), `LPDiD` (default + `"cluster"`, its Stata-anchored convention) and, since the 4.0 cut (item 7c, [M-154]), + `SpilloverDiD` - its Wave D GMM sandwich lanes make no shared-CR1 call but resolve the same + fallback df through `utils.resolve_tail_df`: `G − 1` on the unit auto-cluster / an explicit + `cluster=`, inert on Conley (an explicit product-kernel cluster included) and on + `cluster=False`. `"cluster"` switches clustered analytical t/p/CI to the Stata/fixest `G − 1` convention and is inert on unclustered and conley fits; `"normal"` is deliberate normal-theory z at the fallback level on EVERY fit (clustered, unclustered, and conley alike — on lanes that were already t it is a new deliberate option, @@ -656,8 +660,7 @@ This matches the behavior of R's `fixest::feols()` with absorbed FE. `docs/v4-deprecations.yaml` M-004..M-006 + M-128..M-131, pinned by `tests/test_v4_inference_defaults.py`; the flip moved every clustered p-value/CI, which is why it waited for a major version), and `"residual"` restores the 3.x residual-df inference - bit-for-bit. **Out of the knob's scope** (own inference stacks): `SpilloverDiD` - (until [M-154], item 7c - its default CR1 lane runs residual-df t at 7b), the IF-based + bit-for-bit. **Out of the knob's scope** (own inference stacks): the IF-based family — `CallawaySantAnna`, `TwoStageDiD`, `ChaisemartinDHaultfoeuille`, `EfficientDiD`, `ContinuousDiD`, `ChangesInChanges`, `SyntheticDiD`, `HeterogeneousAdoptionDiD` — and the OLS-based `TripleDifference` / `TROP` (own `safe_inference` df conventions, outside the @@ -7074,12 +7077,27 @@ should be a deliberate user choice. shared `str | None | False` contract, [M-155]). Results report `cluster_name="unit"` / `n_clusters=G` on the default lane. Roster in `docs/methodology/variance-conventions.md` "Default clustering roster". -- **Note:** inference df unchanged at 7b - `t(n_obs_eff - k_effective)` on every - non-survey lane INCLUDING the new default CR1 lane (the survey-design df keeps its - precedence under `survey_design=`, as before); SpilloverDiD has no `df_convention` - knob yet, so it is the one auto-clustered default not on t(G-1); [M-154] / item 7c - adds the knob with the 4.0 default `"cluster"` (t(G-1) on the clustered lanes, inert - on Conley, survey df still first). +- **Note (inference df, [M-154], item 7c, 2026-10-29):** ONE df feeds every + `safe_inference` call (aggregate `tau_total`, per-ring `delta_j`, event-study per-cell + `tau_k` / `delta_jk` and the scalar lincom `att`), resolved by the pure helper + `_resolve_inference_df` (`diff_diff/spillover.py`) with the precedence ladder: (1) a + `survey_design` df (`ResolvedSurveyDesign.df_survey`) wins under every knob value; (2) a + saturated stage-2 sample (`n_obs_eff - k_effective <= 0`) warns and forces the `df = 0` + all-NaN sentinel under every value (never normal theory); (3) otherwise the shared + three-value `df_convention` knob via `utils.resolve_tail_df` - `"cluster"` (the 4.0 + default) = `t(G − 1)` with `G` the effective cluster count of the Wave D `"cluster"` + variance mode (the unit auto-cluster or an explicit `cluster=`; reported as + `n_clusters`), `G` forced to `None` (residual df) on EVERY Conley fit - an explicit + product-kernel `cluster=` included, the SunAbraham precedent - and on the `cluster=False` + HC1 lane; `"residual"` = `t(n_obs_eff - k_effective)` (the 3.x and 7b inference); + `"normal"` = z. Point estimates, SEs and t-statistics never move. Results carry + `df_convention` (the configuration) and `inference_df` (the df actually used; `None` + under z and the NaN sentinel). Pre-existing provenance gap, unchanged here: the + event-study container's `results_base._resolve_scalar_df_survey` reads a `df_inference` + attribute while `DiDResults` carriers expose `inference_df`, so a non-survey Spillover + event-study container reports `df_survey=None` (tracked in `TODO.md`, the df-provenance + completion row). Pinned by `tests/test_v4_spillover_df_convention.py` and the + variance-conventions rows `spillover_cluster_unit` / `spillover_cluster_unit_default_df`. - **Note (rank-guarded Wave D bread):** The Wave D variance bread `A_22 = (X_2'WX_2)^{-1}` is inverted by the shared `_rank_guarded_inv` (`diff_diff/linalg.py`) on the already-column-dropped kept submatrix. `np.linalg.solve` raised only on an exactly-singular bread (prior fallback: dense `lstsq`); a **near**-singular kept Gram would otherwise flow a garbage inverse (~1e13) into the SE. The rank-guard truncates redundant directions → finite SE on the identified subspace (NaN at rank 0), re-inflated to (k, k) with NaN at the dropped positions, and warns. See the CallawaySantAnna "rank-guarded IF standard errors" Note. Sibling of axis-A finding #17. **Identification spec (committed):** @@ -7187,7 +7205,7 @@ The correction applies unconditionally (no opt-out kwarg). Point estimates (`tau The `_compute_stratified_meat_from_psu_scores` helper at `diff_diff/survey.py` implements the Binder TSL formula and handles `lonely_psu ∈ {"remove", "certainty", "adjust"}` and FPC consistently across the package. -Degrees of freedom for the t-distribution lookup use `ResolvedSurveyDesign.df_survey` (the standard survey 4-way branch: PSU+strata → `n_PSU - n_strata`; PSU only → `n_PSU - 1`; strata only → `n_obs - n_strata`; neither → `n_obs - 1`). Threaded through all four `safe_inference` call sites: aggregate `tau_total`, per-ring `delta_j`, event-study per-event-time `tau_k` / `delta_jk`, and the scalar `att` lincom in event-study mode. +Degrees of freedom for the t-distribution lookup use `ResolvedSurveyDesign.df_survey` (precedence over the `df_convention` knob under every value, [M-154]; the standard survey 4-way branch: PSU+strata → `n_PSU - n_strata`; PSU only → `n_PSU - 1`; strata only → `n_obs - n_strata`; neither → `n_obs - 1`). Threaded through all four `safe_inference` call sites: aggregate `tau_total`, per-ring `delta_j`, event-study per-event-time `tau_k` / `delta_jk`, and the scalar `att` lincom in event-study mode. - **Note (documented synthesis):** Wave E.1 composes Gerber (2026, arXiv:2605.04124) Proposition 1 — Binder Taylor Series Linearization for IF representations of smooth functionals; explicitly derived for TwoStageDiD in the paper's Appendix — with the Wave D Gardner GMM first-stage uncertainty correction (Butts 2021 §3.1 + Gardner 2022 §4) applied to SpilloverDiD's ring-indicator stage-2 design. The composition is mechanical: SpilloverDiD's Wave D Psi is aggregated to PSU level and passed to the audited Binder TSL meat helper. Survey weights enter via Hájek normalization at the gamma_hat solve, eps construction, and bread inversion. No reference software combines all ingredients; Wave E.2 extends with the Conley × survey product-kernel composition (cross-sectional `conley_lag_cutoff = 0`) — see "Variance (Wave E.2)" subsection below; Wave E.2 follow-up further extends with the within-PSU serial Bartlett HAC for `conley_lag_cutoff > 0` — see "Variance (Wave E.2 follow-up)" subsection below. - **Note (warn-and-use-PSU for cluster + survey):** when both `cluster=` and `survey_design.psu` are supplied with **different groupings**, the cluster argument emits a `UserWarning` and is overridden by PSU (mirrors `TwoStageDiD._resolve_effective_cluster`). PSU is the design-relevant cluster on survey panels; `cluster=` on SpilloverDiD is more often a spatial / unit-level label, so the design constraint wins. When both knobs are supplied with the **same** groupings, no warning fires and PSU still takes precedence (the inference is unchanged either way). @@ -7273,7 +7291,7 @@ Degrees of freedom for the t-distribution lookup use `ResolvedSurveyDesign.df_su - **Restrictions / unchanged from Wave E.1/E.2/follow-up:** - Replicate-weight variance (BRR / Fay / JK1 / JKn / SDR) raises `NotImplementedError` (Wave E.1 gate at `spillover.py:2400` inherited; per-replicate refit + subpopulation is a separate follow-up). - TwoStageDiD's analogous always-treated design-subset pattern was shipped in a follow-up PR (TwoStageDiD Wave E.3 parity) — see "Note (documented synthesis — Wave E.3 parity, full-domain survey design under always-treated drop)" in the TwoStageDiD section above. The two estimators now share the full-design retention contract; the trigger differs (TwoStageDiD: always-treated unit detection; SpilloverDiD: `finite_mask` warn-and-drop / subpopulation zero-weight rows) but the invariant is identical (zero-pad scores at the meat-helper boundary; retain full-domain `resolved_survey.psu / strata / fpc`). - - `df_for_inference` at `spillover.py:3229-3234` reads `resolved_survey_fit.df_survey` which is the full-domain value under Wave E.3 (since `resolved_survey_fit = resolved_survey`). All four `safe_inference` call sites (aggregate `tau_total`, per-ring `delta_j`, event-study per-event-time `tau_k` / `delta_jk`, scalar `att` lincom) inherit the full-domain df. + - the shared df ladder `_resolve_inference_df` (`diff_diff/spillover.py`) reads `resolved_survey_fit.df_survey` first, which is the full-domain value under Wave E.3 (since `resolved_survey_fit = resolved_survey`). All four `safe_inference` call sites (aggregate `tau_total`, per-ring `delta_j`, event-study per-event-time `tau_k` / `delta_jk`, scalar `att` lincom) inherit the full-domain df. **Implementation:** `spillover.py:2845-2896` design-subset block deleted; `survey_weights_fit = survey_weights[finite_mask]` retained for the stage-2 OLS solve (which still operates on the fit sample); `cluster_ids_full[finite_mask]` subset dropped on the survey path (`cluster_ids_fit = cluster_ids_full` under survey). The `_compute_gmm_corrected_meat` call at `spillover.py:3163` is rewired to receive SURVEY-FINITE-MASK gamma_hat-construction inputs (`X_1_sparse_fit`, `X_10_sparse_fit`, `eps_10_fit` built on `survey_finite_mask`; `X_2_kept_gamma`, `eps_2_fit_gamma`, `survey_weights_fit_gamma` projected from the fit-sample frame down to survey_finite_mask) plus FULL-LENGTH kernel-dispatch arrays (`cluster_ids_for_meat`, `conley_coords_for_meat`, `conley_time_for_meat`, `conley_unit_for_meat`, `resolved_survey_fit`) plus the new `score_pad_mask=survey_finite_mask` kwarg; no-survey path passes `score_pad_mask=None` and uses fit-length variables throughout (bit-identical to pre-E.3). `_compute_gmm_corrected_meat` at `two_stage.py` adds one new optional kwarg `score_pad_mask: Optional[np.ndarray] = None` and one post-Psi-construction zero-pad block; the `_validate_conley_kwargs` call uses `n_for_conley = len(score_pad_mask)` when the kwarg is set. Within-unit-constancy validator at `spillover.py:2913` updated to operate on full-length unit array (matches full-length cluster array under Wave E.3). Second `compute_survey_metadata` recompute at `spillover.py:2954-2959` now uses full-length `raw_w` (no `[finite_mask]` subset). Tests: `TestSpilloverDiDWaveE3SubpopulationFullDesign` + `TestSpilloverDiDWaveE3SubpopulationFullDesignEventStudy` at `tests/test_spillover.py`. @@ -7295,7 +7313,7 @@ Degrees of freedom for the t-distribution lookup use `ResolvedSurveyDesign.df_su - `survey_design=` for `vcov_type ∈ {"hc1"}` (plus `cluster=` for CR1) SHIPPED in Wave E.1 — see "Variance (Wave E.1)" subsection below. Threads Hájek-normalized survey weights through stage-1 FE estimation, gamma_hat solve, eps construction, and bread inversion; aggregates the Wave D Psi to PSU totals and routes through the audited `_compute_stratified_meat_from_psu_scores` Binder TSL meat helper. `vcov_type="conley"` combined with `survey_design=` SHIPPED in Wave E.2 for cross-sectional Conley (`conley_lag_cutoff = 0`) — see "Variance (Wave E.2)" subsection below (stratified-Conley sandwich on PSU totals). Wave E.2 follow-up adds the panel-block composition (`conley_lag_cutoff > 0`) via spatial + serial Bartlett HAC — see "Variance (Wave E.2 follow-up)" subsection below. `SurveyDesign.subpopulation()` and warn-and-drop full-design retention via zero-pad scores SHIPPED in Wave E.3 — see "Variance (Wave E.3)" subsection below (matches R `survey::svyrecvar(subset())` + in-library precedent at `imputation.py:2175-2183` and `prep.py:1401-1432`). Replicate-weight variance (BRR / Fay / JK1 / JKn / SDR) raises `NotImplementedError` — Gerber (2026) Appendix A notes the IF-reweighting shortcut does NOT apply to TwoStageDiD-class estimators because `gamma_hat` is weight-sensitive; correct support requires per-replicate full re-fit and is queued as a follow-up. - `covariates=` raises `NotImplementedError` — Gardner-style stage-1 residualization not yet wired through; planned follow-up. - `ring_method="count"` not exposed — only the nearest-treated-ring specification. -- `vcov_type` ∈ {`"hc2"`, `"hc2_bm"`, `"hc3"`, `"classical"`} raises `NotImplementedError` — `hc2`/`hc2_bm` (at construction) because current stage-2 inference uses generic residual df rather than per-coefficient Bell-McCaffrey / CR2 DOF; `hc3` (at construction) because it is not implemented for the two-stage spillover variance; `classical` (at fit) because the Wave D Gardner GMM first-stage correction has not been derived for the classical homoskedastic variance (different meat structure `sigma_hat^2 * (X_10' X_10)` vs the Wave D IF outer product `Psi' Psi`). Use `"hc1"` or `"conley"`, or pair with `cluster=` for CR1 — all three apply the Wave D GMM correction. +- `vcov_type` ∈ {`"hc2"`, `"hc2_bm"`, `"hc3"`, `"classical"`} raises `NotImplementedError` — `hc2`/`hc2_bm` (at construction) because stage-2 inference uses one fallback df per fit (the `df_convention`-resolved df, [M-154]) rather than per-coefficient Bell-McCaffrey / CR2 DOF; `hc3` (at construction) because it is not implemented for the two-stage spillover variance; `classical` (at fit) because the Wave D Gardner GMM first-stage correction has not been derived for the classical homoskedastic variance (different meat structure `sigma_hat^2 * (X_10' X_10)` vs the Wave D IF outer product `Psi' Psi`). Use `"hc1"` or `"conley"`, or pair with `cluster=` for CR1 — all three apply the Wave D GMM correction. - **`rings[0]` must equal 0** — the partition must cover treated locations (`d_it = 0` belongs to Ring 1). Rings starting at a nonzero inner edge would leave units in `0 <= d_it < rings[0]` as exposed-but-unmodeled, silently biasing the estimator. Validator rejects such inputs. - **Balanced panel required (Wave B MVP)** — every unit must observe every period. An unbalanced (unit, time) Ω₀ bipartite graph can produce disconnected FE components and unidentified stage-1 residuals on treated rows. Exact graph-connectivity-based identification (which would relax this to a strictly weaker condition) is queued as a follow-up extension. Validator rejects unbalanced inputs. - **One row per `(unit, time)` cell required** — duplicate cells silently re-weight stage-1 FE estimation AND stage-2 OLS. Validator rejects duplicate cells. diff --git a/docs/methodology/variance-conventions.md b/docs/methodology/variance-conventions.md index f7692f0d0..c3ec2ddd3 100644 --- a/docs/methodology/variance-conventions.md +++ b/docs/methodology/variance-conventions.md @@ -19,6 +19,11 @@ y = N(0,1) + 0.3 unit + 0.2 t + 2 treated; n = 360, G = 60 under cluster="unit". Each row's exact fit kwargs are in the test file — the k values and SEs in any claim below come from the SAME fit (mixing panels produced wrong figures three separate times while this inventory was drafted). +The two SpilloverDiD rows fit the same panel with a deterministic coordinate +overlay (`_with_coords`): ever-treated units at lat = 0.001 x unit, lon = 0 (a +~7 km cluster), never-treated units alternately at lat 0.5 (~55 km, inside the +100 km ring) and lat 3.0 (~330 km, the far-away Omega_0 controls); +rings=[0, 100]. | surface | CR1 `k` (multiset) | tail df (multiset) | status | reason | |---|---|---|---|---| @@ -41,6 +46,7 @@ produced wrong figures three separate times while this inventory was drafted). | `dml_did_bad_control` | — | None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None | **legitimate** | L3: Caetano et al. (2026) Eq. 11 augmented-score plug-in variance (same psi_bar - D*theta/pi centering as Chang Thm 2; per-unit influence function; normal-theory safe_inference on no-design fits; cluster= fits use the CR1 per-cell variance with df=df_survey t-inference) and the per-cell ATT_X diagnostic tuple sharing the same branch and df (one extra safe_inference call per retained cell) | | `dml_did_rcs` | — | None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None | **legitimate** | L3: Chang (2020) Thm 2 lambda-corrected augmented-score plug-in variance (Case 2; per-observation influence function; normal-theory safe_inference on no-design fits; survey_design=/cluster= fits use the design-based CR1/weighted-IF per-cell variance — replicate designs the IF-reweighting per-cell variance — with df=df_survey t-inference outside the shared CR1 sandwich) | | `duration_did` | — | None, None | **legitimate** | L3: Deaner & Ku (2026) whole-individual pooled bootstrap (centered absolute-deviation pointwise and simultaneous bands); no CR1 sandwich, normal-theory safe_inference gate only | +| `spillover_cluster_unit` | — | 358, 358 | **legitimate** | L3: Wave D Gardner-GMM two-stage sandwich with its own CR1 multiplier G/(G-1)*(n-1)/(n-p_2) (two_stage meat, never the shared CR1). Explicit df_convention='residual' since the 4.0 default is G-1 (see the _default_df twin). | | `did_absorb_hc1_cluster_unit_default_df` | 7 | 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 | | `twfe_hc1_cluster_unit_default_df` | 3 | 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 | | `wooldridge_hc1_within_default_df` | 15 | 59, 59, 59, 59, 59, 59, 59, 59, 59, 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 | @@ -49,6 +55,7 @@ produced wrong figures three separate times while this inventory was drafted). | `twfe_es_within_hc1_cluster_unit_default_df` | 11 | 59, 59, 59, 59, 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 | | `twfe_es_pooled_hc1_cluster_time_default_df` | 7 | 5, 5, 5, 5, 5 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 | | `imputation_pretrends_event_study_default_df` | unpinned | 59, 59, 59, None, None, None, None, None | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 | +| `spillover_cluster_unit_default_df` | — | 59, 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-154 - SpilloverDiD gained the knob at the cut) - no shared-CR1 call, as the explicit-'residual' twin; tail df G-1 | cr1_k is the sorted multiset of K_reference counts reaching the shared clustered CR1 denominator — visible columns + the signed cluster_k_adjustment @@ -139,11 +146,15 @@ output). library-wide 4.0 flip — bit-identical, no numbers moved there), with "residual"/"normal" as opt-ins; the degenerate lanes (unclustered refit, RA G<=1, saturated early return) keep literal df=None under every value. -- **L3 — CallawaySantAnna / DMLDiD / TwoStageDiD / ImputationDiD (default) / - DurationDiD**: different variance theory (influence functions / Chang (2020) - augmented-score plug-in / two-stage / BJS imputation / Deaner & Ku (2026) - whole-individual pooled bootstrap with centered absolute-deviation bands), - never the shared CR1 sandwich. DurationDiD is bootstrap-only: its single +- **L3 — CallawaySantAnna / DMLDiD / TwoStageDiD / SpilloverDiD / ImputationDiD + (default) / DurationDiD**: different variance theory (influence functions / Chang + (2020) augmented-score plug-in / two-stage / Gardner two-stage + Wave D GMM + first-stage correction / BJS imputation / Deaner & Ku (2026) whole-individual + pooled bootstrap with centered absolute-deviation bands), never the shared CR1 + sandwich. SpilloverDiD's CR1 lane applies its own `G/(G-1) * (n-1)/(n-p_2)` + multiplier inside `two_stage._compute_gmm_corrected_meat`, outside the shared + sandwich, and since 4.0 ([M-154]) resolves its tail df through the shared + `df_convention` knob (its two matrix rows). DurationDiD is bootstrap-only: its single `safe_inference` + `safe_inference_batch` gate passes `df=None` and the centered-bootstrap p-values/intervals override the normal-theory values. CS is anchored to Stata csdid outright; DMLDiD's panel lane to DoubleML at @@ -172,7 +183,12 @@ ONE three-value knob (`df_convention ∈ {"residual", "cluster", "normal"}`) now governs the analytical fallback df on every shared-CR1 surface: DiD/TWFE/LinearRegression + SunAbraham/WooldridgeDiD-OLS/StackedDiD/ ImputationDiD-pretrends and LPDiD, all at the default "cluster" (the -Stata-anchored t(G−1); LPDiD since 3.9, the rest since the 4.0 cut) with +Stata-anchored t(G−1); LPDiD since 3.9, the rest since the 4.0 cut) - plus, +since the 4.0 cut (item 7c, [M-154]), SpilloverDiD, whose Wave D GMM sandwich +lanes make no shared-CR1 call (`cr1_k=()`) but resolve the same fallback df +through `resolve_tail_df`: G−1 on the unit auto-cluster / an explicit cluster=, +inert on Conley (an explicit product-kernel cluster included) and on +cluster=False - with "residual" restoring the 3.x residual df. Survey/replicate df and hc2_bm Bell-McCaffrey DOF keep precedence under every value; "cluster" fails closed warn+NaN at G<=1 and is inert on unclustered/conley fits; "normal" is deliberate z at @@ -211,7 +227,7 @@ column lookup. | Class | `cluster` param? | Default when omitted (v3.13.0 -> 4.0) | `cluster=False` since 4.0 | Category | |---|---|---|---|---| -| SpilloverDiD | yes | observation-level HC1 -> **CR1 at unit since 4.0** (`"unit"`, G; TWFE's carve-outs mirrored; inference df still the residual df until [M-154]) | plain HC1 SE (= the 3.x default) | **a** - default flipped to unit | +| SpilloverDiD | yes | observation-level HC1 -> **CR1 at unit since 4.0** (`"unit"`, G; TWFE's carve-outs mirrored; inference df t(G-1) since 4.0, [M-154]) | plain HC1 SE (= the 3.x default) | **a** - default flipped to unit | | TwoWayFixedEffects (static) | yes | CR1 at unit (`cluster_name="unit"`; reports `n_clusters` since 4.0 on non-survey analytical fits; the survey-lane auto label is None since 4.0) | plain HC sandwich (absorbed-FE full-K rescale, residual-df t; one-way CR2-BM under `hc2_bm`); rejected under `inference="wild_bootstrap"` | c1 | | TwoWayFixedEffects (event-study) | yes | CR1 at unit since 3.9 (M-010) | plain HC sandwich (the core's unclustered lanes) | c1 | | SunAbraham | yes | CR1 at unit (reports `cluster_name="unit"` and the appended `n_clusters` since 4.0) | plain HC sandwich (residual-df t); no unit PSU under `survey_design=`; pairs bootstrap unaffected; rejected with `hc2_bm` | c1 | @@ -249,5 +265,5 @@ than open an unverified variance: `cluster=False` + `vcov_type="hc2_bm"` on SunAbraham / WooldridgeDiD (their Bell-McCaffrey DOF helper is cluster-only; the one-way port is a `TODO.md` row - TWFE's one-way CR2-BM lane already exists), and the bootstraps that draw AT the analytical cluster (DiD/TWFE wild, WooldridgeDiD -multiplier). SpilloverDiD's new default lane keeps residual-df inference until -[M-154] (item 7c) adds its `df_convention` knob. +multiplier). SpilloverDiD's default lane moved from residual-df to t(G-1) +inference when [M-154] (item 7c, 2026-10-29) added its `df_convention` knob. diff --git a/docs/migration-4.0.md b/docs/migration-4.0.md index c35f4f160..1d1145912 100644 --- a/docs/migration-4.0.md +++ b/docs/migration-4.0.md @@ -307,6 +307,8 @@ Two changes alter results without changing any call: leads); `LPDiD` was already there. Clustered analytical p-values and confidence intervals now use the Stata/fixest `t(G − 1)` reference instead of the fitted residual df `t(n − K)`; point estimates, standard errors and t-statistics never move. + (`SpilloverDiD` gained the knob at 4.0 with the same `"cluster"` default, [M-154] - no + flip, a new surface; see the bullet below.) ```python TwoWayFixedEffects().fit(df, outcome="y", treatment="d", post="post", unit="unit") @@ -381,9 +383,14 @@ Two changes alter results without changing any call: auto-cluster is never injected as a PSU, so the former `"unit"` label described ids the regression never received); summaries render ", G=..." and `SunAbraham` summaries gain a "Std. errors:" line on analytical non-survey fits. -- **`SpilloverDiD` p-values and CIs on the new default lane stay on the residual df** in - this release step; the next PR adds its `df_convention` knob with the 4.0 `"cluster"` - default ([M-154]), which moves them to t(G-1) like every other auto-clustered default. +- **`SpilloverDiD` p-values and CIs on its clustered lanes move to t(G − 1)** ([M-154]): + `SpilloverDiD` gained the three-value `df_convention` knob with the 4.0 `"cluster"` + default, so the unit auto-cluster and an explicit `cluster=` now use the same reference + distribution as every other auto-clustered default. `SpilloverDiD(df_convention="residual")` + restores the 3.x residual-df inference bit-for-bit; `"cluster"` falls back to the residual df + on `cluster=False` and Conley fits (`"normal"` still selects z there); the survey design df + keeps precedence under every value. Results + report `df_convention` and `inference_df`. ## Removed functions and aliases @@ -401,8 +408,8 @@ Bacon roster re-homing, and the family-wide `anticipation` validation below. The the ledger-derived removals/flips; [M-144], [M-145], and [M-146] are behavior tightenings, [M-147] is an additive policy, and [M-153] is a results-frame reshape. These have no removal/deprecation fields and appear here only. [M-155] (the shared `cluster` -argument contract, see "Inference defaults that move numbers") is the same kind of -behavior tightening. +argument contract) and [M-154] (the `SpilloverDiD` `df_convention` knob), both in +"Inference defaults that move numbers", are the same kind of behavior tightening. - `WooldridgeDiD(unsupported_period_action="error")` ([M-147]) refuses periods lacking comparison support before removing them. The default `"drop"` preserves filtering and @@ -602,7 +609,7 @@ does not mean no action is required, so read the `Fix` cell. | M-002 | obligation-sdid-params | `diff_diff:SyntheticDiD[zeta]` | — | Drop it. It has been IGNORED since 3.0.0, so 3.x already auto-computes regularization - do NOT copy its value into `zeta_lambda=`, which activates an override that was inert and changes weights, ATT and inference. Set `zeta_lambda=` only as a deliberate new choice. | | M-003 | obligation-sdid-params | `diff_diff:SyntheticDiDResults.placebo_effects` | `diff_diff:SyntheticDiDResults.variance_effects` | Read `variance_effects` instead of `placebo_effects`. | | M-007 | obligation-warning-retirements | `diff_diff.estimators` | — | The MultiPeriodDiD `e=-1` transition FutureWarning is no longer emitted (the class is gone; the event-study mode defaults to the last pre-period silently); drop any warning filter that suppressed it. | -| M-080 | policy-auto-cluster | `diff_diff:SpilloverDiD[cluster]` | — | `SpilloverDiD` now clusters at `unit` when `cluster=` is omitted (TWFE's carve-outs mirrored: explicit `cluster=` wins, dropped on the Conley path, never injected as a survey PSU; its p-values/CIs keep the residual df until [M-154]); pass `cluster=False` for the 3.x observation-level HC standard errors (`None`/omission means auto-cluster, not off). Every other panel estimator already clustered at unit or is unit-level by construction, so its numbers do not move. `cluster=False` is also newly accepted on `TwoWayFixedEffects`/`SunAbraham`/`WooldridgeDiD`/`LPDiD` (plain HC sandwich - new behaviour; residual-df t), as a no-op on the by-construction estimators, on `ImputationDiD`/`TwoStageDiD` and on the 2x2 `DifferenceInDifferences` / 2x2x2 `TripleDifference` surfaces; `StackedDiD` still rejects it. Rejected with `inference="wild_bootstrap"` (DiD/TWFE), `WooldridgeDiD(n_bootstrap > 0)` and `vcov_type="hc2_bm"` on `SunAbraham`/`WooldridgeDiD`. `cluster` accepts `str | None | False` only ([M-155]). | +| M-080 | policy-auto-cluster | `diff_diff:SpilloverDiD[cluster]` | — | `SpilloverDiD` now clusters at `unit` when `cluster=` is omitted (TWFE's carve-outs mirrored: explicit `cluster=` wins, dropped on the Conley path, never injected as a survey PSU; its p-values/CIs use t(G − 1) since [M-154], item 7c); pass `cluster=False` for the 3.x observation-level HC standard errors (`None`/omission means auto-cluster, not off). Every other panel estimator already clustered at unit or is unit-level by construction, so its numbers do not move. `cluster=False` is also newly accepted on `TwoWayFixedEffects`/`SunAbraham`/`WooldridgeDiD`/`LPDiD` (plain HC sandwich - new behaviour; residual-df t), as a no-op on the by-construction estimators, on `ImputationDiD`/`TwoStageDiD` and on the 2x2 `DifferenceInDifferences` / 2x2x2 `TripleDifference` surfaces; `StackedDiD` still rejects it. Rejected with `inference="wild_bootstrap"` (DiD/TWFE), `WooldridgeDiD(n_bootstrap > 0)` and `vcov_type="hc2_bm"` on `SunAbraham`/`WooldridgeDiD`. `cluster` accepts `str | None | False` only ([M-155]). | | M-032 | renames-cohort | `diff_diff:WooldridgeDiD.fit[cohort]` | `diff_diff:WooldridgeDiD.fit[first_treat]` | Rename the keyword: `cohort=` becomes `first_treat=`. | | M-035 | renames-col-suffix | `diff_diff:HeterogeneousAdoptionDiD.fit[outcome_col]` | `diff_diff:HeterogeneousAdoptionDiD.fit[outcome]` | Rename the keyword: `outcome_col=` becomes `outcome=`. | | M-036 | renames-col-suffix | `diff_diff:HeterogeneousAdoptionDiD.fit[dose_col]` | `diff_diff:HeterogeneousAdoptionDiD.fit[dose]` | Rename the keyword: `dose_col=` becomes `dose=`. | diff --git a/docs/tutorials/23_spillover_tva.ipynb b/docs/tutorials/23_spillover_tva.ipynb index 59755a2a9..4d641a92c 100644 --- a/docs/tutorials/23_spillover_tva.ipynb +++ b/docs/tutorials/23_spillover_tva.ipynb @@ -383,7 +383,10 @@ "spillover-aware estimator removes the bias the naive TWFE suffered.\n", "The standard errors in the table are the 4.0 default: cluster-robust\n", "(CR1) at the unit level, like `TwoWayFixedEffects` - `SpilloverDiD`\n", - "auto-clusters at `unit` when `cluster=` is omitted.\n", + "auto-clusters at `unit` when `cluster=` is omitted. Its p-values and\n", + "confidence intervals use the t(G-1) reference distribution\n", + "(`df_convention=\"cluster\"`, the 4.0 default, with G = 200 units);\n", + "`df_convention=\"residual\"` restores the residual-df inference.\n", "\n", "The interpretation is straightforward: every period in the post window,\n", "treated units lose 7.4 units of $y$ on average from the policy, AND\n", diff --git a/docs/v4-deprecations.yaml b/docs/v4-deprecations.yaml index dc364e3b2..8da38d381 100644 --- a/docs/v4-deprecations.yaml +++ b/docs/v4-deprecations.yaml @@ -1058,7 +1058,7 @@ rows: phase: 5 test_ref: tests/test_v4_cluster_policy.py code_refs: [diff_diff/spillover.py, diff_diff/twfe.py, diff_diff/estimators.py, diff_diff/sun_abraham.py, diff_diff/lpdid.py, diff_diff/stacked_did.py, diff_diff/wooldridge.py, diff_diff/imputation.py, diff_diff/two_stage.py, diff_diff/staggered.py, diff_diff/efficient_did.py, diff_diff/dml_did.py, diff_diff/lwdid.py, diff_diff/had.py, diff_diff/triple_diff.py, diff_diff/_staggered_triple_diff_engine.py, diff_diff/chaisemartin_dhaultfoeuille.py, docs/methodology/variance-conventions.md] - notes: "DONE at the 4.0 cut (item 7b, 2026-10-29): SpilloverDiD auto-clusters at unit when cluster= is omitted (TWFE carve-outs mirrored: explicit cluster= wins, dropped on the Conley path, never injected as a survey PSU; inference df unchanged - residual df - pending [M-154]); uniform cluster=False via utils.validate_cluster - the shared contract is str | None | False ([M-155]), so non-str column labels are rejected everywhere (DMLDiD existing rule generalized) - plain HC on a/c1 incl. the new LPDiD / WooldridgeDiD-GLM unclustered lanes, the Wooldridge within-transform absorbed-FE full-K rescale and TWFE one-way hc2_bm; accepted no-op on b/c2 and the 2x2 surfaces (incl. the CS RC auto-PSU path, EfficientDiD.hausman_pretest and the LWDiD sensitivity helpers); rejected on StackedDiD (own vocabulary); the DiD/TWFE wild and WooldridgeDiD multiplier bootstraps and the cluster-only hc2_bm lanes of SunAbraham / WooldridgeDiD reject it (SunAbraham pairs bootstrap: accepted no-op, the hc2 precedent); metadata: DiD/TWFE analytical n_clusters from the regression (non-survey fits), SunAbrahamResults.n_clusters (appended) + cluster_name on clustered analytical fits, TWFE survey auto-label corrected to None. Pinned by tests/test_v4_cluster_policy.py; notebook 23 re-executed. RE-SCOPED 2026-10-18 (4.0-cut framing, user decision; measured roster in docs/methodology/variance-conventions.md and spec section 7). The original premise - static TwoWayFixedEffects runs HC-robust by default - was already false: static TWFE auto-clusters at unit (twfe.py, since 2026-01). Measured categories: (a) SpilloverDiD is the ONE required-unit estimator whose default SE moves (observation-level HC -> unit clustering, mirroring TWFE's carve-outs: explicit cluster= wins, Conley drops it, never injected as survey PSU); (c1) TWFE/SunAbraham/WooldridgeDiD/LPDiD already cluster at unit by regression-level default - cluster=False yields the plain HC sandwich; (c2) ImputationDiD/TwoStageDiD default cluster_var=unit INSIDE unit-level influence-function machinery - no unclustered variance exists, cluster=False is an accepted explicit no-op; (c3) StackedDiD clusters by construction and its validator accepts only unit/unit_subexp - cluster=False is REJECTED (REGISTRY Note); (b) CS/EfficientDiD/DMLDiD/dCDH/staggered-DDD (per-unit IF) and LWDiD/HAD (regression on a unit-level cross-section) are unit-level by construction - cluster=False is an accepted explicit no-op; (d) ContinuousDiD/DurationDiD/SDiD/SC/TROP/CiC have no cluster param; (n/a) cross-sectional DiD/2x2x2 DDD/RDD stay HC-robust or unclustered. Categories b and d carry NO auto-cluster metadata by decision (cluster_name stays None where the field exists; no field is added where it does not). The 4.0 flip therefore = the SpilloverDiD default + the uniform cluster=False off-switch on a/b/c1/c2 (today accepted nowhere: LPDiD's `self.cluster or unit` treats False as auto-cluster, most others look up a column named False, DMLDiD/StackedDiD reject on type, dCDH raises on any non-None) + metadata gaps (TWFE/DiD n_clusters on the analytical path; SunAbraham cluster_name + n_clusters). ORIGINAL NOTE: ONE auto-cluster policy at 4.0: panel estimators (required unit column) default to clustering at unit with cluster_name/n_clusters metadata; cluster=False disables; 2x2 cross-sectional estimators stay HC-robust. Changes some default SEs - migration guide with [M-004..M-006]. SCOPE NOTE (Phase 3(a), user decision 2026-08-07): the TWFE EVENT-STUDY mode shipped already auto-clustered at 3.9 (new API adopts the end-state immediately, with the static carve-outs mirrored - see [M-010]), so this 4.0 flip does not re-touch it." + notes: "DONE at the 4.0 cut (item 7b, 2026-10-29): SpilloverDiD auto-clusters at unit when cluster= is omitted (TWFE carve-outs mirrored: explicit cluster= wins, dropped on the Conley path, never injected as a survey PSU; inference df moved to t(G-1) by [M-154] in 7c); uniform cluster=False via utils.validate_cluster - the shared contract is str | None | False ([M-155]), so non-str column labels are rejected everywhere (DMLDiD existing rule generalized) - plain HC on a/c1 incl. the new LPDiD / WooldridgeDiD-GLM unclustered lanes, the Wooldridge within-transform absorbed-FE full-K rescale and TWFE one-way hc2_bm; accepted no-op on b/c2 and the 2x2 surfaces (incl. the CS RC auto-PSU path, EfficientDiD.hausman_pretest and the LWDiD sensitivity helpers); rejected on StackedDiD (own vocabulary); the DiD/TWFE wild and WooldridgeDiD multiplier bootstraps and the cluster-only hc2_bm lanes of SunAbraham / WooldridgeDiD reject it (SunAbraham pairs bootstrap: accepted no-op, the hc2 precedent); metadata: DiD/TWFE analytical n_clusters from the regression (non-survey fits), SunAbrahamResults.n_clusters (appended) + cluster_name on clustered analytical fits, TWFE survey auto-label corrected to None. Pinned by tests/test_v4_cluster_policy.py; notebook 23 re-executed. RE-SCOPED 2026-10-18 (4.0-cut framing, user decision; measured roster in docs/methodology/variance-conventions.md and spec section 7). The original premise - static TwoWayFixedEffects runs HC-robust by default - was already false: static TWFE auto-clusters at unit (twfe.py, since 2026-01). Measured categories: (a) SpilloverDiD is the ONE required-unit estimator whose default SE moves (observation-level HC -> unit clustering, mirroring TWFE's carve-outs: explicit cluster= wins, Conley drops it, never injected as survey PSU); (c1) TWFE/SunAbraham/WooldridgeDiD/LPDiD already cluster at unit by regression-level default - cluster=False yields the plain HC sandwich; (c2) ImputationDiD/TwoStageDiD default cluster_var=unit INSIDE unit-level influence-function machinery - no unclustered variance exists, cluster=False is an accepted explicit no-op; (c3) StackedDiD clusters by construction and its validator accepts only unit/unit_subexp - cluster=False is REJECTED (REGISTRY Note); (b) CS/EfficientDiD/DMLDiD/dCDH/staggered-DDD (per-unit IF) and LWDiD/HAD (regression on a unit-level cross-section) are unit-level by construction - cluster=False is an accepted explicit no-op; (d) ContinuousDiD/DurationDiD/SDiD/SC/TROP/CiC have no cluster param; (n/a) cross-sectional DiD/2x2x2 DDD/RDD stay HC-robust or unclustered. Categories b and d carry NO auto-cluster metadata by decision (cluster_name stays None where the field exists; no field is added where it does not). The 4.0 flip therefore = the SpilloverDiD default + the uniform cluster=False off-switch on a/b/c1/c2 (today accepted nowhere: LPDiD's `self.cluster or unit` treats False as auto-cluster, most others look up a column named False, DMLDiD/StackedDiD reject on type, dCDH raises on any non-None) + metadata gaps (TWFE/DiD n_clusters on the analytical path; SunAbraham cluster_name + n_clusters). ORIGINAL NOTE: ONE auto-cluster policy at 4.0: panel estimators (required unit column) default to clustering at unit with cluster_name/n_clusters metadata; cluster=False disables; 2x2 cross-sectional estimators stay HC-robust. Changes some default SEs - migration guide with [M-004..M-006]. SCOPE NOTE (Phase 3(a), user decision 2026-08-07): the TWFE EVENT-STUDY mode shipped already auto-clustered at 3.9 (new API adopts the end-state immediately, with the static carve-outs mirrored - see [M-010]), so this 4.0 flip does not re-touch it." - id: M-081 kind: behavior group: policy-n-bootstrap @@ -1919,10 +1919,11 @@ rows: introduced_in: "4.0" deprecated_in: null removed_in: null - status: planned + status: done phase: 5 + test_ref: tests/test_v4_spillover_df_convention.py code_refs: [diff_diff/spillover.py, diff_diff/utils.py, docs/methodology/variance-conventions.md] - notes: "PLANNED (item 7c, the PR after 7b; user decision 2026-10-29): SpilloverDiD gains the three-value df_convention knob with the 4.0 default 'cluster' - t(G-1) on its clustered analytical lanes (the 4.0 unit auto-cluster and explicit cluster=), residual df under 'residual' and on unclustered (cluster=False) fits, z under 'normal'; inert on Conley (G forced to None even with an explicit product-kernel cluster, the SunAbraham precedent); survey df keeps precedence. Closes the gap 7b documents: at 7b the default CR1 lane runs t(n_obs_eff - k_effective) while every other auto-clustered default runs t(G-1). Moves p/CI on clustered SpilloverDiD fits; notebook 23 re-executed again in 7c; the REGISTRY TWFE knob Note moves SpilloverDiD from its out-of-scope list to the knob roster." + notes: "DONE at the 4.0 cut (item 7c, 2026-10-29): SpilloverDiD(df_convention='cluster') - t(G-1) on the clustered analytical lanes (the unit auto-cluster and explicit cluster=), residual df under 'residual' and on cluster=False, z under 'normal'; inert (residual df) on every Conley fit incl. an explicit product-kernel cluster=; survey df keeps precedence; the saturated-sample all-NaN contract is knob-independent (ladder: survey df -> saturation -> utils.resolve_tail_df, in spillover._resolve_inference_df); SpilloverDiDResults carries df_convention / inference_df; variance-conventions matrix rows spillover_cluster_unit / spillover_cluster_unit_default_df; notebook 23 re-validated (it prints no p-values or CIs). ORIGINAL: PLANNED (item 7c, the PR after 7b; user decision 2026-10-29): SpilloverDiD gains the three-value df_convention knob with the 4.0 default 'cluster' - t(G-1) on its clustered analytical lanes (the 4.0 unit auto-cluster and explicit cluster=), residual df under 'residual' and on unclustered (cluster=False) fits, z under 'normal'; inert on Conley (G forced to None even with an explicit product-kernel cluster, the SunAbraham precedent); survey df keeps precedence. Closes the gap 7b documents: at 7b the default CR1 lane runs t(n_obs_eff - k_effective) while every other auto-clustered default runs t(G-1). Moves p/CI on clustered SpilloverDiD fits; notebook 23 re-executed again in 7c; the REGISTRY TWFE knob Note moves SpilloverDiD from its out-of-scope list to the knob roster." - id: M-155 kind: behavior group: policy-auto-cluster diff --git a/docs/v4-design.md b/docs/v4-design.md index 8030ac5f2..5f6edc295 100644 --- a/docs/v4-design.md +++ b/docs/v4-design.md @@ -864,18 +864,22 @@ domain vocabulary, not drift. with `hc2_bm` on SunAbraham/WooldridgeDiD pending the one-way BM DOF port (TODO.md); DiD/TWFE analytical `n_clusters`, `SunAbrahamResults.n_clusters` + `cluster_name`, the TWFE survey auto-label corrected to None; pinned by - `tests/test_v4_cluster_policy.py`. SpilloverDiD's default CR1 lane keeps - residual-df inference until [M-154].) -- **SpilloverDiD df_convention** [M-154] (item 7c, the PR after 7b; user - decision 2026-10-29): SpilloverDiD gains the three-value `df_convention` knob + `tests/test_v4_cluster_policy.py`. SpilloverDiD's default CR1 lane kept + residual-df inference until [M-154] shipped in 7c.) +- **SpilloverDiD df_convention** [M-154] (SHIPPED item 7c, 2026-10-29; pinned by + `tests/test_v4_spillover_df_convention.py` and the variance-conventions rows + `spillover_cluster_unit` / `spillover_cluster_unit_default_df`; notebook 23 + re-validated - it prints no p-values or CIs): SpilloverDiD gained the three-value + `df_convention` knob with the 4.0 default `"cluster"` - t(G-1) on its clustered analytical lanes (the unit auto-cluster and explicit `cluster=`), residual df under `"residual"` and on unclustered (`cluster=False`) fits, z under `"normal"`; inert on Conley (G forced to None even with an explicit product-kernel cluster, the SunAbraham precedent); survey df keeps precedence. Closes the one gap 7b leaves: at 7b the default CR1 lane runs t(n_obs_eff - - k_effective) while every other auto-clustered default runs t(G-1). Moves - p/CI on clustered SpilloverDiD fits; notebook 23 is re-executed again there. + k_effective) while every other auto-clustered default ran t(G-1). Moved + p/CI on clustered SpilloverDiD fits; the saturated-sample all-NaN contract is + knob-independent (ladder: survey df -> saturation -> `utils.resolve_tail_df`). - **df_convention default flip** [M-004]..[M-006] + [M-128]..[M-131]: `"residual"` -> `"cluster"` (G-1) at 4.0 - the locked #663 direction. PR C ([M-127], 3.9) resolved the "evaluate extending the knob" item EARLY: @@ -1232,8 +1236,9 @@ tests-first PR that flips no rows). df_convention flips [M-004]..[M-006] + [M-128]..[M-131], pinned by `tests/test_v4_inference_defaults.py`; 7b SHIPPED 2026-10-29: [M-080] + the shared cluster contract [M-155], pinned by - `tests/test_v4_cluster_policy.py`, notebook 23 re-executed; 7c = [M-154] - SpilloverDiD df_convention, the next PR.) + `tests/test_v4_cluster_policy.py`, notebook 23 re-executed; 7c SHIPPED + 2026-10-29: [M-154] SpilloverDiD df_convention, pinned by + `tests/test_v4_spillover_df_convention.py`; item 7 complete.) 8. [M-093] sentinel retirement + event-study schema enforcement. 9. [M-090] docs reorganization: Bacon to the diagnostics family, catalogue counts, JOSS prose, REGISTRY archive framing for the removed classes. @@ -1244,8 +1249,8 @@ tests-first PR that flips no rows). 04_parallel_trends, 05_honest_did, 07_pretrends_power, 09_real_world_examples, 11_imputation_did, 13_stacked_did, 16_wooldridge_etwfe (NOT 16_survey_did: the survey df wins) - plus any - 7c/8/9 movers; 24 is handled in 7a and 23 in 7b and again in 7c (each - re-executed there). 7b's label changes also drift every committed tutorial + 8/9 movers; 24 is handled in 7a and 23 in 7b and re-validated in 7c (it + prints no p-values or CIs, so 7c moved no committed output). 7b's label changes also drift every committed tutorial that renders an analytical DiD/TWFE summary (", G=" on the variance line) or a SunAbraham summary (the new "Std. errors:" line, e.g. 02) - all within the nine-notebook set above. diff --git a/tests/test_spillover.py b/tests/test_spillover.py index 35d069305..aa2b4cea5 100644 --- a/tests/test_spillover.py +++ b/tests/test_spillover.py @@ -555,6 +555,7 @@ def test_get_params_returns_constructor_args(self): "conley_cutoff_km", "conley_lag_cutoff", "cluster", + "df_convention", "alpha", "anticipation", "event_study", diff --git a/tests/test_t23_spillover_tva_drift.py b/tests/test_t23_spillover_tva_drift.py index 57d88d7a6..a8041bc4c 100644 --- a/tests/test_t23_spillover_tva_drift.py +++ b/tests/test_t23_spillover_tva_drift.py @@ -796,3 +796,29 @@ def test_spillover_conley_fit_warning_policy_post_filter_clean(panel): _silence_spillover_matmul_warnings() # mirror notebook §6 filter est.fit(panel, outcome="y", unit="unit", time="time", treatment="D") _assert_post_filter_warning_surface_is_clean(w) + + +# ---------------------------------------------------------------------------- +# 7c ([M-154]): the df_convention knob on the tutorial's fits +# ---------------------------------------------------------------------------- + + +def test_default_inference_is_t_g_minus_1(spillover_fit, panel): + """The default (CR1 at unit) lane runs t(G-1) since 4.0: G = 200 units.""" + from scipy import stats + + G = panel["unit"].nunique() + assert spillover_fit.df_convention == "cluster" + assert spillover_fit.inference_df == G - 1 == spillover_fit.n_clusters - 1 == 199 + np.testing.assert_allclose( + spillover_fit.p_value, + 2 * stats.t.sf(abs(spillover_fit.t_stat), G - 1), + rtol=1e-12, + ) + + +def test_hc1_and_conley_lanes_keep_residual_df(spillover_hc1_fit, spillover_conley_lag0_fit, panel): + """cluster=False and every Conley fit are knob-inert: residual df n - k (k = 2).""" + n_obs = len(panel) + assert spillover_hc1_fit.inference_df == n_obs - 2 + assert spillover_conley_lag0_fit.inference_df == n_obs - 2 diff --git a/tests/test_v4_cluster_policy.py b/tests/test_v4_cluster_policy.py index 5f91a6be1..b3df0f7b3 100644 --- a/tests/test_v4_cluster_policy.py +++ b/tests/test_v4_cluster_policy.py @@ -896,5 +896,5 @@ def test_ledger_rows_are_done_and_point_here(): assert row["test_ref"] == THIS_TEST_REF, rid assert rows["M-080"]["old"] == "diff_diff:SpilloverDiD[cluster]" assert rows["M-155"]["old"] == "diff_diff.utils:validate_cluster" - assert rows["M-154"]["status"] == "planned" # 7c: the SpilloverDiD df_convention knob + assert rows["M-154"]["status"] == "done" # 7c shipped the SpilloverDiD df_convention knob assert rows["M-154"]["old"] == "diff_diff:SpilloverDiD[df_convention]" diff --git a/tests/test_v4_matrix.py b/tests/test_v4_matrix.py index 155fc5fd4..ecb77d101 100644 --- a/tests/test_v4_matrix.py +++ b/tests/test_v4_matrix.py @@ -143,7 +143,7 @@ # to_dataframe level twin (M-150, 4.0-cut framing) = 132; + the LWDiD # results-contract rows (M-151/M-152 alias deprecations, M-153 to_dataframe # reshape; 4.0-cut PR 1) = 135; + the 7b cluster-policy pair (M-154 SpilloverDiD -# df_convention, planned for 7c; M-155 the shared cluster contract) = 137. +# df_convention, shipped 7c; M-155 the shared cluster contract) = 137. # Ids are never reused and terminal rows are never deleted, so the ledger # only grows - raise the floor when rows are added; a lower parse count # means scanner/format drift or an illegal row deletion. diff --git a/tests/test_v4_spillover_df_convention.py b/tests/test_v4_spillover_df_convention.py new file mode 100644 index 000000000..7b8ce1563 --- /dev/null +++ b/tests/test_v4_spillover_df_convention.py @@ -0,0 +1,435 @@ +"""Item 7c of the 4.0 cut: the SpilloverDiD ``df_convention`` knob (ledger row M-154). + +7b made SpilloverDiD auto-cluster at ``unit`` but left its inference on the OLS +residual df ``t(n_obs_eff - k_effective)``; 7c adds the shared three-value knob +with the 4.0 default ``"cluster"`` so the clustered lanes (the unit auto-cluster +and an explicit ``cluster=``) run the Stata/fixest ``t(G - 1)`` like every other +auto-clustered default. What moves: p-values and CIs on clustered fits. What does +not: point estimates, SEs and t-statistics; ``cluster=False`` and EVERY Conley fit +(G is forced to None, the SunAbraham precedent - an explicit product-kernel +``cluster=`` included); the survey ``df_survey`` (precedence under every value); +the saturated-sample all-NaN contract. ``df_convention="residual"`` restores the +3.x / 7b inference bit-for-bit. + +The df ladder is the pure helper ``diff_diff.spillover._resolve_inference_df`` +(survey df -> saturation -> ``utils.resolve_tail_df``); its saturation branch is +pinned here directly because a saturated SpilloverDiD FIT is unreachable through +the balanced-panel + far-away validation (no test constructs one). + +The ledger import lives inside its test: ``tests.test_v4_matrix`` skips at import +when ``docs/v4-deprecations.yaml`` is absent (the wheel-install CI legs copy only +``tests/``), and the behavioral pins must still run there. +""" + +from __future__ import annotations + +import inspect +import warnings + +import numpy as np +import pytest +from scipy import stats + +from diff_diff import SpilloverDiD, SurveyDesign +from diff_diff.spillover import _resolve_inference_df +from diff_diff.utils import safe_inference +from tests._dgp_utils import generate_butts_staggered_dgp +from tests.test_t23_spillover_tva_drift import ( + D_BAR_KM, + _build_t23_panel, + _silence_spillover_matmul_warnings, +) + +THIS_TEST_REF = "tests/test_v4_spillover_df_convention.py" +SP_KW = dict(outcome="y", unit="unit", time="time", treatment="D") +N_RINGS = 1 # rings=[0, D_BAR_KM] -> one ring coefficient; k_effective = 1 + N_RINGS + + +def _t_tail(t, df): + return 2.0 * stats.t.sf(np.abs(t), df) + + +def _z_tail(t): + return 2.0 * stats.norm.sf(np.abs(t)) + + +def _fit(panel, *, survey_design=None, **kwargs): + est = SpilloverDiD(rings=[0.0, D_BAR_KM], conley_coords=("lat", "lon"), **kwargs) + with warnings.catch_warnings(): + _silence_spillover_matmul_warnings() + warnings.simplefilter("ignore", UserWarning) + if survey_design is None: + return est.fit(panel, **SP_KW) + return est.fit(panel, survey_design=survey_design, **SP_KW) + + +CONLEY_KW = dict(vcov_type="conley", conley_cutoff_km=D_BAR_KM, conley_lag_cutoff=0) + + +@pytest.fixture(scope="module") +def panel(): + df = _build_t23_panel() + idx = df["unit"].str[1:].astype(int) + df["blk"] = idx % 10 # a unit-constant coarser cluster: 10 labels + df["w"] = 1.0 + (idx % 5) * 0.2 # unit-constant survey weights + df["stratum"] = idx % 4 + return df + + +@pytest.fixture(scope="module") +def default(panel): + return _fit(panel) + + +@pytest.fixture(scope="module") +def residual(panel): + return _fit(panel, df_convention="residual") + + +# --------------------------------------------------------------------------- # +# (1) Signature / validation +# --------------------------------------------------------------------------- # + + +class TestKnobSurface: + def test_signature_default_is_cluster(self): + assert inspect.signature(SpilloverDiD).parameters["df_convention"].default == "cluster" + + def test_get_set_params_round_trip(self): + est = SpilloverDiD(rings=[0.0, 50.0]) + assert est.get_params()["df_convention"] == "cluster" + est.set_params(df_convention="residual") + assert est.df_convention == "residual" == est.get_params()["df_convention"] + + def test_constructor_rejects_unknown_value(self): + with pytest.raises(ValueError, match="df_convention must be one of"): + SpilloverDiD(rings=[0.0, 50.0], df_convention="bogus") + + def test_set_params_is_transactional(self): + est = SpilloverDiD(rings=[0.0, 50.0]) + with pytest.raises(ValueError, match="df_convention must be one of"): + est.set_params(df_convention="bogus") + assert est.df_convention == "cluster" + + def test_direct_mutation_rejected_at_fit(self, panel): + est = SpilloverDiD(rings=[0.0, D_BAR_KM], conley_coords=("lat", "lon")) + est.df_convention = "bogus" + with pytest.raises(ValueError, match="df_convention must be one of"): + est.fit(panel, **SP_KW) + + +# --------------------------------------------------------------------------- # +# (2)-(4) The three values on the default (CR1 at unit) lane +# --------------------------------------------------------------------------- # + + +class TestDefaultLane: + def test_default_is_t_g_minus_1(self, default, panel): + G = panel["unit"].nunique() + assert default.cluster_name == "unit" and default.n_clusters == G + assert default.df_convention == "cluster" + assert default.inference_df == G - 1 == default.n_clusters - 1 + np.testing.assert_allclose(default.p_value, _t_tail(default.t_stat, G - 1), rtol=1e-12) + lo, hi = default.conf_int + np.testing.assert_allclose( + (hi - lo) / 2, stats.t.ppf(1 - default.alpha / 2, G - 1) * default.se, rtol=1e-12 + ) + + def test_ring_rows_share_the_df(self, default, panel): + G = panel["unit"].nunique() + eff = default.spillover_effects + assert eff is not None and len(eff) == N_RINGS + for _, row in eff.iterrows(): + np.testing.assert_allclose(row["p_value"], _t_tail(row["t_stat"], G - 1), rtol=1e-12) + np.testing.assert_allclose( + (row["ci_high"] - row["ci_low"]) / 2, + stats.t.ppf(1 - default.alpha / 2, G - 1) * row["se"], + rtol=1e-12, + ) + + def test_explicit_unit_cluster_is_bit_equal(self, default, panel): + explicit = _fit(panel, cluster="unit") + assert (explicit.p_value, explicit.conf_int, explicit.inference_df) == ( + default.p_value, + default.conf_int, + default.inference_df, + ) + + def test_to_dict_carries_both_fields(self, default, panel): + d = default.to_dict() + assert d["df_convention"] == "cluster" + assert d["inference_df"] == panel["unit"].nunique() - 1 + + def test_explicit_coarser_cluster_uses_its_own_g(self, panel): + """G comes from the effective cluster labels, not the unit count.""" + blk = _fit(panel, cluster="blk") + assert blk.cluster_name == "blk" and blk.n_clusters == 10 + assert blk.inference_df == 9.0 + np.testing.assert_allclose(blk.p_value, _t_tail(blk.t_stat, 9), rtol=1e-12) + blk_res = _fit(panel, cluster="blk", df_convention="residual") + assert blk_res.se == blk.se + assert blk_res.inference_df == blk.n_obs - (1 + N_RINGS) + + def test_residual_restores_the_7b_inference(self, default, residual): + k = 1 + N_RINGS + assert residual.df_convention == "residual" + assert residual.inference_df == residual.n_obs - k + np.testing.assert_allclose( + residual.p_value, _t_tail(residual.t_stat, residual.n_obs - k), rtol=1e-12 + ) + assert residual.att == default.att and residual.se == default.se + assert residual.t_stat == default.t_stat + assert residual.p_value != default.p_value + assert residual.conf_int != default.conf_int + + def test_normal_is_deliberate_z(self, default, panel): + z = _fit(panel, df_convention="normal") + assert z.inference_df is None and z.df_convention == "normal" + assert z.se == default.se and z.t_stat == default.t_stat + np.testing.assert_allclose(z.p_value, _z_tail(z.t_stat), rtol=1e-12) + lo, hi = z.conf_int + np.testing.assert_allclose( + (hi - lo) / 2, stats.norm.ppf(1 - z.alpha / 2) * z.se, rtol=1e-12 + ) + assert "inference_df" not in z.to_dict() + + +# --------------------------------------------------------------------------- # +# (5) Inert lanes: cluster=False and every Conley fit +# --------------------------------------------------------------------------- # + + +class TestInertLanes: + def test_cluster_false_is_knob_inert(self, panel): + off = _fit(panel, cluster=False) + off_res = _fit(panel, cluster=False, df_convention="residual") + assert off.n_clusters is None and off.cluster_name is None + assert off.inference_df == off.n_obs - (1 + N_RINGS) + assert (off.p_value, off.conf_int, off.inference_df) == ( + off_res.p_value, + off_res.conf_int, + off_res.inference_df, + ) + + def test_conley_auto_is_knob_inert(self, panel): + c = _fit(panel, **CONLEY_KW) + c_res = _fit(panel, df_convention="residual", **CONLEY_KW) + assert c.n_clusters is None + assert c.inference_df == c.n_obs - (1 + N_RINGS) + assert (c.p_value, c.conf_int, c.inference_df) == ( + c_res.p_value, + c_res.conf_int, + c_res.inference_df, + ) + + def test_conley_with_explicit_cluster_keeps_residual_df(self, panel): + """The product-kernel cluster is reported (n_clusters) but never a df source.""" + cu = _fit(panel, cluster="unit", **CONLEY_KW) + cu_res = _fit(panel, cluster="unit", df_convention="residual", **CONLEY_KW) + assert cu.n_clusters == panel["unit"].nunique() and cu.cluster_name == "unit" + assert cu.inference_df == cu.n_obs - (1 + N_RINGS) + assert (cu.p_value, cu.conf_int) == (cu_res.p_value, cu_res.conf_int) + + def test_conley_normal_is_z(self, panel): + cz = _fit(panel, df_convention="normal", **CONLEY_KW) + assert cz.inference_df is None + np.testing.assert_allclose(cz.p_value, _z_tail(cz.t_stat), rtol=1e-12) + + +# --------------------------------------------------------------------------- # +# (6) Survey df keeps precedence under every value +# --------------------------------------------------------------------------- # + + +class TestSurveyPrecedence: + @pytest.mark.parametrize( + "design, expected_df", + [ + (SurveyDesign(weights="w"), lambda df: len(df) - 1), # implicit per-obs PSUs + ( + SurveyDesign(weights="w", strata="stratum", psu="unit"), + lambda df: df["unit"].nunique() - df["stratum"].nunique(), + ), + ], + ids=["weights_only", "psu_strata"], + ) + def test_survey_df_wins(self, panel, design, expected_df): + fits = { + v: _fit(panel, df_convention=v, survey_design=design) + for v in ("cluster", "residual", "normal") + } + exp = expected_df(panel) + for v, r in fits.items(): + assert r.survey_metadata is not None and r.survey_metadata.df_survey == exp, v + assert r.inference_df == exp, v + base = fits["cluster"] + for v in ("residual", "normal"): + assert (fits[v].p_value, fits[v].conf_int) == (base.p_value, base.conf_int), v + # The PSU design's df (G - n_strata) is distinguishable from G - 1. + assert exp != panel["unit"].nunique() - 1 or design.psu is None + + +# --------------------------------------------------------------------------- # +# (7) Event-study mode +# --------------------------------------------------------------------------- # + + +def _fit_es(df, **kwargs): + est = SpilloverDiD( + rings=[0.0, 50.0, 200.0], + d_bar=200.0, + conley_coords=("lat", "lon"), + event_study=True, + horizon_max=2, + **kwargs, + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + warnings.simplefilter("ignore", RuntimeWarning) + return est.fit(df, outcome="y", unit="unit", time="time", first_treat="first_treat") + + +def _k_effective(res): + """Finite stage-2 coefficients - the synthetic 'ATT' lincom entry excluded.""" + return sum(1 for name, v in res.coefficients.items() if name != "ATT" and np.isfinite(v)) + + +@pytest.fixture(scope="module") +def es_panel(): + return generate_butts_staggered_dgp(seed=0) + + +class TestEventStudy: + def test_cluster_default_is_g_minus_1(self, es_panel): + res = _fit_es(es_panel) + G = es_panel["unit"].nunique() + assert res.n_clusters == G and res.inference_df == G - 1 + dyn = res.att_dynamic + finite = dyn["p_value"].notna() & (dyn["se"] > 0) + assert finite.sum() >= 3 + np.testing.assert_allclose( + dyn.loc[finite, "p_value"], _t_tail(dyn.loc[finite, "t_stat"], G - 1), rtol=1e-12 + ) + np.testing.assert_allclose(res.p_value, _t_tail(res.t_stat, G - 1), rtol=1e-12) + + def test_residual_uses_n_minus_k(self, es_panel): + res = _fit_es(es_panel, df_convention="residual") + k = _k_effective(res) + assert res.inference_df == res.n_obs - k + dyn = res.att_dynamic + finite = dyn["p_value"].notna() & (dyn["se"] > 0) + np.testing.assert_allclose( + dyn.loc[finite, "p_value"], + _t_tail(dyn.loc[finite, "t_stat"], res.n_obs - k), + rtol=1e-12, + ) + + def test_normal_is_z_through_the_extractor(self, es_panel): + res = _fit_es(es_panel, df_convention="normal") + assert res.inference_df is None + dyn = res.att_dynamic + finite = dyn["p_value"].notna() & (dyn["se"] > 0) + np.testing.assert_allclose( + dyn.loc[finite, "p_value"], _z_tail(dyn.loc[finite, "t_stat"]), rtol=1e-12 + ) + np.testing.assert_allclose(res.p_value, _z_tail(res.t_stat), rtol=1e-12) + + +# --------------------------------------------------------------------------- # +# (8) The df ladder itself (saturation is unreachable through a fit) +# --------------------------------------------------------------------------- # + + +KNOBS = ("cluster", "residual", "normal") + + +class TestResolveInferenceDf: + @pytest.mark.parametrize("knob", KNOBS) + @pytest.mark.parametrize( + "n_obs_eff, k_effective, substring", + [ + (4, 4, "inference df = 0 (n_obs=4, effective_rank=4"), + (5, 8, "inference df = -3 (n_obs=5, effective_rank=8"), + ], + ) + def test_saturated_sample_is_nan_sentinel(self, knob, n_obs_eff, k_effective, substring): + with pytest.warns(UserWarning, match=substring.replace("(", r"\(")): + df = _resolve_inference_df( + df_convention=knob, + n_obs_eff=n_obs_eff, + k_effective=k_effective, + n_clusters=20, + vcov_mode="cluster", + survey_df=None, + has_survey=False, + ) + assert df == 0.0 + + @pytest.mark.parametrize("knob", KNOBS) + def test_survey_saturated_is_nan_sentinel(self, knob): + with pytest.warns(UserWarning, match="df_survey = 0"): + df = _resolve_inference_df( + df_convention=knob, + n_obs_eff=302, + k_effective=2, + n_clusters=20, + vcov_mode="cluster", + survey_df=0, + has_survey=True, + ) + assert df == 0.0 + + @pytest.mark.parametrize("knob", KNOBS) + def test_survey_df_beats_every_knob(self, knob): + df = _resolve_inference_df( + df_convention=knob, + n_obs_eff=302, + k_effective=2, + n_clusters=20, + vcov_mode="cluster", + survey_df=12, + has_survey=True, + ) + assert df == 12.0 + + @pytest.mark.parametrize( + "vcov_mode, n_clusters, expected", + [ + ("cluster", 20, {"cluster": 19.0, "residual": 300.0, "normal": None}), + ("conley", 20, {"cluster": 300.0, "residual": 300.0, "normal": None}), + ("hc1", None, {"cluster": 300.0, "residual": 300.0, "normal": None}), + ], + ids=["cluster_mode", "conley_mode", "hc1_mode"], + ) + def test_knob_branches(self, vcov_mode, n_clusters, expected): + for knob, want in expected.items(): + got = _resolve_inference_df( + df_convention=knob, + n_obs_eff=302, + k_effective=2, + n_clusters=n_clusters, + vcov_mode=vcov_mode, + survey_df=None, + has_survey=False, + ) + assert got == want, (vcov_mode, knob) + + def test_zero_sentinel_is_all_nan_in_safe_inference(self): + t, p, ci = safe_inference(1.0, 0.5, alpha=0.05, df=0.0) + assert np.isnan(t) and np.isnan(p) and np.isnan(ci[0]) and np.isnan(ci[1]) + + +# --------------------------------------------------------------------------- # +# (9) Ledger +# --------------------------------------------------------------------------- # + + +def test_ledger_row_is_done_and_points_here(): + from tests.test_v4_matrix import ROWS as LEDGER_ROWS + + row = {r["id"]: r for r in LEDGER_ROWS}["M-154"] + assert row["kind"] == "behavior" + assert row["status"] == "done" + assert row["test_ref"] == THIS_TEST_REF + assert row["old"] == "diff_diff:SpilloverDiD[df_convention]" + assert row["introduced_in"] == "4.0" diff --git a/tests/test_variance_conventions.py b/tests/test_variance_conventions.py index 3612769b3..b1bae41b6 100644 --- a/tests/test_variance_conventions.py +++ b/tests/test_variance_conventions.py @@ -78,6 +78,21 @@ def make_panel() -> pd.DataFrame: return pd.DataFrame(rows) +def _with_coords(df: pd.DataFrame) -> pd.DataFrame: + """Deterministic coordinate overlay for the SpilloverDiD rows (documented in + docs/methodology/variance-conventions.md "Shared DGP"): ever-treated units sit + in a ~7 km cluster at lat = 0.001 * unit, lon = 0; never-treated units alternate + between a near band (lat 0.5, ~55 km, inside the 100 km ring) and a far band + (lat 3.0, ~330 km, the Omega_0 far-away controls present in every period).""" + out = df.copy() + ever = out["first_treat"] > 0 + out["lat"] = np.where( + ever, 0.001 * out["unit"], np.where((out["unit"] // 4) % 2 == 0, 0.5, 3.0) + ) + out["lon"] = 0.0 + return out + + # --------------------------------------------------------------------------- # Programmatic discovery of instrumentation targets. # --------------------------------------------------------------------------- @@ -545,6 +560,22 @@ def snapshot(self): "no CR1 sandwich, normal-theory safe_inference gate only" ), ), + dict( + key="spillover_cluster_unit", + fit=lambda df: diff_diff.SpilloverDiD( + rings=[0.0, 100.0], conley_coords=("lat", "lon"), df_convention="residual" + ).fit(_with_coords(df), outcome="y", unit="unit", time="time", first_treat="first_treat"), + cr1_k=(), + # Two safe_inference calls (aggregate tau_total + the one ring), both at + # the stage-2 residual df n - k = 360 - 2. + tail_df=(358.0, 358.0), + status="legitimate", + reason=( + "L3: Wave D Gardner-GMM two-stage sandwich with its own CR1 multiplier " + "G/(G-1)*(n-1)/(n-p_2) (two_stage meat, never the shared CR1). Explicit " + "df_convention='residual' since the 4.0 default is G-1 (see the _default_df twin)." + ), + ), # ---- 4.0 default twins: identical fits WITHOUT the explicit knob, so the # tail df is G-1 (G = clusters with positive weight on each surface's # clustered sample). The cr1_k literals are copied from the residual twins @@ -672,6 +703,20 @@ def snapshot(self): "same CR1 k as the explicit-'residual' twin, tail df G-1" ), ), + dict( + key="spillover_cluster_unit_default_df", + fit=lambda df: diff_diff.SpilloverDiD(rings=[0.0, 100.0], conley_coords=("lat", "lon")).fit( + _with_coords(df), outcome="y", unit="unit", time="time", first_treat="first_treat" + ), + cr1_k=(), + # G-1 with G = 60 units (the 4.0 unit auto-cluster), on both calls. + tail_df=(59.0, 59.0), + status="legitimate", + reason=( + "t(G-1): the 4.0 df_convention default (M-154 - SpilloverDiD gained the knob at the " + "cut) - no shared-CR1 call, as the explicit-'residual' twin; tail df G-1" + ), + ), ] _FAST_KEYS = { @@ -685,6 +730,8 @@ def snapshot(self): "wooldridge_hc1_within_default_df", "lpdid_pre2_post2", "callaway_santanna_default", + "spillover_cluster_unit", + "spillover_cluster_unit_default_df", }