diff --git a/CHANGELOG.md b/CHANGELOG.md index a5d2dcea..7434a3cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **`ChaisemartinDHaultfoeuille.by_path` is now compatible with `trends_linear` (DID^{fd} group-specific linear trends) and `trends_nonparam` (state-set trends).** For `trends_linear`, the first-differencing transform runs once globally before path enumeration, so per-path raw second-differences `DID^{fd}_{path, l}` surface on `path_effects[path]["horizons"][l]` automatically. Per-path **cumulated level effects** `delta_{path, l} = sum_{l'=1..l} DID^{fd}_{path, l'}` (the quantity R returns under `did_multiplegt_dyn(..., by_path, trends_lin)`) surface on the new `results.path_cumulated_event_study[path][l]` field, mirroring the global `linear_trends_effects` cumulation. `to_dataframe(level="by_path")` exposes `cumulated_effect` / `cumulated_se` columns (always present, NaN-when-None — mirrors the `cband_*` convention from PR #374); `summary()` renders a "Cumulated Level Effects (DID^{fd}, trends_linear)" sub-section under each per-path block. SE on the cumulated layer is the conservative upper bound (sum of per-horizon component SEs, NaN-consistent), matching the global `linear_trends_effects` convention. Path enumeration runs on the post-first-differenced `N_mat_fd`: switchers with `F_g==2` fail the window-eligibility check and are dropped from path enumeration entirely (the existing global `F_g >= 3` warning still surfaces the issue), so a path whose switchers all have `F_g < 3` is silently absent from `path_effects` rather than present-with-NaN. Placebo under `trends_linear` returns RAW per-horizon values — there is no per-path placebo cumulation surface in either Python or R. For `trends_nonparam`, the set membership column is validated and stored once globally as `set_ids_arr`; the `set_ids` parameter is now threaded through the four per-path IF helpers (`_compute_path_effects`, `_compute_path_placebos`, `_collect_path_bootstrap_inputs`, `_collect_path_placebo_bootstrap_inputs`) so per-path analytical SE, bootstrap, placebos, and sup-t bands all consume the set-restricted control pool automatically. Per-period effects remain unadjusted under both extensions, consistent with the existing per-period DID contract. Validated against R via two new golden-value scenarios: `single_baseline_multi_path_by_path_trends_lin` (n_periods=13, F_g >= 4, cohort-single-path; per-path cumulated point estimates match R bit-exactly with `POINT_RTOL=1e-9`, cumulated SE within `CUM_SE_RTOL=0.20`) and `multi_path_reversible_by_path_trends_nonparam` (per-path point estimates AND placebos match R bit-exactly with `POINT_RTOL=1e-9`, per-path SE within `SE_RTOL=0.15`). **F_g=3 boundary-case divergence (`by_path + trends_linear`):** `F_g=3` switchers have only 1 valid pre-window Z value after first-differencing, triggering 30%+ relative divergence between Python and R per-path point estimates on paths whose switchers include `F_g=3`. A targeted `UserWarning` fires at fit-time on this regime; R parity is asserted only on the `F_g >= 4` parity fixture. Placebo parity for `trends_linear` is intentionally skipped (R's per-path placebo computation re-runs on the path-restricted subsample with different control eligibility than Python's global-then-disaggregate architecture surfaces; placebo + `trends_linear` is exercised via internal regression only). Cross-path cohort-sharing SE deviation from R documented for `path_effects` is inherited unchanged. Gates at `chaisemartin_dhaultfoeuille.py:1014-1023` removed; `by_path` docstring updated to add the two new compatibility paragraphs and remove `trends_linear` / `trends_nonparam` from the incompatible list. R-parity tests at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsLinear` and `::TestDCDHDynRParityByPathTrendsNonparam`; cross-surface regressions at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsLinear` and `::TestByPathTrendsNonparam`. See `docs/methodology/REGISTRY.md` §ChaisemartinDHaultfoeuille `Note (Phase 3 by_path ...)` → "Per-path linear-trends DID^{fd}" and "Per-path state-set trends" for the full contract. - **HAD `trends_lin=True` linear-trend detrending mode** on `HeterogeneousAdoptionDiD.fit(aggregate="event_study")`, `joint_pretrends_test`, and `joint_homogeneity_test`. Mirrors R `DIDHAD::did_had(..., trends_lin=TRUE)` (paper Eq. 17 / Eq. 18 / page 32 joint-Stute homogeneity-with-trends). Per-group linear-trend slope estimated as `Y[g, F-1] - Y[g, F-2]` and applied as `(t - base) × slope` adjustment to per-event-time outcome evolutions. Requires F ≥ 3 (panel must contain F-2). The "consumed" placebo at our event-time `e=-2` is auto-dropped (R reduces max placebo lag by 1 with the same effect). Mutually exclusive with survey weighting (`survey_design` / `survey` / `weights`): raises `NotImplementedError` per `feedback_per_method_survey_element_contract` (weighted slope estimator not derived from paper; tracked in TODO.md as a follow-up). Bit-exact backcompat for `trends_lin=False` (default). Patch-level (additive keyword-only kwarg). - **HAD R-package end-to-end parity test** vs `DIDHAD` v2.0.0 (`Credible-Answers/did_had`) on the **`design="continuous_at_zero"` (Design 1') surface**. New parity fixture `benchmarks/data/did_had_golden.json` generated by `benchmarks/R/generate_did_had_golden.R` covers 3 paper-derived synthetic DGPs (Uniform, Beta(2,2), Beta(0.5,1)) × 5 method combinations (overall, event-study, placebo, yatchew, trends_lin). The harness explicitly forces `HeterogeneousAdoptionDiD(design="continuous_at_zero")` because R `did_had` always evaluates the local-linear at `d=0` regardless of dose distribution; our default `design="auto"` may legitimately choose `continuous_near_d_lower` or `mass_point` on dose distributions with boundary density bounded away from zero (e.g., Beta(2,2)) and thereby diverge from R numerically — that divergence is methodologically defensible but out of scope for this parity test. Python parity test `tests/test_did_had_parity.py` asserts point estimate / SE / CI bounds at `atol=1e-8` and Yatchew T-stat at `atol=1e-10` after a documented `× G/(G-1)` finite-sample convention shift. Two intentional convention deviations from R, documented in `docs/methodology/REGISTRY.md`: (a) we report the bias-corrected point estimate (modern CCF 2018 convention; R's `Estimate` column reports the conventional estimate with the bias-corrected CI separately — our `att` matches R's CI midpoint); (b) Yatchew uses paper Appendix E's literal (1/G) variance-denominator convention while R uses base-R `var()`'s (1/(N-1)) sample-variance convention (parity is bit-exact after the `× G/(G-1)` shift). Yatchew on placebos with R's mean-independence null (`order=0`) is not yet exposed in our `yatchew_hr_test` (we currently only support the linearity null) and is skipped in the parity test; tracked as TODO follow-up. diff --git a/benchmarks/R/generate_dcdh_dynr_test_values.R b/benchmarks/R/generate_dcdh_dynr_test_values.R index 69d37bf2..1c237cd1 100644 --- a/benchmarks/R/generate_dcdh_dynr_test_values.R +++ b/benchmarks/R/generate_dcdh_dynr_test_values.R @@ -753,6 +753,180 @@ scenarios$multi_path_reversible_by_path_controls <- list( results = extract_dcdh_by_path(res16, n_effects = 3) ) +# Scenario 17: single-baseline multi-path + by_path=3 + trends_lin=TRUE +# (Phase 3 Wave 3 #6: by_path + DID^{fd} group-specific linear trends). +# Custom inline single-baseline DGP — `multi_path_reversible` (Scenarios +# 13-16) concentrates each path on 1-2 F_g values, which after R's +# per-path subset + trends_lin's F_g==2 filter collapses to a single F_g, +# violating the dCDH staggered design restriction inside R's per-path +# `did_multiplegt_main` call. `mixed_single_switch` (Scenario 17's first +# attempt) is MULTI-baseline (joiners + leavers), which triggers the +# documented multi-baseline divergence between Python's global-then- +# disaggregate architecture and R's per-path full-pipeline call (same +# divergence pattern as `controls`; rel diffs of 7-19% on point estimates +# observed). To get clean single-baseline parity AND avoid the trends_lin +# F_g concentration trap, this scenario uses a custom DGP: all groups +# start at D=0 (single baseline), 3 paths span F_g ∈ {3,4,5} (all >= 3 +# so trends_lin's F_g==2 filter is a no-op), per-path counts unequal so +# top-k ranking is deterministic. **R returns the cumulated level effect +# delta_l per horizon, NOT the raw second-difference DID^{fd}_l** — +# verified empirically against the existing `joiners_only_trends_lin` +# parity test (`tests/test_chaisemartin_dhaultfoeuille_parity.py:403-409` +# documents this convention). The Python parity test compares Python's +# `path_cumulated_event_study[path][l]` against R's per-path Effect_l. +# Placebos under trends_lin remain RAW per-horizon (no cumulated placebo +# surface in R either), so the Python parity test compares +# `path_placebo_event_study[path][-l]` against R's per-path Placebo_l +# directly. Cumulated SE_RTOL is widened (~0.20 vs 0.12 used for +# non-cumulated by_path parity) because the conservative upper-bound SE +# (sum of per-horizon component SEs) compounds the cross-path +# cohort-sharing deviation under summation. +cat(" Scenario 17: single_baseline_multi_path_by_path_trends_lin\n") +{ + # Custom DGP: 80 switchers across 3 paths × 2 distinct F_g per path, + # all single-baseline (D_{g,1}=0). Each F_g maps to exactly ONE path + # (cohort-single-path, eliminating the cross-path cohort-sharing + # deviation from R that PR #360 documented for path_effects). All + # F_g >= 3 so trends_lin's F_g==2 filter is a no-op. Two distinct + # F_g per path satisfies R's staggered-design requirement inside the + # per-path `did_multiplegt_main` call. n_periods=11, L_max=3 gives + # F_g in [2,8] = 7 values; we use F_g in {3..8} = 6 values, two per + # path. Plus 20 never-treated + 20 always-treated controls + # (n_realized_groups = 120). + set.seed(117) + n_periods17 <- 13 + L_max17 <- 3 + target_paths17 <- list( + c(0L, 1L, 1L, 1L), # path 1, sustained on (rank 1) + c(0L, 1L, 1L, 0L), # path 2, on-then-off (rank 2) + c(0L, 1L, 0L, 0L) # path 3, on briefly (rank 3) + ) + # F_g-to-path mapping (each F_g unique to one path). + # All F_g >= 4 to avoid the F_g=3 boundary case under trends_lin — + # F_g=3 leaves only 1 valid pre-window Z value after the time==1 + # filter, causing R's per-path call (which re-runs the full pipeline + # on the path subset) to handle pre-window/control eligibility + # differently from Python's global first-differencing. F_g >= 4 gives + # both implementations 2+ valid pre-window Z values, eliminating the + # boundary-case divergence. + # F_g=4 -> path 1, F_g=5 -> path 1 (path 1 = 38) + # F_g=6 -> path 2, F_g=7 -> path 2 (path 2 = 24) + # F_g=8 -> path 3, F_g=9 -> path 3 (path 3 = 18) + # Total switchers = 80 + fg_path_counts17 <- list( + list(F_g = 4L, path_idx = 1L, count = 20L), + list(F_g = 5L, path_idx = 1L, count = 18L), + list(F_g = 6L, path_idx = 2L, count = 13L), + list(F_g = 7L, path_idx = 2L, count = 11L), + list(F_g = 8L, path_idx = 3L, count = 11L), + list(F_g = 9L, path_idx = 3L, count = 7L) + ) + n_switchers17 <- sum(sapply(fg_path_counts17, function(x) x$count)) + stopifnot(n_switchers17 == 80L) + D17 <- matrix(0L, nrow = n_switchers17, ncol = n_periods17) + g17 <- 1L + for (entry in fg_path_counts17) { + F_g <- entry$F_g + target <- target_paths17[[entry$path_idx]] + n_here <- entry$count + for (k in seq_len(n_here)) { + # Pre-baseline [1..F_g-2]: D=0 (single-baseline contract) + if (F_g >= 3L) D17[g17, 1:(F_g - 2L)] <- 0L + # Window [F_g-1..F_g-1+L_max]: target path + for (j in 0:L_max17) D17[g17, F_g - 1L + j] <- target[j + 1L] + # Post-window: stable at path[L_max+1] + if (F_g + L_max17 <= n_periods17) { + D17[g17, (F_g + L_max17):n_periods17] <- target[L_max17 + 1L] + } + g17 <- g17 + 1L + } + } + # Append 20 never-treated and 20 always-treated controls + D17 <- rbind(D17, + matrix(0L, nrow = 20L, ncol = n_periods17), + matrix(1L, nrow = 20L, ncol = n_periods17)) + n_total17 <- nrow(D17) + # Generate fixed effects, treatment effects, outcomes (mirror gen_reversible + # parameters: group_fe_sd=2.0, treatment_effect=2.0, time_trend=0.1, noise_sd=0.5) + set.seed(117L) + group_fe17 <- rnorm(n_total17, 0, 2.0) + noise17 <- matrix(rnorm(n_total17 * n_periods17, 0, 0.5), + nrow = n_total17, ncol = n_periods17) + period_arr17 <- 0:(n_periods17 - 1L) + Y17 <- 10.0 + + matrix(group_fe17, nrow = n_total17, ncol = n_periods17) + + matrix(0.1 * period_arr17, nrow = n_total17, ncol = n_periods17, byrow = TRUE) + + 2.0 * D17 + + noise17 + # Build long data frame + d17 <- data.frame( + group = rep(seq_len(n_total17) - 1L, each = n_periods17), + period = rep(period_arr17, n_total17), + treatment = as.vector(t(D17)), + outcome = as.vector(t(Y17)) + ) + # Inject per-group linear trends (Scenario 11 pattern) + set.seed(217L) + groups17 <- sort(unique(d17$group)) + g_trends17 <- setNames(rnorm(length(groups17), 0, 0.5), + as.character(groups17)) + d17$outcome <- d17$outcome + + g_trends17[as.character(d17$group)] * d17$period + res17 <- did_multiplegt_dyn( + df = d17, outcome = "outcome", group = "group", time = "period", + treatment = "treatment", effects = 3, placebo = 1, by_path = 3, + trends_lin = TRUE, ci_level = 95 + ) + scenarios$single_baseline_multi_path_by_path_trends_lin <- list( + data = list( + group = as.numeric(d17$group), + period = as.numeric(d17$period), + treatment = as.numeric(d17$treatment), + outcome = as.numeric(d17$outcome) + ), + params = list(pattern = "single_baseline_multi_path", + n_switcher_groups = 80L, n_realized_groups = 120L, + n_periods = 13L, seed = 117L, effects = 3, placebo = 1, + by_path = 3, trends_lin = TRUE, ci_level = 95), + results = extract_dcdh_by_path(res17, n_effects = 3, n_placebos = 1) + ) +} + +# Scenario 18: multi_path_reversible + by_path=3 + trends_nonparam="state" +# (Phase 3 Wave 3 #7: by_path + state-set trends). Same deterministic DGP +# and n_periods=10 as Scenarios 16/17, with a 3-state column added +# (deterministic per-group assignment via `((group - 1) %% 3) + 1` so +# within-set controls are guaranteed to exist for each path). **R does +# NOT cumulate or first-difference under trends_nonparam** — Effect_l +# per horizon is a normal DID with set-restricted control pool. The +# Python parity test compares per-path raw DID per (path, l) directly +# against R's per-path Effect_l. Placebos likewise are raw per-horizon. +# Per-path R parity matches exactly on single-baseline panels. +cat(" Scenario 18: multi_path_reversible_by_path_trends_nonparam\n") +d18 <- gen_reversible(n_groups = N_GOLDEN, n_periods = 10, + pattern = "multi_path_reversible", seed = 118, + L_max = 3) +d18$state <- ((d18$group - 1) %% 3) + 1 +res18 <- did_multiplegt_dyn( + df = d18, outcome = "outcome", group = "group", time = "period", + treatment = "treatment", effects = 3, placebo = 1, by_path = 3, + trends_nonparam = "state", ci_level = 95 +) +scenarios$multi_path_reversible_by_path_trends_nonparam <- list( + data = list( + group = as.numeric(d18$group), + period = as.numeric(d18$period), + treatment = as.numeric(d18$treatment), + outcome = as.numeric(d18$outcome), + state = as.numeric(d18$state) + ), + params = list(pattern = "multi_path_reversible", + n_switcher_groups = N_GOLDEN, n_realized_groups = N_GOLDEN + 40L, + n_periods = 10, seed = 118, effects = 3, placebo = 1, + by_path = 3, trends_nonparam = "state", ci_level = 95), + results = extract_dcdh_by_path(res18, n_effects = 3, n_placebos = 1) +) + # --------------------------------------------------------------------------- # Write output # --------------------------------------------------------------------------- diff --git a/benchmarks/data/dcdh_dynr_golden_values.json b/benchmarks/data/dcdh_dynr_golden_values.json index da2a821a..ccc0a3d5 100644 --- a/benchmarks/data/dcdh_dynr_golden_values.json +++ b/benchmarks/data/dcdh_dynr_golden_values.json @@ -1023,6 +1023,283 @@ } ] } + }, + "single_baseline_multi_path_by_path_trends_lin": { + "data": { + "group": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, 74, 74, 74, 74, 74, 74, 74, 74, 74, 74, 74, 74, 74, 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 82, 82, 82, 82, 82, 82, 82, 82, 82, 82, 82, 82, 82, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 87, 87, 87, 87, 87, 87, 87, 87, 87, 87, 87, 87, 87, 88, 88, 88, 88, 88, 88, 88, 88, 88, 88, 88, 88, 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 90, 90, 90, 90, 90, 90, 90, 90, 90, 90, 90, 90, 90, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 93, 93, 93, 93, 93, 93, 93, 93, 93, 93, 93, 93, 93, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 95, 95, 95, 95, 95, 95, 95, 95, 95, 95, 95, 95, 95, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 97, 97, 97, 97, 97, 97, 97, 97, 97, 97, 97, 97, 97, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 107, 107, 107, 107, 107, 107, 107, 107, 107, 107, 107, 107, 107, 108, 108, 108, 108, 108, 108, 108, 108, 108, 108, 108, 108, 108, 109, 109, 109, 109, 109, 109, 109, 109, 109, 109, 109, 109, 109, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 111, 111, 111, 111, 111, 111, 111, 111, 111, 111, 111, 111, 111, 112, 112, 112, 112, 112, 112, 112, 112, 112, 112, 112, 112, 112, 113, 113, 113, 113, 113, 113, 113, 113, 113, 113, 113, 113, 113, 114, 114, 114, 114, 114, 114, 114, 114, 114, 114, 114, 114, 114, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119], + "period": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], + "treatment": [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], + "outcome": [11.2208972868, 12.0206443959, 12.6301270946, 13.1565227702, 13.5587980475, 13.510509835, 14.0176180446, 13.1336328668, 13.1718499882, 13.6168121118, 13.5033945559, 14.0985627668, 12.9722815957, 10.3177671361, 10.5257057947, 9.4460689914, 11.66056927, 11.4588750304, 11.419245921, 10.1310637016, 11.4178393307, 10.1366659696, 9.7942379556, 9.8238450153, 8.6610625563, 9.8828458342, 12.2913223204, 10.1290882712, 10.0965666101, 11.793675632, 10.6527735376, 9.7276139476, 9.7143534347, 8.2285250015, 8.1586190348, 6.9008742444, 6.9019587686, 7.298865831, 5.3911273211, 3.2827743615, 3.6546382036, 4.4151625416, 7.2478589361, 7.5698003451, 8.1590439789, 7.7640070158, 8.0250246807, 8.1131208756, 9.1025841847, 9.3058811172, 10.1071726181, 10.3255279043, 9.9569128425, 9.5796056338, 10.6979520087, 12.965919732, 12.882958344, 13.1197885246, 14.2788465993, 13.3364211656, 15.3037091875, 16.0669560537, 16.20396553, 16.2482868725, 17.747872621, 7.7624424947, 7.2708979318, 7.3161612384, 8.6005585346, 7.3830904807, 8.0190544411, 6.9008265029, 7.2066784448, 7.08482464, 6.6376465495, 5.9276708584, 5.5671272486, 4.9878042545, 10.2273828784, 10.5849825725, 10.6818624734, 12.9187015862, 13.8500002904, 13.5836303765, 13.9998870066, 13.0472573467, 14.6736631881, 14.5343789369, 14.7738589871, 15.6132666477, 14.9101764018, 10.4424940426, 9.2142112181, 10.2515647602, 11.0396255132, 10.7987035547, 10.4610862533, 11.0398507007, 10.8332985823, 10.0256816882, 9.7919549704, 9.6466133248, 9.1638379833, 8.9223513418, 11.6614528975, 11.9729938537, 11.6055299667, 14.1998559966, 13.0394809577, 14.3351089549, 13.8811942737, 14.8724503042, 14.6277098107, 14.3163693458, 15.5916146531, 14.5889261013, 15.6858546372, 12.7640202801, 11.9559334425, 13.0181984877, 14.6881269813, 14.4024417688, 14.4304105413, 14.3223097209, 13.960962386, 13.9524397112, 13.5395360253, 13.4769332894, 13.6564103531, 13.2775217462, 11.5242725731, 11.4360323315, 12.1441055507, 13.54157956, 13.6625724162, 13.3458955977, 13.5888674204, 13.7870035751, 14.0054922902, 14.6263951025, 14.3742509296, 14.099238024, 14.4890883379, 7.4219422611, 7.3224215073, 7.5497033508, 10.6154497949, 11.1539514963, 11.1638825929, 12.2955295584, 12.4306424802, 12.8779650735, 12.4909913467, 13.5803854335, 14.1674120163, 13.3470132055, 9.6888883794, 9.4954891474, 10.2274878342, 12.1089701534, 11.4662385551, 13.0466812809, 12.6671813852, 13.0038468732, 12.7004473063, 12.3629566928, 12.9279683787, 12.7796169272, 13.8670819786, 10.8774837346, 10.5512000974, 8.7751778142, 11.2732335913, 10.1731591655, 11.0121544833, 10.2712278224, 9.6821465271, 8.731760143, 8.137571352, 9.3949682482, 7.6833922944, 6.5217067974, 9.8103389277, 9.3569078303, 8.4603362534, 8.8197431064, 9.7182120831, 9.1296779572, 7.0648526003, 6.0646164436, 7.0968878083, 5.6600495183, 4.1917737623, 3.5383845559, 3.866433817, 8.4030265817, 8.9859007469, 10.7532684703, 13.5533934785, 13.9716621058, 14.6220834627, 15.4326157428, 16.7784020874, 17.4334568624, 17.7269503581, 18.2068132165, 18.9723658974, 19.0037157173, 13.0142465943, 13.7455201233, 13.6209702638, 16.5173489674, 16.1407328615, 16.781290435, 16.7968443661, 17.8815698661, 17.2868213616, 17.6958243025, 18.0594443022, 18.4247749175, 18.6949348931, 8.8194249864, 8.9144358806, 8.944889639, 10.0029529709, 9.7561965099, 10.6602573597, 10.5466244609, 11.1093075767, 9.9729135675, 10.1059413043, 10.2089903309, 9.4832716341, 10.4741268046, 8.4580754273, 8.1885755932, 8.9164897117, 11.601759862, 12.4379685503, 13.2533113919, 13.7129086679, 14.6900151716, 15.1079081178, 15.1625532579, 15.6090116352, 16.3561017913, 16.7128021607, 9.2622065037, 8.7312409231, 9.0316202102, 10.907769724, 11.6262842853, 11.4167707207, 11.7311453224, 11.3242092932, 11.5814760258, 10.3144918317, 11.2797658908, 11.0676693145, 11.5404147318, 10.8641758973, 11.0582462771, 11.7951294576, 12.3848522519, 15.5054911097, 16.57883609, 17.2926661484, 18.454119555, 19.7616202732, 20.3440308471, 22.170257415, 23.2716688608, 23.5157229935, 14.0962136118, 13.7451912824, 14.7069384931, 14.5918696842, 16.3828334345, 15.4864498183, 16.0956871599, 15.290937802, 15.9454729124, 16.4082820453, 16.1893519817, 16.1463091025, 16.1233097045, 10.5735214278, 11.0835511498, 10.4007175424, 10.6023872632, 13.5070955302, 13.0215356156, 13.3021953542, 13.3713123995, 14.176498484, 14.2031669551, 14.2293141307, 14.908811386, 15.3398488573, 9.9715806802, 9.9950979741, 9.7498535307, 9.7377219758, 11.8120519858, 12.1061169446, 12.8022852224, 13.1637549151, 11.7372209107, 12.4231473946, 12.55040912, 12.1411285193, 12.6437789173, 11.9105805988, 11.6204651173, 12.2512710437, 13.2693027527, 14.6593840269, 15.5002991244, 15.0080648891, 15.460530497, 15.452832216, 16.1468754091, 16.0380561113, 16.2875412895, 16.7280838916, 9.8152896195, 10.1596379561, 10.43826009, 10.4765458271, 12.433091536, 12.5457801085, 12.8011926617, 13.1480348046, 13.1492035808, 13.1971966525, 12.6050273469, 13.5203422773, 13.372899162, 13.5029333191, 14.2831907672, 15.0235671805, 14.0379183777, 16.7904564688, 16.9659304112, 16.9961305166, 17.5816698475, 17.688292882, 17.8179004092, 18.0330095407, 18.7277625641, 18.4738494249, 12.6145711666, 13.0320298981, 13.3964751526, 14.582566413, 18.0303720559, 18.9472654792, 20.642580214, 20.9566231587, 21.2362994351, 22.7834199958, 25.0314077641, 24.1352988138, 25.6652246364, 9.1384212276, 8.2472172271, 9.1787296793, 8.6914770541, 11.7617708568, 11.7315274896, 10.823326501, 12.1286034303, 12.0073084493, 11.9126668564, 12.5525164077, 12.8539017978, 12.3655725111, 8.8541675424, 9.4736020418, 9.4769696081, 10.6729305408, 12.9151952882, 12.7233728221, 12.4340919411, 12.8673994799, 13.0040233618, 14.0800854321, 14.4053312426, 15.0690719917, 14.2054864481, 9.3175011639, 8.864969157, 9.8725307939, 10.2414081485, 12.3704741991, 12.2280616707, 12.1184225327, 12.6357729233, 13.8203984258, 14.0516177031, 13.900162931, 14.6747599538, 14.4671340333, 10.1357750071, 10.5575526433, 11.6838304766, 14.0721250571, 16.3020278439, 17.0685939187, 18.2059741754, 19.1903393413, 19.2386287245, 22.1181906024, 22.1197423526, 23.3989767137, 24.4776769802, 11.1370245422, 12.3994005438, 13.1370673801, 13.6747964186, 16.8931057238, 16.5648495537, 18.2514080317, 18.4978704971, 19.8740908241, 21.504901198, 21.8379085058, 23.0437747284, 24.1755601858, 7.4016914469, 8.2766939849, 8.7989432159, 9.3388479496, 11.8725753312, 11.3805901882, 12.6797803661, 13.82299728, 13.7909363467, 13.6713260998, 15.3291751121, 16.0130828505, 16.4575800779, 12.067159354, 12.4597392055, 12.2278254315, 12.0331565974, 14.416794686, 15.165320719, 15.9687080142, 15.5045102365, 15.4848988639, 16.2776649556, 16.5387423735, 16.6953816625, 17.06453748, 7.9773733234, 7.4765234116, 9.4160828253, 8.5562706166, 11.6521601298, 11.66773942, 11.4499628206, 12.4964567559, 12.9333407838, 13.6943978136, 13.4299597014, 14.351932569, 14.1406406098, 12.8295578841, 12.2170343356, 11.23331624, 10.0444027203, 12.170188108, 11.7524904365, 11.2422060317, 9.8414778724, 10.5900057568, 9.336718387, 10.2182242442, 9.4205125535, 7.7609544389, 11.3562279231, 10.7007541462, 10.3957773458, 8.8561425995, 10.4948352089, 9.6913930252, 10.2049519686, 9.2390801271, 9.1511256317, 7.8886846627, 7.5254697134, 7.0467817644, 5.5975407329, 9.4872549905, 9.9152640509, 9.7843238146, 9.6098976694, 10.3863986841, 12.0667471912, 12.3446449057, 10.476025711, 10.703779366, 10.9150324463, 10.5102783027, 11.5299819236, 11.1398095959, 11.6869037136, 10.8145563396, 11.2718901268, 9.3499835051, 11.1214432618, 12.1173896327, 11.7651432267, 9.7191565104, 9.3632323281, 8.9517103466, 9.403839794, 8.4703508359, 8.1916999589, 8.1461256223, 6.7451339201, 7.2381239382, 7.2004762477, 6.2502971224, 8.4943063396, 7.4605426474, 5.8434598387, 5.0578461141, 4.0223376852, 3.5045912786, 3.2633470173, 2.1032968572, 10.5835090208, 8.9489449204, 7.8643664573, 7.8688870611, 7.9200890976, 9.0087648388, 7.8263202807, 5.3264084614, 4.8273753401, 4.4629639441, 3.6563377114, 3.9680947871, 2.0443308677, 11.5401476051, 10.6931089492, 12.1559357664, 13.4391248924, 12.8605787109, 16.3117118788, 17.0591918024, 15.4679813778, 17.2302041948, 17.0680090505, 18.3633076987, 18.4654960588, 19.066670872, 12.4793576073, 12.5814399499, 11.4647675468, 12.2061496425, 12.0911525903, 14.4143868927, 12.7046285381, 11.3132060943, 12.7179537859, 12.8508471167, 11.0068847716, 10.5269852828, 11.6018512949, 11.9364491274, 12.5930138393, 14.1534101932, 12.8527148228, 13.7612142075, 17.0805691305, 16.2075167944, 13.3934514758, 14.0867626519, 14.2032598771, 15.0857006213, 15.013552305, 14.0313412547, 12.135491212, 12.2369652007, 12.5848578865, 14.3665545212, 14.6512326177, 17.9473021393, 17.637910592, 17.6450058815, 16.9151730536, 17.8454971675, 17.3634348619, 18.9307199633, 19.8120854632, 13.3347977563, 12.0984053648, 11.7573330508, 12.2786179158, 12.6869932445, 12.9298230164, 12.9021355724, 11.1932336972, 10.4018082907, 10.6433904918, 8.9665956669, 10.0441422023, 9.8411021397, 11.156915432, 12.2922813129, 12.7324133307, 12.4333374363, 14.2182749693, 16.3032982255, 15.7153357102, 13.4877969044, 14.003474673, 14.7386747868, 15.4382201198, 15.3021222949, 15.5810272566, 15.2587873664, 13.9998742543, 13.9330954835, 14.7715389242, 14.3395174171, 15.7056347997, 15.8164101836, 13.0788439655, 12.8818766903, 12.7131049318, 11.902917027, 12.1086560283, 11.1001343106, 11.1628694474, 12.2665434897, 11.3948223324, 13.0975296026, 13.5888086787, 14.7223710566, 15.7528611112, 14.6706920293, 15.2461993302, 15.8276626851, 16.6185302321, 16.9725149208, 17.0253666422, 13.7096314523, 14.21691556, 14.2145289832, 15.2566130373, 15.451120014, 18.7176806109, 19.465523395, 18.1363037193, 18.2890479586, 18.6966817149, 19.1050225495, 20.1510012827, 19.0969177797, 8.8256660729, 10.7287064848, 11.1755957551, 12.5464431168, 13.5782400504, 14.3291705054, 16.0643922456, 18.1054627279, 16.5904674129, 17.8479395297, 19.4626128056, 19.9053385751, 20.5619644474, 5.8689882072, 6.419458589, 5.7884214763, 6.1436117487, 5.2972203925, 4.0015013308, 5.3610282842, 6.3477478478, 4.4363084527, 4.5711234493, 3.6783037054, 1.49758312, 2.7985082951, 7.8458113232, 8.7957079815, 7.4685124709, 7.7663097412, 8.310616763, 8.6237472826, 11.6793804837, 11.7533028027, 9.3986950206, 9.2164817918, 9.7784216823, 9.5776700884, 9.9675528149, 4.0857594941, 3.966427364, 4.0574059547, 3.7650364281, 2.3161792071, 2.9744195178, 4.1134825491, 4.4449831942, 1.5772856441, 1.6075214855, 1.0095410832, -0.20286194429, -0.27927002244, 5.751774652, 7.1693585826, 6.338941419, 6.8341811743, 6.7081733263, 6.7802494346, 9.3554452682, 10.1673631622, 8.4010359979, 10.017923456, 8.6609767305, 8.8043748242, 9.1006546393, 10.1618114335, 9.8475459937, 9.596396349, 10.6148371891, 9.0421272043, 10.0214730789, 12.1582125581, 11.957119126, 10.1901741794, 10.4154248574, 10.4236657686, 9.5930626831, 9.8634089502, 10.0806884768, 11.1477828058, 11.285961495, 11.2800736003, 13.0324951082, 14.1062091173, 15.3557285681, 17.0233049324, 16.4401174052, 16.7586282798, 16.7643484107, 18.1621955952, 18.34205094, 6.9186830262, 5.4977495501, 6.3177651341, 4.9802183326, 5.1945180272, 5.2401130314, 5.020047615, 6.8168983426, 3.1791936295, 3.4557030444, 2.9900046754, 2.5467846727, 2.662920327, 7.6918999027, 8.1394448182, 8.824080282, 7.6897509077, 8.2437685642, 7.2430013261, 10.0570725365, 9.7007103716, 7.2333956045, 7.867652728, 7.7183550843, 7.2155505451, 7.4567469703, 12.1107327669, 12.5382549801, 11.2588880084, 11.344137983, 12.130773221, 11.5732531898, 12.4932271519, 13.012476073, 10.5853174423, 10.0537673316, 10.5960543789, 10.0384921123, 9.6750547156, 12.7069622893, 13.0257737182, 12.6935210752, 12.9195272472, 12.9627617752, 13.8658942697, 14.8848141705, 13.6729008998, 12.8208553361, 13.1218208741, 12.1215391775, 13.3673030886, 12.5430419823, 7.8383215906, 9.3267241714, 7.6927836703, 9.6043619253, 8.6404389367, 8.0308227714, 8.1798088698, 11.2741834129, 8.214432482, 8.5225844922, 8.6749589213, 8.5075598971, 9.3348364919, 10.2107325619, 10.68042971, 8.6084055548, 9.0596159016, 8.1602198671, 9.1865560824, 8.7512475028, 9.7533570034, 8.7357998193, 7.6085966546, 7.4980174013, 7.8450805261, 7.7878665842, 11.4048734776, 10.9547617118, 10.0367390754, 9.646233326, 9.2037793885, 8.7336549422, 8.2656234594, 9.4346406281, 6.3727513689, 6.7669102751, 5.8711323163, 5.0563131474, 4.5152199425, 10.6042230801, 9.1739066507, 10.3398737417, 9.2334002895, 8.9877236634, 9.825245037, 9.8666688631, 12.0381905606, 9.1294587164, 8.7222634717, 8.0554941767, 8.1039125366, 8.0648519808, 13.0353972861, 10.9978552921, 11.0198822119, 9.9498688882, 8.5343869743, 8.808835218, 7.6009575751, 8.6002825289, 5.692536701, 4.2725734326, 4.1625328981, 3.7313643346, 1.9934532155, 10.7484409319, 10.0766671341, 9.9243402936, 9.9190350575, 10.5082028603, 10.6693778488, 10.1151316342, 12.3274461455, 11.05807031, 10.8655911755, 9.5687231492, 10.6388918102, 11.447011835, 12.245484534, 11.4305908147, 12.3191562301, 11.5701471191, 12.3770834441, 13.1193351299, 12.5340111603, 14.3087123062, 12.4541999972, 13.6224870131, 12.8627379091, 13.3352774577, 12.1274767639, 10.9184854943, 11.552079352, 10.5449370513, 9.3827594481, 10.0341456262, 9.9270791243, 7.4652735767, 9.5676320471, 6.6380715538, 6.5027384501, 5.423155321, 5.2886499153, 5.4842826357, 10.3106702003, 11.7358729681, 10.3599968551, 10.8979112993, 12.1040897063, 12.8352707203, 12.1864278597, 15.0459635138, 12.9583254258, 12.9486526963, 13.9260468147, 13.7872054009, 14.278483649, 8.0286415476, 6.870117379, 7.4235540576, 7.5222012785, 7.8598040482, 8.2152767429, 7.2603675831, 7.8613302922, 7.6616180169, 7.6878482936, 7.3776131705, 8.1214447162, 6.9041907289, 6.5423624133, 7.3299981066, 8.2056976072, 9.4083100826, 10.0659514559, 10.4712167813, 11.2509087546, 13.9075772087, 12.5874372813, 12.7225403466, 13.8360266177, 14.886388434, 15.3464979698, 10.355791191, 11.098943682, 9.3694116313, 9.9451388117, 10.2122193829, 9.8285337175, 9.5238237116, 10.8418499062, 12.1132650914, 9.9936583776, 9.2731338079, 9.1372733153, 9.1694269619, 10.9933564726, 10.9397267771, 11.8049274392, 10.059589735, 11.1666881949, 10.7655252739, 10.3224749537, 10.4172366954, 12.8191059172, 10.5733247682, 10.2052803129, 9.598060354, 9.7693231592, 12.286603291, 12.6714691982, 12.6022447108, 13.5063887924, 13.4024039762, 13.5144838387, 13.0686230524, 13.8631259537, 16.7905690752, 15.3030762649, 16.3718092289, 15.3352781689, 15.5896841804, 7.4065307804, 8.2075382521, 8.7742700003, 9.5897033703, 9.2787434, 10.4599613013, 10.8587443597, 9.6613694978, 13.0298875613, 10.4060639058, 11.654313024, 12.1120551935, 12.266249734, 7.1426595065, 7.0353656463, 7.0901093071, 7.7089044079, 7.6034627364, 7.5838074633, 7.2987243441, 8.043293194, 10.8930580207, 7.0151092738, 8.0577685007, 8.1332261466, 8.4097450774, 10.7500657958, 11.4887522297, 12.2294637568, 13.9352106976, 15.208432485, 15.796327763, 16.6027229742, 17.4781846203, 21.0359763252, 19.7425058671, 21.0734655021, 22.3637380584, 24.0099262683, 9.878386664, 11.4625991976, 13.5113623654, 14.1824707472, 13.9854971488, 16.0425280254, 16.9886875806, 18.3742312399, 21.905350513, 20.7050782072, 21.2203507328, 22.7696702531, 24.3353465141, 11.1873299907, 10.9159224178, 10.1342750634, 10.4807362246, 11.3821027424, 11.3075400089, 12.5172085435, 11.8874912857, 13.0620402531, 13.3982751158, 11.4837320839, 13.7011838806, 12.9454560599, 9.5449722621, 10.2099566122, 10.6086722668, 10.9340610948, 12.0119283512, 12.7340474229, 13.6352670023, 13.1270262075, 14.117418674, 14.7060503218, 15.4871387752, 17.1092879104, 16.729328644, 9.7531102147, 10.3050597944, 10.2546270982, 9.7007845084, 9.5040469112, 8.4947090914, 10.2868020671, 10.4735128586, 9.5355935109, 9.367632533, 9.3044261601, 9.8777323451, 8.3360040816, 6.8814745641, 8.2749321748, 9.5817246261, 9.0856737138, 8.8730171781, 11.4925856497, 11.3673089404, 12.1913414502, 12.4721907209, 12.6965935168, 13.4356696665, 14.3953862904, 14.0120352774, 12.2791464525, 12.0689280903, 12.0235083035, 11.8669940719, 10.5123811725, 11.1885845023, 10.9818152993, 10.0912326804, 9.8973239629, 9.2837431144, 9.9012526631, 10.1063567302, 8.9826588299, 7.6181215937, 6.5025268312, 6.9875695791, 5.8406095235, 5.144122502, 5.6003257619, 4.2894699793, 3.9228803275, 2.68398141, 3.6307268237, 2.7706940513, 2.1555136647, 3.3392309197, 12.3774678777, 11.577220471, 11.4519339595, 12.1575702562, 11.4936852429, 11.5278524855, 11.0816568977, 10.7052069759, 10.0215844127, 10.7010206877, 10.3588671456, 11.2869406567, 9.9315615839, 8.5016153099, 9.4285049394, 8.7942181184, 8.5801159729, 7.6481997993, 9.1864095514, 7.5988698644, 7.4207375398, 8.2143833098, 8.0063610743, 7.1892767185, 7.1623250686, 7.8980730803, 10.5944656541, 11.0746849765, 11.676422781, 12.7241677065, 11.3901635095, 12.8717200681, 12.8080723782, 13.7064474665, 14.2112359404, 14.8618309336, 14.7663065719, 15.7988169267, 17.2102377244, 10.1856546874, 10.7659206734, 9.8488795844, 9.0516176703, 10.5595271132, 9.8446395315, 10.9704649097, 9.4114886453, 9.9963468045, 9.9152093963, 9.7269373222, 8.582331589, 10.0308636791, 8.1992212171, 8.9127910639, 9.6949770201, 10.6668510575, 10.5023137726, 10.9679038799, 11.1875993148, 11.3730600077, 11.886951974, 12.3119599065, 12.4870088156, 12.2024815129, 13.5886010329, 12.0173024969, 13.5149584188, 14.928347202, 15.6543215076, 16.5807274285, 17.5204797174, 18.4661011532, 19.842900659, 20.34978303, 21.7472572872, 23.2848189341, 24.3110890655, 24.5032550016, 10.7702759171, 9.6266787027, 9.700450938, 8.1772543173, 9.3102384179, 8.88448859, 8.3561828656, 7.3300957018, 8.0364338879, 6.9758267845, 6.6885746043, 7.0154682671, 7.2485721667, 11.1708915917, 12.8224824453, 10.8864400839, 10.2505275169, 10.4885400086, 9.4593201261, 9.0232273756, 8.7310753964, 7.4991032819, 7.9417049466, 6.7464953876, 6.0856179362, 5.9406078573, 8.6963386078, 8.1817487916, 8.3065717351, 8.2525720029, 8.6781272205, 8.8942680041, 7.756629398, 7.1514447911, 7.8725821443, 7.1873562298, 7.5694079188, 5.9062180298, 7.3991831088, 12.2512810564, 12.5358946756, 13.1764707132, 13.2310238087, 12.7965947185, 13.2321082666, 14.279010779, 14.2258196477, 14.5687716182, 15.4391492212, 15.4986050214, 15.7229331276, 16.8252177666, 10.677835411, 10.5865473422, 10.7386307618, 10.0467163088, 10.483534209, 9.5402006465, 9.0399018147, 9.2923804225, 8.1228494304, 9.2108353, 8.5242310396, 8.1943443123, 8.2011902406, 8.1606361659, 9.9413095246, 11.3690539321, 12.8570434462, 13.222611943, 14.9317353685, 15.2900516997, 16.4652672537, 17.1128738069, 18.2770387938, 20.8487871863, 21.168835117, 21.9896945237, 7.9683334265, 8.9286010811, 7.9414433348, 9.4221446053, 8.5952851517, 8.5620712096, 8.9865508267, 8.5875560014, 8.6802607213, 9.1153467321, 8.3104638056, 8.1011075451, 8.1976526566, 12.6015976087, 13.79227206, 13.9629800622, 14.5045338669, 15.5967884041, 17.7566746186, 16.6224306462, 17.4067446383, 19.1516580274, 19.4880246074, 20.9623522274, 21.389297902, 21.8916940126, 10.2543669148, 11.3544826344, 11.2090670401, 10.5403224623, 10.317748646, 11.6964107758, 11.2873719474, 11.4817951524, 11.3607011577, 11.2260568179, 11.8968321717, 12.0030855402, 11.7847335925, 11.7654465038, 11.2792011425, 11.1719738364, 10.8812596505, 10.6119696267, 10.4530976372, 10.3164725013, 8.8860712233, 8.9490659362, 9.4496490292, 7.2112084668, 7.7703364724, 6.2482649657, 7.4498620651, 7.8531943895, 6.4629629082, 7.5915861068, 7.3755905585, 6.6797001355, 5.885431931, 6.7673075331, 5.1236509706, 5.2116006893, 4.9220057977, 4.5147092403, 3.5564939509, 12.7033224566, 13.1005622573, 12.6259229094, 14.3623920431, 14.9494769937, 14.7238667877, 16.4896377019, 16.3746379926, 17.3650828848, 17.5993786417, 18.9841400735, 19.3401835525, 20.0741429263, 13.0358786792, 13.5812812468, 12.6757023233, 12.6858936956, 12.9600155629, 12.8730459845, 12.6564905567, 14.4681817677, 13.4693444149, 13.5056231222, 14.3006800182, 13.9919420335, 14.4509859864, 10.0787030077, 11.0161427676, 11.1119517838, 10.8579673759, 10.2985348273, 11.1914059893, 9.2908299066, 10.8041817581, 11.2094913091, 11.262756338, 10.6984580332, 10.5651758642, 12.3961388555, 11.9783101789, 11.4024197209, 12.9084448379, 13.2980743843, 14.7949171879, 15.2549854442, 16.1696227688, 16.8559293926, 18.3405411483, 19.0304754457, 19.7963980744, 20.884841575, 22.6615368536, 12.3893114289, 13.2804834232, 12.9792535328, 12.8822554812, 12.4631645167, 12.8310171743, 11.9084046496, 12.4102447176, 11.4105803678, 11.4803168059, 11.7054512527, 12.0013667517, 11.4685976555, 10.4651777397, 11.6016323952, 12.4935880485, 14.115516861, 13.1919519345, 15.2595409343, 16.5883674215, 15.0010454235, 17.4447985144, 17.5860870957, 17.8717063104, 18.7597564076, 18.8299762674, 11.6037620359, 11.729854798, 11.9464129839, 11.2993717837, 11.7019828075, 11.3040298165, 12.7056472626, 11.9592391836, 13.0002399896, 12.63142183, 12.6831100199, 12.4155007826, 13.3389318405, 16.5385762363, 17.7161034113, 17.8544416414, 17.3711408951, 17.4646147229, 19.1901743302, 20.2221530273, 20.7122584599, 21.1926970236, 21.8628213564, 23.2805023046, 22.8636879272, 24.4813867863, 9.703422155, 9.8237687398, 7.3261880274, 6.0866958333, 4.6977926579, 3.7899967322, 1.8573929341, 1.4632938456, -0.33060845913, -2.6716526341, -2.875818796, -3.8816767824, -6.2939096705, 11.728301412, 11.7270607069, 11.9840197335, 12.5893341927, 13.0926188763, 13.8462203925, 13.8606332705, 13.1710828702, 13.5973457855, 13.4141818947, 13.7996913399, 14.0421633713, 14.9621847921, 10.2175561819, 9.8516496525, 10.1012586589, 9.5637506618, 9.3787330852, 7.2875530432, 8.2021433852, 8.1433780874, 7.1140048924, 6.9534033236, 6.5956183438, 7.0706625583, 5.57373403, 10.1664266247, 9.6334848938, 10.111719932, 9.9460919758, 10.3751090495, 10.3047060884, 11.5179895732, 10.7413683653, 11.5627917548, 11.4909172596, 12.243994045, 11.6123582858, 12.5642071488, 10.9678818059, 10.3523422131, 9.3943538891, 9.2289652227, 7.8304703, 7.8201427455, 8.2254757096, 6.0980640882, 6.723780218, 7.0672446627, 6.0374875544, 5.1550728691, 3.8688472359, 10.4995887723, 10.2276841479, 10.9763093737, 11.3347930967, 10.439423484, 10.5904570889, 11.3930835134, 9.8106326994, 10.3748657462, 10.7280209244, 10.1806222811, 10.0998178044, 10.6244914378, 14.4075963107, 13.8341779792, 12.7900733848, 12.7189917504, 12.1805277156, 11.3342923226, 10.0679542582, 9.707101245, 8.9579520295, 8.5054648222, 8.4177618953, 7.1743043198, 6.3508384026, 12.1102413302, 11.3747284025, 11.00934426, 10.0455096307, 8.557398718, 7.3979316434, 7.0939584047, 5.4301425524, 4.1776005389, 3.4468950865, 2.5155258906, 1.3713112491, 0.63716517791, 11.081480697, 10.9502572618, 10.5629441958, 10.4119661371, 9.417541206, 9.2043965901, 9.4069344883, 8.3498664087, 7.782845323, 8.2296020119, 6.6900470024, 6.7848884914, 5.8301339359] + }, + "params": { + "pattern": "single_baseline_multi_path", + "n_switcher_groups": 80, + "n_realized_groups": 120, + "n_periods": 13, + "seed": 117, + "effects": 3, + "placebo": 1, + "by_path": 3, + "trends_lin": true, + "ci_level": 95 + }, + "results": { + "by_path": [ + { + "path": "0,1,1,1", + "frequency_rank": 1, + "horizons": { + "1": { + "effect": 1.9473624681, + "se": 0.21052169486, + "ci_lo": 1.5347475282, + "ci_hi": 2.359977408, + "n_switchers": 38, + "n_obs": 180 + }, + "2": { + "effect": 1.5423817614, + "se": 0.34099545257, + "ci_lo": 0.87404295547, + "ci_hi": 2.2107205673, + "n_switchers": 38, + "n_obs": 149 + }, + "3": { + "effect": 1.5916235802, + "se": 0.47409210676, + "ci_lo": 0.6624201256, + "ci_hi": 2.5208270348, + "n_switchers": 38, + "n_obs": 125 + }, + "-1": { + "effect": 0.43629889827, + "se": 0.30710654377, + "ci_lo": -0.16561886693, + "ci_hi": 1.0382166635, + "n_switchers": 18, + "n_obs": 80 + } + } + }, + { + "path": "0,1,1,0", + "frequency_rank": 2, + "horizons": { + "1": { + "effect": 1.6776380774, + "se": 0.3124181582, + "ci_lo": 1.0653097392, + "ci_hi": 2.2899664155, + "n_switchers": 24, + "n_obs": 111 + }, + "2": { + "effect": 1.8275981267, + "se": 0.50680823802, + "ci_lo": 0.83427223313, + "ci_hi": 2.8209240203, + "n_switchers": 24, + "n_obs": 89 + }, + "3": { + "effect": -0.19775938148, + "se": 0.68369459271, + "ci_lo": -1.5377761596, + "ci_hi": 1.1422573966, + "n_switchers": 24, + "n_obs": 71 + }, + "-1": { + "effect": 0.073204143596, + "se": 0.34538448969, + "ci_lo": -0.60373701702, + "ci_hi": 0.75014530421, + "n_switchers": 24, + "n_obs": 111 + } + } + }, + { + "path": "0,1,0,0", + "frequency_rank": 3, + "horizons": { + "1": { + "effect": 2.242169714, + "se": 0.3303303128, + "ci_lo": 1.5947341979, + "ci_hi": 2.8896052301, + "n_switchers": 18, + "n_obs": 65 + }, + "2": { + "effect": 0.025221819683, + "se": 0.53975149737, + "ci_lo": -1.0326716758, + "ci_hi": 1.0831153151, + "n_switchers": 18, + "n_obs": 58 + }, + "3": { + "effect": 0.07893553466, + "se": 0.79067071104, + "ci_lo": -1.4707505826, + "ci_hi": 1.6286216519, + "n_switchers": 18, + "n_obs": 58 + }, + "-1": { + "effect": 0.066712859101, + "se": 0.24331825321, + "ci_lo": -0.41018215398, + "ci_hi": 0.54360787218, + "n_switchers": 18, + "n_obs": 65 + } + } + } + ] + } + }, + "multi_path_reversible_by_path_trends_nonparam": { + "data": { + "group": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 36, 36, 36, 36, 36, 36, 36, 36, 36, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 41, 41, 41, 41, 41, 41, 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 67, 67, 67, 67, 67, 67, 67, 67, 67, 67, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 69, 69, 69, 69, 69, 69, 69, 69, 69, 69, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, 74, 74, 74, 74, 74, 74, 74, 74, 74, 74, 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 82, 82, 82, 82, 82, 82, 82, 82, 82, 82, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 87, 87, 87, 87, 87, 87, 87, 87, 87, 87, 88, 88, 88, 88, 88, 88, 88, 88, 88, 88, 89, 89, 89, 89, 89, 89, 89, 89, 89, 89, 90, 90, 90, 90, 90, 90, 90, 90, 90, 90, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 93, 93, 93, 93, 93, 93, 93, 93, 93, 93, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 95, 95, 95, 95, 95, 95, 95, 95, 95, 95, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 97, 97, 97, 97, 97, 97, 97, 97, 97, 97, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 107, 107, 107, 107, 107, 107, 107, 107, 107, 107, 108, 108, 108, 108, 108, 108, 108, 108, 108, 108, 109, 109, 109, 109, 109, 109, 109, 109, 109, 109, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 111, 111, 111, 111, 111, 111, 111, 111, 111, 111, 112, 112, 112, 112, 112, 112, 112, 112, 112, 112, 113, 113, 113, 113, 113, 113, 113, 113, 113, 113, 114, 114, 114, 114, 114, 114, 114, 114, 114, 114, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119], + "period": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9], + "treatment": [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], + "outcome": [6.4634047155, 8.9447509959, 8.4923639645, 9.4320694104, 8.8595036256, 9.6611385119, 9.8473224039, 10.1843034702, 9.2491353232, 9.017723347, 10.6124116198, 12.6882113704, 13.6666296494, 12.6122833286, 12.8625322133, 13.1868030093, 12.1672766759, 12.9146539636, 13.6109369269, 12.8751178951, 10.6772766646, 11.9357543055, 11.6374530164, 12.5042088418, 12.3812814051, 12.2295406672, 12.5124219971, 13.1474581858, 12.8069178505, 12.8418105121, 10.7013559575, 12.2677900506, 12.6502304002, 13.2624696471, 13.1892754712, 12.6683055091, 13.3881709153, 13.3731078988, 12.7552105522, 13.7214559835, 6.0721478453, 8.4788747191, 8.7868740948, 8.250194387, 8.6021056708, 8.6417846479, 8.5967373897, 9.3829374711, 9.7399883388, 10.5137623125, 7.9559639743, 9.9489682412, 10.1612284327, 9.7840950365, 8.916389572, 10.7535755539, 10.5076604565, 9.7393074281, 10.2776586149, 11.5633443391, 10.2662377104, 11.1842439446, 11.9437806392, 11.6122988913, 11.8932478009, 12.0095797455, 12.3739130869, 12.8506330506, 12.5033859798, 12.6490510108, 11.6894180868, 13.3608314367, 13.2295006198, 14.2755553694, 13.1992279174, 13.1717857943, 13.8973926888, 13.4990602621, 13.9096859483, 13.695658739, 9.0355346401, 11.1934994867, 11.5785789744, 11.3123756559, 10.4280449116, 11.8607019874, 11.4744780946, 11.4564089544, 11.8321415468, 11.2977297636, 11.3094576688, 13.4376629676, 13.4415718968, 13.0846966926, 14.3522724652, 14.2750768782, 13.976166324, 13.1906749936, 14.029248159, 13.3259649925, 6.9868222042, 8.2172426214, 9.2710080464, 9.7373683752, 9.2913840235, 9.3578953869, 9.4253702727, 9.0663851526, 10.2418163139, 10.1222739879, 11.0235629287, 13.8057970509, 13.1745895862, 13.4013003593, 12.9426148012, 13.1502823424, 13.3042290167, 13.7082183208, 13.4615667708, 14.1911984177, 11.2708962386, 13.5035479213, 13.3770300191, 14.2794912894, 13.85315257, 13.2059479665, 13.0516502452, 13.6451763235, 13.9161025637, 14.1226078331, 9.4742545682, 12.1475433419, 11.7983000919, 11.4785981358, 11.9981043264, 12.1742463767, 12.5934330356, 11.9280639362, 12.2809134953, 12.9794234124, 11.0317885313, 12.4851680102, 13.0175592996, 12.3055885632, 13.3063204983, 13.526566489, 13.1768138284, 13.3096135938, 14.3498882403, 12.8662376796, 10.4893550989, 13.7239159655, 13.0928751659, 13.1181222807, 13.5392580851, 13.3046061032, 13.5055863898, 14.8688037708, 14.517046827, 15.0614363281, 12.0905575816, 12.9216645581, 14.7681420509, 14.2576239976, 14.04823184, 14.4549410656, 14.6357766757, 14.997526491, 14.1526256638, 14.7414254625, 11.6948600267, 12.9299404978, 13.8423585868, 13.5521756522, 12.0795510558, 13.9309432861, 14.2616636368, 13.9462133066, 14.2838125161, 14.3927365425, 11.1314192188, 13.044698825, 13.760191851, 14.5764532812, 14.2456033578, 13.9468348948, 13.5176141049, 14.5705671429, 13.4859552099, 14.722728218, 11.5410413746, 13.9879586871, 14.0006342622, 13.6831624351, 14.8312538419, 13.7578631343, 14.3754907007, 13.7581860903, 14.9274846885, 14.1241395999, 13.9743044234, 13.9542521985, 14.9915845699, 15.4849085273, 16.1248369801, 16.6962014768, 15.9290826004, 16.4958896067, 17.1723795041, 16.2076757098, 10.3624070634, 10.8388037076, 12.3054528755, 12.6874374779, 13.1113859074, 12.912797312, 13.2100506719, 12.8452245582, 14.5382602992, 13.2951101003, 11.5657441636, 12.9944937965, 13.8366688794, 15.0093193619, 14.626013212, 14.5139860339, 13.7458783372, 14.480278726, 15.8220164046, 14.6574257032, 11.9353804259, 12.479279303, 13.4109138625, 15.0827678557, 14.5686511639, 13.9497958712, 14.5334415131, 15.2254362119, 14.8102726333, 15.1016824121, 10.080005987, 10.4859062832, 14.2946102951, 14.6858580746, 13.2545468068, 13.5352606496, 13.1186218798, 14.3206765963, 13.9489866518, 13.7056744914, 8.899709491, 9.9613631662, 12.1428008126, 12.4627323979, 12.3083491316, 13.5342505911, 13.0544758493, 12.5042339106, 13.5082445426, 13.6723882293, 9.4175920782, 8.9001805244, 12.1449781703, 12.2561253156, 11.8755838436, 11.7062942382, 12.2640785206, 12.7948904777, 11.7724140498, 12.5483593901, 11.6151852887, 11.4891872291, 11.9881212172, 13.1908868164, 12.1486855607, 12.7477499463, 13.1853553361, 13.468884115, 12.5674310496, 12.6650954881, 9.4654674497, 11.3001631345, 12.3452042848, 12.7934446238, 12.3364367587, 14.1307363775, 13.5711684091, 13.623303528, 13.0905339616, 13.4138237981, 6.5150674389, 6.6917771659, 9.5548382221, 8.468157604, 8.47938515, 9.9418374095, 9.1235111346, 9.6387695795, 9.9146707408, 9.4605024156, 7.8729538964, 9.1099426769, 11.0408102834, 11.3374520484, 10.9944659538, 11.2782982058, 11.4261825688, 10.5214456001, 11.4537551354, 12.403260222, 5.9564266599, 6.3982667378, 8.4513324426, 8.5504662403, 8.7337270892, 8.4149537, 8.7397737411, 8.4408621813, 8.186887403, 9.2580798005, 6.6620609462, 5.313984736, 9.0474074101, 7.6208811262, 9.2286813499, 9.6893994059, 8.4498504926, 8.9086513492, 8.6355108531, 8.6965177852, 14.41680428, 13.8382362241, 15.7279702931, 16.7081668696, 17.1897425191, 16.5892256536, 16.5696130764, 16.148221531, 16.9050871542, 17.189579536, 12.6030374263, 11.8829050982, 15.5457743376, 14.8111253291, 14.19662849, 15.3826933896, 15.3174180155, 15.6771055191, 14.5408237459, 15.1890817129, 9.4498330498, 9.4989555546, 12.0940275663, 10.9818885177, 12.1272550786, 11.875879157, 11.1974284042, 12.2486548619, 11.8814738668, 12.111668881, 8.5647366034, 8.6628778544, 11.0025942549, 10.4488442311, 10.9050573724, 11.0493478146, 10.8560049615, 11.7371794174, 11.4941831304, 11.2200982861, 11.7838334497, 10.9931018984, 13.9549316537, 14.5943980163, 13.3046509087, 14.2724200813, 13.8185244077, 14.1226887733, 15.1717286485, 14.5038148668, 8.1685559886, 9.0116366418, 11.2051493256, 11.3273588848, 11.1076544889, 11.0372904384, 12.1452504207, 10.7943953761, 12.1396342524, 11.2182662803, 6.6134687078, 6.4666729895, 9.9357245058, 8.7486203219, 9.7406313079, 10.1091228238, 9.4232589048, 9.2215931031, 9.9840534396, 9.7483893743, 13.2178707714, 12.5487873843, 13.3514003061, 15.62235035, 15.4981466315, 13.0826897739, 14.2917327538, 14.129373652, 14.6727016247, 14.7339212132, 14.4082198006, 13.5402763326, 14.4352321307, 15.4946329445, 16.3674138996, 14.8661127898, 15.0348227162, 15.2892278092, 14.3894317542, 14.5155445843, 15.7804485051, 15.3033781442, 16.0896439298, 17.5926581753, 17.3536794059, 15.238162834, 15.1493878972, 15.8155552053, 15.8004269498, 15.8919112925, 9.542859233, 9.2901585933, 8.8582121092, 12.0548030064, 11.9987131246, 9.9834998088, 10.2698871615, 10.4568757682, 10.951831549, 10.5463892726, 11.3735582805, 11.3346464621, 11.8150443043, 13.4643227564, 13.0533481249, 11.0665501594, 11.4808275399, 11.9307681842, 12.2800548175, 11.7292036594, 5.2898803995, 4.9830425649, 5.1985806253, 8.1907290892, 7.6177281858, 4.8873892386, 5.2518455206, 5.7013978698, 5.5290136821, 6.9931478905, 7.644503296, 8.5036766685, 9.0041517118, 11.4840607895, 11.2936137278, 8.616111333, 9.2887474484, 9.0800027883, 9.5124285954, 8.5497284387, 8.4842252156, 8.5518977155, 9.1874361525, 11.2090670117, 10.9066335693, 8.9192637554, 9.1598349647, 9.2000659095, 9.870565493, 9.545315057, 6.3890737541, 6.4714606377, 5.9040636196, 8.9715684238, 8.888383127, 6.756442361, 7.9617943785, 7.1361743502, 6.5888038005, 7.2112043151, 10.4273261906, 10.1914253706, 10.0000907067, 12.4252624805, 12.7157421505, 11.0824121671, 10.012741297, 11.4842134407, 10.9673174772, 11.2943505903, 8.0851184806, 8.5062261319, 9.0835553687, 10.0090403522, 10.8556157027, 8.2180009688, 8.5147887021, 8.8773322864, 9.2711951125, 8.5810055246, 6.3954388055, 7.1336061267, 7.9420079804, 9.12528467, 9.8849087519, 8.2053153221, 7.4072483328, 7.4717937563, 8.2540280743, 8.0338750219, 8.90938829, 10.1115594111, 9.7517130846, 12.3969495018, 11.8955620152, 11.2526532979, 11.4481027185, 10.4226627701, 10.6027053414, 10.2914661234, 10.8615629311, 10.6963222978, 10.450867039, 13.0155381631, 14.7824087348, 11.856029576, 11.6832091599, 12.4752374788, 11.5629757885, 13.1183557733, 12.6508657423, 13.5007471256, 13.2720898366, 14.9748870016, 15.2405396706, 13.4638520423, 14.0936499209, 13.8368184339, 14.5570491702, 12.7476507599, 10.1247810636, 12.0359541543, 11.8867009674, 11.6650605946, 13.9269171844, 14.4399495134, 12.5424676759, 12.5964665142, 11.9677467582, 12.2978034144, 8.9458886814, 8.3402568034, 9.7836817579, 7.7457544845, 11.0117768199, 12.2860756886, 9.3682689438, 9.5949702035, 9.6010341355, 9.3282396282, 12.9966391497, 12.9977095876, 12.9696027929, 13.5069520696, 15.6065338341, 15.6777827051, 14.2404400544, 13.6814598693, 14.638720975, 14.7305775247, 7.8202198546, 8.0419709789, 7.6842907804, 7.6824286764, 10.1881268434, 9.9057448234, 8.190705737, 7.8882084097, 8.3779243902, 8.5504958193, 9.7173423113, 9.77119863, 9.9829547863, 10.2566386764, 11.3393527142, 11.3048445246, 9.9003477966, 9.6689890686, 10.3934859364, 9.888426243, 11.4463695292, 12.9973004361, 12.2077881155, 12.2170606001, 14.4139602675, 14.7453786384, 12.2136998951, 12.966989354, 13.0072534368, 13.4667443713, 11.0305099096, 10.0366878464, 11.0234264902, 10.4684807334, 13.7227446073, 12.9712630789, 10.6701171833, 10.9918104747, 11.5295734257, 10.6211701405, 4.1105704185, 4.296267329, 5.0151193711, 4.8734003914, 7.2784861665, 6.7536159051, 5.4246986988, 4.6654179469, 4.6348146307, 4.7435636902, 10.5548374034, 10.848237225, 12.0781738915, 10.4291107164, 12.6721525389, 14.3689365342, 11.7988032899, 11.5372700779, 11.6120062766, 11.6556206929, 7.8852858352, 7.8146537884, 9.1974132686, 8.0380037936, 10.7216279347, 10.9567811523, 9.0039154702, 9.3971628881, 8.7898225065, 8.9704413208, 6.2690417141, 6.3725658071, 7.3360351858, 6.6382521993, 7.7077647555, 8.9467890098, 6.8488464423, 7.2784230143, 6.8631206729, 8.4582260212, 8.3779353389, 8.387762414, 9.1792836439, 9.5740915701, 9.474222541, 11.0629286745, 8.8194799917, 10.0873362621, 9.4536103619, 9.1863563054, 12.7481035661, 12.2114212211, 12.2537183508, 13.0249477467, 12.357909268, 15.0287044863, 12.9488353349, 12.1733306028, 13.1844584041, 13.024041906, 8.3169488649, 8.0909312063, 8.84944762, 8.9510499493, 8.0113812439, 10.4516023037, 9.9159955844, 9.1906744601, 8.6812325573, 9.1762495119, 8.6150444483, 8.5634895827, 8.7716325951, 8.6745280606, 8.0744009868, 10.1472302122, 9.9615850693, 8.9837568965, 8.7154254875, 9.8754181885, 9.2220846368, 10.0541745693, 11.0986557412, 10.0219968515, 9.8121536315, 11.0743324325, 10.5855446852, 10.7457951581, 11.4851478856, 10.3019803429, 8.0273966815, 8.4628137516, 8.0906891783, 7.7169373251, 8.0065795269, 10.8708142283, 7.8022033124, 9.1978191457, 9.0528261318, 8.5810623344, 10.5420793762, 10.5760292074, 10.2343977459, 9.7230440297, 10.2268209564, 12.3985959331, 10.361121358, 11.7504404946, 10.7513645911, 12.1320323003, 10.7098329829, 9.772105815, 9.7710012638, 10.2509136461, 10.364871815, 12.0129752871, 11.1653793532, 11.4508073765, 10.1576478517, 11.6389211294, 12.59652846, 12.853082822, 13.1502885787, 12.0290402692, 12.1657142727, 15.8550532945, 13.327104726, 12.7314935057, 13.9793838103, 13.8586173093, 8.4506277755, 8.3647464323, 9.040994314, 8.8908595178, 9.2848354217, 9.5085498772, 11.2473640979, 8.8242729185, 10.8556643995, 10.2431689303, 9.8899425463, 10.0120036066, 9.8170585846, 10.8473422411, 9.9276517689, 9.7495607064, 11.9234844459, 9.7312609083, 11.9451846117, 12.3891628262, 10.8602492683, 10.5455800339, 11.3114987548, 11.3304495882, 11.1855933474, 11.0922964572, 14.0981108558, 12.5768471215, 14.0952735074, 13.8087384721, 11.9083145385, 11.7624560915, 11.6836134044, 12.569161788, 12.4433782856, 12.0757803627, 14.4695755112, 13.0932392235, 14.3262651395, 14.4133759657, 7.2890605177, 7.3311765632, 8.0100800446, 7.2895917042, 6.9759091909, 8.1200275184, 10.6821954449, 8.5019946025, 9.5574546755, 10.9213387329, 12.1399126555, 12.5514499201, 12.3203497348, 12.6470864783, 13.1437804821, 13.3698642347, 13.190701541, 13.0447528965, 13.0591205218, 13.3341675484, 11.658843697, 11.8825386947, 11.8592141795, 12.1213291737, 10.9958031061, 12.518941957, 12.3297276782, 12.6660350204, 12.8559183917, 11.8257606481, 9.3873701725, 9.8643883504, 9.5696692452, 9.6871218676, 10.4273884357, 10.6057701788, 9.6284115568, 9.3147818771, 11.5172179771, 11.1410737656, 10.8211899145, 10.863087663, 12.1890386704, 12.5576841593, 12.0217913326, 12.6645140613, 11.8837738848, 13.3089811296, 12.2277063819, 12.2505745057, 9.4971816863, 8.5550158582, 10.3092682472, 10.1296693349, 11.0268863808, 9.7044568548, 9.8878635457, 9.7641510113, 9.7070970946, 10.2999988992, 6.9331725544, 5.9971964731, 6.7580903235, 7.4644717605, 8.6747459582, 8.4311356095, 7.693790941, 8.4280730427, 8.8012261983, 8.4571130812, 7.5166679177, 7.6973110072, 6.5532651804, 7.2733802857, 6.6322147081, 7.5019540228, 7.7757350376, 8.3302026483, 7.4841549858, 7.374931026, 13.1030947964, 12.4702390619, 11.90720807, 12.6776958772, 13.0743221975, 13.8629738986, 13.0877801478, 13.4095478045, 13.6613515429, 13.8448532427, 9.5522837323, 9.01132359, 9.4896440956, 10.0312241835, 9.3856389155, 10.4013418786, 9.7537373124, 9.7803696742, 10.2003591331, 10.8887958432, 9.3164529757, 8.7810802338, 8.9600350079, 9.3585730514, 9.7178189338, 9.2227036469, 8.96213626, 8.967259658, 10.0037697823, 9.7108925705, 12.2396169025, 13.8039238255, 12.0438161718, 12.7117572058, 12.5270868266, 13.1837086073, 13.0821847678, 13.0461332057, 13.31614128, 13.1304930314, 11.9953920202, 11.7696275661, 11.8603914788, 11.7175157844, 12.3576185514, 11.6549483729, 12.1514619505, 12.7530538929, 12.9835511417, 12.2940382323, 11.814128859, 11.3040386343, 11.8475306098, 11.6422473584, 11.6140513144, 11.3013085497, 11.1077733541, 12.5963327352, 12.5660110403, 12.9591231342, 12.0063720367, 10.7589481116, 10.4293773156, 10.8805866849, 11.2915954786, 11.4913078048, 11.9369737636, 10.7442303832, 12.7541683658, 11.7543173031, 6.4352381258, 6.1653728588, 6.6392920122, 6.8063176135, 7.3497182958, 6.4832244793, 7.4001352798, 6.7584668659, 7.6446019222, 6.9779000593, 11.1696659557, 10.3511563618, 11.050405813, 11.2348820772, 12.190797708, 11.7950336879, 11.8500587574, 11.3210841208, 12.2274503482, 12.0324694956, 9.9845717088, 10.138958858, 10.6368110906, 9.7047126475, 10.352419417, 10.6498228427, 9.9005882986, 11.5954624628, 10.6131229737, 10.8355877229, 9.651450992, 9.546130513, 10.2939340822, 10.2106337186, 9.5287950307, 10.3051567293, 10.3971994556, 10.4365063753, 10.134289619, 10.7723677739, 8.0023483179, 8.7072829822, 8.2714647585, 8.7975260874, 9.129579977, 8.0555446906, 8.6461380665, 9.2986304845, 8.4437044253, 10.1042230185, 7.6050267719, 8.6348693182, 9.0664999666, 9.4737154251, 9.4935821587, 8.1160274639, 9.7684411002, 9.2483829892, 9.5850417365, 10.0062886135, 8.8665536431, 10.1433128425, 10.9493199763, 10.7686272701, 10.475016802, 10.5520044273, 11.0137044966, 10.7394555593, 10.9917121273, 10.5620791246, 9.5835028783, 9.1732986225, 9.7430371593, 9.819364505, 9.9068872408, 10.2121307275, 10.2199168161, 10.8238043464, 9.9359262456, 10.0792554393, 10.7745696212, 9.7348502985, 9.4781591877, 10.4261281772, 10.8134336638, 10.5741516575, 10.8809436548, 10.3500180171, 10.5476148014, 10.3241537582, 10.2077723969, 10.8933285961, 11.8042343448, 11.2038521088, 11.2661780617, 12.4645419955, 11.9727609067, 11.9997502394, 11.8063904679, 11.5535685989, 11.6845591957, 12.0572626241, 10.958137042, 11.2107163572, 11.4056205483, 11.8184945298, 12.4119603827, 12.4665166418, 10.7731176586, 11.7291150648, 12.8184965545, 13.3043881404, 12.7800690397, 11.8337507736, 13.2510766052, 12.2894095358, 13.4256300983, 12.9975751814, 13.494538209, 12.6142088529, 13.5584041913, 12.8219220805, 13.6229300624, 14.0511561866, 13.4488099607, 13.0748411895, 14.3760677538, 13.8849019873, 14.2061521212, 13.4568420992, 15.4356681866, 16.1244603017, 17.0811086611, 16.5037287902, 16.2477432658, 16.6276553466, 16.8022315263, 17.0592736232, 15.9462370419, 17.0471345246, 13.3845064595, 12.3068281215, 13.8259593665, 13.9937773924, 13.6759763247, 14.345944256, 14.4152866157, 13.758144164, 14.2598786229, 12.8311675055, 10.8748187776, 12.109088842, 11.5995618021, 11.789368618, 12.6630689508, 11.2869594168, 12.3902568821, 11.8752055512, 11.5508802683, 11.4598140671, 15.7095731498, 14.8653180881, 15.2700536558, 15.3067075109, 14.9138972737, 15.8559701563, 15.352165616, 15.654558688, 15.4576547553, 14.8297961816, 10.9204972623, 11.1996886711, 11.3861179126, 11.2018078632, 10.8022419011, 11.9819486943, 12.0961687427, 11.5719976392, 11.6498162479, 12.3108096652, 13.728835702, 14.172686378, 14.995018131, 13.8560574922, 15.2531284001, 14.5830251371, 15.3641892167, 15.1941008231, 14.3271657163, 14.2854126186, 10.8435876316, 11.6574535903, 11.8062184041, 11.0660706113, 11.5771243477, 12.2164034487, 11.3393024591, 12.0813207683, 11.50379269, 13.1056806721, 9.1894197215, 9.9090581954, 10.1732846163, 10.3132184859, 10.6417383274, 9.3626122835, 9.3241130219, 11.1177017109, 10.5147978139, 9.6986122468, 14.8673303899, 15.0193775706, 14.7076823949, 14.7826755022, 16.3384563977, 15.4670016774, 15.8177344417, 16.5668452813, 17.0011184292, 14.9626278196, 10.5090802857, 10.6805209253, 10.8530468889, 10.2898258713, 10.6448670787, 10.6448775323, 11.6222329113, 11.1594515216, 10.957468941, 11.3825617988, 9.585488171, 9.0806715399, 9.548035496, 8.9946370013, 9.3910814761, 9.818734874, 10.0147418513, 10.5046277625, 10.1552702419, 9.6833826161, 8.2013462109, 9.2511643198, 8.1733568218, 8.8303259944, 8.4386220459, 9.8745861792, 9.0506323324, 10.2118190629, 9.1762936507, 8.7744836141, 12.5056849155, 12.9451963894, 12.7074548413, 12.4724888699, 13.7884197229, 13.188770297, 13.5414202728, 14.0740790446, 13.1685451109, 14.3996041582], + "state": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] + }, + "params": { + "pattern": "multi_path_reversible", + "n_switcher_groups": 80, + "n_realized_groups": 120, + "n_periods": 10, + "seed": 118, + "effects": 3, + "placebo": 1, + "by_path": 3, + "trends_nonparam": "state", + "ci_level": 95 + }, + "results": { + "by_path": [ + { + "path": "0,1,1,1", + "frequency_rank": 1, + "horizons": { + "1": { + "effect": 1.9079103903, + "se": 0.16211305415, + "ci_lo": 1.5901746427, + "ci_hi": 2.2256461378, + "n_switchers": 40, + "n_obs": 180 + }, + "2": { + "effect": 1.9966511573, + "se": 0.12330160938, + "ci_lo": 1.7549844437, + "ci_hi": 2.2383178709, + "n_switchers": 40, + "n_obs": 145 + }, + "3": { + "effect": 1.9302008986, + "se": 0.14424606897, + "ci_lo": 1.6474837985, + "ci_hi": 2.2129179987, + "n_switchers": 40, + "n_obs": 120 + }, + "-1": { + "effect": -0.19417742088, + "se": 0.2009313907, + "ci_lo": -0.58799571003, + "ci_hi": 0.19964086826, + "n_switchers": 20, + "n_obs": 80 + } + } + }, + { + "path": "0,1,1,0", + "frequency_rank": 2, + "horizons": { + "1": { + "effect": 2.2178798117, + "se": 0.16313871231, + "ci_lo": 1.8981338111, + "ci_hi": 2.5376258123, + "n_switchers": 25, + "n_obs": 105 + }, + "2": { + "effect": 2.2516014036, + "se": 0.21746552506, + "ci_lo": 1.8253768066, + "ci_hi": 2.6778260006, + "n_switchers": 25, + "n_obs": 85 + }, + "3": { + "effect": 0.080409496379, + "se": 0.18661231574, + "ci_lo": -0.28534392154, + "ci_hi": 0.4461629143, + "n_switchers": 25, + "n_obs": 70 + }, + "-1": { + "effect": 0.2734916973, + "se": 0.17927884821, + "ci_lo": -0.077888388381, + "ci_hi": 0.62487178299, + "n_switchers": 25, + "n_obs": 105 + } + } + }, + { + "path": "0,1,0,0", + "frequency_rank": 3, + "horizons": { + "1": { + "effect": 2.1334803711, + "se": 0.29050443384, + "ci_lo": 1.5641021434, + "ci_hi": 2.7028585988, + "n_switchers": 10, + "n_obs": 35 + }, + "2": { + "effect": 0.59466153494, + "se": 0.37303668216, + "ci_lo": -0.13647692701, + "ci_hi": 1.3257999969, + "n_switchers": 10, + "n_obs": 30 + }, + "3": { + "effect": 0.53993019623, + "se": 0.29804900455, + "ci_lo": -0.044235118322, + "ci_hi": 1.1240955108, + "n_switchers": 10, + "n_obs": 30 + }, + "-1": { + "effect": 0.14583236091, + "se": 0.24030710977, + "ci_lo": -0.32516091946, + "ci_hi": 0.61682564128, + "n_switchers": 10, + "n_obs": 35 + } + } + } + ] + } } }, "generator": "generate_reversible_did_data v1", diff --git a/diff_diff/chaisemartin_dhaultfoeuille.py b/diff_diff/chaisemartin_dhaultfoeuille.py index 9d138fed..2e6fe940 100644 --- a/diff_diff/chaisemartin_dhaultfoeuille.py +++ b/diff_diff/chaisemartin_dhaultfoeuille.py @@ -408,10 +408,9 @@ class ChaisemartinDHaultfoeuille(ChaisemartinDHaultfoeuilleBootstrapMixin): the object of interest) and ``L_max >= 1`` (the path window depends on ``L_max``). Binary treatment only — non-binary treatment + ``by_path`` is deferred. Also incompatible with - ``trends_linear``, ``trends_nonparam``, ``heterogeneity``, - ``design2``, ``honest_did``, and ``survey_design`` (each - combination raises ``NotImplementedError`` in the current - release). + ``heterogeneity``, ``design2``, ``honest_did``, and + ``survey_design`` (each combination raises + ``NotImplementedError`` in the current release). Compatible with ``controls`` (DID^X residualization) -- the per-baseline OLS residualization runs once on first-differenced @@ -439,6 +438,56 @@ class ChaisemartinDHaultfoeuille(ChaisemartinDHaultfoeuilleBootstrapMixin): configuration is detected. SE inherits the cross-path cohort- sharing deviation from R documented for ``path_effects``. + Compatible with ``trends_linear`` (DID^{fd} group-specific + linear trends) -- first-differencing replaces ``Y`` with + ``Z = Y_t - Y_{t-1}`` once globally before path enumeration, + so per-path raw second-differences DID^{fd}_{path, l} surface + on ``path_effects[path]["horizons"][l]`` automatically. Per-path + cumulated level effects ``delta_{path, l} = sum_{l'=1..l} + DID^{fd}_{path, l'}`` are surfaced on the new + ``results.path_cumulated_event_study[path][l]`` field + (mirroring the global ``linear_trends_effects`` cumulation; + inner dict keyed by horizon directly, no ``"horizons"`` wrapper). + SE on the cumulated layer is the conservative upper bound + (sum of per-horizon component SEs, NaN-consistent), matching + the global ``linear_trends_effects`` SE convention. Path + enumeration runs on the post-first-differenced ``N_mat_fd``: + switchers with ``F_g==2`` fail the window-eligibility check + and are dropped from path enumeration entirely, so a path + whose switchers all have ``F_g < 3`` is silently absent from + ``path_effects`` (the existing global ``F_g < 3`` warning + still fires). Per-path R parity matches R + ``did_multiplegt_dyn(..., by_path, trends_lin)`` on per-path + cumulated point estimates under single-baseline panels with + sufficient pre-window depth (``F_g >= 4`` for every selected- + path switcher). R re-runs the per-path full pipeline on each + path's restricted subsample; same multi-baseline divergence + pattern as ``controls`` (a ``UserWarning`` fires when switcher + baselines take multiple values). **F_g=3 boundary-case + divergence:** `F_g=3` switchers have only 1 valid pre-window + Z value after first-differencing and the ``time==1`` filter, + which causes Python's global-then-disaggregate architecture + to diverge from R's per-path full-pipeline call (30%+ on + point estimates observed empirically). A separate + ``UserWarning`` fires at fit-time when the panel includes any + `F_g=3` switchers and `by_path + trends_linear` is set, so + practitioners hitting this boundary regime see the divergence + flag explicitly. **Placebo under trends_linear returns RAW + per-horizon values, not cumulated** -- there is no per-path + placebo cumulation surface (verified empirically against R + via the existing ``joiners_only_trends_lin`` parity scenario). + + Compatible with ``trends_nonparam`` (state-set trends) -- the + set membership column is validated and stored once globally + (time-invariance, NaN rejection, partition coarseness checks + unchanged); per-path analytical SE, bootstrap SE, per-path + placebos, and per-path sup-t bands all inherit the + set-restricted control pool automatically through the + ``set_ids`` parameter threaded through the per-path IF + helpers. Per-path R parity matches R + ``did_multiplegt_dyn(..., by_path, trends_nonparam)`` on + per-path point estimates under single-baseline panels. + Compatible with ``n_bootstrap > 0`` -- the top-k paths are enumerated once on the observed data (paths held fixed across bootstrap draws, matching R ``did_multiplegt_dyn(..., by_path, @@ -1011,16 +1060,6 @@ def fit( "[F_g - 1, F_g - 1 + L_max] and therefore depends on " "the event-study horizon. Set L_max when calling fit()." ) - if trends_linear: - raise NotImplementedError( - "by_path combined with trends_linear (DID^{fd}) is " - "deferred to a future release." - ) - if trends_nonparam is not None: - raise NotImplementedError( - "by_path combined with trends_nonparam (state-set " - "trends) is deferred to a future release." - ) if heterogeneity is not None: raise NotImplementedError( "by_path combined with heterogeneity testing is " @@ -1626,6 +1665,79 @@ def fit( UserWarning, stacklevel=2, ) + # Multi-baseline switcher panel detection (`by_path + trends_linear`): + # mirror the analogous warning fired by `by_path + controls` at + # `:1565-1584`. Python first-differences once globally before + # path enumeration; R `did_multiplegt_dyn(..., by_path, trends_lin)` + # re-runs the full pipeline (including first-differencing) per + # path's restricted subsample. On single-baseline switcher + # panels the two architectures coincide; on multi-baseline + # switcher panels (some switchers have `D_{g,1}=0`, others + # `D_{g,1}=1`) per-path point estimates can diverge — warn the + # user explicitly so they don't silently consume estimates + # that disagree with R. The check filters to switcher groups + # only (never-switchers / always-treated controls don't + # contribute to switcher baseline multiplicity). + if self.by_path is not None: + _switcher_mask_tl = first_switch_idx_arr >= 0 + if _switcher_mask_tl.any(): + _switcher_baselines_tl = baselines[_switcher_mask_tl] + if np.unique(_switcher_baselines_tl).size > 1: + warnings.warn( + "by_path + trends_linear: switcher baselines " + "D_{g,1} take multiple values in this panel. " + "Python first-differences once on the full " + "panel before path enumeration; R " + "`did_multiplegt_dyn(..., by_path, trends_lin)` " + "re-runs the full pipeline (including " + "first-differencing) on each path's restricted " + "subsample, so per-path point estimates can " + "diverge between Python and R on this panel. " + "See `docs/methodology/REGISTRY.md` " + "(`Note (Phase 3 by_path ...)` -> Per-path " + "linear-trends DID^{fd}) for the full " + "deviation contract.", + UserWarning, + stacklevel=2, + ) + # F_g=3 boundary-case divergence (`by_path + trends_linear`). + # `F_g=3` switchers have exactly 2 pre-switch periods, + # which after trends_linear's first-difference and + # `time != 1` filter leaves only 1 valid pre-window Z + # value. R re-runs the full pipeline on each path's + # restricted subsample (path's switchers + same-baseline + # yet-to-treat controls), and this single-pre-period + # regime triggers different control-eligibility + # treatment in R's per-path call than Python's global- + # then-disaggregate architecture. Empirically observed + # 30-165% rel diff on path 1 of the parity fixture's + # earlier `F_g=3` variant; the shipped parity scenario + # uses `F_g >= 4` exclusively. Fire a targeted warning + # whenever the panel contains any `F_g=3` switchers AND + # `by_path` is requested, so practitioners hitting this + # boundary regime see the divergence flag explicitly. + _f_g_three_count = int((first_switch_idx_arr == 2).sum()) + if _f_g_three_count > 0: + warnings.warn( + f"by_path + trends_linear: {_f_g_three_count} " + f"switching group(s) have F_g=3 (exactly 2 " + f"pre-switch periods). After first-differencing " + f"and the time==1 filter, these groups have " + f"only 1 valid pre-window Z value, which " + f"triggers a documented boundary-case " + f"divergence between Python's global-then-" + f"disaggregate architecture and R's per-path " + f"full-pipeline call. Per-path point estimates " + f"on paths whose switchers include F_g=3 can " + f"diverge from `did_multiplegt_dyn(..., " + f"by_path, trends_lin)` by 30%+ on point " + f"estimates. See `docs/methodology/REGISTRY.md` " + f"(`Note (Phase 3 by_path ...)` -> Per-path " + f"linear-trends DID^{{fd}}) for the full " + f"deviation contract.", + UserWarning, + stacklevel=2, + ) N_mat_orig = N_mat.copy() Y_mat, N_mat = _compute_first_differenced_matrix(Y_mat, N_mat) @@ -2048,6 +2160,9 @@ def fit( # by_path disaggregation by observed treatment trajectory path_effects: Optional[Dict[Tuple[int, ...], Dict[str, Any]]] = None path_placebos: Optional[Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]] = None + path_cumulated_event_study: Optional[ + Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]] + ] = None if ( self.by_path is not None and L_max is not None @@ -2070,7 +2185,15 @@ def fit( all_groups=all_groups, alpha=self.alpha, df_inference=_inference_df(_df_s_bp, resolved_survey), + set_ids=set_ids_arr, ) + # NOTE: per-path cumulated layer is computed AFTER the + # bootstrap propagation block below (search for + # `path_cumulated_event_study =`) so it reads the final + # post-bootstrap per-horizon SEs rather than the analytical + # ones that path_effects was just populated with. This + # mirrors the global `linear_trends_effects` cumulation + # which also runs after the event_study bootstrap propagation. # Phase 2: placebos, normalized effects, cost-benefit delta multi_horizon_placebos: Optional[Dict[int, Dict[str, Any]]] = None @@ -2223,6 +2346,7 @@ def fit( multi_horizon_placebos=multi_horizon_placebos, alpha=self.alpha, df_inference=_inference_df(_df_s_bp_pl, resolved_survey), + set_ids=set_ids_arr, ) # Normalized effects DID^n_l (suppressed under trends_linear @@ -2766,6 +2890,7 @@ def fit( eligible_mask_var=eligible_mask_var, multi_horizon_dids=multi_horizon_dids, path_effects=path_effects, + set_ids=set_ids_arr, ) # Sibling collector for per-path backward placebos. Mirrors @@ -2796,6 +2921,7 @@ def fit( eligible_mask_var=eligible_mask_var, multi_horizon_placebos=multi_horizon_placebos, path_placebos=path_placebos, + set_ids=set_ids_arr, ) br = self._compute_dcdh_bootstrap( @@ -3039,6 +3165,42 @@ def fit( ) path_effects[path_key]["horizons"][l_h]["t_stat"] = np.nan + # Per-path cumulated layer (under trends_linear). Computed AFTER + # the bootstrap propagation block above so the cumulated SE / t / + # p / CI are derived from the FINAL post-bootstrap per-horizon + # path SEs rather than the analytical ones path_effects was + # initially populated with at fit-time. Mirrors the global + # `linear_trends_effects` placement at `:3405-3454` which also + # runs after the event_study bootstrap propagation. Honors the + # library-wide NaN-on-invalid bootstrap contract: any non-finite + # component SE in the running-sum upper bound yields a NaN + # cumulated SE / t / p / CI, regardless of whether the source + # was an analytical singularity or a non-finite bootstrap draw. + if ( + self.by_path is not None + and _is_trends_linear + and L_max is not None + and L_max >= 1 + and multi_horizon_dids is not None + and path_effects is not None + and len(path_effects) > 0 + ): + _df_s_bp_cum = _effective_df_survey( + resolved_survey, _replicate_n_valid_list + ) + path_cumulated_event_study = _compute_path_cumulated_event_study( + D_mat=D_mat, + N_mat=N_mat, + first_switch_idx=first_switch_idx_arr, + switch_direction=switch_direction_arr, + L_max=L_max, + by_path=self.by_path, + multi_horizon_dids=multi_horizon_dids, + path_effects=path_effects, + alpha=self.alpha, + df_inference=_inference_df(_df_s_bp_cum, resolved_survey), + ) + # Phase 3: propagate bootstrap results to per-path placebos # (by_path + placebo). Sibling of the path_effects propagation # block above. Library-wide NaN-on-invalid bootstrap contract: @@ -3746,6 +3908,7 @@ def fit( ), path_effects=path_effects, path_placebo_event_study=path_placebos, + path_cumulated_event_study=path_cumulated_event_study, path_sup_t_bands=( # When by_path + n_bootstrap > 0 is active, surface a # dict (possibly empty) — preserving the documented @@ -5404,6 +5567,7 @@ def _compute_path_effects( all_groups: List[Any], alpha: float, df_inference: Optional[int] = None, + set_ids: Optional[np.ndarray] = None, ) -> Optional[Dict[Tuple[int, ...], Dict[str, Any]]]: """ Compute per-path event-study effects using the joiners/leavers IF pattern. @@ -5489,7 +5653,7 @@ def _compute_path_effects( switch_direction=switch_direction, T_g=T_g, L_max=L_max, - set_ids=None, + set_ids=set_ids, compute_per_period=False, switcher_subset_mask=switcher_mask, ) @@ -5572,6 +5736,133 @@ def _compute_path_effects( return path_effects +def _compute_path_cumulated_event_study( + D_mat: np.ndarray, + N_mat: np.ndarray, + first_switch_idx: np.ndarray, + switch_direction: np.ndarray, + L_max: int, + by_path: int, + multi_horizon_dids: Dict[int, Dict[str, Any]], + path_effects: Dict[Tuple[int, ...], Dict[str, Any]], + alpha: float, + df_inference: Optional[int] = None, +) -> Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]: + """ + Per-path cumulated level effects under ``trends_linear=True``. + + Mirrors the global ``linear_trends_effects`` cumulation + (``chaisemartin_dhaultfoeuille.py:3340-3398``): for each enumerated + path, accumulate per-group running sums of ``DID^{fd}_{g, l'}`` over + ``l' = 1..l``, then average over the path's switchers eligible at + horizon ``l``. SE is the conservative upper bound (sum of per-horizon + component SEs from ``path_effects[path]["horizons"][l']["se"]``, + NaN-consistent: if any component SE is non-finite the cumulated SE is + NaN). Inference (t-stat, p-value, CI) via ``safe_inference``. + + Returns ``{path: {horizon: {effect, se, t_stat, p_value, conf_int, + n_obs}}}`` directly (no ``horizons`` wrapper), aligned with the + ``path_placebo_event_study`` and global ``linear_trends_effects`` + shapes. The outer keys match ``path_effects.keys()``; horizons that + have NaN cumulated values still appear (for to_dataframe alignment). + R parity: matches R ``did_multiplegt_dyn(..., by_path, trends_lin)`` + which returns cumulated ``Effect_l`` per path under single-baseline + panels (validated against ``joiners_only_trends_lin`` empirically). + """ + from diff_diff.utils import safe_inference + + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + selected_paths, path_to_group_mask, _ = _enumerate_treatment_paths( + D_mat=D_mat, + first_switch_idx=first_switch_idx, + N_mat=N_mat, + L_max=L_max, + by_path=by_path, + ) + + n_groups_total = D_mat.shape[0] + S_arr = switch_direction.astype(float) + + path_cumulated: Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]] = {} + + for path in selected_paths: + if path not in path_effects: + continue + switcher_mask = path_to_group_mask[path] + # Per-group running sum of DID^{fd}_{g, l'} for path switchers + running_per_group = np.zeros(n_groups_total) + path_horizons_anal = path_effects[path].get("horizons", {}) + cumulated_path: Dict[int, Dict[str, Any]] = {} + + for l_h in range(1, L_max + 1): + if l_h not in multi_horizon_dids: + continue + mh = multi_horizon_dids[l_h] + did_g_l = mh.get("did_g_l") + eligible_global = mh.get("eligible_mask") + if did_g_l is None or eligible_global is None: + cumulated_path[l_h] = { + "effect": float("nan"), + "se": float("nan"), + "t_stat": float("nan"), + "p_value": float("nan"), + "conf_int": (float("nan"), float("nan")), + "n_obs": 0, + } + continue + # Add this horizon's per-group DID to running sum (NaN -> 0 + # for accumulation, matching the global cumulation pattern) + increment = np.where(np.isfinite(did_g_l), did_g_l, 0.0) + running_per_group += increment + # Path-restricted eligible mask at horizon l + eligible_path = switcher_mask & eligible_global + n_l_path = int(eligible_path.sum()) + if n_l_path == 0: + cumulated_path[l_h] = { + "effect": float("nan"), + "se": float("nan"), + "t_stat": float("nan"), + "p_value": float("nan"), + "conf_int": (float("nan"), float("nan")), + "n_obs": 0, + } + continue + cum_effect = float( + np.sum(S_arr[eligible_path] * running_per_group[eligible_path]) + / n_l_path + ) + # Conservative SE upper bound: sum of per-horizon component + # SEs from path_effects (matches global formula at :3402-3413). + # NaN-consistency: any non-finite component SE -> cumulated NaN. + component_ses = [ + path_horizons_anal.get(ll, {}).get("se", float("nan")) + for ll in range(1, l_h + 1) + ] + if all(np.isfinite(s) for s in component_ses): + cum_se = float(sum(component_ses)) + else: + cum_se = float("nan") + cum_t, cum_p, cum_ci = safe_inference( + cum_effect, + cum_se, + alpha=alpha, + df=df_inference, + ) + cumulated_path[l_h] = { + "effect": cum_effect, + "se": cum_se, + "t_stat": cum_t, + "p_value": cum_p, + "conf_int": cum_ci, + "n_obs": n_l_path, + } + + path_cumulated[path] = cumulated_path + + return path_cumulated + + def _compute_path_placebos( D_mat: np.ndarray, Y_mat: np.ndarray, @@ -5586,6 +5877,7 @@ def _compute_path_placebos( multi_horizon_placebos: Dict[int, Dict[str, Any]], alpha: float, df_inference: Optional[int] = None, + set_ids: Optional[np.ndarray] = None, ) -> Optional[Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]]: """ Compute per-path backward-horizon placebos ``DID^{pl}_{path, l}``. @@ -5674,7 +5966,7 @@ def _compute_path_placebos( switch_direction=switch_direction, T_g=T_g, L_max=L_max, - set_ids=None, + set_ids=set_ids, compute_per_period=False, switcher_subset_mask=switcher_mask, ) @@ -5758,6 +6050,7 @@ def _collect_path_bootstrap_inputs( eligible_mask_var: np.ndarray, multi_horizon_dids: Dict[int, Dict[str, Any]], path_effects: Dict[Tuple[int, ...], Dict[str, Any]], + set_ids: Optional[np.ndarray] = None, ) -> Dict[Tuple[int, ...], Dict[int, Tuple[np.ndarray, int, float, None]]]: """ Collect per-(path, horizon) inputs for the bootstrap mixin. @@ -5838,7 +6131,7 @@ def _collect_path_bootstrap_inputs( switch_direction=switch_direction, T_g=T_g, L_max=L_max, - set_ids=None, + set_ids=set_ids, compute_per_period=False, switcher_subset_mask=switcher_mask, ) @@ -5882,6 +6175,7 @@ def _collect_path_placebo_bootstrap_inputs( eligible_mask_var: np.ndarray, multi_horizon_placebos: Dict[int, Dict[str, Any]], path_placebos: Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]], + set_ids: Optional[np.ndarray] = None, ) -> Dict[Tuple[int, ...], Dict[int, Tuple[np.ndarray, int, float, None]]]: """ Collect per-(path, lag) inputs for the placebo bootstrap mixin @@ -5960,7 +6254,7 @@ def _collect_path_placebo_bootstrap_inputs( switch_direction=switch_direction, T_g=T_g, L_max=L_max, - set_ids=None, + set_ids=set_ids, compute_per_period=False, switcher_subset_mask=switcher_mask, ) diff --git a/diff_diff/chaisemartin_dhaultfoeuille_results.py b/diff_diff/chaisemartin_dhaultfoeuille_results.py index 8f090a58..6ddfb0d6 100644 --- a/diff_diff/chaisemartin_dhaultfoeuille_results.py +++ b/diff_diff/chaisemartin_dhaultfoeuille_results.py @@ -418,6 +418,26 @@ class ChaisemartinDHaultfoeuilleResults: cohort-sharing SE deviation from R documented for ``path_effects``. See REGISTRY.md ``Note (Phase 3 by_path ...)`` → "Per-path placebos". + path_cumulated_event_study : dict, optional + Per-path cumulated level effects ``delta_{path, l} = + sum_{l'=1..l} DID^{fd}_{path, l'}`` for ``l = 1..L_max``, + keyed by observed treatment trajectory (tuple of int). Inner + dict is keyed by horizon directly (no ``"horizons"`` wrapper); + each entry holds ``{"effect", "se", "t_stat", "p_value", + "conf_int", "n_obs"}``. Populated when ``by_path`` is a + positive int AND ``trends_linear=True`` AND ``L_max >= 1``; + ``None`` otherwise. Mirrors the global ``linear_trends_effects`` + cumulation: SE on the cumulated layer is the conservative + upper bound (sum of per-horizon component SEs from + ``path_effects[path]["horizons"][l]["se"]``, NaN-consistent). + Built AFTER bootstrap propagation so the cumulated SE / t / p + / CI are derived from the FINAL post-bootstrap per-horizon + SEs when ``n_bootstrap > 0``. Surfaced as ``cumulated_effect`` + / ``cumulated_se`` columns on + ``to_dataframe(level="by_path")`` (always-present, NaN-when- + None) and as a per-path "Cumulated Level Effects" sub-section + in ``summary()``. See REGISTRY.md ``Note (Phase 3 by_path + ...)`` → "Per-path linear-trends DID^{fd}". path_sup_t_bands : dict, optional Per-path joint sup-t simultaneous-band metadata, keyed by observed treatment trajectory (tuple of int). Each entry holds @@ -552,6 +572,22 @@ class ChaisemartinDHaultfoeuilleResults: path_placebo_event_study: Optional[Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]]] = field( default=None, repr=False ) + # Per-path cumulated event study (level effects under `trends_linear` + # = True). `path_effects[path]["horizons"][l]` surfaces raw + # `DID^{fd}_l` per path; this field surfaces the cumulated level + # effect `delta_l = sum_{l'=1..l} DID^{fd}_{path, l'}` per (path, + # horizon), mirroring the global `linear_trends_effects` cumulation + # for non-by_path fits. Inner dict keyed by horizon directly (no + # `horizons` wrapper). `None` when not requested (`by_path is None` + # or `trends_linear=False`); `{}` is impossible (the field follows + # `path_effects` so a populated `path_effects` plus `trends_linear` + # always populates this). SE on the cumulated layer is the + # conservative upper bound (sum of per-horizon component SEs, + # NaN-consistent), matching the global `linear_trends_effects` + # convention. + path_cumulated_event_study: Optional[ + Dict[Tuple[int, ...], Dict[int, Dict[str, Any]]] + ] = field(default=None, repr=False) # Per-path joint sup-t simultaneous-band metadata. Keyed by path # tuple; each entry holds `{"crit_value", "alpha", "n_bootstrap", # "method", "n_valid_horizons"}`. Populated when `by_path` is a @@ -1289,6 +1325,32 @@ def _render_path_effects_section( h["p_value"], ) ) + # Per-path cumulated level effects (under trends_linear). + # Mirrors the global linear_trends_effects rendering inside + # the per-path block: appears as a labeled sub-block right + # after the per-horizon DID^{fd}_l rows. Skip silently when + # path_cumulated_event_study is None or this path lacks an + # entry (the latter shouldn't happen, but kept as a guard). + if ( + self.path_cumulated_event_study is not None + and path in self.path_cumulated_event_study + ): + cum_horizons = self.path_cumulated_event_study[path] + if cum_horizons: + lines.append( + " Cumulated Level Effects (DID^{fd}, trends_linear):" + ) + for l_h in sorted(cum_horizons.keys()): + ce = cum_horizons[l_h] + lines.append( + _format_inference_row( + f" Level_{l_h}", + ce["effect"], + ce["se"], + ce["t_stat"], + ce["p_value"], + ) + ) # Per-path joint sup-t critical value (when populated). # Mirrors the OVERALL sup-t crit print at line ~1019. if self.path_sup_t_bands is not None and path in self.path_sup_t_bands: @@ -1377,15 +1439,21 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: ``path``, ``frequency_rank``, ``n_groups``, ``horizon``, ``effect``, ``se``, ``t_stat``, ``p_value``, ``conf_int_lower``, ``conf_int_upper``, ``n_obs``, - ``cband_lower``, ``cband_upper``. The ``horizon`` column - takes negative ints for placebo rows when - ``placebo=True``. The ``cband_*`` columns mirror the - OVERALL ``level="event_study"`` schema (joint sup-t - simultaneous bands); they are populated for positive- - horizon rows of paths with a finite per-path sup-t crit - (``n_bootstrap > 0``) and NaN otherwise (placebo rows, - unbanded paths, or the requested-but-empty fallback - DataFrame). + ``cband_lower``, ``cband_upper``, ``cumulated_effect``, + ``cumulated_se``. The ``horizon`` column takes negative + ints for placebo rows when ``placebo=True``. The + ``cband_*`` columns mirror the OVERALL + ``level="event_study"`` schema (joint sup-t simultaneous + bands); they are populated for positive-horizon rows of + paths with a finite per-path sup-t crit (``n_bootstrap > + 0``) and NaN otherwise (placebo rows, unbanded paths, or + the requested-but-empty fallback DataFrame). The + ``cumulated_*`` columns mirror the global + ``linear_trends_effects`` cumulation; populated for + positive-horizon rows when ``trends_linear=True`` is + also set, NaN for placebo rows or non-trends_linear fits + (always-present, NaN-when-None — same convention as + ``cband_*``). Returns ------- @@ -1645,6 +1713,8 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: "n_obs", "cband_lower", "cband_upper", + "cumulated_effect", + "cumulated_se", ] ) rows = [] @@ -1666,6 +1736,13 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: if self.path_placebo_event_study is not None else {} ) + # Per-path cumulated entries (under trends_linear). Always- + # present, NaN-when-None mirrors the cband_* convention. + path_cumulated = ( + self.path_cumulated_event_study.get(path, {}) + if self.path_cumulated_event_study is not None + else {} + ) for lag_key in sorted(placebo_horizons.keys()): ph_entry = placebo_horizons[lag_key] # Placebos do not get joint sup-t bands in this @@ -1673,6 +1750,9 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: # mirrors OVERALL placebo / event-study sup-t # convention). Emit NaN cband columns for schema # parity with the OVERALL level="event_study" table. + # Placebo + cumulated is also NaN: there is no per- + # path placebo cumulation surface (placebo under + # trends_lin returns RAW per-horizon values per R). ph_cband = ph_entry.get("cband_conf_int", (np.nan, np.nan)) rows.append( { @@ -1689,6 +1769,8 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: "n_obs": ph_entry["n_obs"], "cband_lower": ph_cband[0] if ph_cband else np.nan, "cband_upper": ph_cband[1] if ph_cband else np.nan, + "cumulated_effect": np.nan, + "cumulated_se": np.nan, } ) for l_h in sorted(horizons.keys()): @@ -1698,6 +1780,7 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: # key / missing path entry -> NaN columns. Pinned at # `TestByPathSupTBands::test_path_sup_t_to_dataframe_emits_cband_columns`. h_cband = h_entry.get("cband_conf_int", (np.nan, np.nan)) + cum_entry = path_cumulated.get(l_h, {}) rows.append( { "path": path, @@ -1713,6 +1796,8 @@ def to_dataframe(self, level: str = "overall") -> pd.DataFrame: "n_obs": h_entry["n_obs"], "cband_lower": h_cband[0] if h_cband else np.nan, "cband_upper": h_cband[1] if h_cband else np.nan, + "cumulated_effect": cum_entry.get("effect", np.nan), + "cumulated_se": cum_entry.get("se", np.nan), } ) return pd.DataFrame(rows) diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 7d3cf4c7..9060def0 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -638,7 +638,7 @@ The guard is fired by `_survey_se_from_group_if` (analytical and replicate) and - **Note (Phase 3 Design-2 switch-in/switch-out):** Convenience wrapper for Web Appendix Section 1.6 (Assumption 16). Identifies groups with exactly 2 treatment changes (join then leave), reports switch-in and switch-out mean effects. This is a descriptive summary, not a full re-estimation with specialized control pools as described in the paper. **Always uses raw (unadjusted) outcomes** regardless of active `controls`, `trends_linear`, or `trends_nonparam` options - those adjustments apply to the main estimator surface but not to the Design-2 descriptive block. For full adjusted Design-2 estimation with proper control pools, the paper recommends "running the command on a restricted subsample and using `trends_nonparam` for the entry-timing grouping." Activated via `design2=True` in `fit()`, requires `drop_larger_lower=False` to retain 2-switch groups. -- **Note (Phase 3 `by_path` per-path event-study disaggregation):** Per-path disaggregation of the multi-horizon event study, mirroring R `did_multiplegt_dyn(..., by_path=k)`. Activated via `ChaisemartinDHaultfoeuille(by_path=k, drop_larger_lower=False)` where `k` is a positive integer (top-k most common observed paths by switcher-group frequency). **Window convention:** the path tuple for a switcher group `g` is `(D_{g, F_g-1}, D_{g, F_g}, ..., D_{g, F_g-1+L_max})` — length `L_max + 1`, matching R's window `[F_{g-1}, F_{g-1+l}]`. **Ranking:** paths are ranked by descending frequency; ties are broken lexicographically on the path tuple for deterministic ordering, so every selected path has a unique `frequency_rank`. If `by_path` exceeds the number of observed paths, all observed paths are returned with a `UserWarning`. **Per-path SE convention (joiners/leavers precedent):** the per-path influence function follows the joiners-only / leavers-only IF construction at `chaisemartin_dhaultfoeuille.py:5495-5504`: the switcher-side contribution `+S_g * (Y_{g,out} - Y_{g,ref})` is zeroed for groups whose observed trajectory is NOT the selected path; control contributions and the full cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. After applying the singleton-baseline eligible mask and cohort-recentering with the original cohort IDs, the plug-in SE uses the path-specific divisor `N_l_path` (count of path switchers eligible at horizon `l`) — same pattern as `joiners_se` using `joiner_total`. This gives the **within-path mean** estimand `DID_{path,l}` as the within-path average of `DID_{g,l}`. **Degenerate-cohort behavior per path:** when a path's centered IF at some horizon is identically zero (every variance-eligible path switcher forms its own `(D_{g,1}, F_g, S_g)` cohort, or the path has a single contributing group), SE / t_stat / p_value / conf_int are NaN-consistent and a `UserWarning` is emitted scoped to `(path, horizon)`. This mirrors the overall-path degenerate-cohort surface and is common for rare paths with few contributing groups. **Empty-state contract:** `results.path_effects` distinguishes "not requested" (`None`) from "requested but empty" (`{}` — all switchers have windows outside the panel or unobserved cells). The empty-dict case emits a `UserWarning` at fit-time and renders as an explicit "no observed paths" notice in `summary()`; `to_dataframe(level="by_path")` returns an empty DataFrame with the canonical column set (mirrors the `linear_trends` pattern when `trends_linear=True` but no horizons survive). **Requirements:** `drop_larger_lower=False` (multi-switch groups are the object of interest; default `True` filters them out) and `L_max >= 1` (path window depends on the horizon). **Scope:** binary treatment only; combinations with `trends_linear`, `trends_nonparam`, `heterogeneity`, `design2`, `honest_did`, and `survey_design` remain gated behind explicit `NotImplementedError` (deferred to follow-up wave PRs). `n_bootstrap > 0` is now supported — see the **Bootstrap SE** paragraph below. `placebo=True` is now supported per-path — see the **Per-path placebos** paragraph below. **TWFE diagnostic** remains a sample-level summary (not computed per path) in this release. Results are exposed on `results.path_effects` as `Dict[Tuple[int, ...], Dict[str, Any]]` with nested `horizons` dicts per horizon `l`, and on `results.to_dataframe(level="by_path")` as a long-format table with columns `[path, frequency_rank, n_groups, horizon, effect, se, t_stat, p_value, conf_int_lower, conf_int_upper, n_obs, cband_lower, cband_upper]` (the last two are added by the joint sup-t Note below; populated for positive-horizon rows of paths with a finite sup-t crit, NaN otherwise). Gated tests live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathGates` / `::TestByPathBehavior` / `::TestByPathEdgeCases`. **R-parity** against `DIDmultiplegtDYN 2.3.3` is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPath` via two scenarios: `mixed_single_switch_by_path` (2 paths, `by_path=2`) and `multi_path_reversible_by_path` (4 paths, `by_path=3`; path-assignment deterministic on `F_g` so each `(D_{g,1}, F_g, S_g)` cohort contains switchers from a single path). Per-path point estimates and per-path switcher counts match R exactly; per-path SE matches within the Phase 2 multi-horizon SE envelope (observed rtol ≤ 10.2% on the 2-path mixed scenario, ≤ 4.2% on the 4-path cohort-clean scenario). **Deviation from R (cross-path cohort-sharing SE):** our analytical SE is the marginal variance of the path-contribution estimator cohort-centered on the *full-panel* cohort structure (joiners/leavers precedent — non-path switchers contribute to cohort means via their zeroed switcher row). R's `did_multiplegt_dyn(..., by_path=k)` re-runs the estimator per path, so cohort means are computed over the path's own switchers only. When a cohort `(D_{g,1}, F_g, S_g)` spans multiple observed paths, Python and R SE diverge materially (our empirical probes with random post-window toggling saw rtol > 100%); when every cohort is single-path (scenario 13 by design, scenario 14 by construction), the two approaches coincide up to the documented Phase 2 envelope. Practitioners with cohort structures that mix paths should interpret the per-path SE as a within-full-panel marginal variance, not a per-path conditional variance. **Bootstrap SE:** when `n_bootstrap > 0` is set, the top-k paths are enumerated once on the observed data (R-faithful: matches `did_multiplegt_dyn(..., by_path=k, bootstrap=B)`'s path-stability convention — verified empirically against DIDmultiplegtDYN 2.3.3) and the multiplier bootstrap (`bootstrap_weights ∈ {"rademacher", "mammen", "webb"}`) runs per `(path, horizon)` target via the shared `_bootstrap_one_target` / `compute_effect_bootstrap_stats` helpers. Point estimates are unchanged from the analytical path. Bootstrap SE replaces the analytical SE in `path_effects[path]["horizons"][l]["se"]`, and `p_value` / `conf_int` are taken as the **bootstrap percentile** statistics, matching the Round-10 library convention for overall / joiners / leavers / multi-horizon bootstrap (see the `Note (bootstrap inference surface)` elsewhere in this file and the pinned regression `test_bootstrap_p_value_and_ci_propagated_to_top_level`). `t_stat` is SE-derived via `safe_inference` per the anti-pattern rule. Interpretation: inference is *conditional on the observed path set*. **SE inherits the analytical cross-path cohort-sharing deviation:** the bootstrap input is the exact same full-panel cohort-centered path IF that the analytical path computes (`_collect_path_bootstrap_inputs` reuses the same enumeration / cohort IDs / IF construction), so the bootstrap SE is a Monte Carlo analog of the analytical SE — it inherits the same cross-path cohort-sharing deviation from R's per-path re-run convention documented above. On single-path-cohort panels (scenarios 13 and 14 of the R-parity fixture, and any DGP where `(D_{g,1}, F_g, S_g)` cohorts never span multiple observed paths), bootstrap SE tracks analytical SE up to Monte Carlo noise and both coincide with R up to the Phase 2 envelope. On cross-path cohort panels, bootstrap SE inherits the >100% rtol divergence from R that analytical already has. **Deviation from R (CI method):** R's per-path CI is normal-theory around the bootstrap SE (half-width ≈ `1.96·se`); ours is the bootstrap percentile CI, intentionally diverging from R to keep the dCDH inference surface internally consistent across all bootstrap targets. Practitioners who want *unconditional* inference capturing path-selection uncertainty need a pairs-bootstrap (deferred — no R precedent). Positive regressions live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathBootstrap` (gated `@pytest.mark.slow`): point-estimate invariance, finite positive SE on non-degenerate panels, SE-within-30%-rtol of analytical on cohort-clean fixtures, degenerate-cohort NaN propagation, Rademacher/Mammen/Webb parity, seed reproducibility, and percentile-vs-normal-theory CI pinning. **Per-path placebos:** when `placebo=True` (and `L_max >= 1`) is combined with `by_path=k`, per-path backward-horizon placebos `DID^{pl}_{path, l}` for `l = 1..L_max` are computed using the same joiners/leavers IF precedent applied to `_compute_per_group_if_placebo_horizon` (with the new `switcher_subset_mask` parameter): switcher contributions are zeroed for groups not in the path; the control pool and the variance-eligible cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. Plug-in SE uses the path-specific divisor `N^{pl}_{l, path}` (count of path switchers eligible at backward lag `l`). Surfaced on `results.path_placebo_event_study[path][-l]` with the same `{effect, se, t_stat, p_value, conf_int, n_obs}` shape as `placebo_event_study` (negative-int inner keys parallel the existing per-path event-study positive-int keys, so a unified forward+backward view is well-formed). **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` (same convention applied backward); tracks R within numerical tolerance on single-path-cohort panels and diverges on cohort-mixed panels. Multiplier bootstrap (when `n_bootstrap > 0`) runs per `(path, lag)` target via the same `_bootstrap_one_target` dispatch used for the per-path event-study, with the canonical NaN-on-invalid contract. The bootstrap SE is a Monte Carlo analog of the analytical placebo SE — same per-path centered IF input — and inherits the same deviation. Surfaced through `summary()` (negative-keyed rows rendered alongside positive-keyed event-study rows under each path block) and `to_dataframe(level="by_path")` (`horizon` column takes negative ints for placebo rows). **Empty-state contract:** `results.path_placebo_event_study` mirrors `path_effects` — `None` when `by_path + placebo` was not requested, `{}` when requested but no observed path has a complete window within the panel (same regime that returns `{}` for `path_effects`, with the same fit-time `UserWarning`). R-parity is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathPlacebo` on the `multi_path_reversible_by_path_placebo` scenario; positive analytical + bootstrap invariants live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathPlacebo` (with the gated `::TestByPathPlacebo::TestBootstrap` subclass). **Per-path covariate residualization (DID^X):** when `controls=[...]` is set with `by_path=k`, the per-baseline OLS residualization (Web Appendix Section 1.2) runs once on the first-differenced outcome BEFORE path enumeration. All four downstream surfaces — analytical per-path SE, bootstrap SE, per-path placebos, and per-path joint sup-t bands — consume the residualized `Y_mat` automatically (Frisch-Waugh-Lovell). Per-period effects remain unadjusted, consistent with the existing `controls` + per-period DID contract (per-period DID does not support residualization). Failed-stratum baselines (rank-deficient X) zero out `N_mat` for affected groups, which the path enumeration treats as ineligible per its existing convention. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, controls)` re-runs the per-baseline residualization on each path's restricted subsample (`R/R/did_multiplegt_dyn.R` lines 401-405: rows of the path's switchers OR rows where `yet_to_switch=1 AND baseline matches the path's baseline`). The first-stage residualization sample R uses for path B equals: pre-switch rows of all switchers with matching baseline + all rows of never-switchers with matching baseline — bit-identical to our global first-stage sample under single-baseline switcher panels (every switcher shares the same `D_{g,1}`, regardless of how `F_g` or path identity varies across switchers). Per-path point estimates therefore coincide with R on those panels up to the existing **DID^X first-stage cell-weighting deviation** documented above in `Note (Phase 3 DID^X covariate adjustment)` (Python's first-stage OLS uses equal cell weights — one observation per `(g, t)` cell, consistent with the library's cell-aggregated input convention; R weights by `N_gt`). On panels with one observation per `(g, t)` cell (the common case after the cell-aggregation step in `fit()`), Python matches R bit-exactly: the `multi_path_reversible_by_path_controls` parity fixture has 4 paths with switcher `F_g` values spanning [0..6] under `D_{g,1}=0` and Python matches R to rtol ~1e-11. On multi-baseline switcher panels (some switchers have `D_{g,1}=0`, others have `D_{g,1}=1`) R's per-path subset drops switchers whose baseline differs from the path's baseline, so the per-baseline regression coefficients diverge per path under R and point estimates can diverge between Python and R — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R. The warning filters to switcher groups only; never-switchers (never-treated + always-treated controls) at multiple baseline values do NOT trigger the warning because they don't affect R's per-path subset construction. **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` — bootstrap SE, placebo SE, and sup-t crit are Monte Carlo / joint-distribution analogs of the same residualized analytical IF and carry the same deviation. R-parity is confirmed against `did_multiplegt_dyn(..., by_path=3, controls="X1")` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathControls` on the `multi_path_reversible_by_path_controls` scenario (single-baseline DGP, exact point-estimate match measured rtol ~1e-11); cross-surface inheritance and the multi-baseline warning are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathControls` (analytical + bootstrap + placebo + sup-t + `to_dataframe(level="by_path")` cband columns + multi-baseline `UserWarning`). +- **Note (Phase 3 `by_path` per-path event-study disaggregation):** Per-path disaggregation of the multi-horizon event study, mirroring R `did_multiplegt_dyn(..., by_path=k)`. Activated via `ChaisemartinDHaultfoeuille(by_path=k, drop_larger_lower=False)` where `k` is a positive integer (top-k most common observed paths by switcher-group frequency). **Window convention:** the path tuple for a switcher group `g` is `(D_{g, F_g-1}, D_{g, F_g}, ..., D_{g, F_g-1+L_max})` — length `L_max + 1`, matching R's window `[F_{g-1}, F_{g-1+l}]`. **Ranking:** paths are ranked by descending frequency; ties are broken lexicographically on the path tuple for deterministic ordering, so every selected path has a unique `frequency_rank`. If `by_path` exceeds the number of observed paths, all observed paths are returned with a `UserWarning`. **Per-path SE convention (joiners/leavers precedent):** the per-path influence function follows the joiners-only / leavers-only IF construction at `chaisemartin_dhaultfoeuille.py:5495-5504`: the switcher-side contribution `+S_g * (Y_{g,out} - Y_{g,ref})` is zeroed for groups whose observed trajectory is NOT the selected path; control contributions and the full cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. After applying the singleton-baseline eligible mask and cohort-recentering with the original cohort IDs, the plug-in SE uses the path-specific divisor `N_l_path` (count of path switchers eligible at horizon `l`) — same pattern as `joiners_se` using `joiner_total`. This gives the **within-path mean** estimand `DID_{path,l}` as the within-path average of `DID_{g,l}`. **Degenerate-cohort behavior per path:** when a path's centered IF at some horizon is identically zero (every variance-eligible path switcher forms its own `(D_{g,1}, F_g, S_g)` cohort, or the path has a single contributing group), SE / t_stat / p_value / conf_int are NaN-consistent and a `UserWarning` is emitted scoped to `(path, horizon)`. This mirrors the overall-path degenerate-cohort surface and is common for rare paths with few contributing groups. **Empty-state contract:** `results.path_effects` distinguishes "not requested" (`None`) from "requested but empty" (`{}` — all switchers have windows outside the panel or unobserved cells). The empty-dict case emits a `UserWarning` at fit-time and renders as an explicit "no observed paths" notice in `summary()`; `to_dataframe(level="by_path")` returns an empty DataFrame with the canonical column set (mirrors the `linear_trends` pattern when `trends_linear=True` but no horizons survive). **Requirements:** `drop_larger_lower=False` (multi-switch groups are the object of interest; default `True` filters them out) and `L_max >= 1` (path window depends on the horizon). **Scope:** binary treatment only; combinations with `heterogeneity`, `design2`, `honest_did`, and `survey_design` remain gated behind explicit `NotImplementedError` (deferred to follow-up wave PRs). `n_bootstrap > 0` is now supported — see the **Bootstrap SE** paragraph below. `placebo=True` is now supported per-path — see the **Per-path placebos** paragraph below. **TWFE diagnostic** remains a sample-level summary (not computed per path) in this release. Results are exposed on `results.path_effects` as `Dict[Tuple[int, ...], Dict[str, Any]]` with nested `horizons` dicts per horizon `l`, and on `results.to_dataframe(level="by_path")` as a long-format table with columns `[path, frequency_rank, n_groups, horizon, effect, se, t_stat, p_value, conf_int_lower, conf_int_upper, n_obs, cband_lower, cband_upper, cumulated_effect, cumulated_se]` (the `cband_*` columns are added by the joint sup-t Note below, populated for positive-horizon rows of paths with a finite sup-t crit and NaN otherwise; the `cumulated_*` columns are added by the per-path linear-trends Note below, populated for positive-horizon rows when `trends_linear=True` is set and NaN otherwise). Gated tests live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathGates` / `::TestByPathBehavior` / `::TestByPathEdgeCases`. **R-parity** against `DIDmultiplegtDYN 2.3.3` is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPath` via two scenarios: `mixed_single_switch_by_path` (2 paths, `by_path=2`) and `multi_path_reversible_by_path` (4 paths, `by_path=3`; path-assignment deterministic on `F_g` so each `(D_{g,1}, F_g, S_g)` cohort contains switchers from a single path). Per-path point estimates and per-path switcher counts match R exactly; per-path SE matches within the Phase 2 multi-horizon SE envelope (observed rtol ≤ 10.2% on the 2-path mixed scenario, ≤ 4.2% on the 4-path cohort-clean scenario). **Deviation from R (cross-path cohort-sharing SE):** our analytical SE is the marginal variance of the path-contribution estimator cohort-centered on the *full-panel* cohort structure (joiners/leavers precedent — non-path switchers contribute to cohort means via their zeroed switcher row). R's `did_multiplegt_dyn(..., by_path=k)` re-runs the estimator per path, so cohort means are computed over the path's own switchers only. When a cohort `(D_{g,1}, F_g, S_g)` spans multiple observed paths, Python and R SE diverge materially (our empirical probes with random post-window toggling saw rtol > 100%); when every cohort is single-path (scenario 13 by design, scenario 14 by construction), the two approaches coincide up to the documented Phase 2 envelope. Practitioners with cohort structures that mix paths should interpret the per-path SE as a within-full-panel marginal variance, not a per-path conditional variance. **Bootstrap SE:** when `n_bootstrap > 0` is set, the top-k paths are enumerated once on the observed data (R-faithful: matches `did_multiplegt_dyn(..., by_path=k, bootstrap=B)`'s path-stability convention — verified empirically against DIDmultiplegtDYN 2.3.3) and the multiplier bootstrap (`bootstrap_weights ∈ {"rademacher", "mammen", "webb"}`) runs per `(path, horizon)` target via the shared `_bootstrap_one_target` / `compute_effect_bootstrap_stats` helpers. Point estimates are unchanged from the analytical path. Bootstrap SE replaces the analytical SE in `path_effects[path]["horizons"][l]["se"]`, and `p_value` / `conf_int` are taken as the **bootstrap percentile** statistics, matching the Round-10 library convention for overall / joiners / leavers / multi-horizon bootstrap (see the `Note (bootstrap inference surface)` elsewhere in this file and the pinned regression `test_bootstrap_p_value_and_ci_propagated_to_top_level`). `t_stat` is SE-derived via `safe_inference` per the anti-pattern rule. Interpretation: inference is *conditional on the observed path set*. **SE inherits the analytical cross-path cohort-sharing deviation:** the bootstrap input is the exact same full-panel cohort-centered path IF that the analytical path computes (`_collect_path_bootstrap_inputs` reuses the same enumeration / cohort IDs / IF construction), so the bootstrap SE is a Monte Carlo analog of the analytical SE — it inherits the same cross-path cohort-sharing deviation from R's per-path re-run convention documented above. On single-path-cohort panels (scenarios 13 and 14 of the R-parity fixture, and any DGP where `(D_{g,1}, F_g, S_g)` cohorts never span multiple observed paths), bootstrap SE tracks analytical SE up to Monte Carlo noise and both coincide with R up to the Phase 2 envelope. On cross-path cohort panels, bootstrap SE inherits the >100% rtol divergence from R that analytical already has. **Deviation from R (CI method):** R's per-path CI is normal-theory around the bootstrap SE (half-width ≈ `1.96·se`); ours is the bootstrap percentile CI, intentionally diverging from R to keep the dCDH inference surface internally consistent across all bootstrap targets. Practitioners who want *unconditional* inference capturing path-selection uncertainty need a pairs-bootstrap (deferred — no R precedent). Positive regressions live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathBootstrap` (gated `@pytest.mark.slow`): point-estimate invariance, finite positive SE on non-degenerate panels, SE-within-30%-rtol of analytical on cohort-clean fixtures, degenerate-cohort NaN propagation, Rademacher/Mammen/Webb parity, seed reproducibility, and percentile-vs-normal-theory CI pinning. **Per-path placebos:** when `placebo=True` (and `L_max >= 1`) is combined with `by_path=k`, per-path backward-horizon placebos `DID^{pl}_{path, l}` for `l = 1..L_max` are computed using the same joiners/leavers IF precedent applied to `_compute_per_group_if_placebo_horizon` (with the new `switcher_subset_mask` parameter): switcher contributions are zeroed for groups not in the path; the control pool and the variance-eligible cohort structure `(D_{g,1}, F_g, S_g)` are unchanged. Plug-in SE uses the path-specific divisor `N^{pl}_{l, path}` (count of path switchers eligible at backward lag `l`). Surfaced on `results.path_placebo_event_study[path][-l]` with the same `{effect, se, t_stat, p_value, conf_int, n_obs}` shape as `placebo_event_study` (negative-int inner keys parallel the existing per-path event-study positive-int keys, so a unified forward+backward view is well-formed). **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` (same convention applied backward); tracks R within numerical tolerance on single-path-cohort panels and diverges on cohort-mixed panels. Multiplier bootstrap (when `n_bootstrap > 0`) runs per `(path, lag)` target via the same `_bootstrap_one_target` dispatch used for the per-path event-study, with the canonical NaN-on-invalid contract. The bootstrap SE is a Monte Carlo analog of the analytical placebo SE — same per-path centered IF input — and inherits the same deviation. Surfaced through `summary()` (negative-keyed rows rendered alongside positive-keyed event-study rows under each path block) and `to_dataframe(level="by_path")` (`horizon` column takes negative ints for placebo rows). **Empty-state contract:** `results.path_placebo_event_study` mirrors `path_effects` — `None` when `by_path + placebo` was not requested, `{}` when requested but no observed path has a complete window within the panel (same regime that returns `{}` for `path_effects`, with the same fit-time `UserWarning`). R-parity is confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathPlacebo` on the `multi_path_reversible_by_path_placebo` scenario; positive analytical + bootstrap invariants live in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathPlacebo` (with the gated `::TestByPathPlacebo::TestBootstrap` subclass). **Per-path covariate residualization (DID^X):** when `controls=[...]` is set with `by_path=k`, the per-baseline OLS residualization (Web Appendix Section 1.2) runs once on the first-differenced outcome BEFORE path enumeration. All four downstream surfaces — analytical per-path SE, bootstrap SE, per-path placebos, and per-path joint sup-t bands — consume the residualized `Y_mat` automatically (Frisch-Waugh-Lovell). Per-period effects remain unadjusted, consistent with the existing `controls` + per-period DID contract (per-period DID does not support residualization). Failed-stratum baselines (rank-deficient X) zero out `N_mat` for affected groups, which the path enumeration treats as ineligible per its existing convention. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, controls)` re-runs the per-baseline residualization on each path's restricted subsample (`R/R/did_multiplegt_dyn.R` lines 401-405: rows of the path's switchers OR rows where `yet_to_switch=1 AND baseline matches the path's baseline`). The first-stage residualization sample R uses for path B equals: pre-switch rows of all switchers with matching baseline + all rows of never-switchers with matching baseline — bit-identical to our global first-stage sample under single-baseline switcher panels (every switcher shares the same `D_{g,1}`, regardless of how `F_g` or path identity varies across switchers). Per-path point estimates therefore coincide with R on those panels up to the existing **DID^X first-stage cell-weighting deviation** documented above in `Note (Phase 3 DID^X covariate adjustment)` (Python's first-stage OLS uses equal cell weights — one observation per `(g, t)` cell, consistent with the library's cell-aggregated input convention; R weights by `N_gt`). On panels with one observation per `(g, t)` cell (the common case after the cell-aggregation step in `fit()`), Python matches R bit-exactly: the `multi_path_reversible_by_path_controls` parity fixture has 4 paths with switcher `F_g` values spanning [0..6] under `D_{g,1}=0` and Python matches R to rtol ~1e-11. On multi-baseline switcher panels (some switchers have `D_{g,1}=0`, others have `D_{g,1}=1`) R's per-path subset drops switchers whose baseline differs from the path's baseline, so the per-baseline regression coefficients diverge per path under R and point estimates can diverge between Python and R — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R. The warning filters to switcher groups only; never-switchers (never-treated + always-treated controls) at multiple baseline values do NOT trigger the warning because they don't affect R's per-path subset construction. **Inherits the cross-path cohort-sharing SE deviation from R** documented above for `path_effects` — bootstrap SE, placebo SE, and sup-t crit are Monte Carlo / joint-distribution analogs of the same residualized analytical IF and carry the same deviation. R-parity is confirmed against `did_multiplegt_dyn(..., by_path=3, controls="X1")` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathControls` on the `multi_path_reversible_by_path_controls` scenario (single-baseline DGP, exact point-estimate match measured rtol ~1e-11); cross-surface inheritance and the multi-baseline warning are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathControls` (analytical + bootstrap + placebo + sup-t + `to_dataframe(level="by_path")` cband columns + multi-baseline `UserWarning`). **Per-path linear-trends DID^{fd}:** when `trends_linear=True` is set with `by_path=k`, the first-differencing transform at `chaisemartin_dhaultfoeuille.py:1599-1630` runs once globally BEFORE path enumeration (replaces `Y_mat` with `Z_mat = Y_t - Y_{t-1}` and shrinks the time axis by one), so per-path raw second-differences `DID^{fd}_{path, l}` surface on `path_effects[path]["horizons"][l]` automatically. Per-path cumulated level effects `delta_{path, l} = sum_{l'=1..l} DID^{fd}_{path, l'}` (the quantity R returns under `did_multiplegt_dyn(..., by_path, trends_lin)` per the existing parity test pivot at `tests/test_chaisemartin_dhaultfoeuille_parity.py:403-409`) surface on the new `results.path_cumulated_event_study[path][l]` field — a per-group running sum of `DID^{fd}_{g, l'}` averaged over the path's switchers eligible at horizon `l`, mirroring the global `linear_trends_effects` cumulation logic at `chaisemartin_dhaultfoeuille.py:3340-3398`. SE on the cumulated layer is the conservative upper bound (sum of per-horizon component SEs from `path_effects[path]["horizons"][l]["se"]`, NaN-consistent: any non-finite component yields a NaN cumulated SE). **Post-bootstrap recomputation:** the cumulated layer is built AFTER the bootstrap propagation block at `chaisemartin_dhaultfoeuille.py:3034-3081` so it reads the FINAL post-bootstrap per-horizon SEs (mirrors the global `linear_trends_effects` placement). When `n_bootstrap > 0`, cumulated SE / t / p / CI are derived from bootstrap per-horizon SEs; when bootstrap produces non-finite SE (e.g., `n_bootstrap=1` degenerate distribution), the cumulated layer's full inference tuple is NaN per the library-wide NaN-on-invalid bootstrap contract. `to_dataframe(level="by_path")` exposes `cumulated_effect` and `cumulated_se` columns (always present, NaN-when-None — mirrors the `cband_*` always-present convention from PR #374). `summary()` renders a `Cumulated Level Effects (DID^{fd}, trends_linear)` sub-section under each per-path block. **Path enumeration uses the post-first-differenced `N_mat_fd`**: switchers with `F_g==2` fail the window-eligibility check and are dropped from path enumeration entirely (the existing global `F_g >= 3` warning at line 1620 surfaces the issue), so a path whose switchers all have `F_g < 3` is silently absent from `path_effects` rather than present-with-NaN. **F_g=3 boundary-case divergence (`by_path + trends_linear`):** `F_g=3` switchers have exactly 2 pre-switch periods, which after first-differencing and the `time==1` filter leaves only 1 valid pre-window Z value. R's per-path full-pipeline call handles this single-pre-period regime differently from Python's global-then-disaggregate architecture, producing 30%+ relative divergence on point estimates for paths whose switchers include `F_g=3` (empirically observed on the parity fixture's earlier `F_g=3` variant). A separate `UserWarning` fires at fit-time when the panel includes any `F_g=3` switcher AND `by_path + trends_linear` is set, mirroring the `F_g < 3` exclusion warning. The shipped parity fixture (`single_baseline_multi_path_by_path_trends_lin`) restricts to `F_g >= 4` exclusively to avoid this regime; per-path R parity is asserted only there. **Placebo under `trends_linear` returns RAW per-horizon values** (no per-path placebo cumulation surface) — verified empirically against the existing `joiners_only_trends_lin` parity fixture: R's per-path Placebo_l matches Python's `path_placebo_event_study[path][-l]` (raw) bit-exactly under non-`by_path` trends_lin. **Deviation from R on multi-baseline switcher panels (point estimates):** R `did_multiplegt_dyn(..., by_path, trends_lin)` re-runs the full pipeline (including first-differencing) on each path's restricted subsample, so it operates on different switcher samples per path when switchers have different baseline values `D_{g,1}`. Python first-differences once globally before path enumeration. On single-baseline switcher panels the two architectures coincide; on multi-baseline switcher panels per-path point estimates can diverge — a `UserWarning` is emitted at fit-time when this configuration is detected so practitioners do not silently consume estimates that disagree with R (mirroring the analogous `by_path + controls` warning). Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_lin=TRUE, placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsLinear` on the `single_baseline_multi_path_by_path_trends_lin` scenario (single-baseline + cohort-single-path + `F_g >= 4` DGP designed to eliminate the multi-baseline divergence, the cross-path cohort-sharing deviation, and the F_g=3 boundary case under R's per-path full-pipeline call). Per-path cumulated point estimates match R bit-exactly (rtol ~1e-9) on event horizons under those conditions; cumulated SE_RTOL is widened to `0.20` (vs `0.12` used for non-cumulated by_path parity) because the conservative upper-bound SE compounds the cross-path cohort-sharing deviation under summation. **Placebo parity is intentionally skipped for `trends_linear`**: R's per-path placebo computation re-runs on the path-restricted subsample with different control eligibility than Python's global-then-disaggregate architecture surfaces, producing a sign-and-magnitude divergence on paths whose switchers have minimal pre-window depth (e.g., `F_g=4` switchers). Placebo under `by_path + trends_linear` is exercised via internal regression in `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsLinear` (finite values, bootstrap inheritance) but not pinned to R bit-by-bit. Cross-surface invariants (analytical + bootstrap + placebo + sup-t + `path_cumulated_event_study` + `to_dataframe` columns + `summary()` rendering) are regression-tested at `TestByPathTrendsLinear`. **Per-path state-set trends:** when `trends_nonparam="state_col"` is set with `by_path=k`, the set membership column is validated and stored once globally as `set_ids_arr` (time-invariance, NaN rejection, partition-coarseness checks unchanged from the non-by_path path). The `set_ids` parameter is threaded through the four per-path IF helpers (`_compute_path_effects`, `_compute_path_placebos`, `_collect_path_bootstrap_inputs`, `_collect_path_placebo_bootstrap_inputs`) so per-path analytical SE, bootstrap, placebos, and sup-t bands all consume the set-restricted control pool automatically. R does NOT first-difference and does NOT cumulate under `trends_nonparam` (unlike `trends_lin`); per-horizon `Effect_l` is a normal DID with set-restricted controls. Per-path R parity is confirmed against `did_multiplegt_dyn(..., by_path=3, trends_nonparam="state", placebo=1)` at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsNonparam` on the `multi_path_reversible_by_path_trends_nonparam` scenario; per-path point estimates AND placebos match R bit-exactly (rtol ~1e-9), per-path SE matches within the Phase 2 envelope (~13% rtol observed). Cross-surface invariants are regression-tested at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathTrendsNonparam`. - **Note (Phase 3 `by_path` per-path joint sup-t bands):** When `n_bootstrap > 0` is set with `by_path=k`, per-path joint sup-t simultaneous confidence bands are computed across horizons `1..L_max` within each path. **Methodology:** a single `(n_bootstrap, n_eligible)` multiplier weight matrix (using the estimator's configured `bootstrap_weights` — Rademacher / Mammen / Webb) is drawn per path and broadcast across all horizons of that path, producing correlated bootstrap distributions across horizons within the path. The path-specific critical value `c_p = quantile(max_l |t_l|, 1 - α)` is then used to construct symmetric joint bands `effect_l ± c_p · se_l` per horizon, surfaced in `path_effects[path]["horizons"][l]["cband_conf_int"]` and at top-level `results.path_sup_t_bands[path] = {"crit_value", "alpha", "n_bootstrap", "method", "n_valid_horizons"}`. **Gates:** a path must have `>= 2` valid horizons (finite bootstrap SE > 0) AND a strict majority (more than 50%) of finite sup-t draws to receive a band; otherwise the path is absent from `path_sup_t_bands`. Both gates mirror the OVERALL `event_study_sup_t_bands` semantics at `chaisemartin_dhaultfoeuille_bootstrap.py:605,612`: `len(valid_horizons) >= 2` AND `finite_mask.sum() > 0.5 * n_bootstrap`. Exactly half-finite draws are NOT enough — the gate is strictly greater than half. **Empty-state contract:** `path_sup_t_bands is None` when not requested (no bootstrap or `by_path is None`); `{}` when requested but no path passes both gates. **`to_dataframe(level="by_path")` integration:** the table now includes `cband_lower` / `cband_upper` columns for parity with OVERALL `level="event_study"`; populated for positive-horizon rows of paths with a finite sup-t crit, NaN for placebo rows / unbanded paths / the requested-but-empty fallback DataFrame. **Methodology asymmetry vs OVERALL:** OVERALL sup-t reuses the same multi-horizon shared-draw distribution for both the SE in the t-stat denominator and the bootstrap distribution in the numerator. The per-path sup-t draws a fresh shared weight matrix per path AFTER the per-path SE bootstrap block has already populated `results.path_ses` via independent per-(path, horizon) draws — numerator: fresh shared draws, denominator: bootstrap SEs from the earlier independent draws. Asymptotically equivalent to OVERALL's self-consistent reuse, but NOT bit-identical. The fresh draw is intentional: it preserves RNG-state isolation and keeps every existing per-path SE seed-reproducibility test bit-stable post-implementation. **Inherited deviation from R:** the bootstrap SE used as the t-stat denominator carries the cross-path cohort-sharing SE deviation from R documented for `path_effects` above; the per-path sup-t crit therefore inherits the same deviation. **Interpretation:** the band covers joint inference *within a single path across horizons*; it does NOT provide simultaneous coverage *across paths* (a different inference target requiring a `path × horizon` re-derivation, deferred to a future wave). **Deviation from R:** `did_multiplegt_dyn` provides no joint / sup-t / simultaneous bands at any surface — this is a Python-only methodology extension, consistent with the existing OVERALL `event_study_sup_t_bands` (also Python-only). Regression test anchor: `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSupTBands`. diff --git a/tests/test_chaisemartin_dhaultfoeuille.py b/tests/test_chaisemartin_dhaultfoeuille.py index 8df6954c..e09c185c 100644 --- a/tests/test_chaisemartin_dhaultfoeuille.py +++ b/tests/test_chaisemartin_dhaultfoeuille.py @@ -3786,15 +3786,18 @@ def test_requires_lmax(self): @pytest.mark.parametrize( "fit_kwargs, msg", [ - # NB: the prior `controls` entry was removed when the - # `by_path + controls` gate was lifted (Wave 3 #5). The - # entry used `controls=["outcome"]` as a "any column works - # because the gate fires first" shortcut; after gate - # removal the column-existence path runs and `outcome` is - # itself a valid column, so there is no equivalent - # NotImplementedError-raising input to migrate to. - ({"trends_linear": True}, "trends_linear"), - ({"trends_nonparam": "group"}, "trends_nonparam"), + # NB: prior `controls` (Wave 3 #5), `trends_linear`, and + # `trends_nonparam` (Wave 3 #6+#7) entries were removed + # when their gates were lifted. After gate removal, those + # combinations either fit successfully (controls passes + # column-validation; trends_linear/trends_nonparam route + # to their respective code paths) or raise a non- + # NotImplementedError specific to the parameter (e.g., + # trends_nonparam=group raises a partition-coarseness + # ValueError because the set partition equals the group + # partition). Coverage for those combinations now lives + # in `TestByPathControls`, `TestByPathTrendsLinear`, and + # `TestByPathTrendsNonparam`. ({"heterogeneity": "group"}, "heterogeneity"), ({"design2": True}, "design2"), ({"honest_did": True}, "honest_did"), @@ -6755,3 +6758,1345 @@ def test_single_baseline_heterogeneous_F_g_does_not_warn(self): f"path={path} l={l_h}: effect not finite under " f"single-baseline + heterogeneous F_g" ) + + +# --------------------------------------------------------------------------- +# Wave 3 #6+#7: by_path + trends_linear (DID^{fd}) and by_path + +# trends_nonparam (state-set trends) +# --------------------------------------------------------------------------- + + +def _by_path_data_with_trends_linear(seed: int = 42) -> pd.DataFrame: + """Multi-path single-baseline panel with F_g spread for trends_linear. + + Mirrors the parity fixture structure: 80 switchers across 3 paths × 2 + distinct F_g per path (all F_g >= 4 to keep trends_linear's + F_g==2 filter a no-op and provide >= 2 valid pre-window Z values). + n_periods=13. 20 never-treated + 20 always-treated controls. + Per-group linear trends injected. + """ + rng = np.random.default_rng(seed) + n_periods = 13 + target_paths = [ + (0, 1, 1, 1), # path 1, sustained on + (0, 1, 1, 0), # path 2, on then off + (0, 1, 0, 0), # path 3, on briefly + ] + fg_path_counts = [ + (4, 0, 20), (5, 0, 18), # path 1 = 38 + (6, 1, 13), (7, 1, 11), # path 2 = 24 + (8, 2, 11), (9, 2, 7), # path 3 = 18 + ] + rows = [] + g_id = 0 + for F_g, path_idx, count in fg_path_counts: + target = target_paths[path_idx] + L_max = 3 + for _ in range(count): + D_row = [0] * n_periods + for j in range(L_max + 1): + D_row[F_g - 1 + j] = target[j] + for t in range(F_g + L_max, n_periods): + D_row[t] = target[L_max] + for t, d in enumerate(D_row): + rows.append({"group": g_id, "period": t, "treatment": d}) + g_id += 1 + # Never-treated and always-treated controls + for _ in range(20): + for t in range(n_periods): + rows.append({"group": g_id, "period": t, "treatment": 0}) + g_id += 1 + for _ in range(20): + for t in range(n_periods): + rows.append({"group": g_id, "period": t, "treatment": 1}) + g_id += 1 + df = pd.DataFrame(rows) + n_groups = df["group"].nunique() + group_fe = rng.normal(0, 2.0, size=n_groups) + g_trends = rng.normal(0, 0.5, size=n_groups) + df["outcome"] = ( + 10.0 + + group_fe[df["group"].values] + + 0.1 * df["period"].values + + 2.0 * df["treatment"].values + + rng.normal(0, 0.5, size=len(df)) + + g_trends[df["group"].values] * df["period"].values + ) + return df + + +def _by_path_data_with_trends_nonparam(seed: int = 43) -> pd.DataFrame: + """Multi-path panel with a 3-state column for trends_nonparam. + + Mirrors the parity fixture: 80 switchers across 3 paths (uses + `multi_path_reversible`-style structure; F_g distribution gives + cohort-single-path), n_periods=10. State assignment is deterministic + per group (`((group - 1) %% 3) + 1`). + """ + rng = np.random.default_rng(seed) + n_periods = 10 + target_paths = [ + (0, 1, 1, 1), + (0, 1, 1, 0), + (0, 1, 0, 0), + ] + # F_g 2/3 -> path 1 (40), F_g 4/5 -> path 2 (25), F_g 6 -> path 3 (10) + fg_path_counts = [ + (2, 0, 20), (3, 0, 20), + (4, 1, 15), (5, 1, 10), + (6, 2, 10), + (7, 2, 5), # rank 4-equivalent absorbed into path 3 cluster (kept by path 3 by frequency) + ] + # Adjust to ensure top 3 paths have unique counts: + # path 1 = 40, path 2 = 25, path 3 = 15 + fg_path_counts = [ + (2, 0, 20), (3, 0, 20), + (4, 1, 15), (5, 1, 10), + (6, 2, 8), (7, 2, 7), + ] + rows = [] + g_id = 0 + for F_g, path_idx, count in fg_path_counts: + target = target_paths[path_idx] + L_max = 3 + for _ in range(count): + D_row = [0] * n_periods + for j in range(L_max + 1): + D_row[F_g - 1 + j] = target[j] + for t in range(F_g + L_max, n_periods): + D_row[t] = target[L_max] + for t, d in enumerate(D_row): + rows.append({"group": g_id, "period": t, "treatment": d}) + g_id += 1 + for _ in range(20): + for t in range(n_periods): + rows.append({"group": g_id, "period": t, "treatment": 0}) + g_id += 1 + for _ in range(20): + for t in range(n_periods): + rows.append({"group": g_id, "period": t, "treatment": 1}) + g_id += 1 + df = pd.DataFrame(rows) + n_groups = df["group"].nunique() + group_fe = rng.normal(0, 2.0, size=n_groups) + df["outcome"] = ( + 10.0 + + group_fe[df["group"].values] + + 0.1 * df["period"].values + + 2.0 * df["treatment"].values + + rng.normal(0, 0.5, size=len(df)) + ) + df["state"] = (df["group"].values % 3) + 1 + return df + + +def _load_by_path_trends_lin_scenario(): + """Load golden-value scenario for by_path + trends_linear.""" + golden_path = ( + Path(__file__).parents[1] + / "benchmarks" + / "data" + / "dcdh_dynr_golden_values.json" + ) + if not golden_path.exists(): + pytest.skip( + f"dCDH golden values file not found at {golden_path}; " + "run: Rscript benchmarks/R/generate_dcdh_dynr_test_values.R" + ) + with open(golden_path) as f: + sc = json.load(f)["scenarios"].get( + "single_baseline_multi_path_by_path_trends_lin" + ) + if sc is None: + pytest.skip( + "scenario 'single_baseline_multi_path_by_path_trends_lin' absent" + ) + return pd.DataFrame(sc["data"]) + + +def _load_by_path_trends_nonparam_scenario(): + """Load golden-value scenario for by_path + trends_nonparam.""" + golden_path = ( + Path(__file__).parents[1] + / "benchmarks" + / "data" + / "dcdh_dynr_golden_values.json" + ) + if not golden_path.exists(): + pytest.skip( + f"dCDH golden values file not found at {golden_path}; " + "run: Rscript benchmarks/R/generate_dcdh_dynr_test_values.R" + ) + with open(golden_path) as f: + sc = json.load(f)["scenarios"].get( + "multi_path_reversible_by_path_trends_nonparam" + ) + if sc is None: + pytest.skip( + "scenario 'multi_path_reversible_by_path_trends_nonparam' absent" + ) + return pd.DataFrame(sc["data"]) + + +class TestByPathTrendsLinear: + """Wave 3 #6: ``by_path`` + ``trends_linear`` (DID^{fd}). + + Validates the gate-lift PR. The first-differencing transform at + ``chaisemartin_dhaultfoeuille.py:1599-1630`` runs once globally + BEFORE path enumeration, so per-path raw DID^{fd}_l surfaces on + ``path_effects[path]["horizons"][l]`` automatically. The new + ``path_cumulated_event_study`` field surfaces the cumulated level + effect ``delta_l = sum_{l'=1..l} DID^{fd}_{path, l'}`` per path + (mirrors the global ``linear_trends_effects`` cumulation at + ``:3340-3398``); the cumulated layer is keyed by horizon directly + (no ``"horizons"`` wrapper). + + R parity for per-path cumulated point estimates is validated + separately at + ``tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsLinear``. + """ + + def test_no_longer_raises(self): + """by_path + trends_linear no longer raises NotImplementedError.""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + + def test_path_effects_present_under_trends_linear(self): + """path_effects populated; per-horizon entries are DID^{fd}_l.""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + assert res.path_effects is not None and len(res.path_effects) > 0 + for path, entry in res.path_effects.items(): + for l_h, vals in entry["horizons"].items(): + assert np.isfinite(vals["effect"]), ( + f"path={path} l={l_h}: DID^{{fd}}_l not finite" + ) + + def test_path_cumulated_event_study_present(self): + """path_cumulated_event_study populated under trends_linear=True.""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + assert res.path_cumulated_event_study is not None + assert set(res.path_cumulated_event_study.keys()) == set( + res.path_effects.keys() + ) + for path, h_dict in res.path_cumulated_event_study.items(): + assert set(h_dict.keys()) == {1, 2, 3}, ( + f"path={path}: expected horizons 1..3, got {sorted(h_dict.keys())}" + ) + for l_h, vals in h_dict.items(): + assert np.isfinite(vals["effect"]), ( + f"path={path} l={l_h}: cumulated effect not finite" + ) + + def test_path_cumulated_is_none_without_trends_linear(self): + """path_cumulated_event_study is None when trends_linear=False.""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + ) + assert res.path_cumulated_event_study is None + + def test_path_cumulated_se_is_conservative_upper_bound(self): + """Cumulated SE per (path, l) equals sum of per-horizon DID^{fd} SEs.""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + for path, cum in res.path_cumulated_event_study.items(): + horizons = res.path_effects[path]["horizons"] + running_sum = 0.0 + for l_h in (1, 2, 3): + running_sum += horizons[l_h]["se"] + np.testing.assert_allclose( + cum[l_h]["se"], + running_sum, + rtol=1e-12, + err_msg=f"path={path} l={l_h}: cumulated SE not running sum", + ) + + def test_path_cumulated_recovers_per_group_running_sum(self): + """Cumulated point estimate matches the per-path running sum + of raw DID^{fd}_l values within rounding error. + """ + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + # Note: cumulated[l] is NOT exactly sum_{l'=1..l} path_effects[l'] + # (that would mix different N_l_path eligible sets across horizons). + # It IS the per-group running sum averaged at each horizon's + # eligible set. Verify cumulated is monotone-ish in magnitude + # vs the per-horizon DID values (sanity check; exact running-sum + # match is checked indirectly via R parity). + for path, cum in res.path_cumulated_event_study.items(): + horizons = res.path_effects[path]["horizons"] + # At horizon 1, eligible set ⊇ horizon 2's ⊇ horizon 3's, + # so cumulated[1] (single-horizon) should equal DID^{fd}_1 + # for groups eligible at horizon 1. + assert np.isfinite(cum[1]["effect"]) + assert np.isfinite(horizons[1]["effect"]) + + def test_to_dataframe_by_path_with_trends_linear(self): + """to_dataframe(level='by_path') exposes cumulated_effect/se columns.""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + df_bp = res.to_dataframe(level="by_path") + assert "cumulated_effect" in df_bp.columns + assert "cumulated_se" in df_bp.columns + positive = df_bp[df_bp["horizon"] > 0] + assert positive["cumulated_effect"].notna().all() + assert positive["cumulated_se"].notna().all() + + def test_to_dataframe_cumulated_columns_nan_when_no_trends_linear(self): + """cumulated_* columns are always present, NaN when trends_linear=False.""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + ) + df_bp = res.to_dataframe(level="by_path") + assert "cumulated_effect" in df_bp.columns + assert "cumulated_se" in df_bp.columns + assert df_bp["cumulated_effect"].isna().all() + assert df_bp["cumulated_se"].isna().all() + + def test_summary_renders_path_cumulated_block(self): + """summary() includes a cumulated sub-section under each path.""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + text = res.summary() + assert "Cumulated Level Effects (DID^{fd}, trends_linear)" in text + assert "Level_1" in text + assert "Level_2" in text + assert "Level_3" in text + + def test_per_period_effects_unaffected_by_trends_linear_by_path(self): + """``per_period_effects`` is unaffected by the by_path + + trends_linear combo. The per-period DID path uses raw ``Y_mat`` + per the comment at ``chaisemartin_dhaultfoeuille.py:1493-1496``; + first-differencing only affects the multi-horizon path. Adding + ``by_path`` is a layer on top of multi-horizon, so per-period + effects should be bit-identical with vs without by_path. + """ + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est_no_bp = ChaisemartinDHaultfoeuille(drop_larger_lower=False) + res_no_bp = est_no_bp.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + est_bp = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res_bp = est_bp.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + # Per-period effects should be bit-identical: by_path doesn't + # touch the per-period DID path; trends_linear doesn't either + # (per the contract at :1493-1496). + no_bp_pp = res_no_bp.per_period_effects + bp_pp = res_bp.per_period_effects + assert (no_bp_pp is None) == (bp_pp is None), ( + f"per_period_effects presence differs (no_bp={no_bp_pp is not None} " + f"vs bp={bp_pp is not None})" + ) + if no_bp_pp is not None and bp_pp is not None: + assert set(no_bp_pp.keys()) == set(bp_pp.keys()), ( + f"per_period_effects horizon set differs" + ) + for t_h in no_bp_pp: + for field_name in ("did_plus_t", "did_minus_t"): + if field_name not in no_bp_pp[t_h]: + continue + no_v = no_bp_pp[t_h][field_name] + bp_v = bp_pp[t_h][field_name] + if isinstance(no_v, dict) and "effect" in no_v: + no_v = no_v["effect"] + bp_v = bp_v["effect"] + if no_v is not None and np.isfinite(no_v): + np.testing.assert_allclose( + bp_v, + no_v, + rtol=1e-12, + err_msg=( + f"per_period_effects[{t_h}][{field_name}] " + f"differs under by_path + trends_linear " + f"(no_bp={no_v} vs bp={bp_v})" + ), + ) + + @pytest.mark.slow + def test_bootstrap_with_trends_linear_finite_se(self): + """Bootstrap SE finite per (path, horizon) under trends_linear.""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, n_bootstrap=200, seed=42 + ) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + for path, entry in res.path_effects.items(): + for l_h, vals in entry["horizons"].items(): + assert np.isfinite(vals["se"]), ( + f"path={path} l={l_h}: bootstrap SE not finite" + ) + + def test_per_path_placebos_with_trends_linear_present(self): + """``path_placebo_event_study`` populated under ``by_path + + trends_linear + placebo=True`` with finite point estimates and + finite SEs on negative-horizon entries (raw per-horizon, NOT + cumulated — per the documented R contract). The R-parity test + skips negative-horizon rows because of the documented + Python-vs-R per-path placebo divergence; this test pins the + Python-side population invariant so the surface itself doesn't + regress to None / empty / all-NaN. + """ + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, placebo=True + ) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + assert res.path_placebo_event_study is not None + assert len(res.path_placebo_event_study) > 0 + # At least one path × negative-lag pair should have a finite + # point estimate (raw per-horizon, not cumulated). Negative + # keys mirror the placebo_event_study convention. + any_finite = False + for path, lag_dict in res.path_placebo_event_study.items(): + assert all(k < 0 for k in lag_dict.keys()), ( + f"path={path}: placebo lag keys must be negative ints" + ) + for lag_k, vals in lag_dict.items(): + if np.isfinite(vals["effect"]): + any_finite = True + # When effect is finite, SE should also be finite + # (NaN-consistent contract: finite point + NaN SE + # is not allowed) + assert np.isfinite(vals["se"]), ( + f"path={path} lag={lag_k}: finite effect " + f"({vals['effect']}) with non-finite SE " + f"({vals['se']})" + ) + assert any_finite, ( + "All placebo cells are non-finite; the trends_linear + " + "placebo path may have regressed." + ) + + @pytest.mark.slow + def test_per_path_placebos_with_trends_linear_bootstrap_inference(self): + """Bootstrap-derived inference fields populated on negative- + horizon ``path_placebo_event_study`` rows under ``by_path + + trends_linear + placebo + n_bootstrap > 0``. Pins the placebo + bootstrap collector path that consumes the first-differenced + ``Y_mat`` AND the bootstrap propagation block at + ``chaisemartin_dhaultfoeuille.py:3097-`` for negative horizons. + Without this, a silent regression in the placebo bootstrap + propagation would surface analytical SEs on a bootstrap fit. + """ + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est_a = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, placebo=True + ) + res_a = est_a.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + est_b = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + by_path=3, + placebo=True, + n_bootstrap=200, + seed=42, + ) + res_b = est_b.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + # Negative-horizon placebo rows must exist and carry bootstrap- + # derived inference. Verify by comparing analytical-only fit's + # SEs to bootstrap-fit's SEs on the same negative-horizon + # entries: bootstrap should differ (non-bit-identical) since + # the propagation block overwrites SE / p_value / conf_int. + assert res_b.path_placebo_event_study is not None + any_se_diff = False + any_finite = False + for path, lag_dict in res_b.path_placebo_event_study.items(): + for lag_k, vals_b in lag_dict.items(): + if not np.isfinite(vals_b["se"]): + continue + any_finite = True + vals_a = res_a.path_placebo_event_study.get(path, {}).get(lag_k) + if vals_a is None or not np.isfinite(vals_a["se"]): + continue + if abs(vals_b["se"] - vals_a["se"]) > 1e-10: + any_se_diff = True + break + if any_se_diff: + break + assert any_finite, "No finite negative-horizon bootstrap SEs surfaced" + assert any_se_diff, ( + "Bootstrap fit produced bit-identical SEs to analytical fit on " + "every negative-horizon placebo cell; the placebo bootstrap " + "propagation block under trends_linear may not be running." + ) + + @pytest.mark.slow + def test_per_path_placebos_with_trends_linear_bootstrap_nan_consistent(self): + """``n_bootstrap=1`` produces NaN-consistent inference on + negative-horizon ``path_placebo_event_study`` rows under + ``by_path + trends_linear + placebo``. Pins the library-wide + NaN-on-invalid bootstrap contract on the new placebo path. + """ + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", (UserWarning, RuntimeWarning)) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + by_path=3, + placebo=True, + n_bootstrap=1, + seed=42, + ) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + assert res.path_placebo_event_study is not None + # n_bootstrap=1 → degenerate bootstrap distribution → NaN SE / + # p_value / conf_int on every negative-horizon entry. + for path, lag_dict in res.path_placebo_event_study.items(): + for lag_k, vals in lag_dict.items(): + assert not np.isfinite(vals["se"]), ( + f"path={path} lag={lag_k}: SE finite ({vals['se']}) " + "under n_bootstrap=1; expected NaN" + ) + assert not np.isfinite(vals["p_value"]), ( + f"path={path} lag={lag_k}: p_value finite under n_bootstrap=1" + ) + + @pytest.mark.slow + def test_sup_t_bands_with_trends_linear_finite_crit(self): + """Per-path joint sup-t bands populated under ``by_path + + trends_linear + n_bootstrap > 0``. Pins the bootstrap-collector + path that consumes the first-differenced ``Y_mat``. + """ + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + by_path=3, + n_bootstrap=400, + seed=42, + ) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + assert res.path_sup_t_bands is not None + any_finite = False + for path, info in res.path_sup_t_bands.items(): + crit = info.get("crit_value", np.nan) + if np.isfinite(crit) and crit > 0: + any_finite = True + break + assert any_finite, ( + "No path produced a finite sup-t crit value under " + "trends_linear + bootstrap" + ) + df_bp = res.to_dataframe(level="by_path") + positive = df_bp[df_bp["horizon"] > 0] + assert positive["cband_lower"].notna().any(), ( + "No positive-horizon cband rows populated under " + "trends_linear + bootstrap" + ) + + @pytest.mark.slow + def test_bootstrap_cumulated_uses_post_bootstrap_per_horizon_se(self): + """Cumulated SE under bootstrap equals running sum of bootstrap per-horizon SEs. + + Regression for the post-bootstrap propagation invariant: the + per-path cumulated layer must be derived from the FINAL post- + bootstrap per-horizon SEs, not the analytical SEs that + path_effects was initially populated with. Mirrors the global + `linear_trends_effects` post-bootstrap recomputation contract. + """ + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, n_bootstrap=200, seed=42 + ) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + # Sanity: bootstrap path produced the cumulated layer. + assert res.path_cumulated_event_study is not None + for path, cum in res.path_cumulated_event_study.items(): + horizons = res.path_effects[path]["horizons"] + running = 0.0 + for l_h in (1, 2, 3): + bs_se = horizons[l_h]["se"] + assert np.isfinite(bs_se), ( + f"path={path} l={l_h}: bootstrap SE not finite " + "(precondition for cumulated assertion)" + ) + running += bs_se + np.testing.assert_allclose( + cum[l_h]["se"], + running, + rtol=1e-12, + err_msg=( + f"path={path} l={l_h}: cumulated SE not equal " + f"to running sum of post-bootstrap per-horizon SEs " + f"(cum_se={cum[l_h]['se']:.6f}, " + f"sum_bs_se={running:.6f}). The cumulated layer " + "must be recomputed AFTER bootstrap propagation." + ), + ) + + def test_multi_baseline_panel_emits_r_deviation_warning(self): + """When ``by_path + trends_linear`` is fit on a panel where + switchers have multiple ``D_{g,1}`` baseline values, the + estimator must emit a ``UserWarning`` documenting the + deviation from R's per-path full-pipeline call. Mirrors the + analogous ``by_path + controls`` warning at + ``test_multi_baseline_panel_emits_r_deviation_warning`` in + ``TestByPathControls``. + """ + # 3 joiners (D_{g,1}=0) + 3 leavers (D_{g,1}=1) + 4 always- + # treated + 4 never-treated controls; F_g >= 3 so trends_lin's + # F_g==2 filter doesn't drop everyone. + rng = np.random.default_rng(7) + rows = [] + + def _add(group, treatment_path): + for t, d in enumerate(treatment_path): + y = d * 2.0 + rng.normal(0, 0.1) + 0.1 * t + rows.append( + {"group": group, "period": t, "treatment": d, "outcome": y} + ) + + # F_g=3 joiners (path 0,0,1,1,1,1) + for g in (1, 2, 3): + _add(g, [0, 0, 1, 1, 1, 1]) + # F_g=4 leavers (path 1,1,1,0,0,0) + for g in (4, 5, 6): + _add(g, [1, 1, 1, 0, 0, 0]) + # Always-treated controls (D_{g,1}=1) + for g in (7, 8, 9, 10): + _add(g, [1, 1, 1, 1, 1, 1]) + # Never-treated controls (D_{g,1}=0) + for g in (11, 12, 13, 14): + _add(g, [0, 0, 0, 0, 0, 0]) + data = pd.DataFrame(rows) + + # Sanity: switchers have both D_{g,1}=0 and D_{g,1}=1 baselines + switcher_ids = data[data["group"].isin([1, 2, 3, 4, 5, 6])] + baselines = ( + switcher_ids[switcher_ids["period"] == 0] + .groupby("group")["treatment"] + .first() + ) + assert sorted(baselines.unique()) == [0, 1] + + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + + deviation_msgs = [ + str(w.message) + for w in caught + if issubclass(w.category, UserWarning) + and "by_path + trends_linear" in str(w.message) + and "switcher baselines" in str(w.message) + ] + assert deviation_msgs, ( + "Expected a UserWarning mentioning 'by_path + trends_linear' " + "and 'switcher baselines D_{g,1}' on a multi-baseline panel. " + f"Captured warnings: {[str(w.message) for w in caught]}" + ) + + def test_single_baseline_panel_does_not_emit_r_deviation_warning(self): + """The multi-baseline R-deviation warning must NOT fire on a + single-baseline panel under ``by_path + trends_linear``. Pinned + against the standard fixture (all joiners, ``D_{g,1}=0``).""" + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + deviation_msgs = [ + str(w.message) + for w in caught + if issubclass(w.category, UserWarning) + and "by_path + trends_linear" in str(w.message) + and "switcher baselines" in str(w.message) + ] + assert not deviation_msgs, ( + "Multi-baseline trends_linear deviation warning fired on a " + f"single-baseline panel: {deviation_msgs}" + ) + + def test_F_g_three_boundary_case_emits_warning(self): + """When ``by_path + trends_linear`` is fit on a panel that + includes ``F_g=3`` switchers, the estimator must emit a + targeted ``UserWarning`` documenting the boundary-case + divergence from R. Locks the warning predicate + (``first_switch_idx_arr == 2``) on a fixture that includes + F_g=3 + F_g=4 switchers. + """ + # 4 F_g=3 switchers (path 0,0,1,1,1,1,1,1) + 4 F_g=4 + # switchers (path 0,0,0,1,1,1,1,1) + 5 never-treated + + # 5 always-treated controls. n_periods=8, all + # single-baseline (D_{g,1}=0) so the multi-baseline warning + # does NOT fire — only the new F_g=3 boundary warning. + rng = np.random.default_rng(13) + rows = [] + + def _add(group, treatment_path): + for t, d in enumerate(treatment_path): + y = d * 2.0 + 0.05 * group + rng.normal(0, 0.1) + rows.append( + {"group": group, "period": t, "treatment": d, "outcome": y} + ) + + for g in (1, 2, 3, 4): + _add(g, [0, 0, 1, 1, 1, 1, 1, 1]) # F_g=3 path + for g in (5, 6, 7, 8): + _add(g, [0, 0, 0, 1, 1, 1, 1, 1]) # F_g=4 path + for g in (9, 10, 11, 12, 13): + _add(g, [1, 1, 1, 1, 1, 1, 1, 1]) + for g in (14, 15, 16, 17, 18): + _add(g, [0, 0, 0, 0, 0, 0, 0, 0]) + data = pd.DataFrame(rows) + + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + + boundary_msgs = [ + str(w.message) + for w in caught + if issubclass(w.category, UserWarning) + and "by_path + trends_linear" in str(w.message) + and "F_g=3" in str(w.message) + ] + assert boundary_msgs, ( + "Expected a UserWarning naming 'by_path + trends_linear' " + "and 'F_g=3' on a panel that includes F_g=3 switchers. " + f"Captured warnings: {[str(w.message) for w in caught]}" + ) + + def test_single_baseline_heterogeneous_F_g_does_not_warn(self): + """Single-baseline switcher panel with HETEROGENEOUS ``F_g`` + across paths must NOT trigger the multi-baseline trends_linear + warning, even though F_g varies. Pin the precise warning + condition: it's switcher-baseline multiplicity, NOT F_g + multiplicity, that triggers the divergence pattern.""" + # _by_path_data_with_trends_linear has F_g in {4,5,6,7,8,9} + # across 3 paths; all switchers have D_{g,1}=0. + data = _by_path_data_with_trends_linear() + # Sanity: F_g varies across switchers + switcher_first_treat = ( + data[data["treatment"] == 1].groupby("group")["period"].min() + ) + all_groups_first_treat = ( + data.groupby("group")["treatment"].agg(lambda x: x.iloc[0]) + ) + # Drop always-treated (D_{g,1}=1) groups to isolate switchers + switcher_groups = all_groups_first_treat[all_groups_first_treat == 0].index + switcher_F_g = switcher_first_treat[switcher_first_treat.index.isin(switcher_groups)] + # _by_path_data_with_trends_linear has 80 switchers, 20 always- + # treated, 20 never-treated; switchers all have D_{g,1}=0 and + # F_g spans {4,5,6,7,8,9} (6 distinct values) + assert switcher_F_g.nunique() >= 2, ( + "Test fixture pre-condition violated: F_g should be heterogeneous " + "across switchers" + ) + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + deviation_msgs = [ + str(w.message) + for w in caught + if issubclass(w.category, UserWarning) + and "by_path + trends_linear" in str(w.message) + and "switcher baselines" in str(w.message) + ] + assert not deviation_msgs, ( + f"Heterogeneous F_g triggered the warning: {deviation_msgs}" + ) + # Sanity: fit produced finite per-path effects + assert res.path_effects is not None and len(res.path_effects) >= 1 + + @pytest.mark.slow + def test_bootstrap_cumulated_nan_consistent_when_n_bootstrap_one(self): + """n_bootstrap=1: bootstrap SE non-finite → cumulated SE/t/p/CI NaN. + + Locks the library-wide NaN-on-invalid bootstrap contract on the + new `path_cumulated_event_study` surface. With n_bootstrap=1 the + bootstrap SE is degenerate (computed from a single draw); the + bootstrap propagation block writes NaN to the per-horizon SE, + and the cumulated layer's running-sum SE must be NaN-consistent + rather than retaining the analytical value. + """ + data = _by_path_data_with_trends_linear() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", (UserWarning, RuntimeWarning)) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, n_bootstrap=1, seed=42 + ) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + assert res.path_cumulated_event_study is not None + # n_bootstrap=1 should produce NaN per-horizon SEs (degenerate + # bootstrap distribution); the cumulated layer must propagate + # NaN through SE / t_stat / p_value / conf_int. + for path, cum in res.path_cumulated_event_study.items(): + for l_h, vals in cum.items(): + assert not np.isfinite(vals["se"]), ( + f"path={path} l={l_h}: cumulated SE finite " + f"({vals['se']}) under n_bootstrap=1; expected NaN per " + "the NaN-on-invalid bootstrap contract" + ) + assert not np.isfinite(vals["t_stat"]), ( + f"path={path} l={l_h}: cumulated t_stat not NaN" + ) + assert not np.isfinite(vals["p_value"]), ( + f"path={path} l={l_h}: cumulated p_value not NaN" + ) + ci_lo, ci_hi = vals["conf_int"] + assert not (np.isfinite(ci_lo) and np.isfinite(ci_hi)), ( + f"path={path} l={l_h}: cumulated conf_int not NaN" + ) + + +class TestByPathTrendsNonparam: + """Wave 3 #7: ``by_path`` + ``trends_nonparam`` (state-set trends). + + Validates the gate-lift PR + ``set_ids`` threading. The + ``set_ids_arr`` array is computed once globally at + ``chaisemartin_dhaultfoeuille.py:1722`` and threaded through the + per-path IF helpers so per-path analytical SE, bootstrap, placebos, + and sup-t bands all consume the set-restricted control pool. + + R parity is validated separately at + ``tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathTrendsNonparam``. + """ + + def test_no_longer_raises(self): + """by_path + trends_nonparam no longer raises NotImplementedError.""" + data = _by_path_data_with_trends_nonparam() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + + def test_set_restriction_changes_per_path_estimates(self): + """Fitting with vs without trends_nonparam changes per-path estimates.""" + data = _by_path_data_with_trends_nonparam() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est_no_set = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3 + ) + res_no = est_no_set.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + ) + est_set = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3 + ) + res_set = est_set.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + # At least one (path, horizon) should differ: set restriction + # shrinks the control pool and produces different DIDs. + any_diff = False + for path in res_no.path_effects: + if path not in res_set.path_effects: + continue + for l_h in res_no.path_effects[path]["horizons"]: + eff_no = res_no.path_effects[path]["horizons"][l_h]["effect"] + eff_set = res_set.path_effects[path]["horizons"][l_h]["effect"] + if np.isfinite(eff_no) and np.isfinite(eff_set): + if abs(eff_no - eff_set) > 1e-6: + any_diff = True + break + if any_diff: + break + assert any_diff, ( + "Expected at least one per-path estimate to differ when " + "trends_nonparam restricts the control pool, but all match. " + "set_ids may not be threading through to the per-path IF helpers." + ) + + def test_per_path_se_finite(self): + """Per-path analytical SE finite under trends_nonparam.""" + data = _by_path_data_with_trends_nonparam() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + for path, entry in res.path_effects.items(): + for l_h, vals in entry["horizons"].items(): + assert np.isfinite(vals["se"]) and vals["se"] > 0, ( + f"path={path} l={l_h}: SE not positive-finite" + ) + + def test_time_varying_set_with_by_path_raises(self): + """time-varying set assignment still rejected.""" + data = _by_path_data_with_trends_nonparam() + # Make state vary within group 0 + data.loc[(data["group"] == 0) & (data["period"] == 0), "state"] = 99 + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + with pytest.raises(ValueError, match="time-invariant"): + est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + + def test_missing_set_column_with_by_path_raises(self): + """missing column still rejected.""" + data = _by_path_data_with_trends_nonparam() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=2) + with pytest.raises(ValueError, match="not found"): + est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="missing_column", + L_max=3, + ) + + @pytest.mark.slow + def test_bootstrap_with_trends_nonparam_finite_se(self): + """Bootstrap SE finite per (path, horizon) under trends_nonparam.""" + data = _by_path_data_with_trends_nonparam() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, n_bootstrap=200, seed=42 + ) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + for path, entry in res.path_effects.items(): + for l_h, vals in entry["horizons"].items(): + assert np.isfinite(vals["se"]), ( + f"path={path} l={l_h}: bootstrap SE not finite" + ) + + def test_per_period_effects_unaffected_by_trends_nonparam_by_path(self): + """``per_period_effects`` is unaffected by the by_path + + trends_nonparam combo. Symmetric pin to the trends_linear + version; per-period DID does not consume ``set_ids`` (the + set-restriction only affects the multi-horizon path). + """ + data = _by_path_data_with_trends_nonparam() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est_no_bp = ChaisemartinDHaultfoeuille(drop_larger_lower=False) + res_no_bp = est_no_bp.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + est_bp = ChaisemartinDHaultfoeuille(drop_larger_lower=False, by_path=3) + res_bp = est_bp.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + no_bp_pp = res_no_bp.per_period_effects + bp_pp = res_bp.per_period_effects + assert (no_bp_pp is None) == (bp_pp is None) + if no_bp_pp is not None and bp_pp is not None: + assert set(no_bp_pp.keys()) == set(bp_pp.keys()) + for t_h in no_bp_pp: + for field_name in ("did_plus_t", "did_minus_t"): + if field_name not in no_bp_pp[t_h]: + continue + no_v = no_bp_pp[t_h][field_name] + bp_v = bp_pp[t_h][field_name] + if isinstance(no_v, dict) and "effect" in no_v: + no_v = no_v["effect"] + bp_v = bp_v["effect"] + if no_v is not None and np.isfinite(no_v): + np.testing.assert_allclose( + bp_v, no_v, rtol=1e-12, + err_msg=( + f"per_period_effects[{t_h}][{field_name}] " + f"differs under by_path + trends_nonparam" + ), + ) + + @pytest.mark.slow + def test_sup_t_bands_with_trends_nonparam_finite_crit(self): + """Per-path joint sup-t bands populated under + ``by_path + trends_nonparam + n_bootstrap > 0``. Pins the + bootstrap-collector path that consumes the set-restricted IF + through the threaded ``set_ids`` parameter. + """ + data = _by_path_data_with_trends_nonparam() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + by_path=3, + n_bootstrap=400, + seed=42, + ) + res = est.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + # path_sup_t_bands should be populated; at least one path + # passes the strict-majority gate from PR #374. + assert res.path_sup_t_bands is not None + any_finite = False + for path, info in res.path_sup_t_bands.items(): + crit = info.get("crit_value", np.nan) + if np.isfinite(crit) and crit > 0: + any_finite = True + break + assert any_finite, ( + "No path produced a finite sup-t crit value under " + "trends_nonparam + bootstrap; the set_ids threading may " + "not be reaching the per-path bootstrap collector." + ) + # to_dataframe(level="by_path") cband columns should be + # populated for at least one positive-horizon row. + df_bp = res.to_dataframe(level="by_path") + assert "cband_lower" in df_bp.columns + assert "cband_upper" in df_bp.columns + positive = df_bp[df_bp["horizon"] > 0] + assert positive["cband_lower"].notna().any(), ( + "No positive-horizon cband rows populated under " + "trends_nonparam + bootstrap" + ) + + @pytest.mark.slow + def test_per_path_placebos_with_trends_nonparam_bootstrap_inference(self): + """Bootstrap-derived inference fields populated on negative- + horizon ``path_placebo_event_study`` rows under ``by_path + + trends_nonparam + placebo + n_bootstrap > 0``. + + Pins the ``set_ids`` threading into + ``_collect_path_placebo_bootstrap_inputs`` (line 5963 in the + diff): without that threading, the placebo bootstrap collector + would re-compute the per-group placebo IF with set_ids=None, + bypassing the set-restricted control pool. We verify by + comparing two bootstrap fits — one with trends_nonparam, one + without — and asserting at least one negative-horizon SE + differs (the set restriction must propagate through the + placebo bootstrap path) AND remains finite. + """ + data = _by_path_data_with_trends_nonparam() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + est_no_set = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + by_path=3, + placebo=True, + n_bootstrap=200, + seed=42, + ) + res_no = est_no_set.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + L_max=3, + ) + est_set = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, + by_path=3, + placebo=True, + n_bootstrap=200, + seed=42, + ) + res_set = est_set.fit( + data, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + assert res_set.path_placebo_event_study is not None + assert res_no.path_placebo_event_study is not None + any_diff = False + any_finite = False + for path, lag_dict in res_set.path_placebo_event_study.items(): + for lag_k, vals_set in lag_dict.items(): + if not np.isfinite(vals_set["se"]): + continue + any_finite = True + vals_no = res_no.path_placebo_event_study.get(path, {}).get( + lag_k + ) + if vals_no is None or not np.isfinite(vals_no["se"]): + continue + # Set restriction shrinks the control pool; with the + # same seed, the bootstrap distribution should differ. + if abs(vals_set["se"] - vals_no["se"]) > 1e-10: + any_diff = True + break + if any_diff: + break + assert any_finite, ( + "No finite negative-horizon bootstrap SEs surfaced under " + "trends_nonparam + placebo + bootstrap" + ) + assert any_diff, ( + "Bootstrap placebo SEs are bit-identical with vs without " + "trends_nonparam restriction; set_ids may not be reaching " + "the per-path placebo bootstrap collector." + ) diff --git a/tests/test_chaisemartin_dhaultfoeuille_parity.py b/tests/test_chaisemartin_dhaultfoeuille_parity.py index 550a43c6..0a916cc3 100644 --- a/tests/test_chaisemartin_dhaultfoeuille_parity.py +++ b/tests/test_chaisemartin_dhaultfoeuille_parity.py @@ -22,6 +22,7 @@ import json from pathlib import Path +from typing import List import pandas as pd import pytest @@ -362,6 +363,26 @@ def _golden_to_df_with_covariates(data_dict: dict) -> pd.DataFrame: return pd.DataFrame(cols) +def _golden_to_df_with_extra( + data_dict: dict, extra_cols: List[str] +) -> pd.DataFrame: + """Reconstruct a panel DataFrame including arbitrary extra columns. + + Used for scenarios that ship non-covariate side columns (e.g., + ``state`` for ``trends_nonparam``). + """ + cols = { + "group": data_dict["group"], + "period": data_dict["period"], + "treatment": data_dict["treatment"], + "outcome": data_dict["outcome"], + } + for c in extra_cols: + if c in data_dict: + cols[c] = data_dict[c] + return pd.DataFrame(cols) + + class TestDCDHDynRParityPhase3: """ Phase 3 parity tests: covariates (DID^X) and linear trends (DID^{fd}). @@ -874,3 +895,294 @@ def test_parity_multi_path_reversible_by_path_controls(self, golden_values): f"path={path_key} h={h} SE: " f"py={py_se:.4f} vs r={r_se:.4f}" ) + + +class TestDCDHDynRParityByPathTrendsLinear: + """ + Parity tests for ``by_path + trends_linear`` (DID^{fd} group-specific + linear trends) against R DIDmultiplegtDYN 2.3.3. + + R's ``did_multiplegt_dyn(..., by_path=k, trends_lin=TRUE)`` re-runs + the estimator per path with a path-restricted subsample + (``R/R/did_multiplegt_dyn.R`` lines 226-257 dispatcher → per-path + ``did_multiplegt_main()`` call with ``trends_lin=TRUE``). Inside + ``did_multiplegt_main``, R first-differences the outcome and drops + ``F_g==2`` switchers and ``time==1`` rows, then loops through + horizons computing per-horizon DID values and **cumulating** them + (R's ``Effect_l`` under ``trends_lin=TRUE`` is the cumulated level + effect ``delta_l``, NOT the raw second-difference DID^{fd}_l — + verified by the existing parity test at + ``test_parity_joiners_only_trends_lin`` lines 403-409). + + Python's architecture first-differences once globally before path + enumeration, then disaggregates per path on the global + ``multi_horizon_dids`` (raw DID^{fd}_l surfaces on + ``path_effects``). The new ``path_cumulated_event_study`` field + surfaces the cumulated ``delta_l`` per path (sum of per-horizon + DID^{fd} values for the path's switchers, eligibility-restricted at + each horizon — mirrors the global ``linear_trends_effects`` + cumulation at ``chaisemartin_dhaultfoeuille.py:3340-3398``). + + The fixture ``single_baseline_multi_path_by_path_trends_lin`` is + designed for clean parity: + - Single-baseline (all switchers ``D_{g,1}=0``) eliminates the + multi-baseline divergence pattern (same architectural family as + ``controls`` — see ``TestDCDHDynRParityByPathControls``) + - Cohort-single-path (each ``F_g`` maps to exactly one path) + eliminates the cross-path cohort-sharing deviation that PR #360 + documented for ``path_effects`` + - All ``F_g >= 4`` keeps trends_lin's ``F_g==2`` filter a no-op + and gives both Python and R 2+ valid pre-window Z values + (avoids the boundary-case divergence at ``F_g=3``) + + Per-path cumulated point estimates match R bit-exactly (rtol ~1e-9) + on event horizons under these conditions. Cumulated SE inherits the + cross-path cohort-sharing deviation amplified by summation across + horizons; ``CUM_SE_RTOL=0.20`` is widened from the 0.12 used for + non-cumulated by_path parity (per the conservative upper-bound SE + formula at ``_compute_path_cumulated_event_study``). + + **Placebo divergence (documented):** under ``trends_linear=True`` + + ``by_path``, the per-path placebo at lag 1 diverges between Python + and R for paths whose switchers have minimal pre-window depth + (e.g., ``F_g=4`` switchers in path 1 of this fixture). R's per-path + placebo computation re-runs on the path-restricted subsample with + different control eligibility than Python's global-then-disaggregate + architecture surfaces. The placebo parity assertion is therefore + skipped here; placebo + ``trends_linear`` is exercised via internal + regression in ``TestByPathTrendsLinear`` (finite values, bootstrap + inheritance) but not pinned to R bit-by-bit. See REGISTRY.md + "Note (Phase 3 by_path)" sub-paragraph "Per-path linear-trends + DID^{fd}" for the full deviation documentation. + """ + + POINT_RTOL = 1e-9 + POINT_ATOL = 1e-9 + CUM_SE_RTOL = 0.20 + + def _path_key_from_r_label(self, r_label: str): + return tuple(int(x) for x in r_label.split(",")) + + def test_parity_single_baseline_multi_path_trends_lin(self, golden_values): + """3-path single-baseline trends_lin: by_path=3, trends_linear=True.""" + import math + import warnings + + scenario = golden_values.get( + "single_baseline_multi_path_by_path_trends_lin" + ) + if scenario is None: + pytest.skip( + "scenario 'single_baseline_multi_path_by_path_trends_lin' " + "not in golden values" + ) + + df = _golden_to_df(scenario["data"]) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, placebo=True + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + results = est.fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_linear=True, + L_max=3, + ) + + r_by_path = scenario["results"]["by_path"] + assert results.path_cumulated_event_study is not None, ( + "path_cumulated_event_study should be populated under " + "by_path + trends_linear=True" + ) + py_cum = results.path_cumulated_event_study + + py_keys = set(py_cum.keys()) + r_keys = {self._path_key_from_r_label(e["path"]) for e in r_by_path} + assert py_keys == r_keys, ( + f"Path-set mismatch.\n" + f" Python only: {py_keys - r_keys}\n" + f" R only: {r_keys - py_keys}" + ) + + for r_path_entry in r_by_path: + path_key = self._path_key_from_r_label(r_path_entry["path"]) + py_path_cum = py_cum[path_key] + + for h_str, r_h in r_path_entry["horizons"].items(): + h = int(h_str) + # Skip placebo horizons (negative keys) — see class + # docstring for the documented deviation. + if h <= 0: + continue + assert h in py_path_cum, ( + f"path={path_key}: horizon {h} missing from " + f"path_cumulated_event_study" + ) + py_h = py_path_cum[h] + + assert py_h["n_obs"] == int(r_h["n_switchers"]), ( + f"path={path_key} h={h}: switcher-count mismatch " + f"py={py_h['n_obs']} vs r={int(r_h['n_switchers'])}" + ) + + assert py_h["effect"] == pytest.approx( + r_h["effect"], rel=self.POINT_RTOL, abs=self.POINT_ATOL + ), ( + f"path={path_key} h={h}: cumulated effect mismatch " + f"py={py_h['effect']:.6f} vs r={r_h['effect']:.6f}" + ) + + py_se = py_h["se"] + r_se = r_h["se"] + py_finite_positive = math.isfinite(py_se) and py_se > 0.0 + r_finite_positive = math.isfinite(r_se) and r_se > 0.0 + assert py_finite_positive == r_finite_positive, ( + f"path={path_key} h={h} SE state mismatch " + f"(py_se={py_se}, r_se={r_se})" + ) + if py_finite_positive and r_finite_positive: + assert py_se == pytest.approx( + r_se, rel=self.CUM_SE_RTOL + ), ( + f"path={path_key} h={h} cumulated SE: " + f"py={py_se:.4f} vs r={r_se:.4f}" + ) + + +class TestDCDHDynRParityByPathTrendsNonparam: + """ + Parity tests for ``by_path + trends_nonparam`` (state-set trends) + against R DIDmultiplegtDYN 2.3.3. + + R's ``did_multiplegt_dyn(..., by_path=k, trends_nonparam="state")`` + re-runs the estimator per path with a path-restricted subsample + (same dispatch loop as other ``by_path`` modes). The + ``trends_nonparam`` parameter is passed through to + ``did_multiplegt_main()`` which uses the column as a grouping key + for control-pool restriction (``did_multiplegt_main`` lines + 414-415: ``grp_cols <- c(grp_cols, trends_nonparam)``). R does + NOT first-difference and does NOT cumulate under + ``trends_nonparam``; ``Effect_l`` is a normal per-horizon DID + with set-restricted controls. + + Python's architecture validates the set-membership column once + globally and stores ``set_ids_arr`` (aligned with ``all_groups``); + the ``set_ids`` parameter is threaded through the four per-path IF + helpers (``_compute_path_effects``, + ``_compute_path_placebos``, ``_collect_path_bootstrap_inputs``, + ``_collect_path_placebo_bootstrap_inputs``) so per-path analytical + SE, bootstrap, placebos, and sup-t bands all consume the + set-restricted control pool. Per-path point estimates and placebos + match R bit-exactly under the cohort-single-path + ``multi_path_reversible`` DGP. Per-path SE inherits the documented + cross-path cohort-sharing deviation (Phase 2 envelope, ~13% rtol + on this scenario). + """ + + POINT_RTOL = 1e-9 + SE_RTOL = 0.15 + + def _path_key_from_r_label(self, r_label: str): + return tuple(int(x) for x in r_label.split(",")) + + def test_parity_multi_path_reversible_by_path_trends_nonparam( + self, golden_values + ): + """3-path case with state-set trends: by_path=3, trends_nonparam='state'.""" + import math + import warnings + + scenario = golden_values.get( + "multi_path_reversible_by_path_trends_nonparam" + ) + if scenario is None: + pytest.skip( + "scenario 'multi_path_reversible_by_path_trends_nonparam' " + "not in golden values" + ) + + df = _golden_to_df_with_extra(scenario["data"], extra_cols=["state"]) + est = ChaisemartinDHaultfoeuille( + drop_larger_lower=False, by_path=3, placebo=True + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + results = est.fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + trends_nonparam="state", + L_max=3, + ) + + r_by_path = scenario["results"]["by_path"] + assert results.path_effects is not None + + py_keys = set(results.path_effects.keys()) + r_keys = {self._path_key_from_r_label(e["path"]) for e in r_by_path} + assert py_keys == r_keys, ( + f"Path-set mismatch.\n" + f" Python only: {py_keys - r_keys}\n" + f" R only: {r_keys - py_keys}" + ) + + for r_path_entry in r_by_path: + path_key = self._path_key_from_r_label(r_path_entry["path"]) + py_path = results.path_effects[path_key] + py_placebos = ( + results.path_placebo_event_study.get(path_key, {}) + if results.path_placebo_event_study is not None + else {} + ) + + assert py_path["frequency_rank"] == r_path_entry["frequency_rank"], ( + f"path={path_key}: frequency_rank mismatch " + f"py={py_path['frequency_rank']} vs r={r_path_entry['frequency_rank']}" + ) + + for h_str, r_h in r_path_entry["horizons"].items(): + h = int(h_str) + if h > 0: + assert h in py_path["horizons"], ( + f"path={path_key}: horizon {h} missing from " + f"Python path_effects" + ) + py_h = py_path["horizons"][h] + else: + assert h in py_placebos, ( + f"path={path_key}: placebo {h} missing from " + f"Python path_placebo_event_study" + ) + py_h = py_placebos[h] + + assert py_h["n_obs"] == int(r_h["n_switchers"]), ( + f"path={path_key} h={h}: switcher-count mismatch " + f"py={py_h['n_obs']} vs r={int(r_h['n_switchers'])}" + ) + + assert py_h["effect"] == pytest.approx( + r_h["effect"], rel=self.POINT_RTOL + ), ( + f"path={path_key} h={h}: " + f"py={py_h['effect']:.6f} vs r={r_h['effect']:.6f}" + ) + + py_se = py_h["se"] + r_se = r_h["se"] + py_finite_positive = math.isfinite(py_se) and py_se > 0.0 + r_finite_positive = math.isfinite(r_se) and r_se > 0.0 + assert py_finite_positive == r_finite_positive, ( + f"path={path_key} h={h} SE state mismatch " + f"(py_se={py_se}, r_se={r_se})" + ) + if py_finite_positive and r_finite_positive: + assert py_se == pytest.approx(r_se, rel=self.SE_RTOL), ( + f"path={path_key} h={h} SE: " + f"py={py_se:.4f} vs r={r_se:.4f}" + )