Chalkbeat deep dives — analysis + views plan

Status: working plan (2026-07-08). Scope agreed with the Chalkbeat team after the checkpoint meeting. These three deep dives extend the descriptives packet (/bakeoff/02_descriptives_packet.html, memo ../01_descriptives.md). Nothing here is published yet.

Goal

Three requested deeper views on the descriptives packet:

1. Grade-stage differences & the PK/K distortion — PK/K vs 1–8 vs 9–12, and how much including PK (a non-mandatory grade) inflates a school's / district's / city's headline rate. Extends Figure 2a.
2. Distributional shift & the >50% chronic tail, split ES vs HS — how many schools sit above 50% chronic absenteeism and how the whole distribution moved. Extends Figure 6.
3. Descriptives by accountability unit (district + borough) — the full descriptive battery per unit, plus an explicit view of where trends are shared vs. unit-specific.

Decisions locked (checkpoint follow-up)

• Deliverable: three dedicated static HTML packets (one per deep dive), linked from a short index — matching the existing 02_descriptives_packet.html convention. Not one combined packet; not interactive app pages (yet).
• Grade cut (DD1): groups are PK/K vs 1–8 vs 9–12. Headline distortion metric is the aggregate rate recomputed ex-PK, then ex-PK/K, reported as the delta vs the all-grades rate.
• Units (DD3): 32 community-school districts + 5 boroughs. School-type / admission / charter splits deferred to a possible later pass.

Constraints & how this fits the existing workflow

• Compute substrate. All descriptive numbers are computed against a local Postgres replica (edu_data_local). This replica can now be stood up inside the web/sandbox environment via scripts/analysis/bootstrap_replica.sh: start Postgres 16 (preinstalled) → prisma db push → run the schools / metric-definitions / demographics / attendance loaders. It loads the absenteeism subset (schools + per-grade + subgroup CA/ADA) and reproduces the published descriptives to the digit (citywide Table 1, per-grade Table 4, the >50% tail, and the subgroup rows all match 01_descriptives.md exactly). Panel CSVs and source bytes stay gitignored/local; the replica is rebuilt from source, not committed.
  • One sandbox caveat: the NYCENET LCGMS live-directory enrichment (an ASP.NET postback export) is unreachable through the proxy, so the bootstrap runs ALLOW_MISSING_LCGMS=1 and loads the wg9x (2019-20) school base only. In practice this loses nothing for absenteeism — school/student counts match the memo exactly across all seven years (e.g. 1,450 schools / 836,268 students in 2024-25) — but post-2019-20 openings could in principle be absent, so it's flagged.
• Source of truth = committed numbers JSON. Each deep dive writes a committed data/analysis/*.json (like absenteeism-descriptives.json), which the render scripts read. Raw source bytes and the replica stay out of git.
• Views = static HTML. Render scripts emit HTML to a local working dir; a public-safe copy (localhost debug links stripped, noindex forced) is written to public/bakeoff/ by the existing export_public_views.py pattern, and served via Vercel.
• No new prod dependency. These are analysis artifacts, not app features. The app build is untouched except for the static files under public/bakeoff/.

Universe & conventions (inherited from 01_descriptives.md)

• Universe: NYC district schools (include_in_default_comparisons = true; no charters, D75, D79, alt), subgroup ALL, non-suppressed, value + denominator non-null.
• Weighted = student-weighted by CA denominator (headline); unweighted school mean + percentiles reported as companions.
• 2019-20 and 2020-21 excluded from trend interpretation (COVID-truncated / remote attendance). Shown for completeness only.
• Suppression stays visible: report cell/coverage counts, never silently drop.
• NYSED asterisk carries forward: the 2024-25 elementary/middle (grades 1–8) DOE-vs-NYSED divergence flagged in 01_descriptives.md applies to DD1's "1–8" group and DD3's EM rows. HS agreement is excellent, so 9–12 findings are unaffected. Keep the asterisk on any "continued 2024-25 improvement" claim for grades 1–8.

Deep dive 1 — Grade-stage decomposition & the PK/K distortion

Question. PK CA is far above the school average (48.2% vs 32.3% citywide in 2024-25) and PK is not mandatory. How much does including PK — and K — inflate the headline rate, and for which schools/districts does it matter most?

Why it's not just citywide. Citywide the effect is modest: dropping PK moves the weighted rate ~0.6pp (32.3 → ~31.6), dropping PK+K ~1.1pp (→ ~31.1), because PK is only ~3.8% of students. The story is the school-level heterogeneity — a school with a large PK/K program and high early-grade absenteeism can move several points. So the headline is the distribution of the per-school distortion, with the citywide/district numbers as context.

Reporting-convention framing (decided; computed in compute_dd1_grade.py). The three grade cuts each map to a real publisher, so we present them as labeled conventions rather than an invented adjustment (established in verify/METHODOLOGY.md + NYSED/ESSA docs; confirmed on the replica):

• PK-12 — the DOE InfoHub attendance bulk file "All Grades", PK-inclusive. This is what our current descriptives headline uses (confirmed empirically: the authoritative all-grades denominator equals the sum of PK-12 grade cells, not K-12 or 1-12).
• K-8 — the NYC School Quality Snapshot / SQR basis (K-inclusive, PK-excluded on the EMS report; 9-12 on the HS report).
• 1-8 — the NYSED / ESSA accountability basis (grades 1-8 and 9-12; PK and K excluded).

So the bulk-file headline is the only one of the three that carries PK. We decompose the distortion into a PK-step and a separate K-step rather than forcing K into one bucket: K is non-mandatory by NY statute (compulsory attendance starts at grade 1) yet empirically closer to grade 1 than to PK (2024-25 weighted: PK 48.2, K 37.8, grade 1 33.2). The K boundary is material — removing K moves 459 of 782 elementary/K8 schools by >1pp and 76 by >3pp, and the top movers are schools with kindergarten CA of 55-90% (e.g. 02M001 P.S. 001: K-8 57.1% → 1-8 50.0%). The per-school full PK/K removal has median +2.7pp, up to +14.8pp (100 schools >5pp). The distortion is also concentrated in high-CA, high-poverty districts (D5 −2.7pp vs D2 −0.1pp), so including PK/K widens apparent between-district gaps.

New computation.

• School × grade CA cells + denominators already exist in the replica (they back per_grade).
• Per unit (school, district, city) compute three student-weighted rates: R_all (PK–12), R_exPK (K–12), R_exPKK (1–12). Distortion metrics: dPK = R_all − R_exPK, dPKK = R_all − R_exPKK (pp).
• Regroup grades into PK/K, 1–8, 9–12; weighted CA per group per year + recovery vs 2018-19 and vs 2021-22 peak.
• Rank schools by dPKK; correlate with PK/K enrollment share (expect ES/K8 schools with big early-grade programs to move most).
• District-level: R_all vs R_exPKK, ranked, showing any reordering of the district table.
• Validity crosscheck (must pass before publishing): does Σ(grade-cell denominators) ≈ school-wide CA denominator? The ex-PK reweighting is only clean if grade cells partition the school-wide population; report the reconciliation gap and any suppression-driven shortfall (PK is the smallest, most-suppressed cell — 632 of ~1,450 schools have a PK cell).

Figures (packet 05_grade_pk_distortion.html).

• F1 — Grade-group trend: weighted CA for PK/K, 1–8, 9–12 by year (the reframed 2a).
• F2 — Citywide headline under 3 inclusion rules, by year (shows the ~0.6/1.1pp citywide gap).
• F3 — Per-school distortion dPKK distribution, 2024-25, + scatter vs PK/K enrollment share.
• F4 — District rate all-grades vs ex-PK/K, ranked dot plot, highlighting reordering.
• Tables — top-N most-distorted schools; district distortion table; grade-group recovery.

Deep dive 2 — Distributional shift & the >50% chronic tail (ES vs HS)

Question. How many schools sit above 50% chronic absenteeism, how did the whole distribution move, and does the tail look different for high schools vs elementary?

What we have. distribution.histograms + distribution.n_above_50_by_year — citywide only. The tail shape is already stark: 137 schools >50% in 2018-19 (9.3%) → 520 at the 2021-22 peak (35.5%) → 231 in 2024-25 (15.9%), still ~1.7× the pre-pandemic count.

New computation.

• Recompute school-level CA histograms per year split by band (ES, HS; MS and K8 for completeness).
• >50% count and share by band and by accountability unit (feeds DD3), per year.
• Distribution stats per band (median, p10/p90, SD, skew) — quantify the HS right-shift (2024-25 unweighted mean HS 39.5 vs ES 34.3).
• Named lists: schools >50% in 2024-25 by band; schools that crossed above / dropped below 50% since 2018-19.

Figures (packet 06_distribution_tail.html).

• F1 — Distribution overlay 2018-19 vs 2024-25, faceted ES / HS (+ MS/K8).
• F2 — >50% count (and share) trend by band.
• F3 — >50% share by district — bridges to DD3.
• Tables — >50% counts by band × year; named >50% list (ranked by CA).

Deep dive 3 — Descriptives by accountability unit (district + borough)

Question. Give a per-district and per-borough view of all the descriptives (incl. DD1/DD2), and show where trends are the same across units vs. where they diverge.

New computation.

• Per unit (32 districts + 5 boroughs), compute the battery: weighted CA trend; grade-group split + PK/K distortion (from DD1); distribution + >50% count (from DD2); subgroup gaps (poverty, STH, SWD, ELL); YoY stability (consecutive r, size-volatility).
• Cross-unit comparison — the "same vs different" view:
  • Between-unit variance share (η²) per descriptive → how much of the variation is unit-level vs within-unit (parallels the 23.3% between-district figure already computed for the level).
  • Direction agreement: does every unit show the same sign of change 2021-22→2024-25 for each descriptive (recovery, tail, poverty gap, PK distortion)?
  • Classify each descriptive as shared (homogeneous across units) vs local (unit-specific), with the units that are outliers on each dimension named.

Figures (packet 07_accountability_units.html).

• F1 — Small-multiples trend grid: one sparkline per district, boroughs highlighted.
• F2 — Ranked dot plot of 2024-25 weighted CA by district (with the DD1 ex-PK/K adjustment), boroughs as reference bands.
• F3 — Unit × descriptive heatmap (recovery, tail share, poverty gap, PK distortion), z-scored, to surface which units are outliers on which dimension.
• F4 — "Agreement" summary: between-unit η² per descriptive (shared vs local classification).
• Per-unit mini-cards — one compact block per unit with its key numbers.

Caveat baked in: small units are noisy (e.g., D16 ≈ 5,526 students vs D31 ≈ 58,697); flag low-N units and consider enrollment-weighting the cross-unit comparisons.

Deliverables & file layout

docs/analysis/absenteeism/deep-dives/
  00_PLAN.md                     ← this file
  05_grade_pk_distortion.md      ← working memo (headlines/findings/tables/caveats, 01_ style)
  06_distribution_tail.md
  07_accountability_units.md
  compute_dd1_grade.py           ← replica → data/analysis/dd1_grade_distortion.json
  compute_dd2_distribution.py    ← replica → data/analysis/dd2_distribution_tail.json
  compute_dd3_units.py           ← replica → data/analysis/dd3_accountability_units.json
  render_dd1.py / render_dd2.py / render_dd3.py  ← JSON → HTML

data/analysis/                   ← committed numbers JSON (source of truth for the packets)
  dd1_grade_distortion.json
  dd2_distribution_tail.json
  dd3_accountability_units.json

public/bakeoff/                  ← public-safe static packets (via export_public_views.py)
  05_grade_pk_distortion.html
  06_distribution_tail.html
  07_accountability_units.html

Packet numbering continues the public series (02 descriptives, 04 bakeoff results → 05/06/07).

Sequencing

• Phase A — DD1. Grade cells + distortion. Foundational: DD3 reuses the ex-PK/K rates.
• Phase B — DD2. Band-split distribution + tail + >50% counts. DD3 reuses the per-unit tail.
• Phase C — DD3. Per-unit battery + cross-unit comparison; consumes A and B.

Each phase: compute JSON → write memo → render packet → run crosschecks → public-safe copy.

Crosscheck gates (per phase, before any packet ships).

• DD1 grade-group weighted rates reconcile to the existing per_grade numbers; Σ grade-cell denominators reconcile to school-wide denominators (report the gap).
• DD2 >50% band counts sum to the citywide n_above_50_by_year already in the JSON.
• DD3 unit-level weighted CA reconciles to citywide when pooled; district table matches the existing Table 5 in 01_descriptives.md.

Open questions / risks

• PK/K cell suppression (smallest cells) can bias the grade-group aggregates and the distortion estimate; DD1 must report coverage and a suppression-sensitivity note.
• Enrollment-weighting the cross-unit comparison (DD3): unweighted treats a 5k-student district equal to a 58k one. Propose reporting both; decide the headline before rendering.
• NYSED 2024-25 EM asterisk applies to grades 1–8 (DD1 "1–8" group; DD3 EM rows) — keep the publisher caveat on 2024-25 improvement claims there.
• Charter reference series (DD2/DD3): out of the district universe and on a different (NYSED) measuring stick; excluded from the headline but could appear as a flagged reference, matching Figure 5's treatment — decide per packet.