# Chalkbeat deep dives — analysis + views plan

> **Status: working plan** (2026-07-08). Scope agreed with the Chalkbeat team after the
> checkpoint meeting. These three deep dives extend the descriptives packet
> ([`/bakeoff/02_descriptives_packet.html`](../../../../public/bakeoff/02_descriptives_packet.html),
> memo [`../01_descriptives.md`](../01_descriptives.md)). Nothing here is published yet.

## Goal

Three requested deeper views on the descriptives packet:

1. **Grade-stage differences & the PK/K distortion** — PK/K vs 1–8 vs 9–12, and how much
   including PK (a non-mandatory grade) inflates a school's / district's / city's headline
   rate. Extends Figure 2a.
2. **Distributional shift & the >50% chronic tail**, split ES vs HS — how many schools sit
   above 50% chronic absenteeism and how the whole distribution moved. Extends Figure 6.
3. **Descriptives by accountability unit** (district + borough) — the full descriptive
   battery per unit, plus an explicit view of where trends are shared vs. unit-specific.

## Decisions locked (checkpoint follow-up)

- **Deliverable:** three dedicated static HTML packets (one per deep dive), linked from a
  short index — matching the existing `02_descriptives_packet.html` convention. Not one
  combined packet; not interactive app pages (yet).
- **Grade cut (DD1):** groups are **PK/K vs 1–8 vs 9–12**. Headline distortion metric is the
  aggregate rate recomputed **ex-PK**, then **ex-PK/K**, reported as the delta vs the
  all-grades rate.
- **Units (DD3):** **32 community-school districts + 5 boroughs.** School-type / admission /
  charter splits deferred to a possible later pass.

## Constraints & how this fits the existing workflow

- **Compute substrate.** All descriptive numbers are computed against a local Postgres replica
  (`edu_data_local`). This replica can now be **stood up inside the web/sandbox environment**
  via [`scripts/analysis/bootstrap_replica.sh`](../../../../scripts/analysis/bootstrap_replica.sh):
  start Postgres 16 (preinstalled) → `prisma db push` → run the schools / metric-definitions /
  demographics / attendance loaders. It loads the absenteeism subset (schools + per-grade +
  subgroup CA/ADA) and **reproduces the published descriptives to the digit** (citywide Table 1,
  per-grade Table 4, the >50% tail, and the subgroup rows all match `01_descriptives.md`
  exactly). Panel CSVs and source bytes stay gitignored/local; the replica is rebuilt from
  source, not committed.
  - **One sandbox caveat:** the NYCENET **LCGMS** live-directory enrichment (an ASP.NET postback
    export) is unreachable through the proxy, so the bootstrap runs `ALLOW_MISSING_LCGMS=1` and
    loads the wg9x (2019-20) school base only. In practice this loses nothing for absenteeism —
    school/student counts match the memo exactly across all seven years (e.g. 1,450 schools /
    836,268 students in 2024-25) — but post-2019-20 openings could in principle be absent, so
    it's flagged.
- **Source of truth = committed numbers JSON.** Each deep dive writes a committed
  `data/analysis/*.json` (like `absenteeism-descriptives.json`), which the render scripts
  read. Raw source bytes and the replica stay out of git.
- **Views = static HTML.** Render scripts emit HTML to a local working dir; a public-safe
  copy (localhost debug links stripped, `noindex` forced) is written to `public/bakeoff/`
  by the existing `export_public_views.py` pattern, and served via Vercel.
- **No new prod dependency.** These are analysis artifacts, not app features. The app build
  is untouched except for the static files under `public/bakeoff/`.

## Universe & conventions (inherited from `01_descriptives.md`)

- Universe: NYC **district schools** (`include_in_default_comparisons = true`; no charters,
  D75, D79, alt), subgroup `ALL`, non-suppressed, value + denominator non-null.
- **Weighted** = student-weighted by CA denominator (headline); unweighted school mean +
  percentiles reported as companions.
- **2019-20 and 2020-21 excluded** from trend interpretation (COVID-truncated / remote
  attendance). Shown for completeness only.
- **Suppression stays visible**: report cell/coverage counts, never silently drop.
- **NYSED asterisk carries forward**: the 2024-25 elementary/middle (grades 1–8) DOE-vs-NYSED
  divergence flagged in `01_descriptives.md` applies to DD1's "1–8" group and DD3's EM rows.
  HS agreement is excellent, so 9–12 findings are unaffected. Keep the asterisk on any
  "continued 2024-25 improvement" claim for grades 1–8.

---

## Deep dive 1 — Grade-stage decomposition & the PK/K distortion

**Question.** PK CA is far above the school average (48.2% vs 32.3% citywide in 2024-25) and
PK is not mandatory. How much does including PK — and K — inflate the headline rate, and for
*which* schools/districts does it matter most?

**Why it's not just citywide.** Citywide the effect is modest: dropping PK moves the weighted
rate ~0.6pp (32.3 → ~31.6), dropping PK+K ~1.1pp (→ ~31.1), because PK is only ~3.8% of
students. **The story is the school-level heterogeneity** — a school with a large PK/K program
and high early-grade absenteeism can move several points. So the headline is the *distribution
of the per-school distortion*, with the citywide/district numbers as context.

**Reporting-convention framing (decided; computed in `compute_dd1_grade.py`).** The three grade
cuts each map to a real publisher, so we present them as labeled conventions rather than an
invented adjustment (established in `verify/METHODOLOGY.md` + NYSED/ESSA docs; confirmed on the
replica):
- **PK-12** — the DOE InfoHub attendance bulk file "All Grades", **PK-inclusive**. This is what
  our current descriptives headline uses (confirmed empirically: the authoritative all-grades
  denominator equals the sum of PK-12 grade cells, not K-12 or 1-12).
- **K-8** — the **NYC School Quality Snapshot / SQR** basis (K-inclusive, PK-excluded on the
  EMS report; 9-12 on the HS report).
- **1-8** — the **NYSED / ESSA accountability** basis (grades 1-8 and 9-12; PK *and* K excluded).

So the bulk-file headline is the only one of the three that carries PK. We **decompose the
distortion into a PK-step and a separate K-step** rather than forcing K into one bucket: K is
non-mandatory by NY statute (compulsory attendance starts at grade 1) yet empirically closer to
grade 1 than to PK (2024-25 weighted: PK 48.2, K 37.8, grade 1 33.2). The K boundary is
material — removing K moves 459 of 782 elementary/K8 schools by >1pp and 76 by >3pp, and the
top movers are schools with kindergarten CA of 55-90% (e.g. 02M001 P.S. 001: K-8 57.1% →
1-8 50.0%). The per-school full PK/K removal has median +2.7pp, up to +14.8pp (100 schools >5pp).
The distortion is also concentrated in high-CA, high-poverty districts (D5 −2.7pp vs D2 −0.1pp),
so including PK/K widens apparent between-district gaps.

**New computation.**
- School × grade CA cells + denominators already exist in the replica (they back `per_grade`).
- Per unit (school, district, city) compute three student-weighted rates:
  `R_all` (PK–12), `R_exPK` (K–12), `R_exPKK` (1–12). Distortion metrics:
  `dPK = R_all − R_exPK`, `dPKK = R_all − R_exPKK` (pp).
- Regroup grades into **PK/K, 1–8, 9–12**; weighted CA per group per year + recovery vs
  2018-19 and vs 2021-22 peak.
- Rank schools by `dPKK`; correlate with PK/K enrollment share (expect ES/K8 schools with big
  early-grade programs to move most).
- District-level: `R_all` vs `R_exPKK`, ranked, showing any reordering of the district table.
- **Validity crosscheck (must pass before publishing):** does Σ(grade-cell denominators) ≈
  school-wide CA denominator? The ex-PK reweighting is only clean if grade cells partition the
  school-wide population; report the reconciliation gap and any suppression-driven shortfall
  (PK is the smallest, most-suppressed cell — 632 of ~1,450 schools have a PK cell).

**Figures (packet `05_grade_pk_distortion.html`).**
- F1 — Grade-group trend: weighted CA for PK/K, 1–8, 9–12 by year (the reframed 2a).
- F2 — Citywide headline under 3 inclusion rules, by year (shows the ~0.6/1.1pp citywide gap).
- F3 — Per-school distortion `dPKK` distribution, 2024-25, + scatter vs PK/K enrollment share.
- F4 — District rate all-grades vs ex-PK/K, ranked dot plot, highlighting reordering.
- Tables — top-N most-distorted schools; district distortion table; grade-group recovery.

---

## Deep dive 2 — Distributional shift & the >50% chronic tail (ES vs HS)

**Question.** How many schools sit above 50% chronic absenteeism, how did the whole
distribution move, and does the tail look different for high schools vs elementary?

**What we have.** `distribution.histograms` + `distribution.n_above_50_by_year` — **citywide
only**. The tail shape is already stark: 137 schools >50% in 2018-19 (9.3%) → 520 at the
2021-22 peak (35.5%) → 231 in 2024-25 (15.9%), still ~1.7× the pre-pandemic count.

**New computation.**
- Recompute school-level CA histograms per year **split by band** (ES, HS; MS and K8 for
  completeness).
- `>50%` count and share **by band** and **by accountability unit** (feeds DD3), per year.
- Distribution stats per band (median, p10/p90, SD, skew) — quantify the HS right-shift
  (2024-25 unweighted mean HS 39.5 vs ES 34.3).
- Named lists: schools >50% in 2024-25 by band; schools that crossed above / dropped below 50%
  since 2018-19.

**Figures (packet `06_distribution_tail.html`).**
- F1 — Distribution overlay 2018-19 vs 2024-25, faceted ES / HS (+ MS/K8).
- F2 — `>50%` count (and share) trend by band.
- F3 — `>50%` share by district — bridges to DD3.
- Tables — >50% counts by band × year; named >50% list (ranked by CA).

---

## Deep dive 3 — Descriptives by accountability unit (district + borough)

**Question.** Give a per-district and per-borough view of all the descriptives (incl. DD1/DD2),
and show where trends are the *same* across units vs. where they *diverge*.

**New computation.**
- Per unit (32 districts + 5 boroughs), compute the battery: weighted CA trend; grade-group
  split + PK/K distortion (from DD1); distribution + >50% count (from DD2); subgroup gaps
  (poverty, STH, SWD, ELL); YoY stability (consecutive r, size-volatility).
- **Cross-unit comparison** — the "same vs different" view:
  - Between-unit variance share (η²) per descriptive → how much of the variation is unit-level
    vs within-unit (parallels the 23.3% between-district figure already computed for the level).
  - Direction agreement: does every unit show the same sign of change 2021-22→2024-25 for each
    descriptive (recovery, tail, poverty gap, PK distortion)?
  - Classify each descriptive as **shared** (homogeneous across units) vs **local**
    (unit-specific), with the units that are outliers on each dimension named.

**Figures (packet `07_accountability_units.html`).**
- F1 — Small-multiples trend grid: one sparkline per district, boroughs highlighted.
- F2 — Ranked dot plot of 2024-25 weighted CA by district (with the DD1 ex-PK/K adjustment),
  boroughs as reference bands.
- F3 — Unit × descriptive heatmap (recovery, tail share, poverty gap, PK distortion), z-scored,
  to surface which units are outliers on which dimension.
- F4 — "Agreement" summary: between-unit η² per descriptive (shared vs local classification).
- Per-unit mini-cards — one compact block per unit with its key numbers.

**Caveat baked in:** small units are noisy (e.g., D16 ≈ 5,526 students vs D31 ≈ 58,697); flag
low-N units and consider enrollment-weighting the cross-unit comparisons.

---

## Deliverables & file layout

```
docs/analysis/absenteeism/deep-dives/
  00_PLAN.md                     ← this file
  05_grade_pk_distortion.md      ← working memo (headlines/findings/tables/caveats, 01_ style)
  06_distribution_tail.md
  07_accountability_units.md
  compute_dd1_grade.py           ← replica → data/analysis/dd1_grade_distortion.json
  compute_dd2_distribution.py    ← replica → data/analysis/dd2_distribution_tail.json
  compute_dd3_units.py           ← replica → data/analysis/dd3_accountability_units.json
  render_dd1.py / render_dd2.py / render_dd3.py  ← JSON → HTML

data/analysis/                   ← committed numbers JSON (source of truth for the packets)
  dd1_grade_distortion.json
  dd2_distribution_tail.json
  dd3_accountability_units.json

public/bakeoff/                  ← public-safe static packets (via export_public_views.py)
  05_grade_pk_distortion.html
  06_distribution_tail.html
  07_accountability_units.html
```

Packet numbering continues the public series (02 descriptives, 04 bakeoff results → 05/06/07).

## Sequencing

- **Phase A — DD1.** Grade cells + distortion. Foundational: DD3 reuses the ex-PK/K rates.
- **Phase B — DD2.** Band-split distribution + tail + >50% counts. DD3 reuses the per-unit tail.
- **Phase C — DD3.** Per-unit battery + cross-unit comparison; consumes A and B.

Each phase: compute JSON → write memo → render packet → run crosschecks → public-safe copy.

**Crosscheck gates (per phase, before any packet ships).**
- DD1 grade-group weighted rates reconcile to the existing `per_grade` numbers; Σ grade-cell
  denominators reconcile to school-wide denominators (report the gap).
- DD2 `>50%` band counts sum to the citywide `n_above_50_by_year` already in the JSON.
- DD3 unit-level weighted CA reconciles to citywide when pooled; district table matches the
  existing Table 5 in `01_descriptives.md`.

## Open questions / risks

- **PK/K cell suppression** (smallest cells) can bias the grade-group aggregates and the
  distortion estimate; DD1 must report coverage and a suppression-sensitivity note.
- **Enrollment-weighting the cross-unit comparison** (DD3): unweighted treats a 5k-student
  district equal to a 58k one. Propose reporting both; decide the headline before rendering.
- **NYSED 2024-25 EM asterisk** applies to grades 1–8 (DD1 "1–8" group; DD3 EM rows) — keep the
  publisher caveat on 2024-25 improvement claims there.
- **Charter reference series** (DD2/DD3): out of the district universe and on a different
  (NYSED) measuring stick; excluded from the headline but could appear as a flagged reference,
  matching Figure 5's treatment — decide per packet.