Find What Works: Schools
← All source documentsRaw file
Source document

docs/qa_reports/snapshot/README.md

Served verbatim from the project repository. Internal working document conventions apply: documents may reference file paths, branch names, and findings-ledger anchors from the repo.

NYC School Quality Snapshot — chronic-absenteeism census & charter provenance

The full-scale (iii) presentation cross-reference for NYC chronic absenteeism: the served values vs the NYC School Quality Snapshot (tools.nycenet.edu), exhaustively, for every comparable school-year — replacing the earlier 300-school sample (docs/qa_reports/chronic_absenteeism_rate__snapshot.{md,json}, kept for the record). Plus a separate provenance study of the Snapshot's charter values, which our site cannot compare like-for-like (charters are served from NYSED).

Contents:

  • CENSUS.md — the district census (6,570 school-year pairs, SY 2018-19 → 2022-23) and the basis findings: the Snapshot displays the same-year grade-band rate (grades K–8, kindergarten-inclusive, on EMS reports; grades 9–12 on HS reports), not the school-wide figure; and the 2024/2025 report-years' renamed feed varnames store the complement (round(100 − band rate)). Hand-verified end-to-end cases in §6; the charter census (descriptive only) in §7.
  • census-stats.json — machine-readable panels (bucket shape compatible with docs/qa_reports/nysed/stats.json), candidate-interpretation tables, worst-case lists, verdicts.
  • PROVENANCE.md — are the Snapshot's charter chronic-absenteeism values an independent series? Verdict: computed, not copied from NYSED's published rates, but reporting outages co-move with the state chain 82/82 — partially independent as a cross-check.
  • provenance-stats.json — all numbers behind PROVENANCE.md.

Method & lineage: the Snapshot side of both studies is data/quality/snapshot-values.json — a committed (~4 MB) bulk fetch of the Snapshot data feed for every school × report years 2019–2025 × report type (band-spanning schools have both an EMS and an HS page; both are fetched), produced by scripts/quality/fetch_snapshot_values.py against the tools.nycenet.edu API via the verify/ connector (raw per-DBN API responses cached under data/reference/snapshot/{EMS,HS}/<reportYear>/, gitignored). The artifact is committed because it is the census's source of record and the verification card reads it directly. The served side is the warehouse (SELECT-only); the NYSED side of PROVENANCE.md is the committed data/sources/nysed/SRC*_CA_nyc.csv extracts. The census/provenance computation scripts (run_census.py, augment_census.py, build_provenance.py, referenced from the reports) live in the analysis workspace, not this repo; the reports were written there and copied verbatim.

Status: one-off studies ahead of harness integration. Follow-up (not done here): verify/snapshot.yaml should re-point the chronic-absenteeism check at the K–8 / 9–12 band aggregates (and decode the 2024/2025 complement varnames) so the recurring (iii) check runs on the established basis.

Headline: ledger A.6 largely explained — school-wide agreement within ±1.5pp is 0.6534 (6,570 pairs; the sample's "~67%" confirmed), rising to 0.8639 pooled / 0.9223 EMS-report (≥0.94 for EMS report-years 2020–2023) on the matching grade band; the entire worst-15 list, including 28Q157 (−26.84pp school-wide = its 35-student grade-9 rate exactly), is band-explained. Residual NOT established: HS reports in SY 2019-20/2021-22 (~0.49, one-sided +1.5pp median shift) and SY 2018-19 (0.80–0.83 both report types). A.7 decoded — val_chronic_absent_{ems,hs}_all = round(100 − same-year K–8 / 9–12 rate), 0.97–0.98 agreement (raw-value candidates max out at 0.042); page-label semantics still unestablished. A.19 settled — charter values are computed (17.5–19.8% rounding-match vs ~100% expected for republication; values exist where NYSED suppresses; K-inclusive enrollment verified at scale), track NYSED only loosely (median |diff| 1.79pp, r 0.73–0.83 vs district 0.99 — why not established), and 82/82 DATA_REP_FLAG=N charter school-years have no Snapshot CA value: partially independent — catches gross serving errors, cannot catch state-chain reporting failures, cannot arbitrate <2pp.

Addendum (2026-06-10): census-stats.json carries two computed sections — band_basis_panels (the established band-basis comparison as signed distribution buckets + worst10 lists, pooled / per-year / per report type) and decoded_2024_2025_panels (same shape for the decoded 2024/2025 complement varnames). First built by augment_band_distributions.py on the one-value-per-school-year artifact (CENSUS.md §9: 8 of the 10 pooled extremes were dual-cached schools whose feed value matched their 9–12 rate — tie-break artifacts), then re-censused type-matched by recensus_both_types.py after the both-report-types fetch for the 201 band-spanning schools (CENSUS.md §10): every report-type value is compared against its own band, spanning schools contribute both pairs. Pooled band agreement 0.8639 → 0.8695 (n 6,964), EMS 0.9261, decode 0.97–0.98; the tie-break artifacts are gone from the worst lists, and the residual pockets (HS reports SY 2019-20/2021-22 ≈0.48–0.50, SY 2018-19 0.80–0.83) persist under type-matching — real, still unexplained. These panels drive the band-basis strips and the extreme-cases list on the metric page's open issue (3).