# probe/ — measurement scripts behind the NYC design docs

Read-only Python scripts that produce the numbers quoted in the numbered design
docs, kept as code so every figure is reproducible (house rule: derive
quantities from the process, don't quote them). The Miami analogue is
`docs/cities/miami/probe/`.

They fetch into a scratch directory — default
`data/reference/sources/_probe/`, git-ignored; override with `NYC_PROBE_DIR` —
and cache every download, so a rerun is offline and the cache is the audit
trail. Both also read the committed parse of the SCA's PDFs
(`data/cities/nyc/reference/parsed/`) and refuse a year whose two-parser gate
did not pass.

| Script | Reads | Produces |
|---|---|---|
| `bluebook_util.py` | SCA Blue Book (`gkd7-3vk7`, `q9xk-w9iv`) + the five PDF-parsed years + the 2019-20 school directory (`wg9x-4ke6`) + three DOE Demographic Snapshot vintages (`s52a-8aq6`, `c7ru-d68s`, the current InfoHub xlsx) | §1–§4 of [`10_nyc_enrollment_capacity_utilization.md`](../10_nyc_enrollment_capacity_utilization.md): panel shape, snapshot → school-year date-stamping, join rate and its failure modes, computed-value check, utilization distributions at both grains, enrollment-vs-capacity decomposition, the two capacity definitions, district breakdown, and the disappearance lift table |
| `bluebook_verify.py` | the same Blue Book datasets + NCES CCD (via the Urban Institute API) + the DOE Building Space Usage room inventory (`wavz-fkw8`) | the cross-source verification in [`10`](../10_nyc_enrollment_capacity_utilization.md) §5.1–§5.3: enrollment against an independent publisher with the DOE's own file as a control, capacity against the physical room inventory (a room set fixed before any comparison; square feet per seat and seats per room are measured, not assumed), and the retired extract against the live dataset |

What neither script produces, and `10` says so where each appears: the §2.1
parse figures (in the gate report beside each parsed CSV), the §4.3 segment
table (the packet's `decomposition.segments`), the §4.2 Miami-Dade and
Philadelphia figures (those cities' packets), the §4.2 charter and District 75
counts, and the Pre-K measurements in §5.2 and §5.4.

Requires `requests`, `openpyxl`, `scipy`.

```bash
python docs/design/probe/bluebook_util.py     # what the SCA publishes
python docs/design/probe/bluebook_verify.py   # is it right
```

Roughly 30 MB cold for `bluebook_util.py` and another ~250 MB for the room
inventory in `bluebook_verify.py`; both are seconds warm. Every section
prints its own method line, so the output is readable without the doc beside it.