Source document

docs/cities/miami/probe/README.md

Served verbatim from the project repository. Internal working document conventions apply: documents may reference file paths, branch names, and findings-ledger anchors from the repo.

probe/ — measurement scripts behind findings.md

Small, read-only Python scripts that produced the numbers in ../findings.md. They are the Miami-Dade analogue of the first-fetch batches recorded in docs/cities/philly/findings.md, kept as code so every figure in the plan is reproducible (house rule: derive quantities from the process, don't quote them).

They run against a scratch directory (default data/cities/miami/sources/_probe/, git-ignored; override with MIAMI_PROBE_DIR) and need only pandas, openpyxl, requests, scipy (pipeline_philly/requirements.txt + scipy).

ScriptFetchesProduces
ccd_probe.pyNCES CCD 2023-24 directory (029), staff (059), lunch (033) zips (download them first, see below)universe counts by status / type / charter / level; ID-scheme checks
ccd_membership_year.py <url> <out.csv>one CCD school-membership zip (052, ~200 MB), streamedthe Miami-Dade (ST_LEAID FL-13) rows only
ccd_small.py— (uses the two above)size buckets by sector × level; the under-300 list; race composition; grade totals; mdc_directory_enrollment_2324.csv
ccd_trend.pyCCD 2018-19 directory zipfive-year school-level change, by sector / level / grade; mdc_district_regular_trend_1819_2324.csv
gis_util.py— (uses gis/layer_0.json)utilization distribution from the county GIS layer; capacity-definition and enrollment-vintage tests
closure_overlap.py—lift of the district's 2026 closure / review lists over the CCD size tail (closure_overlap.json)

Run order, from the repo root:

export MIAMI_PROBE_DIR=data/cities/miami/sources/_probe
mkdir -p $MIAMI_PROBE_DIR/ccd $MIAMI_PROBE_DIR/gis
cd $MIAMI_PROBE_DIR/ccd
for f in ccd_sch_029_2324_w_1a_073124 ccd_sch_059_2324_l_1a_073124 ccd_sch_033_2324_l_1a_073124 ccd_sch_029_1819_w_1a_091019; do
  curl -sS -o $f.zip https://nces.ed.gov/ccd/Data/zip/$f.zip && unzip -o -q $f.zip -d unz_$f
done
mv unz_ccd_sch_029_1819_w_1a_091019 unz_1819_dir
cd -
curl -sS "https://services.arcgis.com/8Pc9XBTAsYuxx9Ny/arcgis/rest/services/SchoolSite_gdb/FeatureServer/0/query?where=1%3D1&outFields=*&returnGeometry=true&outSR=4326&f=json&resultRecordCount=2000" -o $MIAMI_PROBE_DIR/gis/layer_0.json
python docs/cities/miami/probe/ccd_probe.py
python docs/cities/miami/probe/ccd_membership_year.py https://nces.ed.gov/ccd/Data/zip/ccd_sch_052_2324_l_1a_073124.zip ccd/mdc_membership_2324.csv
python docs/cities/miami/probe/ccd_membership_year.py https://nces.ed.gov/ccd/Data/zip/ccd_sch_052_1819_l_1a_091019.zip ccd/mdc_membership_1819.csv
python docs/cities/miami/probe/ccd_small.py
python docs/cities/miami/probe/ccd_trend.py
python docs/cities/miami/probe/gis_util.py
python docs/cities/miami/probe/closure_overlap.py

The sha256 / bytes / retrieved-at of every file fetched on 2026-09-15 are in data/cities/miami/sources/<source_id>/2026-09-15/provenance.json; the derived per-school tables are committed under data/cities/miami/reference/ as the audit trail (same convention as Philly's reference CSVs).

These scripts are throwaway probes, not loaders. The real P1 loaders will live under scripts/loaders/miami/ and write to miami_* tables, per 03_integration_plan.md.