# Findings from first fetch and first measurements (2026-09-15)

What the files and the environment actually look like, measured, before any
loader exists. Same role as `docs/cities/philly/findings.md`: every finding
either changes a planning doc or a `sources.yaml` entry. Scripts in
`probe/`; provenance (sha256, bytes, retrieved-at) for every file under
`data/cities/miami/sources/<source_id>/2026-09-15/provenance.json`; derived
per-school tables in `data/cities/miami/reference/`.

## E. Environment: what a cloud pipeline can and cannot reach

| Host | Result from this sandbox | Consequence |
|---|---|---|
| `www.fldoe.org` (pages and `/file/`, `/core/fileparse.php/` paths) | **403, Akamai "Access Denied"** on every request, any user agent | — |
| `origin.fldoe.org` (same paths) | **200**; served all 12 FLDOE files fetched here, byte-identical file types | `discover.py` for Miami rewrites `www.fldoe.org` → `origin.fldoe.org`; record the canonical `www` URL plus `fetched_from` in provenance |
| `edstats.fldoe.org` | proxy CONNECT rejected (host retired) | successor is `edudata.fldoe.org` (Tableau; interactive only) |
| `eds.fldoe.org` (MSID) | 200; `All_schools.cfm` needs a session cookie from `index.cfm` first | scripted download works |
| `drs.dadeschools.net`, `oada.dadeschools.net`, `oer.dadeschools.net` | proxy CONNECT rejected | the OADA monthly membership portal is unavailable from cloud; the same data reaches us via FLDOE annual files |
| `www.dadeschools.net`, `api.dadeschools.net`, `pdfs.dadeschools.net`, `drs2.dadeschools.net` | 200 | climate-survey workbooks, Statistical Highlights, board PDFs, budget all fetchable |
| `nces.ed.gov`, `civilrightsdata.ed.gov`, `eddataexpress.ed.gov` | 200 | CCD, CRDC, EDFacts fetchable |
| `www2.ed.gov` / `www.ed.gov` | 403 | CRDC state CSVs (2015-16 only) unreachable; national zips on `civilrightsdata.ed.gov` are fine |
| `gis-mdc.opendata.arcgis.com`, `services.arcgis.com` | 200 | REST + CSV export |
| Miami Herald, Axios | 403 | press figures only via syndicated copies |
| `web.archive.org` | resets / 429 | not a usable fallback |

Net: **everything the v1 build needs is fetchable from the cloud** via the
origin host; nothing has to be hand-carried from a laptop.

## M. Measurements (NCES CCD + county GIS, fetched 2026-09-15)

### M.1 Universe — CCD 2023-24 directory, LEAID `1200390` (`ST_LEAID = FL-13`)

| Cut | Count |
|---|---|
| Records | 556 (Open 530 · Closed 19 · Added 5 · New 1 · Future 1) |
| Open with a membership total | 536; total students **337,719** |
| — charter (`CHARTER_TEXT = Yes`) | 158 schools, 85,164 students (25.2%) |
| — non-charter | 378 schools, 252,555 students |
| — district-run *regular* (non-charter, `SCH_TYPE = Regular School`) | **337 schools, 249,922 students** |
| School type (all records) | Regular 501 · Alternative 42 · Special Education 8 · Career/Technical 5 |
| Level (open, non-charter / charter) | ES 219 / 73 · HS 86 / 46 · MS 47 / 30 · Other 18 / 7 · PK 4 / 0 |

Cross-checks: FLDOE School Grades 2025 has 470 graded Dade schools (142
charter); FLDOE Absent-21+ 2024-25 has 518 Dade school rows; M-DCPS
Statistical Highlights 2025-26 counts 503 schools (341 traditional + 162
charter); MSID lists 542 active district-13 schools. The universe is
**~530–560 records, ~500 "real" schools**, of which ~160 are charters —
about 1.7× Philly and 0.3× NYC.

**Identifier.** `ST_SCHID = "FL-13-NNNN"`: district 13 + a 4-digit school
number, the same number M-DCPS, FLDOE, the county GIS layer and CIMS use.
Charter numbers span 0070–7516 and overlap district-run numbers (0041–9733)
— **no numeric range identifies charters**; use the flags (MSID
`CHARTER_SCHL_STAT`, School Grades `Charter School`, CCD `CHARTER_TEXT`).

### M.2 Size distribution — district-run regular schools, Oct 2023

| Students | ES | HS | MS | Other/PK | All |
|---|---|---|---|---|---|
| 0 | 0 | 1 | 0 | 1 | 2 |
| 1–100 | 0 | 1 | 0 | 5 | 6 |
| 101–200 | 6 | 5 | 0 | 1 | 12 |
| 201–300 | 22 | 3 | 0 | 0 | 25 |
| 301–400 | 37 | 4 | 2 | 1 | 44 |
| 401–500 | 33 | 3 | 9 | 0 | 45 |
| 501–750 | 65 | 5 | 14 | 1 | 85 |
| 751–1000 | 32 | 3 | 15 | 1 | 51 |
| 1001–2000 | 20 | 23 | 7 | 0 | 50 |
| 2001+ | 1 | 16 | 0 | 0 | 17 |
| **Total** | 216 | 64 | 47 | 10 | 337 |

Elementary schools under 300: **27 of 216**; under 400: **65 of 216**.
Middle under 500: 11 of 47. High under 800: 21 of 64 (most are the
"School for Advanced Studies" dual-enrollment sites, 92–273 students,
not neighborhood schools). Median district-run school: 533 students;
median charter: 432.

### M.3 Five-year change — Oct 2018 → Oct 2023 (CCD)

| Universe | 2018-19 | 2023-24 | Change |
|---|---|---|---|
| All open Miami-Dade schools | 510 schools · 350,434 | 536 · 337,719 | −3.6% |
| Non-charter | 376 · 281,953 | 378 · 252,555 | **−10.4%** |
| Charter | 134 · 68,481 | 158 · 85,164 | **+24.4%** |
| Continuing district-run regular schools (n = 330) | 277,506 | 247,147 | **−10.9%** |
| Continuing charters (n = 120) | 64,920 | 72,352 | +11.4% |

Schools that left the roster: 25 (14 charter, 11 district; 5,414 students
in 2018-19). New: 51 (38 charter, 13 district; 15,425 students in 2023-24).

School-level change among the 330 continuing district-run regular schools:
median **−12.6%**; 10th percentile −31.7%, 25th −21.5%, 75th −0.7%.
191 schools (58%) lost ≥10%; 94 (28%) lost ≥20%; **69 (21%) lost ≥25%**;
25 (8%) lost ≥33%; 77 grew. By level: ES −11.9% (median school −13.4%),
MS −13.7% (−12.4%), HS −8.1% (−10.5%).

### M.4 Grade pipeline (all Miami-Dade schools incl. charters)

| Grade | 2018-19 | 2023-24 | Change |
|---|---|---|---|
| PK | 9,573 | 9,587 | +0.1% |
| K | 22,854 | 21,202 | **−7.2%** |
| 1 | 24,625 | 23,153 | −6.0% |
| 2 | 25,575 | 24,117 | −5.7% |
| 3 | 26,753 | 25,159 | −6.0% |
| 4 | 26,272 | 24,538 | −6.6% |
| 5 | 27,039 | 23,855 | **−11.8%** |
| 6 | 26,650 | 25,166 | −5.6% |
| 7 | 27,560 | 26,810 | −2.7% |
| 8 | 26,247 | 26,347 | +0.4% |
| 9 | 26,827 | 27,701 | +3.3% |
| 10 | 27,029 | 28,058 | +3.8% |
| 11 | 26,263 | 26,054 | −0.8% |
| 12 | 27,167 | 25,972 | −4.4% |

In 2023-24 the kindergarten cohort is 82% of the grade-12 cohort and 76% of
grade 10. The decline enters at the bottom; elementary schools take it
first, which is why the closure lists are elementary-heavy.

### M.5 Utilization — county GIS `Public School` layer (451 points, edits through 2025-11-13)

322 district-run E/M/S points carry both `CAPACITY` and `ENROLLMNT`; all
322 join to CCD on the 4-digit number; none are charters (charters are a
separate layer).

| Statistic | enrollment ÷ CAPACITY |
|---|---|
| median | 0.671 (mean 0.697) |
| 10th / 25th / 75th / 90th pct | 0.433 / 0.539 / 0.849 / 0.982 |
| below one-third | 13 schools (4.0%) |
| below one-half | **59 schools (18.3%)** |
| below two-thirds | 156 (48.4%) |
| below 80% (the Strategic Plan target) | 224 (69.6%) |
| at or above 100% | 26 (8.1%) |
| sum ÷ sum | 0.704 |

By level: E (216) median 0.675, 16% under half; M (49) median 0.601, 27%
under half; S (57) median 0.789, 19% under half. For comparison, Chalkbeat
reported 30% of Chicago schools at least half-empty and 47 (7%) under
one-third.

**What the two GIS fields are (measured, not assumed):**
- `CAPACITY` = **permanent FISH capacity**: for the nine schools closed for 2026-27 it equals the board's "Perm FISH Cap" exactly in 7 cases (Parkway 478, Lenora B. Smith 754, Wheatley 620, Moton 679, Pine Villa 834, Mandarin Lakes 1,508, Miami Springs MS 1,307); Richmond Heights carries the perm+temp figure (973 vs 914); Rainbow Park differs by vintage (560 vs 542).
- `ENROLLMNT` is **not** the Oct 2023 count (median ratio to CCD 0.988, r = 0.996, but only 26% of schools within ±2%) and sits between CCD 2023-24 and the board's 2025-26 figures (Parkway 216 → 177 → 154) — consistent with a **2024-25** count. Treat it as one labelled year, not "current".

### M.6 Does size predict the district's own lists? (independent reference)

Reference lists, built from the board's June 17, 2026 action (9 closures /
consolidations) and the Aug 25, 2026 letters (12 "reassessed" + 15 "under
review"), matched by name to CCD 2023-24 district-run regular ES + MS
schools with enrollment (n = 263). 9/9 closed and 26/27 review schools
matched (the one left out, "Center for International Education", is a high
school). Predictor: Oct 2023 enrollment, two years before the vote.

| Level | Cut | Below cut | Flagged | At/above cut | Flagged | Lift | Fisher p |
|---|---|---|---|---|---|---|---|
| ES | < 300 | 27 | 14 (52%) | 189 | 14 (7.4%) | **7.0×** | 9e-8 |
| ES | < 400 | 65 | 24 (37%) | 151 | 4 (2.6%) | **13.9×** | 7e-11 |
| MS | < 500 | 11 | 5 (45%) | 36 | 2 (5.6%) | 8.2× | 0.005 |
| MS | < 700 | 23 | 6 (26%) | 24 | 1 (4.2%) | 6.3× | 0.048 |

Base rate: 35 of 263 (13.3%). Five of the nine closed schools were in the
ES < 300 tail in Oct 2023. Caveats: "review" is not closure; name matching
is exact-substring on CCD names (list and matches in
`data/cities/miami/reference/closure_overlap_ccd2324_vs_2026_lists.json`);
utilization was not in this model — §4 of `06_…md` adds it.

### M.7 Composition and what it means for peer groups

All Miami-Dade schools, 2023-24: Hispanic **75.4%**, Black 17.0%, White
6.0%, Asian 1.0%, two-or-more 0.5%. NYC-style race-share covariates have
little variance on White and Asian here; the informative axes are Black
vs Hispanic share, English-learner share, economic disadvantage, and
school type/choice status. Peer-group derivation should be re-tuned on
Miami's distributions, not lifted with Philly's feature weights.

### M.8 Economic-disadvantage measure is direct certification

In CCD 2023-24, the "free lunch qualified" count equals the direct
certification count at sampled schools (e.g. Air Base K-8: 422 = 422 of
461). Florida reports NSLP eligibility from direct certification and CEP,
with a 1.6 multiplier at CEP schools in its own lunch-status files. FRL
shares are therefore not comparable across CEP and non-CEP schools or
across a school's CEP adoption year. Peer-group and residual models must
carry a CEP flag or use the FLDOE "% economically disadvantaged" field
consistently (decision M.8 in `00_overview.md`).

## F. File-shape findings from the FLDOE / M-DCPS first fetch

- **F.1 One school number, two spellings.** Zero-padded text (`0026`) in School Grades, assessment SRS, graduation, learning-gains, ESSA PPE and membership files; unpadded integer (`26`) in Absent-21+, ADA/ADM, lunch, SESIR, discipline, in-field/out-of-field files. Normalize to 4-digit text at the source boundary. Some accountability files add a `Virtual Provider Number` for the virtual programs (7001/7004/7006/7023); include it in the key.
- **F.2 Suppression tokens vary:** `*` (n < 10) in most files; `**` in the 2018-19 membership file; blank (not `*`) in School Grades components; `Less_10` flag in the climate workbooks; no suppression in SESIR, ADA/ADM, in-field/out-of-field; perturbation (noise), not suppression, in EDFacts. The loader censuses every non-numeric token per file (as the Philly full QA does).
- **F.3 Header rows and sheet order change by year.** Membership by school by grade: 2024-25 has sheets `State-District / Schl Total Enrollment / School` with the header on row 2 and zeros for grades not served; 2018-19 has `School Total Enrollment / School / District / State`, header on row 3, blanks for grades not served, `**` suppression. Absent-21+: header row 3. School Grades: header row 4. SRS: header row 5. Graduation: header row 6. Each year's file is a `source_id`; the loader takes the header row from `sources.yaml`.
- **F.4 The membership series is 2016-17 → 2025-26 for school × grade** (`1314MembBySchoolByGrade.xlsx` → 404); by race 2014-15 →; district totals 2013-14 →. CCD covers earlier years at the same grain.
- **F.5 Assessment SRS files carry no subgroups** (all students only; per-grade rows; `Number of Students, Mean Scale Score, % Level 3+, % in each level 1–5`; state row `00/0000`). Subgroup assessment results exist only in the Tableau report cards and SIP PDFs. Graduation files *do* carry race and category subgroups (7 races; ESE, FRL, ELL, migrant, at-risk, sex).
- **F.6 School Grades is the longitudinal school list back to 1999** (`Grade 2025 … Grade 1999` columns per school) and carries `Charter School`, `Title I`, `Alternative/ESE Center`, `School Type` (01 Elem / 02 Middle / 03 High / 04 Combination) and `Percent of Economically Disadvantaged Students`. Dade 2025: 470 graded schools — A 300, B 103, C 65, D 2.
- **F.7 Three utilization measures with three numerators.**
  - FLDOE **FISH Level of Service** report (`2425Dade.pdf`, 20 pp, report date 3/6/2025): per facility `School Capacity, Permanent Capacity, Dining Capacity, Year Round Capacity, Utilization Factor (1.00 elementary / 0.90 combination / 0.95 senior high), Percentage Utilization, Primary Use, Total COFTE, Available Capacity`. **Percentage Utilization = COFTE ÷ School Capacity** (ADA Merritt 766.5 / 707 = 108%; Aventura Waterways 1,858 / 1,485 = 125%); School Capacity includes relocatables (Air Base 1,268 vs permanent 1,170).
  - M-DCPS **ABC book** (`ABC_2026-27_final_reading.pdf`, 78 pp): per targeted school a "Capacity Membership Data" block — `Student Enrollment` (headcount), `Perm FISH Cap`, `% Utilization Perm FISH Cap`, `Perm & Temp FISH Cap`, `% Utilization Perm & Temp`, for 2025-26 and projected 2026-27 / 2027-28. Regex-parseable.
  - County **GIS** layer: enrollment (2024-25 vintage) ÷ permanent FISH capacity.
  The plan stores capacity variants and numerators separately (`06_…md §3`) and never mixes them in one figure.
- **F.8 ESSA per-pupil expenditure file** (`ESSA-2025-School-Per-Pupil-Expenditures.xlsx`, one sheet, header row 5): `District Code, District Name, School Code, School Name`, then `School Costs Per Pupil` (state+local / federal / total) and `District Indirect Costs Per Pupil` (same three) and `Total Costs`. Codes zero-padded. Prior years use the older `/core/fileparse.php/7507/urlt/` URL form.
- **F.9 Climate survey workbooks are long-format** (`2025_Students_Data`: `Key (LOC-Qnn), LOC, school, region, FTEFeb2025, TTL responses, Less_10, SSQNO, SSQLabel, Agree_PCT, DK_PCT, Disagree_PCT, NA_PCT`, plus respondent demographics and the school-grade and recommend items): 17,748 rows, 493 schools, ~32 items. Directly the shape of NYC's survey ingest. Parent and staff workbooks sit alongside; years 2018-19 → 2024-25 as xlsx.
- **F.10 SESIR is unsuppressed incident counts** (≈550 Dade schools × 27 incident types × law-enforcement / substance / bullying / weapon flags, 2005-06 →), while the **discipline-actions file is so heavily suppressed** (44,440 `*` cells statewide) that most Dade in-school-suspension rows are unusable. CRDC (single/multiple OSS, expulsions, arrests by race × sex, IDEA, 504, EL) is the subgroup-level discipline source.
- **F.11 Chronic absenteeism has two state definitions plus two federal ones.** FLDOE: "absent 21+ days" (2010-11 →) and "absent 10% or more of days enrolled, among students enrolled ≥ 10 days" (2017-18 →), both on an annual unduplicated enrollment (~6% larger than October membership; students enrolled at two schools count at each). EDFacts FS195 (EdDataExpress): ESSA 10% definition with subgroups, perturbed counts, 2020-21 → 2022-23. CRDC: absent 15+ days, subgroups, 2015-16 → 2023-24. Dade 2024-25: 17.3% absent 21+, 27.7% absent ≥10%; EDFacts 2022-23: 31.5%.
- **F.12 MSID is a snapshot, not a series** (77 fields incl. charter status and open year, magnet status, Title I, lat/lng, NCES id, `DATE_OPENED/CLOSED`, current school grade); no historical versions are published. Openings and closures over time come from School Grades' grade-history columns, CCD status fields, and the board record.
- **F.13 The 2023 science SRS files lead with a `Grade` column.** Every FAST, EOC and 2024+ science State Report of Schools file has `District Number` in column A; the spring 2023 Statewide Science files (`54G5SRSSpring23.xls`, `55G8SRSSpring23.xls`) put `Grade` in column A and `District Number` in column B, and add a `Number of Points Possible` row under the header. Both the loader and the harness reader now locate the district and school columns by header text; the first pass assumed column A and silently skipped both files (found because the science metrics had two years instead of three).
- **F.14 EDFacts chronic absenteeism counts K-12 only; the state file includes PK.** The federal copy of the 2022-23 "absent 10% or more" count (EdDataExpress FS195/DG814) matches the state count exactly at 83% of schools with no October PK membership and within ±1 student (the federal perturbation) at 95%; at schools with PK the federal count is lower by 0.66 students per PK student (r = 0.86 across 209 schools), which is the PK chronic-absence share. The harness's cross-publisher check therefore compares the no-PK schools exactly and reports the PK gap as a regression (`pipeline_miami/verify/cross_publisher_absenteeism.py`).
- **F.15 Hidden race cells: which allocation rule recovers the truth.** The state race file hides cells under 10 students but the October total is known, so each school's unallocated count U is exact. Three rules were measured against the unsuppressed NCES CCD race subtotals for 2018-19 and 2023-24 before any was adopted (`probe/race_imputation.py`): counting hidden cells as zero (mean error 3.0–3.3 pp on the Black and Hispanic shares, always biased down), allocating U by the visible mix (5.3–5.7 pp, biased toward the majority group), and splitting U equally over the hidden cells (0.8–0.9 pp, 88–90% of schools within 2 pp, median error 0). Splitting per grade using the by-grade totals did worse on the large groups (1.5–1.9 pp) because some grade rows are missing from the race file. Blank cells are grades not served, not hidden counts; in 2022-23 `*` marks zeros as well, and hidden cells in a grade whose by-grade total is already fully visible are resolved as zeros (1,482 cells) before the split. The 43–46 schools with 20 pp or more unallocated (median 60–80 students) still carry 5–6 pp errors and are the reason the harness's pass rule is a per-school process bound (each hidden cell holds 1–9 students) rather than a fixed 2 pp.
- **F.16 Every climate-survey workbook has a different layout.** Twenty workbooks (students, parents, staff × 2018-19 → 2024-25) and seventeen distinct column sets: the long results sheet is named `2019 Stud TBL`, `STUD 2020`, `STUD2021`, `2023_Stud_Data_12June`, `2024_Student_Data`, `2025_Students_Data` for students alone; agreement is a fraction in some years and a percentage in others (decided per file by the median, not the maximum, after a stray count flipped one file); the item number restarts or shifts between years (parents' Q07 in 2024-25 is Q08 in 2023-24), so items are matched on wording. The 2021-22 workbooks publish only pivot tables; their records were read from `xl/pivotCache/pivotCacheRecords1.xml` with field names from the cache definition. No parents workbook exists for 2019-20. The 2023 and 2024 recommend items use five options, 2025 three.
- **F.17 The 2022-23 School Grades file is four sheets with different column layouts.** Elementary, Middle, High and Combination sheets carry different component columns (acceleration, graduation, college readiness), so a loader that reads all sheets with the first sheet's column indexes silently misaligns the later ones. The first load did exactly that: 204 of 470 letter grades and 369 of 470 points percentages, and the residual-z fits for 2022-23 had 261 eligible schools instead of 440 because the economic-disadvantage share was missing. Found through the survey-versus-enrollment funnel (the 2022-23 sample halved at the letter-grade control); every sheet is now indexed on its own header and the harness components for 2022-23 points and science re-read all four sheets independently.

## N. Negative results and things not verified

- No school-level SWD (ESE) membership file from FLDOE (district-level only); SWD denominators come from CRDC / EdDataExpress or graduation-file cohorts.
- No county-level voucher (FES) enrollment table from Step Up or FLDOE reachable; figures are secondary (press, district budget) plus one 2023 district study.
- No per-school capital spending or building age verified (DEFP / FISH inventory are PDF; not parsed yet).
- No public utilization dashboard; the one the board ordered in June 2026 does not exist yet.
- CRDC zip internals not opened (0.6–0.8 GB for some years); EdDataExpress 2023-24 chronic-absenteeism file not yet listed.
- Know Your Data (Tableau) subgroup assessment results were not exercised; treated as out of scope for v1 loaders.
- Residual-z: the pre-registered expectation that a demographic-adjusted residual would track Florida's learning gains at least as well as raw achievement fails in every fitted year (ρ 0.44–0.60 against 0.75–0.84); the state's gains measure itself moves with economic disadvantage (ρ −0.2 to −0.55). Reported on `/miami/methodology/analysis/residual-z`, not tuned away.
- The survey-versus-enrollment expectation was wrong in the other direction: parent sentiment in 2021-22 adds as much explained variance in four-year enrollment change as utilization, grade, level and size together (partial R² 0.11 over a controls R² of 0.11).
- CCD lags ~1.5 years (2023-24 latest); the 2024-25 and 2025-26 picture comes only from FLDOE and M-DCPS files.