# What the review's fixes changed (2026-09-16)

Companion to `08_review_2026-09-16.md`. Every correctness finding in that
review's Parts 1.1–1.4 has been fixed on the branch; the affected sources
were reloaded, the analyses re-run, the harness re-run, and the served
numbers re-published. The refactors in the review's Part 2 are not started.
This file records, per finding, what was done and what it changed, then the
re-validation results, the published conclusions that moved, and what is
still open.

> **Status.** A second correctness review of this state
> ([`10_review_2026-09-16b.md`](10_review_2026-09-16b.md)) found five further
> problems and retracts two claims made below: the invariants step would
> **not** have caught the first review's critical findings (§1.3), and the
> 2021-22 staff "substance abuse" item was not a reversed scale but one symptom
> of that workbook's values being shifted one item against their labels (§1.3).
> No fixes from the second review are applied yet.
> Those findings have since been applied; `10_review_2026-09-16b.md` §7
> records what changed and what it measured.

Reading order for a sceptic: §3 (what moved), §2 (the checks that now hold),
§4 (what is open), then §1 for the detail.

---

## 1. Fixes, by finding

### 1.1 Served numbers (C1–C13)

| # | Fix | Reload | Served effect (before → after) |
|---|---|---|---|
| C1 SESIR total doubled | `sesir.ts` skips the file's `SCHOOL TOTAL` row and uses it as a cross-check that fails the load on any disagreement. | SESIR 2024-25 | Mean `sesir_total_per_100` 3.24 → 1.62 (category sum 1.56; the four categories do not cover every type). All residual-z cells on this outcome refitted. |
| C2 survey extras from the Q99 row | `climate_survey.py` drops the Q00/Q99 lines before anything is aggregated; the respondent count is the item rows' median, else the Q00 line's count. | All 20 workbooks | `survey_staff_response_rate` 2022-23: 342 cells at exactly 100 → 322 cells, mean 71.9. `survey_parents_response_rate` 2020-21: mean 81.8 → 7.4. |
| C3 survey items above 100 | Percent agreeing = agree ÷ (agree + dk + disagree) where the three are published and sum further than 1.5 pp from 100 (the largest deviation three whole-percent roundings can produce); students 2024-25 shares come from the response counts on the same rows; any value still outside [0, 100] is invalid and skipped, counted in the load note. | All | 28 composite cells above 100 → 0. 2024-25 rows renormalised: students 0 (counts used), staff 666; invalid item rows dropped: 68 (staff 2018-19, no don't-know column), 57 (parents 2021-22), 8, 6, 2. |
| C4 parents 2022-23 counts as percents; recommend missing | Spec marks the file's extras as counts over `Total`; recommend = (DefinitelyYes_A + MaybeYes) ÷ Total. | Parents 2022-23 | `survey_parents_grade_ab` 2022-23 mean 48.7 → 86.9; `survey_parents_recommend` 2022-23 now served (368 schools). The pre-registered second cohort exists — see §3. |
| C5 2018-19 respondent count on Q00 only | Covered by the C2 rule (Q00 fallback). | Parents/staff 2018-19 | 8 parent schools served 0.0 → suppressed; 60 staff schools with n < 10 → suppressed; every 2018-19 composite now carries a denominator. |
| C6 whole domains suppressed by the core rule | Domain and reverse-coding are canonical-level in the crosswalk (they were per row, so a canonical could sit in "resources" one year and "teaching" the next, and the union count could never be met). | All | `survey_staff_resources` 2022-23 → 2024-25: 0 served → 411–414 of ~467; `survey_students_resources` 2021-22 → 2024-25: 0 → 383–430. Core items per domain now identical in every year. |
| C7 clamp hid garbage | `climate_survey.ts` never clamps; values outside [0, 100] are skipped and counted. | All | No served survey value above 100. |
| C8 2016-17/2017-18 total sheet missed | `membership.ts` matches `total (enrollment|membership)` and `students|membership`; fails loudly without a total sheet; requires all 14 grade columns. | Membership 2016-17, 2017-18; derived | `enrollment_oct` 2017-18 suppressed 43 → 9; district total 347,420 (454 schools) → 354,807 (488); `district_run_regular` 281,819 → 286,386; `enrollment_change_5yr_pct` 2022-23 436 → 463 rows. The trend chart's 2017-18 dip is gone. |
| C9 residual-z survivors only | The engine fits every school with the outcome that year; the packet reports `n_closed_or_inactive_included` per fit. | Engine, residual load | 18,297 → 19,457 fitted cells (32 closed schools in the 2018-19 ELA fit, 18 in 2024-25). |
| C10 2022-23 race small groups inflated | Two-pass loader: in a year where any school has more hidden cells than hidden students (the file's `*` marks zeros), U is split over hidden groups in proportion to the school's adjacent-year shares, capped at 9 per hidden cell, overflow re-spread by prior then equally. Rule chosen by measurement against CCD (`probe/race_imputation_prior.py`): 2022-23 mean error Black 2.39 → 0.89 pp, Hispanic 2.15 → 0.66, White 1.73 → 0.84, American Indian 1.64 → 0.08; the uncapped prior (1.21 / 0.94 / 1.04) left 2–4% of school-years outside the process bound, the capped one none. Applying the prior in normal years was measured worse (Black 1.10 vs 0.91) and is not done. | Race, all years; engine | `pct_amer_indian` 2022-23 median 0.958 → 0.000 (mean 1.67 → 0.10); 10 uniform-14.29% school-years → 0. Schools whose October total is suppressed now get suppressed shares instead of a stale CCD-derived value (6 school-years). |
| C11 directory re-run reverted FISH capacity | `directory.ts` seeds `fish_capacity_permanent` from GIS only on a new record. | Directory re-run | 0 of 756 records changed on re-run (was 126). Registry text corrected: GIS equals FISH for 200 of 326, not "7 of 9". |
| C12 ELL absence | Universe schools with October membership and no row are served `pct_ell = 0` (no all-zero row exists statewide in any year, so absence is the file's zero); the load note lists them. | ELL, all years | 11–19 schools per year gain a value (School for Advanced Studies campuses, Teenage Parent Program, …). |
| C13 CCD teachers = 0 | A reported 0 at an operating school is served as not reported (suppressed), never 0; the note warns that charter-network reporting makes 25 of 474 ratios exceed 40. | CCD staff | 60 "Teachers (FTE) 0" cells → suppressed. |

Also from §1.4 of the review: `essa_ppe.ts` district filter strips `.0`; `readLatestSourcePath` sorts; `school_events.ts` replaces the table in one transaction; `residual_z.ts` resets every cell carrying residual columns; `school_grades.ts` loads `college_career_acceleration` (83–98 schools per year); `derived.ts` populates `miami_schools.latest_enrollment` (566 schools); CCD 2022-23 membership loaded (534 schools) so the enrollment cross-check now covers three years; `compute_percentiles.ts` kept its bare query (the reconnect loop is refactor #2 in the review).

**New load-time guarantee.** `finishDataLoad` deletes cells still owned by an earlier load of the same source, so a school or value that drops out of a re-run cannot survive as a stale served cell. The first survey reload after the fixes left ~11,000 such cells; the guarantee removed them and the invariants step now checks it (I3).

### 1.2 Method and harness (M1–M12)

| # | Fix |
|---|---|
| M1 R1 not independent on N | `residual_checks.py` re-derives N from the served grade columns under the `n_rule` with its own parser, re-derives eligibility, requires the eligible set to equal the engine's, then refits. Result: 46 fits, eligible sets equal, max |ΔN| 0, max |Δz| 4.1e-11. |
| M2 survey reader shared the parser | `source_file.climate_survey` has its own column map, scale rule, renormalisation, respondent-count rule, suppression and bounds assertion; it imports nothing from the parser and consumes only the committed crosswalk. Base cases: 958 + 906 + 928 cells, 0 mismatches. |
| M3 variance double-count | z = r ÷ √(τ² + se²), τ² = max(0, s² − Σw·se²/Σw), s² with the n/(n−k) correction; packet carries `tau_between`, `mean_sampling_se`, `k_parameters`. |
| M4 science weight | `grades:5,8,9` (measured: Biology testers ≈ 0.75–0.85 × grade-9 membership, 0.39–0.41 × grades 9–10, 0.20–0.21 × 9–12 in the SRS files); the old rule and 9–10 are in the ablation (`n_rule=` rows: 12–18 and 6–8 flag changes). |
| M5 crosswalk splits | Lead-ins dropped, stop-words and light stemming, plus a hand-reviewed alias table (`survey_item_aliases.csv`, 13 rows) for what wording cannot merge; the parser prints every disjoint-year pair it did not merge. Core items: parents 34, staff 31 (safety 3 → 6), students 25. |
| M6 students' response-rate denominator | Computed in the loader against the same year's October membership in grades 4–12; the workbooks' own eligible column is kept in the parsed table but not served. |
| M7 race bound looser than the loader | The harness counts hidden cells as the loader does (tokens in fully visible grades are zeros); adds a uniform-split guard and per-year reporting. |
| M8 suppressed PK | Schools whose PK cell is suppressed are in neither group (34 in 2022-23); no-PK group 298 → 264, exact 83%, within ±1 95.8%. |
| M9 regime windows | `comparability.yaml` separates `boundaries` (regime breaks) from `events` (excluded / flagged years); persistence counts calendar years, so a missing year is a gap, not bridged. |
| M10 served-only never failed | `reconcile.py` fails a component above its `served_only_allowance` (default 0; the four FISH components carry 3, the documented name-override cells). |
| M11 funnel after the join | Counted before the end-year join; the closed/merged drop is explicit (16 and 15 schools). A missing pre-registered predictor now stops the analysis. |
| M12 magnet cells too small | `derive_peer_groups.ts` relaxes the magnet filter to level|sector where the cell has fewer than 5 candidates (22 schools; groups 417 → 438; 9 still ungrouped). Subject inclusion stays as decided (M.9): non-regular schools get no group. |

### 1.3 New: served-value invariants (`served_invariants.py`, step E)

Hard rules that fail the closure verdict: ranges by unit (signed changes and
utilization may exceed the [0, 100] band), identities between cells (SESIR
total ≥ each category; race shares sum to 100; October total = sum of grades
where all are served), and ownership (every row belongs to its source's
latest load). Shift alarms: a year whose median moves further than both
3 × MAD of the other years' medians and half the metric's pooled within-year
IQR, or whose served-school count moves more than 3 × MAD (floor 2%), is
listed; an alarm counts as explained only when
`served_invariants_allowlist.yaml` records its cause (assessment regime
boundaries, ESSER expiry, the parent workbooks' own respondent counts, the
2025-26 capacity fallback, the CCD-verified 2018-19 race shares). Each of the
review's critical findings would have tripped one of these.

The invariants step found one more publisher anomaly the review had not:
the 2021-22 staff workbook's substance-abuse item shows 58% agreement
district-wide (Air Base K-8: 81%) against 0–4% in every other year and 0% for
the same school's violence and gang items that year. It is recorded in
`survey_item_exclusions.csv` and left out of the 2021-22 staff safety
composite (5 of 6 core items remain).

### 1.4 Published text

Stories, recipes, notes, registry descriptions and page text were rewritten
from the re-run packets: the cost-premium story now attributes its dollars to
the school-level series and cites Table 5; the closure recipe filters on the
four actions rather than a year and restricts to regular schools; the sector
recipe separates district-run regular from other; the ablation sentence names
school level and English-learner share (19–26 ELA flag changes) ahead of
economic disadvantage (16–17); the region sentence says central and south
lost more than north (median −18.8 / −17.3 / −14.7); the pre-K sentence says
two in three pre-K students are chronically absent; the CCD agreement is 99%
of 1,533 school-years over three years; k ÷ g5 before the pandemic is
0.85–0.88; the participation story uses like-for-like universes (76 of 130
charters, 310 of 314 district-run regular schools with grade-4+ students);
the pct_* note names the FLDOE race file and the allocation rule; the blank-
component note says blanks are not stored; the annual-enrollment note says
+4.9%; the "not here yet" and "ratings follow" sentences are gone; the
citywide tooltip names Miami-Dade; derived sesir/survey pages name their real
inputs; the outcomes verdict, metric index and invariants report are
versioned; the residual dropdown lists only fitted metrics and years and a
metric change clears the year; nullable sorts put NULLs last;
`sources.yaml` names `survey_students_*`.

---

## 2. Re-validation (all reports under `docs/qa_reports/miami/`)

| Check | Result |
|---|---|
| A base case, 30 components | PASS, 0 mismatches (enrollment_oct 5,099 common; the new `enrollment_oct_g3` component confirms the grade-3 medians the invariants flagged are the file's) |
| B completeness, 10 years | every source school served, every served school in source |
| C FLDOE vs CCD October count | PASS, 98.9% exact over 1,533 school-years (2018-19, 2022-23, 2023-24) |
| D computed values | 0 mismatches |
| E invariants | hard rules 0 violations; 25 shift alarms, 13 with a recorded cause, 12 open (see §4); the outcomes summary therefore reads REVIEW while the closure verdict, which the hard rules gate, reads PASS |
| Absenteeism vs EDFacts | PASS, 95.8% of 264 no-PK schools within ±1; PK slope 0.66, r 0.86 |
| Race shares vs CCD | PASS, 100% within the (tightened) process bound for all three shares; mean error 0.88 / 0.77 / 0.93 pp; 2022-23 alone 0.89 / 0.66 / 0.84 |
| Residual checks | PASS: independent N and eligibility equal, max |Δz| 4.1e-11 over 46 fits; served = committed 19,457 cells |
| Closure verdict | PASS |
| App | `tsc` clean, eslint clean on the touched files (the two pre-existing `set-state-in-effect` errors in the search dialog fixed), 51/51 unit tests |

---

## 3. Published conclusions that moved

- **Parent sentiment → enrollment does not replicate.** With the 2022-23
  parent recommend measure loaded, the pre-registered second cohort gives
  +2.0% per SD (n = 265, p = 0.11) against +6.9% (p < 10⁻⁶) in 2021-22. The
  student recommend share holds in both (+5.5% and +5.3% per SD, p < 0.001),
  as do student safety (+5.0 / +4.2) and staff leadership (+3.4 / +3.2). The
  story, the climate page and the recipe now say so; the pre-registration's
  reason for two cohorts was exactly this.
- **Residual-z:** every z changed (universe, variance, science weights, race
  covariates, SESIR scale); the P2 negative result stands, ρ(z, gains)
  0.45–0.60 against ρ(raw, gains) 0.76–0.85 in all eight ELA/math fits; ELA
  R² 0.68–0.76; flag counts moved by a few per fit.
- **Enrollment packet:** only the 2017-18 point (C8) and the 2022-23
  five-year-change row count; the lift table, Fisher tests and cost figures
  are unchanged.
- **Survey composites:** every served value re-derived; resources domains
  exist; 2018-19 has no served zeros; nothing above 100.
- **Race shares 2022-23:** small groups back to their level; Black/Hispanic
  errors halved against CCD.
- **SESIR total:** halved; its 12 residual flags refitted.

---

## 4. Open items

- **Twelve shift alarms without a recorded cause** (`__served_invariants.md`):
  eleven are served-school counts that move 8–23 between years on SRS and
  School Grades components (biology, geometry, grade 9–10 ELA, college and
  career acceleration, social studies, middle-school acceleration) and on the
  students' 2021-22 extras (430 schools against 393 typical); one is a median,
  the lowest-quartile ELA gains rising to 63 in 2024-25 from 55. Each needs a
  reader, not a loader: nothing in the re-reads disagrees with the files.
- **Survey response-rate history:** on the same grades-4–12 denominator the
  students' median rate is 36–37% in 2018-19 and 2019-20, 50% in 2020-21 and
  62–67% from 2021-22 (means 44–46%, 49%, 57–63%; corrected 2026-09-16b);
  the respondent counts are the workbooks' own. Not an artefact this
  pipeline can identify; reported as is.
- **Charter staffing ratios** above 40 are served as reported with a caveat;
  no external staff count exists to correct them.
- **Refactors** (review Part 2, items 1–10): none started. The two policy
  decisions to record first are unchanged: the reconcile suppression rule
  and the peer-group distance rule.