# Miami build: adversarial review (2026-09-16)

Scope: everything authored on `claude/admiring-darwin-9pzcjp` through commit
`bb0be6b` (loaders, Python pipeline, harness, analyses, app pages, stories,
docs), reviewed for (1) correctness and (2) duplication / opportunities for
shared code across cities and outcome variables.

Method: four independent read-only reviewers, each scoped to one layer
(TypeScript loaders + schema + enrollment packet; Python pipeline + harness +
residual-z + survey; app pages + stories + docs; cross-city duplication),
followed by verification of every critical claim with direct SQL against the
served database and re-reads of the cited lines. Nothing was modified. No
fixes have been applied; a proposed fix order is at the end.

> **Status (applied later on 2026-09-16).** Every correctness item in Parts 1.1–1.4
> has been fixed on the branch and the affected data reloaded, re-validated and
> re-published; the refactors in Part 2 are not started. The record of what
> changed, what the re-runs found and what remains open is in
> [`09_review_fixes_2026-09-16.md`](09_review_fixes_2026-09-16.md).

Bottom line: the closure-track numbers (enrollment, utilization, PPE, events,
lift table, Fisher tests) reconcile against the source cells and hold. The
outcomes side has eleven defects that change served numbers, seven of them in
the climate survey chain and one (SESIR total) a clean 2× error. The residual-z
engine fits on survivors only, and two of its "independent" checks are not
independent on the inputs that matter. Several story sentences contradict the
packets they cite. On duplication: the schema is not what blocks sharing;
copy-paste is, and there is a low-risk order in which to fix that.

---

## Part 1 — Correctness

Severity scale: **critical** = a served number is wrong; **major** = a
statistic, check, or published claim is wrong or biased; **minor** = latent,
cosmetic, or confined to a probe. Confidence is stated where a finding rests
on inference rather than a reproduced number.

### 1.1 Critical: served numbers are wrong

| # | Where | What | Evidence (served DB unless noted) |
|---|---|---|---|
| C1 | `scripts/loaders/miami/sesir.ts:38-41` | `sesir_total_per_100` is exactly 2× the true count. The file has a `SCHOOL TOTAL` row per school and the loop adds every row's count, including that one. Category rates are correct. | Mean `sesir_total_per_100` across 519 schools 3.24 vs sum of the four category rates 1.56. School 0041: type rows sum 5, SCHOOL TOTAL 5, served total ×enr/100 = 10. Downstream: 442 residual-z cells fitted on the doubled scale (z is not scale-invariant because se depends on n). |
| C2 | `pipeline_miami/parse/climate_survey.py:182-190` | School-level extras (`n_responses`, `n_eligible`, response rate, recommend, grade A/B) are aggregated with `max` over **all** rows, including the Q00/Q99 district/summary rows that line 179 drops only from the item table. | Staff 2022-23 school 0041: Q99 "Overall" TTL 2,368 vs 64 respondents → parsed response rate 3,289%. Parents 2020-21 school 0001: n_responses 43,168. Served `survey_staff_response_rate` 2022-23: 342/342 cells = 100 (after the loader clamp in C7); `survey_parents_response_rate` 2020-21 mean 81.8, 293/445 = 100. The n<10 suppression for recommend/grade_ab in those files uses the same wrong n. |
| C3 | `climate_survey.py:119-124,162,170` and `scripts/loaders/miami/climate_survey.ts:57-69` | No range check on item percentages in the parser and no guard in the loader. The 2024-25 student and staff workbooks contain publisher rows with agree share > 1 (students 0441 Q05: Agree_PCT 1.232, NA_PCT −0.303). | Parsed 2024-25: 94 unsuppressed student item rows > 100 at 34 schools (max 166.7), 165 staff rows at 18 schools (max 132.1). Served: 28 composite cells > 100 (`survey_students_teaching` 5091 = 119.30; `survey_staff_leadership` 3661 = 117.13) and every other composite for ~47 schools inflated below the cap. |
| C4 | `climate_survey.py:55` (`parents_2022-23` spec) | `grade_ab=["A","B"]` reads response **counts** (A+B+C+D+F = Total on 100% of rows), so `grade_ab_pct` is a count. The same file has recommend counts but the spec omits `recommend`, so `survey_parents_recommend` 2022-23 is never produced. | Served `survey_parents_grade_ab` 2022-23 mean 48.7 vs 86–88 every other year (0041 parsed 164 → clamped 100). `survey_parents_recommend` exists for 2021-22, 2023-24, 2024-25 only. Consequence for the survey→enrollment test: the pre-registered second cohort (2022-23) is "skipped: fewer than 60 schools" (n = 0), so the headline predictor rests on one cohort despite the docstring's "two survey years so the result is not one cohort's accident". |
| C5 | `climate_survey.py:160-161,174-179` | In the 2018-19 parents/staff workbooks the Ncount sits only on the Q00 row, which is dropped; item rows get n = null and `suppressed` = false everywhere. The district zeroes withheld schools. | 8 parent schools (e.g. 0070, 7027) served `survey_parents_{safety,teaching,leadership}` 2018-19 = 0.0, suppressed = false; 60 staff schools with n < 10 served (0400, n = 9: 100 / 96.3); all 446 + 476 2018-19 composites have NULL denominator. The harness component for `survey_parents_teaching` 2018-19 PASSES because the reader reads n the same way (see M2). |
| C6 | `climate_survey.ts:35,65-68` | `nCore` is the union of "core" canonicals across all years, but the crosswalk's `core` flag is not "asked every year". The `vals.length*2 < nCore` rule then suppresses whole domains, and where a composite does pass its item set changes by year, contradicting the registry text "using only the items asked in every survey year" (`metric-definitions.ts:229`, `metric-notes.ts:131`). | `survey_staff_resources` 467/467, 469/469, 459/459 suppressed in 2022-23..2024-25; `survey_students_resources` 482/482, 447/447, 488/488, 492/492 suppressed from 2021-22. 58 (respondent, year, domain) cells have a different item count than the union. |
| C7 | `climate_survey.ts:81` | `Math.min(100, v)` clamps parser garbage to a valid-looking 100 instead of rejecting it. | `survey_staff_response_rate` 2022-23 342/342 = 100 (raw up to 7,400); `survey_parents_response_rate` 2020-21 66% = 100; `survey_parents_grade_ab` 2022-23 17% = 100 (raw up to 23,516); students response rate 2021-22 27 raw rows > 100. |
| C8 | `scripts/loaders/miami/membership.ts:47,55` | Total-sheet lookup `/total enrollment/i` misses "Schl Total Membership" (2016-17) and "School Total Membership" (2017-18); the value-column regex `/students/i` would miss 2016-17's header too. With no total sheet the tokens-are-zero rule cannot run. | `enrollment_oct` 2017-18 stored suppressed for 43 schools (2–7 in other years); the ignored sheet publishes totals for 34 of them (0092 Sunny Isles K-8 = 2,174; 2191 Spanish Lake ES = 1,577; 0311 = 472 …). Packet: `district_trend` 2017-18 = 347,420 over 454 schools vs 357,288/486 (2016-17) and 350,442/500 (2018-19); "other" sector 6 schools/1,385 vs 30/3,088 and 35/3,406; `enrollment_change_5yr_pct` 2022-23 has 436 rows vs 465/474 adjacent. |
| C9 | `pipeline_miami/analysis/residual_z.py:226` | `activity_code == "A" & closed_at.isna()` drops closed/inactive schools from **every** year's fit, including years when they were open. The baseline is survivors-only and no doc mentions it. | 36 closed schools carry 120 ELA cells with mean 47–51 vs 58–64 for kept schools in each year; they get no z and are absent from the reference distribution. This biases the fitted line upward for the schools most relevant to the closure analysis. |
| C10 | `scripts/loaders/miami/race_membership.ts:397-413` | 2022-23 imputation: that year's file marks zeros with `*` (17,464 tokens vs ~5,900 in other years); the grade-total rule resolves only 1,482 as zeros, so 15,863 "hidden" cells share the unallocated count equally, inflating the small groups. | `pct_amer_indian` median 0.000 (2021-22) → 0.958 (2022-23) → 0.000 (2023-24), mean 0.111 → 1.673 → 0.099 (~15×); `pct_hawaiian_pac_isl` identical; `pct_two_or_more` mean 1.03 → 1.68 → 0.89; `pct_white` median 2.55 → 1.47 → 2.39. 10 school-years (all 2022-23, e.g. 7840, 7855) are served with exactly 14.29% for all seven races. The race cross-check passes them because its bound is looser than the loader (M7). |
| C11 | `scripts/loaders/miami/directory.ts:223` vs `fish_los.ts:64` | `miami_schools.fish_capacity_permanent` is order-dependent: directory.ts sets it from GIS CAPACITY in the **update** payload; fish_los.ts later overwrites with FISH. A documented-idempotent re-run of directory.ts would revert it. | 126 of 326 schools (39%) have GIS ≠ FISH (4491: FISH 1,723 vs GIS 689; 5101: 1,242 vs 403; 0231: 1,485 vs 2,099). Registry text "equals permanent FISH capacity for 7 of 9 checked schools" (`metric-definitions.ts:131`) vs 200/326 (61%) agreement in the loaded data. Current DB is correct only because fish_los ran last. |

Two more that change served numbers but are smaller in reach:

| # | Where | What | Evidence |
|---|---|---|---|
| C12 | `scripts/loaders/miami/ell.ts:189-194` | Schools absent from the ELL file get no `pct_ell` rather than 0. Absence = zero LY students is the evident convention but is **not verified** against a codebook. | 11–19 enrolled schools per year (2024-25: the six School for Advanced Studies campuses, 3029, 6053, 8016). Removes them from peer-group features: 8 of the 45 ungrouped default-included schools. |
| C13 | `scripts/loaders/miami/ccd_staff.ts:228` | TEACHERS = 0 kept as a value where CCD has no report; FTE = 1 yields absurd ratios. | 61 schools served "Teachers (FTE) 0" (0100 Mater Academy, 7042 …); `students_per_teacher` 999 (7016), 487 (6032), 322 (4010). |

### 1.2 Major: statistical method and harness

| # | Where | What | Evidence |
|---|---|---|---|
| M1 | `pipeline_miami/verify/residual_checks.py:81` | R1 "independent refit" copies `n` (the WLS weight and the sampling N) and the eligible set straight from the engine's CSV (`e["n"] = g.n.to_numpy()`). Only the solver is independent; the docstring and `METHODOLOGY.md:240` claim more. It cannot detect a wrong `n_rule`, grade sum, or eligibility rule. | Structural; C1 (doubled sesir scale) and M4 (science n) both pass R1 by construction. |
| M2 | `pipeline_miami/verify/connectors/source_file.py:400-455` | The climate-survey "independent" reader imports the parser's `SPECS`, consumes the parser-built crosswalk, reuses the same median scaling and per-row n. It can only catch an error in the arithmetic mean. `METHODOLOGY.md:241` describes it as "code that shares nothing with the normaliser except the committed crosswalk". | C3 and C5 are present in the raw files and both verified components PASS with 0 mismatches. |
| M3 | `residual_z.py:182,200-202` | `s` is the weighted RMSE of residuals, which already contains the weighted-mean sampling variance, and it is added again in `sqrt(s² + se_i²)`; no (n − k) correction. | Weighted-mean se² is 5.8% of s² at the median but 39–40% for `learning_gains_*_l25` and `attendance_rate` (z understated up to ~18%, fewer flags); graduation (n = 83, k = 14) has s understated ≈10% (z overstated). Direction is conservative for most outcomes. Fix: τ² = max(0, s² − Σw·se²/Σw) with an (n − k) correction. |
| M4 | `pipeline_miami/config/outcomes.yaml:32` with `residual_z.py:143-150` | `science_achievement: grades:5,8,9-12`: Biology EOC is taken by roughly one high-school cohort, so summing 9–12 inflates high-school N (×1.89) and shrinks se_i. | Refit with `9-10` (same engine functions): 34 of 2,065 cells change flag (≈15% of flagged cells); high-school flags 19 → 17, elementary 77 → 60 (high-school weights move the whole fit). Add the n-rule choice to the ablation. |
| M5 | `climate_survey.py:137-141,209-232,249-253` | Jaccard ≥ 0.6 on unstemmed words (stop list lacks feel/there/school/problem) splits the same item across wording changes, and `core` then excludes items that were asked every year. | Confirmed complementary splits: staff_21 "I feel there is a school violence problem." (2018-19→2023-24) vs staff_50 "School violence." (2024-25); staff_23/51 substance abuse; seven more staff pairs from the 2022-23 "I feel…" prefix drop; students_17/30 gangs; parents_06/38, 30/39, 36/44. Consequence: `staff_safety` and `staff_teaching` rest on 3 core items each, `students_leadership` on 3; the reverse-coded violence/substance items drop out of staff safety. Construct validity, not arithmetic. |
| M6 | `climate_survey.py:47-51` | `n_elig` for students alternates between all-grade FTE (2020-21, 2022-23) and grades-4+ counts (2021-22, 2023-24, 2024-25), so `survey_students_response_rate` is one series with two denominators; nothing marks the break. | Served means by year: 24.1 / 57.6 / 29.8 / 52.2 / 56.5. |
| M7 | `pipeline_miami/verify/cross_publisher_race.py:39,183-191` | `hidden` counts every `*` token, whereas the loader resolves tokens in fully-visible grades to zero, so the process bound is looser than the process it certifies and trivial where hidden cells are many. | 4,599 pairs: 13.5% have bound ≥ 10 pp (max 100.5). The 10 uniform-split school-years in C10 pass against a 100.5-pp bound. 36 school-years with minor-group shares summing ≥ 20 pp feed 20 residual-z eligible cells (11 flagged). The check has teeth for the 42.6% of pairs with no hidden cell (bound 0.5 pp, p90 |Δ| = 0). |
| M8 | `pipeline_miami/verify/cross_publisher_absenteeism.py:81` | `(p["value"] or 0.0)` puts schools whose PK cell is suppressed (1–9 PK students; 40 of 521 in 2022-23) into the "no PK" exact-comparison group. | The three largest "no-PK" deltas (5931 −8, 5241 −5, 9732 −3) are exactly such schools. Biases against passing and mislabels the population. |
| M9 | `pipeline_miami/config/comparability.yaml:13-14` + `residual_z.py:86-97,308-315` | The 2020-21 boundary is annotated "flag, do not drop" but `flagged_years` is read nowhere; every boundary starts a new persistence window (2018-19 and 2021-22, both FSA, can never share one; single-year windows make persistence impossible). Persistence counts consecutive fitted rows, not calendar years. | Today all 189 persistent cells span 2–3 calendar years, so no gap artefact yet; a school ineligible in a middle year would be bridged. |
| M10 | `pipeline_miami/verify/reconcile.py:67` | `status = PASS if matches and not mismatches`: served-only and source-only cells never fail a component; only `enrollment_oct` has a completeness check. | Today absorbs only the 3 documented FISH served-only cells per year; a loader that fabricates or misattributes cells for any other metric would pass. |
| M11 | `pipeline_miami/analysis/survey_enrollment.py:75,82` | The funnel's first row is counted after the inner join on 2025-26 enrollment and counts any metric row, not survey rows; 23 district-run schools with 2021-22 membership (5,545 students) drop invisibly. The survivors-only universe is pre-registered, but the funnel hides its size. | `with_membership_both_years` == `district_run_active` by construction. |
| M12 | `scripts/loaders/miami/derive_peer_groups.ts:43,192,214` | Whole-school-magnet hard filter plus the <5-candidate floor leaves 22 fully-featured, active, default-included schools with no group and no peer percentile; separately, the subject filter on `includeInDefaultComparisons` gives the 80 non-regular schools no group at all (NYC and Philly filter candidates only). | Cells: K-8/district/magnet = 4 schools (0041 Air Base K-8, 3101, 3191, 5831); 6-12/district/magnet = 4; K-12/district = 2. School pages show "—". |

### 1.3 Published text that contradicts the data

Stories and notes are served to readers; each of these is a wrong claim on
the page today.

| # | Where | Claim | Measured |
|---|---|---|---|
| S1 | `data/cities/miami/stories/index.ts:79-98`, `recipes.ts:43-52` (small-school-cost-premium) | Dollar figures described as "school costs plus allocated district indirect costs"; recipe selects `per_pupil_expenditure_total`; evidence link goes to `per-pupil-expenditure-total`; cites "Table 7" and says the flagged-school table shows spending. | Every figure ($16,893 / $14,051 / $11,895 / $20.5M) is `per_pupil_expenditure_school` (packet `ppe_vs_size`, script line 198; DB median district ES <300 in 2024-25: _school 16,892.5, _total 17,545). The analysis page says "district indirect costs excluded". The by-size medians are Table 5; Table 7 has no spending column. |
| S2 | `recipes.ts:27` (closure-lists-vs-utilization) | Step 2 flags `school_year = '2026-27'`; step 1 omits `school_type = 'REGULAR'`. | REASSESSED (12) and UNDER_REVIEW (15) are stored under 2027-28; 2026-27 holds BOUNDARY_CHANGE (16) and RECEIVED_CONSOLIDATION (4). The recipe as written returns 261 schools / 23 flagged vs the text's 260 / 35. ESE center 2531 (util 35%) is admitted. |
| S3 | `recipes.ts:37` (charter-growth) | Sector SQL groups by `is_charter` only. | Returns 294,386 → 234,581, not the 291,298 → 231,607 quoted for district-run regular. |
| S4 | `index.ts:155` (adjusted-scores-vs-learning-gains) | "economic disadvantage does most of it" (ablation). | ELA 2024-25: dropping `pct_ell` changes 20 flags, `level` 17, `pct_econ_disadv` 12, `magnet` 5; 2018-19: level 21, ELL 16, econ 13. ELL and level carry more in every year. |
| S5 | `index.ts:195` (charter-participation) | "82 of 157 active charter schools … against 329 of 353 district-run". | 157 is `active_schools.charter`; 353 is `cells` (schools with any survey row). The climate page renders 329 of 385. Two denominators in one sentence. |
| S6 | `index.ts:128`, `recipes.ts:67` (decline-since-pandemic) | Losses "concentrated … in the north of the county". Not in any packet. | District-run REGULAR, 2019-20 → 2025-26, by `miami_schools.region`: median change N −14.5% / C −18.5% / S −16.5% (mean N −12.2 / C −17.8 / S −14.1; share ≤ −25%: N 27% / C 29% / S 33%). North is the least affected. Recipe SQL returns 481 rows (all sectors, suppressed included) vs the text's 325. Confidence: plausible (depends on the GIS region field, but that is the site's own field). |
| S7 | `index.ts:182` (federal-absenteeism-omits-prek) | "four-year-olds are chronically absent at two-thirds the rate of the school as a whole". | The slope 0.66 is chronically absent students per PK student, i.e. ~66% of PK students, which is what the same story's thesis (line 174) says. The two sentences contradict each other. |
| S8 | `data/cities/miami/metric-notes.ts:85-88` | `pct_*` shares "from the NCES Common Core of Data … the 2023-24 vintage is the latest published". | Loaded from the FLDOE race file 2018-19 → 2025-26 with hidden-cell allocation (`race_membership.ts`); CCD is only the cross-check. Also `:20` says blank School Grades components are "stored as suppressed" but `school_grades.ts:346` writes no row for blanks. |
| S9 | Number slips | `index.ts:65,132` k÷g5 "0.86–0.89" → packet 0.845–0.881; `index.ts:62`, `recipes.ts:40` "95%" → `exact_share` 0.9833 on 1,018 school-years in two CCD years; `index.ts:42,50` "31 unlisted … identifiable by name" → Table 7 lists only the 35 flagged, the histogram has no names; label "lift table, §4" → Table 4 in §3; `recipes.ts:60`, `index.ts:105` 2022-23 caveat blames missing letter grades (306 vs 305 schools) when the real reason is C4; "controls explain 11%" mixes `controls_only.r2` (0.110, n 305) with the parent regression's `r2_controls` (0.105, n 256) and "adds another 11%" is partial R² (raw increment 9.5 points); `index.ts:110` "utilization … a weak predictor" → coef 0.00194 (t ≈ 2.9), ≈ +4.6%/SD vs +6.9% for parents; `metric-notes.ts:59` "6% larger" → measured +4.9%; `index.ts:64` "high schools least (−9%)" → 6-12 (−7.4%) and K-12 (−6.3%) fell less; `index.ts:178` "Both files are served here" → EDFacts is not served as a metric; `enrollment/page.tsx:234` scholarship "roughly a quarter" is unsourced; `residual-z/page.tsx:117` P3 "fails (small fits, low R²)" → also fails for science 2024-25 (n 426, R² 0.49). |
| S10 | Stale / mislabelled UI text | `app/miami/methodology/page.tsx:128-130` "What is not here yet: climate survey, residual-z, stories … in progress" (all shipped and linked above); `app/miami/page.tsx:31` "Ratings … follow once test-score loads pass validation" (live); `components/peer-pill.tsx:68` city tooltip says "~2,000 NYC public + charter schools" on every Miami page; `[metric]/page.tsx:62,113` `inputsOf` renders "computed by sesir.ts from sesir_total_per_100" for derived sesir/survey metrics; `climate-survey/page.tsx:40,71` "district-run" denominator 385 includes adult, technical, DJJ and OTHER (321 of 329 with-value schools are REGULAR); `docs/cities/miami/sources.yaml:315` `survey_student_*` (singular) so the 2024-25 student workbook backs no served metric in `metric_index.json`; `survey_items.csv` canonical parents_37 with text "nan" (36 rows from a NaN item number). |

### 1.4 Latent and minor code robustness

- `scripts/loaders/miami/essa_ppe.ts:36` district filter lacks `.replace(/\.0$/, "")`; the other eight loaders have it. A numeric `13.0` cell would silently drop every row.
- `scripts/loaders/miami/compute_percentiles.ts:12-21` dropped the reconnecting `query` loop that NYC/Philly keep for a stated reason (NYC header lines 13–15).
- `scripts/loaders/miami/residual_z.ts:123-126,142-143` reset covers only metric keys present in the CSV, so a metric dropped from the engine keeps stale residual columns; `missing` counts z across all metrics.
- `membership.ts:43` header check accepts 13 of 14 grades; a missing grade column writes 0 (`r[undefined]` → blank → 0). All ten files have 14 today.
- `ccd_membership.ts:139,183-190` + `race_membership.ts:421`: 6 school-years (42 cells) carry CCD-derived `pct_*` with denominators 3–9 next to state-derived rows; race_membership skips those schools so they are never overwritten.
- `nces_ccd_membership_2022-23` is fetched (208 MB) but never loaded; no `enrollment_oct_ccd` for 2022-23.
- `school_events.ts:192-207` deleteMany then per-row create outside a transaction.
- `_lib.ts:72-78` `readLatestSourcePath` returns `readdirSync(...)[0]` unsorted (every dated dir has exactly one file today).
- `compute_percentiles.ts` / `derived.ts` never clear percentiles or derived rows for cells that later become null/suppressed (0 stale today).
- `metric-definitions.ts:187` registers `college_career_acceleration` but `school_grades.ts` COMPONENTS never loads it (0 rows); `miami_schools.latest_enrollment` populated for 0/756.
- `lib/queries/miami-schools.ts:82` `DESC` puts NULLs first for a hand-typed `sort=z` under the peer method (1,067 fitted cells have no peer percentile) → page 1 all "—". Fix `nulls: "last"`.
- `app/miami/ratings/outliers/page.tsx:36-39`, `filters.tsx`, `movers/filters.tsx`: under `method=residual` the metric dropdown lists all 59 rated metrics but only 12 have fits, and the year list includes unfitted years; changing metric does not clear `year`/`from`/`to`.
- `lib/miami/evidence-version.ts:27-37` does not register `__outcomes_validation.json` or `metric_index.json`, so those links carry no `?v=` and the 409 guard cannot catch a stale citation.
- `components/analysis/miami-charts.tsx:141,154,186`, `miami-packet.ts:10` hard-code "2024-25"/"October 2024"/`MIAMI_YEARS` and Table 7 relies on `enrollment_2024_25` field names; consistent today, stale on the next packet run.
- `components/layout/school-search.tsx:51,60` two `react-hooks/set-state-in-effect` eslint errors (pre-existing pattern; the Miami commit only added the bucket).
- `docs/cities/miami/probe/race_imputation.py:174-186` `token_census` counts blank cells as suppressed, contradicting its docstring (probe display only).

### 1.5 What was checked and holds

Loaders reconciled cell-by-cell against source files (3–6 cells each) and
against an independent re-read: absenteeism (all 9 files resolve the same
columns; fractions ×100; n/enr matches %), ESSA PPE (positional columns all
4 years; col4+col5 = col6, col6+col9 = col10 on every Dade row), ADA/ADM,
graduation (2018-19 race header variant handled; `*` → suppressed), school
grades (component prefixes in every year and all four 2022-23 sheets;
204+77+83+106 = 470, no cross-sheet duplicates), SRS (level-3 header unique;
2023 science layout; science_g8), membership apart from C8 (14 grade columns
in all 10 files; 2022-23 `*` → 0 rule; 2025-26 partial suppression),
race_membership apart from C10 (shares sum to exactly 100 in all 3,081
school-years), ELL apart from C12, derived (utilization, 5-yr change, K÷G5;
2025-26 fallbacks = 325 FISH carry-forward + 4), FISH/ABC (arithmetic
identities on 100% of rows), events (56 = 56), directory (dates 629/629 and
209/209; charter 232). Every metrics row is owned by its source's latest load.

Percentiles: direction correct, 0–1 scale matches PeerPill, no stale, no
citywide for excluded schools. Peer groups: 417 groups per k, no hard-filter
violations, no self-inclusion.

Residual-z: `pivot_table(aggfunc="first")` safe under the unique key; binomial
and Poisson SE algebra correct; LOWER_BETTER orientation correct; P1–P5
computed as documented (P3 is unweighted Spearman and does fail: science
2024-25, graduation, sesir); P2 fails in every ELA/math year and is reported as
a negative result on the page; ablation refits with the same weights; R2 and
R3 are real gates; CSV 18,297 = DB cells with z. Survey→enrollment
regression: design full rank, coefficient/SE/p and partial R² reproduced by an
independent QR solve (0.00549 ± 0.00101, p 1.5e-7, partial R² 0.1065).
Pivot-cache reader for 2021-22 verified field by field.

Enrollment packet: Fisher p-values equal scipy for all 7 cuts; lift cells
54/23 vs 206/12, universe 260/35, utilization n 316 median 66.7, excess cost
$20,484,039 (26 small ES, reference median $14,051 over 47), 2025-26 sector
totals all recomputed exactly in SQL. CCD vs state enrollment 2023-24: 518/518
within 5%.

App: `tsc` clean; 51/51 unit tests pass; no Miami page routes PERCENT through
the NYC fraction-rescaling formatter; every number on the enrollment,
residual-z and climate pages recomputes from the packets; robots gates read
the right verdicts; every registered artifact link is versioned; all 30
linked source-doc targets exist; `generateStaticParams` matches the registry
with 0 orphan keys.

### 1.6 Why the harness missed these

The harness is strong where it was designed to be strong (cell-level
reconciliation of a loader against a re-read of the same file) and blind in
four specific ways, each of which let a critical defect through:

1. **Shared inputs in "independent" checks.** The survey reader shares SPECS,
   crosswalk, scale rule and n with the parser (M2); R1 shares n and
   eligibility with the engine (M1). C3, C5, M4 and the C1 scale all pass.
2. **No served-value invariants.** Nothing asserts 0 ≤ PERCENT ≤ 100 (C3, C7),
   total ≥ max(category) and total ≈ Σ categories (C1), or response rate ≤ 100
   (C2).
3. **No year-over-year distribution alarms.** C8 (2017-18 school count drops
   by 32), C10 (a small-group median jumps from 0 to 0.96 for one year),
   M6 (response-rate means alternate 24/58/30/52/57) and C4 (a metric's mean
   halves for one year) are all visible as one-year shifts in a per-metric
   median/count series.
4. **Completeness is checked for one metric.** Served-only and source-only
   cells never fail (M10).

Recommended additions, in the harness rather than in loaders: a
`served_invariants.py` step (range per unit; total-vs-components identities;
per-metric per-year count and median shift flagged at > 3 MAD from the
adjacent years, reported as a lift over the metric's own history); make R1
recompute n from `enrollment_oct_g*` under `outcomes.yaml`; make the survey
reader recompute suppression from Ncount and assert bounds; a per-component
served-only allowance in reconcile.

### 1.7 Which published conclusions survive

- **Enrollment/closure packet (Tables 1–7, Fisher, lift, excess cost):**
  survives. C8 dents the 2017-18 trend point and the 2022-23 five-year-change
  row count; the lift table uses 2024-25/2025-26 and is untouched.
- **Survey → enrollment (parent recommend +6.9%/SD):** the 2021-22 estimate
  reproduces independently and the 2021-22 file's recommend column is
  per-school constant (not affected by C2). But the pre-registered second
  cohort is silently absent (C4), so the result is one cohort's, and the
  story text must say so until 2022-23 is re-parsed.
- **Residual-z (P1–P5, ablation, flags):** every z will move after C9 (fit
  universe), C1 (sesir scale), M3 (variance formula) and M4 (science n).
  The P2 negative result (ρ(z, gains) < ρ(raw, gains)) is a statement about
  correlation ordering and is likely robust, but it must be re-run, not
  assumed. The ablation ordering (ELL, level, econ) is measured on the current
  fit and may change.
- **Climate composites and ratings on `survey_*`:** not trustworthy until
  C2–C7 and M5 are fixed; `survey_*_resources` is empty; 2018-19 has served
  zeros; 2024-25 has values above 100.
- **SESIR total and its residual flags:** wrong by 2× (C1); category rates
  are right.
- **Race shares 2022-23 small groups:** wrong (C10); 2018-19 and 2023-24 are
  within the CCD bound and the other years show no shift.

---

## Part 2 — Duplication and shared code

Measured with two instruments: identical-line count (`diff`) and token
similarity (difflib ratio on comment-stripped, whitespace-collapsed text with
city names and id identifiers normalised). Token similarity is the honest
number where Miami's one-line style defeats line diffs. Git lineage confirms
copy-paste rather than convergence: Philly `_lib.ts` was created from NYC's,
Miami's from Philly's; two NYC percentile fixes were re-applied to Philly by
hand, and Miami's copy dropped the reconnect loop.

### 2.1 Evidence

| Area | Pair | Identical lines | Token sim | Note |
|---|---|---|---|---|
| Loader lib | NYC `_lib.ts` (86) vs Philly (217) | 74 | 0.41 | `gitSha`, `batched`, dotenv preamble byte-identical ×3 |
| Loader lib | Philly vs Miami `_lib.ts` (309) | 128 | 0.43 | `startDataLoad/finishDataLoad` differ only in delegate and `sourceName`/`sourceId` |
| Suppression tokens | 9 definitions, 4 token sets | | | NYC `_fetch.ts:103`; Philly 6 per-loader `toNumber`; Miami `parseCell` (only one distinguishing blank from token); Python ×4 |
| Percentiles | NYC vs Philly `compute-percentiles.ts` | 146/198 | **0.97** | `rank()` byte-identical ×3 plus a 4th copy in `compute-proclivity.ts` |
| Percentiles | Philly vs Miami | | 0.78 | Miami dropped the reconnect loop |
| Peer groups | NYC vs Philly | 79 | 0.54 | Same skeleton; three distance rules; Miami filters the **subject** on `includeInDefaultComparisons`, NYC/Philly the candidates |
| Python sidecars | Philly vs Miami `discover.py` | 117 | 0.61 | Miami is a strict superset |
| Python sidecars | `load_env_local` ×4, `served_db_dsn` ×5 | | | Three reconcile suppression policies (Philly served-side, Miami both-equal, NYC either) |
| Python sidecars | Philly vs Miami `reconcile.py` | 29 | 0.46 | Identical summary key set; Miami's `reader:` dispatch is already generic |
| Stories | Philly vs Miami `[id]/page.tsx` | 49/69 | **0.94** | Differ only in registry import, basePath, badge, Miami's evidence block |
| Stories | `recipe/page.tsx` | 35/48 | **0.99** | `Story` type declared 3× |
| Ratings | movers `filters.tsx` | 38 | 0.87 | Whitespace plus Philly's `cut` select |
| Ratings | outliers `page.tsx` | 60 | 0.67 | Same params/sort/pagination shell; different label columns and filters |
| Queries | `getPhillyOutliers` vs `getMiamiOutliers` | | | Same algorithm and memo wrapper; movers SQL identical modulo table |
| Metric registry | NYC / Philly / Miami | | 0.20 | Same `Def` shape; Philly's cannot be imported by the app (lives in the seed script and binds Prisma) |
| Methodology pages | Miami templated `[metric]` (254) vs Philly hand-written chronic-absenteeism (1,342) | | | Philly has one page for ~40 keys |
| Survey ingest | three architectures | | 0.16–0.24 | Same output contract `survey_{respondent}_{domain}`; NYC has two parsers of the same files |
| Residual-z | Miami engine vs scaffold `engine.py` | | 0.13 | Re-implemented rather than imported; Philly has no engine |
| Miami loaders | 9 per-outcome loaders pairwise | | mean 0.45 | Skeleton (norm, discovery, header search, district filter, pad+universe, parseCell skip, load bookkeeping) copied 6–9×; `norm` has three charsets; `essa_ppe` is the divergent district filter |

Not meaningful duplicates (negative results): `philly-packet.ts` vs
`miami-packet.ts` (0.04), `reconcile_computed.py` pair (0.11), the two
`qa-reports.ts` (0.26), and the school pages (0.38) share only a shell; a full
merge of any of these would regress both cities.

### 2.2 Does the Prisma schema block sharing?

No. For everything written in raw SQL or Python (percentiles, movers, the
verify harness, the residual engine, the metric index) table and column names
are configuration. For Prisma-client code (loader upserts, peer groups,
`lib/queries`) the distinct delegates and unique-key names force a per-city
adapter of roughly 40 lines each, which removes the block without a
migration. What is preventing sharing today is copy-paste.

Schema options, assessed:

- **A (recommended now):** keep the three families; add the adapters.
- **B (two-family):** Philly and Miami are the same shape except Philly's
  `populationCut` and Miami's residual columns. Add `population_cut TEXT NOT
  NULL DEFAULT 'n/a'` to Miami and the residual columns to Philly (both
  additive, small). Do this when Philly gets the residual engine.
- **C (single `school_year_metrics(city, …)`):** requires a union subgroup
  enum with a rename map, resolving NYC's mixed 0–1/0–100 PERCENT storage
  (`lib/format.ts:10`), hand-written migrations (there is no versioned
  migrations directory to build on), a `city =` predicate on every query and
  index (a new bug class: "citywide" silently becoming "all cities"), and
  touching 10 NYC bulk-upsert sites, 8 Philly and 21 Miami loaders. Multi-week
  with high regression risk and little code-sharing gain over A. Only if a
  fourth city arrives, and then build that city on the unified shape rather
  than migrating NYC.

### 2.3 Decisions to record before refactoring (not inside it)

1. **Reconcile suppression policy.** Three today: Philly counts a
   served-suppressed cell as suppressed regardless of the source; Miami
   requires both flags equal else mismatch; NYC treats either side suppressed
   as suppressed. Miami's is the strictest and is the one that would have
   caught C5 had the reader been independent. Proposal: `both_equal` as the
   default, per-city override in `components.yaml`.
2. **Peer-group distance and subject inclusion.** NYC raw Euclidean + topic
   Jaccard; Philly raw Euclidean on percent; Miami z-scored Euclidean with log
   enrollment. Miami excludes non-default subjects from having a group; NYC
   and Philly only exclude them as candidates. Unifying either changes
   rankings in every city; keep `distance` configurable and make
   `subjectMustBeIncluded` an explicit flag, then decide per city.

### 2.4 Ranked refactors

Payoff ≈ lines removable or capability gained; effort in days; ratio is
payoff ÷ effort.

| # | Refactor | Payoff | Effort | Guard |
|---|---|---|---|---|
| 1 | Shared story routes: `data/stories/types.ts` + `components/stories/{story-page,story-list,recipe-page}.tsx` taking a registry; each city's three routes become 5–8-line wrappers | −320 lines at 0.94–0.99 sim; NYC can join | 0.5 | routes unit test + render smoke test with a two-story fake registry |
| 2 | `compute-percentiles.ts --city` with a `PercentileCityConfig` (tables, id column, extra key, combo filter) and `_common/rank.ts` used by `compute-proclivity` too | −300 lines; restores Miami's reconnect; 4 `rank` → 1 | 0.5 | `COPY` percentiles per city before/after, expect byte-equality; unit-test `rank()` on ties |
| 3 | `scripts/loaders/_common/` (city config, `gitSha`/`batched`/provenance readers, one `parseCell` with per-city token additions, `makeFactWriter`) applied to Philly's 8 inline upsert loops and 6 `toNumber`s; fold `sdp_pssa_keystone_2023-24.ts` into a year argument | −430 lines; one suppression definition in TS | 1.5 | each city's `reconcile --all` stays PASS; vitest for `parseCell` (blank ≠ token is load-bearing for `school_grades.ts:67`) |
| 4 | `pipeline_common/{env,sources,verify/served_db,verify/reconcile}` with a frozen `City` dataclass; adopt Miami's `reader:` components schema; explicit suppression policy (decision 1) | −330 lines; 4 `load_env_local` + 5 `served_db_dsn` → 1; unified `__validation.json` so the TS `qa-reports` types merge | 2 | golden test of `reconcile_one` per policy; diff committed `__validation.json` per city |
| 5 | `discover.py` → `pipeline_common/discover.py` (Miami's superset + `City.url_rewrite`/`prime_session`) | −180 lines; Philly gains `--force`, pdf/json inspectors, header detection | 1 | re-run discover for both cities, diff findings |
| 6 | Miami `TableSpec` + `dadeRows` iterator: 5 of 9 loaders become specs (SRS, absenteeism, graduation, ESSA PPE, school-grades components); membership, race, ELL, SESIR keep custom code over the iterator | −150 lines; kills the `essa_ppe` filter-divergence bug class; next FLDOE source ≈ 20 lines | 2 | 25 harness components + fact-level parity dump per loader |
| 7 | Metric registry as pure data per city + shared `components/methodology/metric-page.tsx` (today's Miami page with the three Miami branches behind `qa.crossPublisherFor`) + `export_metric_index --city` | Philly: 1 → ~40 metric pages with QA banners; one seed script | 2 | vitest: every loaded key has a registry entry and vice versa; `generateStaticParams` count snapshot |
| 8 | Ratings pages + queries via `CityRatingsAdapter` / `CityDb` | −420 lines at 0.67–0.87 sim | 3–4 | snapshot `getOutliers` for a fixed (metric, year) per city |
| 9 | City-agnostic residual-z engine built **as Philly's engine** (`FrameSource` protocol; P-checks in `outcomes.yaml`); add residual columns to Philly (schema option B) | avoids a second 384-line engine; brings the scaffold's View-2 trends | 3 | R1/R2 generalised with `City`, run for both |
| 10 | Peer-group engine with `PeerSpec` (decision 2 first) | −275 lines | 2 + decision | dump `(school, k, sorted peers)` before/after; set-equality when distance and inclusion flags match |

Suggested order: 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9 → 10. Items 3 and 4
produce the TS and Python city adapters that 6–9 consume; 1, 2 and 5 are
independent. Two deletions need no refactor: the NYC TS survey parser that
duplicates `survey_to_csv.py`, and `sdp_pssa_keystone_2023-24.ts`.

Survey ingest (three architectures) is deliberately not in the top ten as a
merge: the parsers are per-publisher and irreducible. What can be shared is
the intermediate (`school_id, respondent, year, item_no, item_text,
pct_positive, n_responses, suppressed`), Miami's crosswalk builder (after M5
is fixed) as `pipeline_common/survey/crosswalk.py`, and the composite loader
with a `FactWriter` injected. NYC's regex `MAPPINGS` would become a committed,
reviewable `survey_items.csv`; some NYC values would move, which is a
documented diff, not a regression.

Caveat on the measurement: line counts proxy maintenance cost, they do not
measure it. The stronger argument in each area is the propagation record:
percentile fixes re-applied by hand, Miami's reconnect loop lost, the
`essa_ppe` filter diverging, and Miami's stricter suppression policy not
reaching the other two harnesses.

---

## Part 3 — Proposed fix order (as executed; see 09_review_fixes_2026-09-16.md)

Nothing below has been done.

**Batch A: served numbers (C1–C13).** Fix loaders and parser; reload SESIR,
membership 2016-17/2017-18 + derived, race 2022-23, climate survey (all
years), ELL, CCD staff; remove `fishCapacityPermanent` from the directory
update payload. Then re-run the harness, the residual-z engine (with C9
removed), the survey→enrollment test (now with a real 2022-23 cohort), the
enrollment packet, peer groups and percentiles, and regenerate every packet
JSON. Estimated 1.5 days including re-validation. Every served number that
changes will be listed in the commit.

**Batch B: method and harness (M1–M12 and §1.6).** Independent n in R1;
independent survey reader with bounds and Ncount-based suppression; τ² with
(n − k) correction; science n-rule 9–10 with the choice added to the ablation;
crosswalk stemming/alias table and re-derived `core`; students response-rate
denominator break marked; race bound using the loader's hidden count and a
suppression threshold for high-U schools; PK-suppressed handling; regime
boundaries vs flagged years; served-only allowance; funnel counted before the
join; peer-group fallback for <6-candidate cells. Plus the new
`served_invariants.py` step. Estimated 2 days. Re-run everything from A.

**Batch C: text and docs (S1–S10, §1.4 UI items).** Rewrite the affected
story sentences and recipes from the re-run packets, fix the notes, the stale
page text, the tooltip, `inputsOf`, evidence-version registration, dropdown
filtering, NULLs-last sort, `sources.yaml`. Estimated 0.5 day, after A and B
so the numbers are final.

**Batch D: refactors 1–5** (low risk, ~5.5 days), then 6–7, then decisions
for 8–10.

Order rationale: A first because readers see those numbers today; B before C
because the story text depends on re-run results; D after the numbers are
stable so the parity guards have a correct baseline to compare against.