Miami build: second adversarial review, correctness only (2026-09-16, after the fixes)

Status. Every finding below has since been applied on the branch; §7 records what was done and what it measured, including two further defects found while applying (a float overshoot in the pivot-cache shares that the loader rejected as invalid, and the reworked shift alarm's first pass, which alarmed on 190 metric-years). Sections 1–6 are left as written.

Scope: the branch at commit 7b63db4, i.e. the code, data and text after the fixes recorded in 09_review_fixes_2026-09-16.md. Refactoring is out of scope by agreement. Nothing was modified for this review; the fixes were applied afterwards (§7).

Method: four independent read-only reviewers (TypeScript loaders and the enrollment packet; the Python parser, engine and harness; the app, stories, notes and docs; the statistics and inference), each told to treat the fixes as the most likely place for new bugs and not to trust the 09 record. Every critical claim below was then reproduced by me against the served database or the committed packets; where two reviewers reached the same number independently that is noted.

Bottom line: the fixes hold where they were checked (every loader re-parsed cell-for-cell, the residual engine refitted from scratch, the packet recomputed, 30 base cases at zero mismatches). Five new problems change a served number or a published conclusion. The largest is not a code defect but a publisher one that the pipeline accepted: the 2021-22 staff survey workbook's per-item values are shifted one item against their labels, so all four staff composites for that year are built from the wrong items, and the "substance abuse" exclusion recorded yesterday misdiagnosed the symptom. Two of the analytical sentences added yesterday overreach: the utilization control for the 2021-22 survey cohort sits on the outcome path, and the "parent sentiment does not replicate" verdict is stronger than the numbers allow. Two claims made in the 09 record are false and are retracted here.

1. Critical: a served number or a published conclusion is wrong

2. Major: an inference the numbers do not support, or a method or harness defect

3. Minor

Text and numbers that disagree with the data or the code:

• Residual-z page names science as a P3 failure; science passes P3 in every year (|ρ| ≤ 0.076). The failures are graduation (both years), SESIR, and chronic absenteeism 2024-25.
• metric-notes.ts:87 quotes the pre-cap 2022-23 race error "0.9–1.2 pp"; the report now reads 0.89 / 0.66 / 0.84.
• app/miami/methodology/metrics/enrollment-capacity/page.tsx:80-81 and the gis_schoolsite.ts load note still say GIS capacity equals FISH "for seven of nine schools"; the registry says 200 of 326.
• Story 1: "(74 schools below 50%)" for the state measure is over 316 schools of all levels; on the same 260 it is 60. "12 listed schools are more than half full": one (Golden Glades, 210/420) is exactly half; "at least half full".
• Story 2: "the three years both publish" — CCD publishes every year; three were loaded.
• Story 3 recipe caveat says the ESSA figure "includes allocated district indirect costs" while the query and bullet use the school-level series that excludes them.
• Story 4 recipe pulls only the parents' measure although the story's robust finding is the students'.
• Story 5 and 3 recipe queries as written return 475 and 498 rows (no sector, type or suppression filters); the story numbers need the filters the packet applies.
• Story 8: "the pattern holds in every year since 2018-19" — charter participation among regular schools with grade-4+ students was 87% in 2018-19, 74% in 2019-20, 49–66% since 2020-21; "median around half" for the students' response rate — 2024-25 median is 60%.
• Enrollment page: Fig 6 footnote "n = 316" but 307 points are drawn (6-12, K-12 and Other dropped); Fig 4's "50% (the level below which the district sent 'reassessed' letters)" is not a rule the events support (4 of 12 reassessed schools are at or above 50%).
• 09 record: "pct_amer_indian 2022-23 mean 0.09" is 0.10; the response-rate history "37–41%" then "49–55%" matches neither medians nor means; "eslint clean" is one warning short.
• README status block: pre-fix z formula, "88 files fetched", "~340,000 students".
• college_career_acceleration has no metrics_backed entry in sources.yaml, so its metric page says it "is derived"; utilization_permanent_pct is backed only by the board-book entry meant for its _abc twin. The registry description omits that the component is the state's lagged (prior-year) measure stored under the grade year.
• mean_sampling_se in the packet is the unweighted RMS of se_i while τ² subtracts the weighted mean (ELA 2018-19: 2.67 reported, 2.05 used).
• ineligible_low_n excludes cells with an undefined N (science 2024-25: 19 not fitted, 18 reported).
• Peer-group description string says "hard filters school level + sector + magnet" on the 13 relaxed groups (only methodology_notes.hard_filters is right; neither is rendered).

Code that is wrong or latent:

• survey_students_response_rate is served as 0 (unsuppressed) for two 2018-19 schools whose workbook reports 0 respondents but publishes item shares (7062, 8151); 603 response-rate cells are served with fewer than 10 respondents while the note says the district withholds under 10.
• sesir.ts category regex SEXUAL counts SEXUAL HARASSMENT in both violence and bullying (68 incidents, 26% of the bullying category) while the note lists only "sexual battery, sexual offenses".
• race_membership.ts:163: the overflow branch re-spreads the remainder over all hidden keys by cell count, including groups already at the cap and zero-prior groups (0 overflow cases today); adjacency is by list index, so a year that fails parseYear would silently make a non-adjacent year the prior.
• Re-run ordering: school_grades.ts and fish_los.ts refresh the school record from the last year processed, so running an older year alone overwrites it; finishDataLoad drops the "pruned N" note when the caller passed none.
• Packet utilization.state includes two district-run schools with COFTE ≈ 0 (5931, 7031), pulling the state mean, p10 and the headcount-vs-state correlation down slightly.
• RENORM_TOLERANCE_PP = 1.5 is derived for whole-percent files but applied to files published at 17-decimal precision, where any deviation is a real not-answered share; rows with |sum − 100| in (0, 1.5] are left as published (76 / 36 / 65 / 110 / 6 rows across five files, agreement understated by ≤ 1.5% relative) and integer files contain sums of 102 that three roundings cannot produce.
• Suppression-flag literal list omits low_ncnt (students 2023-24); no served effect because the 105 flagged schools are exactly the 105 with n < 10.
• export_metric_index._pattern mishandles * and {a,b} (not triggered).
• Per-item respondent counts below 10 are never suppressed when the school median is ≥ 10 (457 rows at 293 schools; non-core items today).
• Crosswalk: parents_06 / parents_37 ("adequate resources (textbooks, equipment…)" vs "makes available textbooks, equipment and supplies") are one item split across years (Jaccard 0.18, below the candidate list's 0.3); the stemmer maps class → "clas" but classes → "class".
• sort=grade on the outliers page has no NULLS LAST (257 fitted cells have no letter).
• Movers page does not validate metric against the list.
• directory.ts stores titleI = false rather than null for schools absent from the grades file.
• Unregistered evidence links: the two cross-publisher reports are linked from the metric page without ?v=.
• Checks with limited power, documented but worth stating on the page: the harness's survey reader copies the parser's conventions (tolerance, scaling, flag list, Q00 fallback), so it cannot detect a wrong convention or a label/value misalignment such as N1; residual_checks copies the τ² and se formulas, so it verifies implementation, not method; source_only_allowance defaults to unlimited.

4. What was checked and holds

• Every loader re-parsed with the current code and diffed against served cells: membership 10 years (0 differences), race 2022-23 recomputed for all 520 schools from the file, served totals and adjacent-year shares (0 mismatches, 0 cap violations, 0 overflow), survey composites for four school-years to 4 dp, SESIR three schools, absenteeism and ADA (fractions ×100), ESSA positional columns in all four files, school grades (per-sheet 2022-23 indexes, gains before L25), graduation (provider rows skipped), SRS (63 files), ELL (no all-zero row in any year), CCD membership and staff, FISH/ABC/GIS, derived (0 mismatches on 6,937 cells), residual load (19,457 = 19,457), events (56).
• Ownership: every metrics row belongs to its source's latest load; 0 orphans; no cells in years a loader did not cover.
• Residual engine: from-scratch WLS with N re-derived from served grade columns reproduces eligible sets, betas, τ, R² and flags for four outcome-years (max |Δz| 5e-12); the τ² moment estimator is right under weights n (tr(W M Σ Mᵀ)/Σw·se² = 0.96–0.97 ≈ (n − k)/n, so the dof correction belongs on s² only; the engine's τ is 0.7–1.2% low against the exact estimator); persistence recomputed independently (227 cells, 0 mismatches); flip-diff, P-checks, flag shares all reproduce; the ablation's n_rule= rows reproduce; the science weight (Biology testers 0.73–0.80 × grade-9 membership; grade 5 and 8 testers 0.86–1.01 × membership) is the right order.
• Survey regression: all 12 predictor rows and both controls-only rows reproduce to 1e-5; the funnel and coverage universes (385/157, 322/140, 314/130, 310/76) verified by SQL.
• Enrollment packet: every statistic recomputed from the served tables; Fisher p-values against scipy to 1e-6; sector totals, pipeline ratios, change windows, utilization, lift cuts, cost figure ($20,484,039 as the per-school sum).
• Race shares vs CCD: mean errors 0.89 / 0.66 / 0.84 (2022-23) and 0.83–1.03 (other years) reproduced from the CCD zip; 0 uniform splits; every school-year sums to 100.
• Crosswalk rebuild is byte-identical to the committed file; core sets identical across years except the recorded 2021-22 staff exclusion; every alias group maps to one canonical; every reverse-coded item is genuinely negative.
• App: tsc clean, 51/51 unit tests, eslint one warning (unused variable); every story evidence link resolves; Table and Figure references match the page; residual-method dropdowns list fitted metrics and years; year cleared on metric change; peer, city and z sorts NULLS last.

5. Which published conclusions change

• Staff climate composites for 2021-22 (all four domains, 2,517 cells) are wrong and must be re-derived from re-aligned items; the 2021-22 staff leadership predictor in the survey test moves from +3.42% to +3.43% per SD.
• The parent-survey verdict softens: on the same schools the 2022-23 effect is about half of 2021-22's and still significant; the cohort difference is marginal and the parent measure is the noisiest predictor. The students' result is unchanged.
• The utilization sentence in the survey story goes; with a control off the outcome path, utilization carries about +1% per SD in the 2021-22 cohort.
• The cost-premium figure stays at $20.5M but must be described as a per-school sum ($3,356 per student in the small schools), or recomputed as the median gap × students ($17.3M).
• The adjusted-scores story keeps its negative result but loses its stated mechanism, and its ablation ranges are wrong.
• Two claims in the 09 record are retracted: that the invariants step would have caught the review's critical findings, and that the 2021-22 staff substance-abuse item had a reversed answer scale.

6. Proposed fix order (nothing applied)

1. N1: a per-file label_offset in the staff 2021-22 spec (values on row Q-n belong to Q-(n+1); drop Q36), remove the Q25 exclusion, re-parse, reload, recompute percentiles, re-run the survey test; add the pivot-cache years to the harness's own column map so they are base-cased.
2. N2: clear both percentile columns at the top of compute_percentiles.ts and gate the school-page pills on !suppressed.
3. N3: 2021-22 utilization control = 2021-22 headcount ÷ 2022-23 permanent capacity; republish; rewrite the utilization sentence.
4. N10: refuse the prune when nothing was written or any upsert failed the foreign key; make upsertFacts throw on FK failures.
5. N4, N5, N12, N13 and the §3 text items: rewrite from the packets; add the missing recipe filters; register the cross-publisher reports.
6. N6, N7, N8, N9, N11: the method text and checks — common-school rows and the bootstrap difference in the survey packet and page; the P2 identity and the L25 comparison on the residual page; per-kind allowlist keys, a SESIR total-vs-file identity and scale-aware floors in the invariants; relabel the race bound as a consistency check and add a held-out selection; treat attendance as scale: none.

7. Applied (2026-09-16, later the same day)

Order followed: §6. Every number here is from the re-run after the change.

N1 (staff 2021-22 offset). SPECS["staff_2021-22"] carries label_offset=1; apply_label_offset re-attaches row Q-n's shares to item Q-(n+1) with that item's label and category, drops the Q-36 row's values (no label) and emits a value-less Q01 row per school so the item still counts as asked; survey_item_exclusions.csv now holds that Q01 row (values not in the file) instead of the Q25 "reversed scale" row, which was the misdiagnosis. Measured against the adjacent years (2020-21 and 2022-23 means, unsuppressed schools, 30 canonical items): median absolute gap 10.3 → 7.7 pp (the gap between 2020-21 and 2022-23 themselves is 9.8), median correlation across schools 0.57 → 0.71 (0.42 between the two neighbours); 25 of 30 items closer than before, the other five within 1 pp. Staff core set unchanged (31 items, safety 6); the 2021-22 staff composites were re-derived (420 schools served per domain). The harness now reads the three 2021-22 pivot caches with its own reader (_survey_pivot_cache) and its own copy of the offset, and components.yaml base-cases all three 2021-22 composites; the reader's docstring states that this verifies the implementation of the offset, not the diagnosis. That base case immediately found a further defect: the pivot caches store shares such as 100.00000000000001, the parser wrote them verbatim and the loader's [0, 100] check rejected 34 staff 2021-22 rows, leaving nine leadership composites 0.2–2.8 pp low. Shares are now rounded to 6 decimals with overshoots within 1e-6 clipped, and the base case for all three 2021-22 composites passes.

N2. compute_percentiles.ts clears both percentile columns on every row before ranking; the school page shows no percentile or z pill on a suppressed cell. Suppressed cells with a percentile after the run: 0.

N3. The 2021-22 utilization control is October 2021 headcount ÷ 2022-23 permanent capacity (utilization_basis in the packet meta). Its coefficient in the controls-only fit is −0.00018 per point (se 0.0007; about −0.2% of enrollment change per 10 points, not distinguishable from zero) against +0.0086 per 10 points in 2022-23; the parent and student coefficients are unchanged to the third decimal. The "+4.6% per SD" sentence is gone from the story, the page and the recipe.

N4. The packet carries excess_per_student_usd (3,356), median_gap_usd (2,842) and median_gap_times_students_usd (17,344,726) beside excess_usd (20,484,039); the story, the enrollment page and the recipe describe the per-school sum as such and give the median-gap reading.

N5. The ablation bullet is written from the packet: level 19–30, ELL 14–33, economic disadvantage 10–22, magnet 7–19 flag changes across the five ELA fits against 20–37 flags in a year, with the note that a change counts a flag gained or lost.

N6. survey_enrollment.py adds common_schools (both cohorts refitted on the schools in both regressions; 2,000-resample paired school bootstrap of the difference; year-to-year Pearson r of each predictor; median respondents) and Holm-adjusted p-values over the twelve predictor tests. Parents' recommend on the 242 common schools: +7.1% per SD in 2021-22 (t 5.8) and +3.5% in 2022-23 (t 2.9), ratio 0.50, bootstrap p = 0.003; reliability r 0.60 for the parent share against 0.83–0.87 for the student measures and 0.60 for staff leadership; median 54 parent respondents per school against 160–192 students. Holm: 8 of 12 survive (students' recommend and safety and staff leadership in both cohorts, the two parent measures in 2021-22). The page, story and recipe now say "attenuated by about half and still positive on the same schools; the parent measure is the noisiest", with the same-school table and the Holm column, and label the additions as not pre-registered.

N7. The P2 block reports ρ(fitted, gains), sd(fitted), sd(resid) and fails_by_construction (true in 6 of 8 fits: ρ(fitted, gains) 0.41–0.72 with sd(fitted) ≥ sd(resid) except math 2023-24 and 2024-25), plus the same comparison against the lowest-quartile gains (not pre-registered): the residual beats raw in 2 of 8 fits (math 2023-24, 2024-25), the demographic gradient in L25 gains is −0.03 to −0.26. The page states the identity and reads the check as a test of whether the state's gains carry a demographic gradient; "because" became "consistent with", with the alternative (growth genuinely differing by composition) named. The console summary's KeyError on the new block was fixed.

N8. The shift alarm was rebuilt and then re-tested with the review's injections. Rule: each year's served median (and school count) against the previous served year's, threshold 3 × max(spread, floor) where spread is the MAD of the series' own year-to-year moves not touching the tested year (fewer than three: the unit's pooled prior, measured at run time — 1.24 pp for percents, 4 schools for counts) and the floor is the larger of the unit floor (0.25 pp; 2.5% of level) and the pooled typical move relative to the series' own within-year IQR (0.059 for percents, measured) times that IQR. First pass (own spread with a 0.25 pp floor only): 190 alarms, every FAST recovery year included; with the IQR-scaled floor 144. Injections on the served rows: ELA 2023-24 +8 pp caught (one new alarm); pct_amer_indian 2022-23 +0.96 pp caught; 25 and 40 schools dropped from 2023-24 membership both caught; SESIR total ×2 still not caught by the identity (a doubled total is still ≥ its categories — that defect is caught in the loader, which now refuses a file whose summed types disagree with its published total, and the docstring says so), SESIR total ×0.6 caught by the strengthened identity (total ≥ Σ categories, 328 violations). The allowlist is keyed by (metric, year, kind) with glob and year-list patterns for district-wide causes (FAST baseline and recovery, the 2018-19 vs 2021-22 gap, the excluded pandemic attendance years, survey participation counts and the remote year, the 2022-23 membership file's blank zero cells, Community Eligibility, ESSER, the CCD school list); stale entries are listed in the report and the 14 legacy entries that matched nothing were removed. Result: 144 alarms, 16 unexplained (staff grade A/B 2022-23; grade-cell medians for g3 2016-17 → 2018-19, g4 2020-21 → 2021-22, g1 2021-22; PK cell count 2021-22; parents' safety 2023-24; six small count moves of 8–17 schools in tested-subject and gains series), left as REVIEW. The 09 record's claim that the step would have caught the first review's findings is retracted there.

N9. The race check's bound is labelled a consistency check in the docstring, the verdict note and the report table; a third statement compares the served share's mean |Δ| from CCD with the equal split's (recomputed in the harness from the re-read file): 2022-23 lifts of 63% (Black), 69% (Hispanic) and 51% (White), identical elsewhere as expected; beats_equal_split is part of the verdict. The probe gained a split-half held-out selection: choosing on the even-numbered schools picks prior_year_cap9 (in-sample 0.83 pp) and its error on the odd half is 0.78, the odd half's own best; choosing on the odd half picks the same rule (0.78 in-sample, 0.83 held out), so the in-sample selection carries no measurable optimism. The loader header and the metric note say so.

N10. finishDataLoad refuses the prune when the load wrote nothing (deletes its own load row and throws) and always records the pruned count; upsertFacts throws on a foreign-key failure instead of warning.

N11. attendance_rate has sampling_error: none in outcomes.yaml; the engine's sampling_se returns zeros for it (z = residual ÷ τ). τ 1.30 → 1.63 (2023-24) and 1.62 (2024-25); flags 27 and 34. The harness's independent refit (residual_checks.py) was still using the binomial model for attendance and reported max |Δz| = 1.9 on its first run; it now reads the same sampling_error key (it verifies the implementation of the declared model, not the model), and R1–R3 pass.

N12, N13. The residual page's persistence sentence says 2019-20 and 2020-21 have no fitted outcome; the science P3 sentence is corrected (science passes every year). The coverage table counts only schools active today in the numerator (with_value_active), and the story-8 recipe filters the same way.

§3 items applied. Race note "0.7–0.9 pp"; GIS "200 of 326" on the page and in the load note; story 1 (60 of 260; "at least half full"; Golden Glades exactly half), story 2 (three CCD years loaded), story 3 (title, thesis, directions, recipe caveat and filters), story 4 (recipe pulls all six measures and the capacity series), story 5 (recipe filters), story 8 (participation history 87% / 74% / 49–66%; median 60%); Fig 6 footnote gives the drawn count; Fig 4's 50% line is described as the lift-table cut with 4 of 12 reassessed schools at or above it; the 09 record's mean (0.10) and response-rate history (36–37% / 50% / 62–67% medians) are corrected; README status block (321,000 students, 178 source snapshots, the τ formula); sources.yaml backs college_career_acceleration with the School Grades files (with the lagged-component note) and no longer backs utilization_permanent_pct with the board book; mean_sampling_se is the weighted RMS; ineligible_low_n counts undefined N (science 2024-25: 19); the peer-group description names the filters actually applied; response rates are suppressed under 10 respondents (0 included); the SESIR violence pattern excludes sexual harassment; the race loader's overflow goes to uncapped cells first and adjacency is by calendar year (0 overflow cases, served shares unchanged); school_grades.ts and fish_los.ts refresh the school record only when the latest available year is in the run; the renormalisation tolerance is derived per file from the number of whole-percent share columns (0.55 pp for the 2023-24 staff file, 1.5 for whole-percent files, 0.05 for decimal ones; 569 / 8 / 483 / 143 / 125 rows renormalised in the affected files); low_ncnt and two more flag spellings; _pattern handles * and {a,b}; item rows with their own count under 10 are suppressed; the parents' textbooks item is one canonical across six years (parents core 34 → 35); the stemmer strips -es only after s/x/z/ch/sh and never a final s after ss (no other crosswalk change); sort=grade NULLS last; the movers page validates metric; the two cross-publisher reports are versioned in evidence-version.ts; the survey reader's limits are stated in its docstring.

Not applied. titleI stays false rather than null for schools absent from the grades file: the column is Boolean @default(false) on the shared schema and changing it needs a migration; recorded here instead. source_only_allowance remains unlimited by default (documented in the component config comments, not changed).

Re-validation. Parser, all three survey loads, peer groups, percentiles, residual engine and load, survey test, enrollment packet, run_all, metric index re-run; tsc clean; eslint clean on the touched files; 51/51 unit tests; next build clean. Verdicts after the run are in docs/qa_reports/miami/ (closure track PASS; outcomes REVIEW on the 16 unexplained shift alarms above, every base case and cross-publisher check PASS).