Find What Works: Schools
← All source documentsRaw file
Source document

docs/agents/pass-log/covid-district-18-comeback.md

Served verbatim from the project repository. Internal working document conventions apply: documents may reference file paths, branch names, and findings-ledger anchors from the repo.

Pass log — covid-district-18-comeback

Story: data/stories/answers.ts → "covid-district-18-comeback"

PASS 1 — Editor (2026-05-29)

WHAT I CHANGED

  • TLDR was four sentences but the lead sentence carried multiple comparison numbers (+32.4 vs +22.2 vs +9.7). Three numbers in one sentence is hard. Restructured: open with the headline finding, push the comparisons into a follow-on sentence.

  • "What changed in D18 around 2022" section is the most newsworthy one and was buried at the end. Kept its position (decomposition belongs before mechanism) but added a forward-pointing sentence at the end of the "D18 trajectory" section to set it up.

  • Superintendent Hardrick is named directly. Verified she is on the public NYC DOE D18 leadership page — naming is OK.

  • "Several of D18's most-improved schools appear to have received new principals 2020-2022" is hedged with "appear to" — too soft. Either we verified this or we didn't. Flagged for the expert.

LEGAL REVIEW NOTES

  • Naming Superintendent Tiffany Hardrick is safe (public role, descriptive attribution of timeline).

PASS 2 — Education expert (2026-05-29)

WHAT I VERIFIED OR CORRECTED

  • Superintendent Hardrick was appointed in 2022: verified via NYC DOE Office of Equity press release. Took office July 2022.

  • D18 Lighthouse inclusion: NYC DOE's "Lighthouse Schools" cluster predates Hardrick (started ~2021 under Banks's leadership agenda). D18 was in the original cohort. Verifiable from NYC DOE press releases.

  • ARP stimulus peak timing: federal ARP money was distributed 2021-2024 with the largest single-year disbursement landing in 2022 fiscal year. Citywide context, not D18-specific.

  • "New principals at PS 233, PS 276 2020-2022": I couldn't confirm this from current public records. The current principals may have started later, or earlier. The story should drop the specific claim and reframe as "several D18 schools had leadership changes in the 2020-2022 window per NYC DOE Find-a-School," which is the safer phrasing.

  • D18 demographics: East Flatbush + Canarsie is heavily Caribbean (Haitian, Jamaican, Trinidadian) families — about 85-90% Black; lower poverty than some Brooklyn districts but not low-income. Important context for why the gain matters.

WHAT I ADDED

  • D18 demographic context (Caribbean diaspora, not just "Brooklyn low-income").
  • Verified Hardrick start date (July 2022) — pinpoints inflection.
  • Reframed the "new principals" line to safer phrasing.

PASS 3 — Quantitative (2026-05-29)

WHAT I VERIFIED

  • +32.4 = 64.7 − 32.3: actual 2024-25 D18 mean is 64.68, 2018-19 mean is 32.25. Verified.

  • The 22-point lead over D19 (#2): verified D19 went 27.8 → 50.1 for a +22.2 gain. D18's lead is 32.4 − 22.2 = 10.2 points, consistent with the decomposition table.

  • Citywide +9.7 over the period: verified against the cityWideMath5ByYear data.

  • 12-of-13 schools improving: verified against d18SchoolGains. Only PS 235 was flat at +2.6.

WHAT I CORRECTED OR QUALIFIED

  • The decomposition table is methodologically clean. The conservative "all of citywide trend" subtraction (+22.7 remaining) is the right framing — gives D18 the most defensible "net of measurement" gain.

  • PS 233 Langston Hughes at +65.7 points is the biggest school-level gain. The same caveat that applies to the punching- above story's single-year jumps applies here: a single school going from 15% to 81% over a 6-year window has cohort composition as a candidate. Verify enrollment/demographics stability before profiling.

  • The "13 schools" sample: small district. With n=13, the +32.4 average has wide confidence intervals — the difference between D18 and D19 (~10 points) is large relative to that noise but not enormous. The story could note this.

WHAT I FLAGGED FOR DOWNSTREAM

  • Editor (pass 4): I added a sample-size caveat. Verify it doesn't undermine the headline finding (which is robust).

PASS 4 — Editor (2026-05-29)

WHAT I CHANGED

  • Reframed the "new principals" line per the expert's correction.
  • Added the Caribbean-diaspora demographic context to the TLDR.
  • Tightened the decomposition table caption.
  • The "+10.2 over D19" point now appears in the body, with the table reinforcing it visually.

PASS 5 — Education expert (2026-05-29)

WHAT I VERIFIED OR CORRECTED

  • One more piece of context: D18 also runs a long-standing "Saturday Academy" program (some weekend instruction). This exists in other districts too, so it's not uniquely D18, but was expanded in D18 around 2022. Worth a line.

PASS 6 — Quantitative (2026-05-29)

WHAT I VERIFIED

  • All numerical claims still hold after pass 4-5 additions.

WHAT I CORRECTED OR QUALIFIED

  • Nothing — story is statistically solid.

PASS 7 — Editor (final) (2026-05-29)

WHAT I CHANGED

  • Final polish: shortPreamble lead-sentence verb ("gained") changed to "added" to match the story's vocabulary elsewhere.
  • Caption polish on the school-by-school chart.

FINAL STATUS

Story is publishable. Word delta: ~1700 → ~1850, gained context on Caribbean-diaspora demographics, Hardrick start date, sample-size caveat.

PASS 8 — Plain-language (2026-05-29)

WHAT I CHANGED

  • shortPreamble + TLDR rewritten. Opens with "School District 18 covers East Flatbush and Canarsie in Brooklyn" — geography first, then the finding. The Caribbean-diaspora context now reads as "predominantly Caribbean-American (Haitian, Jamaican, Trinidadian)" instead of "Caribbean-diaspora population."

  • "D18" pseudo-acronym removed everywhere. Replaced with "District 18" throughout. Same for D19, D23, D26, D2.

  • "NYS test recalibration" → "New York State change in how it scores the math test" or "test rescaling."

  • "ARP stimulus dollars" → "federal pandemic-relief dollars (the American Rescue Plan)."

  • "NYC DOE 'Lighthouse' instructional support cluster" → "New York City's 'Lighthouse' instructional-support program."

  • "NYC DOE Find-a-School" → "records on the city's school-finder page."

  • "Galaxy" budget data → "the city's school-budget portal (Galaxy)."

  • "inflection year" → "the steepest single year of climb."

  • "+32.4 points" / "+22.7 points" → "32 percentage points" / "23 percentage points" throughout (cleaner phrasing for general audience).

  • Section headings rewritten:

    • "All NYC districts ranked by grade-5 math gain" → "Every New York City school district, ranked by math gain"
    • "D18 trajectory year-by-year, vs citywide" → "How District 18 moved year by year, compared with the city"
    • "School-by-school: where the gain came from" → "Where the gain came from — school by school"
    • "Decomposing the gain — how much is test recalibration?" → "Pulling apart the gain — how much is the test change?"

JARGON KILLED

  • D18 / D19 / etc. (replaced with "District 18" etc.)
  • NYS test recalibration (replaced with plain description)
  • ARP (defined inline as "American Rescue Plan")
  • aggregate-only analysis
  • "confidence intervals" (kept as "uncertainty")
  • inflection year
  • per-pupil spend trajectory (kept simpler: "per-pupil spending")
  • Caribbean-diaspora (kept the underlying reference, made it "Caribbean-American" with country examples)

ACRONYMS SPELLED OUT (first uses)

  • NYC → New York City
  • NYS → New York State
  • ARP → American Rescue Plan
  • DOE → Department of Education

NEW PLAIN-LANGUAGE PATTERNS I'M ADDING TO THE KNOWLEDGE FILE

  • "D18" / "D19" / similar pseudo-acronyms → always "District 18," "District 19." Even mid-sentence.
  • "Cohort composition changes can drive aggregate gains" → "A district can post bigger averages just by enrolling a different mix of students."
  • "Decompose the gain" → "Pull apart the gain."
  • Geographic / demographic framing comes first when introducing a district: "District 18 covers East Flatbush and Canarsie in Brooklyn — 13 elementary schools serving a student body that's predominantly Caribbean-American."

WHAT I LEFT ALONE AND WHY

  • School names (P.S. 233, P.S. 276, etc.).
  • "Galaxy" (kept; the actual proper noun for the budget portal).
  • "Lighthouse" (kept; the actual program name).
  • Superintendent Tiffany Hardrick (public-facing official, no change).

Net change: ~1850 → ~2150 words.