Find What Works: Schools
← All source documentsRaw file
Source document

docs/agents/pass-log/ell-gap-disaster.md

Served verbatim from the project repository. Internal working document conventions apply: documents may reference file paths, branch names, and findings-ledger anchors from the repo.

Pass log — ell-gap-disaster

Story: data/stories/answers.ts → "ell-gap-disaster"

PASS 1 — Editor (2026-05-29)

WHAT I CHANGED

  • Headline is the right finding (32-point within-school gap) but the prose was structured as a numbered checklist. Reworked into narrative form, leading with what the finding means before the exact numbers.

  • The "disaster" framing in the story id is editorial. Can't change the id, but the prose should not lean into it — "disaster" is the kind of intensifier the style guide flags. The story title can stay, but the body should describe the gap factually.

  • Section title "Top 15 worst within-school ELL math gaps" — "worst" is editorial. Replaced with "largest."

  • The discussion of mechanisms in "Reading the gap honestly" runs long. Each candidate explanation gets a sentence; the section doesn't need four-paragraph treatment.

LEGAL REVIEW NOTES

  • Naming PS 95 Gravesend as the school with the largest within-school ELL gap is fact, supported by the data. The framing should be descriptive (the gap exists at this school) rather than evaluative (this school fails ELL students).
  • "ELL kids in good schools may be getting worse math instruction than their peers" is the kind of claim that could draw a response. Soften to: "ELL students at the same campuses score 60-85 points lower — for reasons the data alone can't separate."

PASS 2 — Education expert (2026-05-29)

WHAT I VERIFIED OR CORRECTED

  • PS 95 Gravesend (Brooklyn District 21) — Bensonhurst/Gravesend area, heavy Chinese, Russian, and Italian immigrant enrollment. The within-school gap reflects a particular dynamic: a high-scoring never-ELL middle-class population alongside a recent-immigrant ELL cohort with very limited English on entry.

  • District 30 (Astoria/LIC) leading the gap by district is consistent with the demographic mix there — significant middle-class/wealthy zoning alongside heavy recent-immigrant enrollment (Greek, South Asian, Latin American).

  • NYC ENL/bilingual policy context: NYC's bilingual programs have been contracting since ~2010, replaced by "ENL" (English as a New Language) pull-out / push-in instruction. The reduction in bilingual program availability is part of the policy backdrop and worth noting.

  • NYS Spanish-language assessment (NYSITELL/NYSESLAT): the story references these. NYSESLAT measures English proficiency, not content knowledge in Spanish. NYC does not administer the math test in Spanish at scale. Verified.

  • "Years-of-ELL-status" framing: NYC tracks ELL status as current ELL (years 1-6 of services), former ELL (exited services in past 4 years), and never ELL. The story should clarify that "current ELL" averages across years 1-6 — a year-1 arriver and a year-5 long-term ELL are different populations.

WHAT I ADDED

  • Context on PS 95 Gravesend's specific demographic mix.
  • Brief note on the NYC ENL vs bilingual program landscape.
  • Clarification on what "current ELL" actually covers (years 1-6 bundled).

PASS 3 — Quantitative (2026-05-29)

WHAT I VERIFIED

  • 31.7-point within-school average gap: verified across 310 schools. The math holds.
  • The 49.8-point ELA gap vs 31.7-point math gap: verified. Math-ELA gap differential is roughly +18 points, consistent with the language-burden-on-the-test framing.
  • District 30 average gap of 45.4 points across 22 schools: verified.

WHAT I CORRECTED OR QUALIFIED

  • The 310-school sample is systematically larger than median NYC schools: subgroup cell suppression below ~5 students means small schools drop out of the gap analysis. The 310 schools are mostly medium-to-large. This biases the gap analysis toward schools where both subgroups exist in measurable numbers, which is not the same as "all NYC schools."

  • "Current ELL" bundles years 1-6: a year-1 student and a year-5 student are at very different points on the language curve. Aggregating them attenuates the effect of time-in-services.

  • PS 122 Mamie Fay at 72.2-point gap (0% ELL proficient) — note that "0%" with a small ELL cohort means a single high-scoring student would have moved that number materially. The displayed precision is misleading at very small cell sizes.

WHAT I FLAGGED FOR DOWNSTREAM

  • Editor (pass 4): Add a note that displayed precision on small cells can mislead. The 0% / 14% / 19% numbers reflect very small cohorts.

PASS 4 — Editor (2026-05-29)

WHAT I CHANGED

  • Adopted the expert's PS 95 Gravesend demographic context.
  • Adopted the "current ELL bundles years 1-6" clarification.
  • Added a sentence on the sample selection bias (systematically larger schools) and the small-cell precision issue.
  • The "Reading the gap honestly" section is now four short bullets instead of four paragraphs.

PASS 5 — Education expert (2026-05-29)

WHAT I VERIFIED OR CORRECTED

  • Cross-checked: PS 748 Brooklyn Global Scholars (number 3 on the largest-gap list) — Sunset Park, dual-language school by name but in practice serves a four-way demographic mix. The within-school ELL gap here reflects that the dual-language program is one of several enrollment tracks at the school, not the whole student body.

  • D30 specifically: Astoria/LIC has had rapid demographic change since ~2015 (gentrification + continued immigrant arrivals). Within-school gaps there reflect that pace of change.

PASS 6 — Quantitative (2026-05-29)

WHAT I VERIFIED

  • All numbers still verified.

WHAT I CORRECTED OR QUALIFIED

  • Story now reads as quantitatively defensible. The small-cell- precision caveat is in.

PASS 7 — Editor (final) (2026-05-29)

WHAT I CHANGED

  • Final polish: caption on the district-level bar chart now clarifies that the district average is across schools with both subgroups measurable (not all schools in the district).
  • shortPreamble final sentence keeps the structural finding ("the gap is largest where the never-ELL population sits at the ceiling, not where the ELL population is uniquely struggling").

KNOWLEDGE-FILE UPDATE

Added "Subgroup-gap stories systematically sample larger schools" to docs/agents/quantitative-knowledge.md.

FINAL STATUS

Story is publishable. Word delta: ~2100 → ~2200, gained context on PS 95 Gravesend demographics, NYC ENL/bilingual policy backdrop, sample-selection caveat.

PASS 8 — Plain-language (2026-05-29)

WHAT I CHANGED

  • "ELL" / "English Language Learner" / "current ELL" / "never-ELL" → human-readable phrasing throughout. "Never needed English- language services" and "currently in English-language services" replaces every instance of the acronyms. Defined in the first sentence of shortPreamble: "English Language Learners — students who haven't yet reached fluency in English."

  • "Subgroup" → "group of students."

  • "Cohort" → "group" / "students."

  • "Cell" / "cell sizes" → "the number of students in the group" + inline explanation of why small cells make exact percentages unreliable.

  • "Disaggregation" → "breakdown."

  • "ENL" defined inline first use: "the program is called English as a New Language, or ENL."

  • "NYSESLAT" spelled out: "the NYSESLAT, the New York State test that measures students' English proficiency directly."

  • "D2 / D20 / D21 / D26 / D30 / D5 / D15" → "District 2" / "District 20" / "District 21" etc., with neighborhood context inline ("District 30 in Astoria and Long Island City").

  • "Pull-out instruction" kept but explained: "students learning English often receive separate language instruction… during portions of the school day — sometimes during the same period when other students are getting core math instruction."

  • Section headings rewritten:

    • "Citywide ELL gap on grade-5 math (2024-25)" → "Citywide: a 32-point gap in fifth-grade math"
    • "Gap trajectory over time" → "How the gap has moved over time"
    • "Math vs ELA: how much of the gap is the language of the test?" → "How much of the gap is the language of the test itself?"
    • "Top 15 largest within-school ELL math gaps" → "The 15 schools with the biggest gaps"
    • "Where the ELL gap is largest — by district" → "Which school districts have the biggest gaps"
    • "Where the gap is smallest — by admission type" → "Schools with selective admissions have smaller gaps"

  • Caveats block fully rewritten in plain language.

JARGON KILLED

  • ELL / current ELL / never-ELL / former ELL (all replaced)
  • subgroup / cohort
  • cell / cell size
  • disaggregation
  • D-numbers for districts
  • "Bensonhurst middle-class never-ELL population" → "long-established middle-class population alongside a more recent immigrant population just starting to learn English"

ACRONYMS SPELLED OUT (first uses)

  • ELL → English Language Learner (and replaced throughout)
  • ENL → English as a New Language
  • NYS → New York State
  • NYSESLAT → with full clause "the NYSESLAT, the New York State test that measures students' English proficiency directly"
  • NYC → New York City

NEW PLAIN-LANGUAGE PATTERNS I'M ADDING TO THE KNOWLEDGE FILE

  • "ELL" / "English Language Learner" → "students learning English" in flowing prose. In data labels: "currently in English-language services" / "never needed English-language services" — much longer but actually parseable.
  • "Subgroup" almost never reads well — replace with "group of students" or just "students" with a modifier.
  • District 5 / District 15 / District 30 etc. — always spell out "District," even in chart labels.
  • "Cell sizes too small" → "when the number of students in the group is very small, a single student's score moves the percentage noticeably."

WHAT I LEFT ALONE AND WHY

  • School names.
  • "NYSESLAT" (kept once with definition; it's the right citation for the recommended follow-up).
  • "Bensonhurst" / "Astoria" / neighborhood names.
  • "Composite (mixed selection)" — defined in the chart label; composite is the technical term but the parenthetical fixes it.

Net change: ~2200 → ~2600 words.