Find What Works: Schools
← All source documentsRaw file
Source document

docs/design/README.md

Served verbatim from the project repository. Internal working document conventions apply: documents may reference file paths, branch names, and findings-ledger anchors from the repo.

Design & review docs

Design and review artifacts for the NYC school-outcomes analysis work on this repo — the spec, the prototype review, the gap analysis, and the integration/contribution plans, plus a small reference Python implementation. These are context and planning; they change no prototype code.

The companion implementation + findings — the read-only data-validation harness — lands separately under verify/ (with verify/METHODOLOGY.md), in the data-validation PR. Where this design diverges from what was actually built, verify/METHODOLOGY.md is authoritative (notably: the shipped (iii) spot check uses the NYC School Snapshot, where this spec had planned a different-publisher NYSED reference — now deferred).

Contents

FileWhat it is
nyc-school-outcomes-spec.mdThe methodology spec — peer groups, covariate-adjusted residual-z outliers, comparability regimes, verification. Cited elsewhere as nyc-school-outcomes-spec.md:<line>.
01_inventory.mdInventory of the prototype: stack, schema, loaders, analysis modules.
02_gap_analysis.mdPrototype vs. spec, dimension by dimension. The largest gap is #9 — correctness vs. reality (now addressed by the validation harness).
03_integration_plan.mdHow to evolve (not rebuild): sequenced, additive, feature-flagged changes (C0–C5).
04_contribution_plan.mdBranch/PR workflow for the shared repo (collaborator, no fork).
05_validation_spec.mdThe data-validation spec — the (i)–(iv) taxonomy; the base case ships first.
reference-scaffold/A small reference Python implementation of the spec's engine + verification (pipeline/..., configs, golden tests). The docs cite scaffold files by their path within this folder — e.g. pipeline/analysis/engine.py means reference-scaffold/pipeline/analysis/engine.py.

Conventions

  • Validation taxonomy: (i) completeness · (ii) correctness — base case (DB vs the exact ingested file) · (iii) correctness — spot check (DB vs a public-facing per-school report) · (iv) correctness — computed values. (05 was originally framed as "Phase 1a/1b/2" and carries a one-line mapping.)
  • Citations are file:line against the audited clone (main @ a4baa1a) and against nyc-school-outcomes-spec.md / the scaffold.
  • Suggested reading order: nyc-school-outcomes-spec.md → 01 → 02 → 03 → 04 → 05.