01 — Prototype Inventory (mwg1378/nyc-edu-data)

Prototype inventory — read-only audit; no prototype code was modified. Clone audited at main @ a4baa1a ("Editorial sweep: finish the remaining ~30 stories"), 157 tracked files, single branch, pushed 2026-05-31.

This is one of the design/review docs in docs/design/. The reference Python scaffold these docs cite lives in docs/design/reference-scaffold/ (scaffold paths like pipeline/... are relative to it); nyc-school-outcomes-spec.md is in this folder. See docs/design/README.md for the map.

1. Stack summary

Entry points / scripts (package.json:5-12): dev (next dev -p 3008), build (prisma generate && next build), start, lint (eslint), test (vitest), prepare (wires .githooks). Data pipeline is not in package.json; loaders run ad-hoc via tsx (e.g. tsx scripts/loaders/test-results.ts), gated by .env.local (DIRECT_URL).

⚠️ Stack mismatch with our scaffold. The prototype's analysis layer is TypeScript + a live Postgres DB. Our reference scaffold (reference-scaffold/) is Python (numpy/pandas/scipy/pyarrow, requirements.txt) emitting static JSON files the site reads. This is the single largest "does it integrate cleanly" question — carried into 03_integration_plan.md §Stack-fit.

2. Data flow (raw → site)

The prototype is DB-centric, not file-centric. There is no analysis_table artifact and no JSON export stage; computed columns are written back into Postgres and the site reads them live.

NYCDOE InfoHub xlsx / NYC Open Data CSV / Socrata / NYSED
        │  (fetchAndCache → data-sources/_cache, gitignored)         scripts/loaders/_fetch.ts:14
        ▼
[ingest+parse, one loader per source]  test-results.ts, demographics.ts, graduation-results.ts,
        │   normalizeDbn / normalizeYearLoose, isSuppressed                school-quality.ts, attendance.ts,
        ▼                                                                  nyc-school-survey.ts, nysed-ssec.ts, hs-directory.ts,
   Postgres (Prisma)  ── school_year_metrics (long format)                 nyc-od-schools.ts, nyc-od-charters.ts, dbn-beds-crosswalk.ts
        │              ── schools, school_year_demographics, survey_*
        │              ── data_loads (provenance row per run)         scripts/loaders/_lib.ts:30-62
        ▼
[derive / compute, in-DB]
   derive-admission-topic.ts   → schools.admission_category, topic_tags
   nycenet-comparison-groups.ts→ comparison_groups (DERIVED, K=40) + schools.comparison_group_id
   compute-proclivity.ts       → schools.proclivity_decile / proclivity_score
   compute-percentiles.ts      → school_year_metrics.comparison_group_percentile / citywide_percentile
   compute-composites.ts       → composite_* metric rows
   compute-survey-metrics.ts   → survey_* metric rows
        ▼
[site reads DB live via Prisma]   lib/queries/schools.ts
   app/ratings/outliers  (decile filter on comparison_group_percentile)
   app/ratings/movers    (raw cross-year value delta)
   app/schools/[dbn], /parents/*, /researchers/*, /stories/*

Sources, vintages & provenance:

• Source catalog: SPECS.md:321-339; per-source notes stub at data-sources/README.md.
• Test results pull years 2018–2025 from two fixed InfoHub xlsx URLs (scripts/loaders/test-results.ts:12-23); ingests grades 3–8 + "All Grades", both % Level 3+4 and Mean Scale Score (test-results.ts:154-182).
• Provenance row per load: source_name, source_url, script_path, git_sha, row counts, notes (_lib.ts:30-62, prisma/schema.prisma:399-419). No source-file content checksum/sha256 is recorded; the cache is keyed by URL only (_fetch.ts:9-29).
• Raw downloads are gitignored (.gitignore:32-46) — not committed.

3. Analysis-relevant module map

Methodology surface to users: app/methodology/page.tsx; the decile rule is the documented v1 method (SPECS.md:300, methodology/README.md:7). Planned methodology write-ups (methodology/README.md:16-24) are not yet authored.

Notable signal — the editorial side already prefers regression residuals. The team's own quant playbook says decile means are a piecewise-constant baseline that distorts the extremes and that "a continuous regression baseline … is the more defensible methodology" (docs/agents/quantitative-knowledge.md:38-47), warns that top-N residual lists are selected-on-outcome (:18-26), and recommends a 3-year rolling residual for stability (:28-36). The published stories speak in "proclivity-adjusted residual" / "gap above expectation" terms (docs/agents/plain-language.md:60,67; data/stories/answers.ts:1216) even though the code computes deciles. The spec's residual-z method formalizes language the project already uses editorially.

4. Contribution conventions (for Phase 5)

House-style implications: new code matching the prototype is TypeScript under scripts/loaders/ (or lib/), with vitest tests under tests/. There is no CI to extend — only the local pre-push hook — so any new gate (golden/determinism, reconcile) must be added as a vitest spec (TS) or a new hook/CI workflow. A Python pipeline (the scaffold) would introduce a second toolchain with no existing runner — see 03_integration_plan.md.

5. README staleness flag

README.md:5 still says "Status: Spec phase. No code yet." — stale; the repo is a fully built Next.js app with a complete loader pipeline and ~30 published stories. Worth correcting in an early housekeeping PR.