NYC School Outcomes — Outlier & Trend Analysis Platform

First-Principles Design Spec & Implementation Plan

Status: Draft v1 for review and Claude Code handoff Scope: A reproducible pipeline + static website that, for each of 8 NYC school outcomes, surfaces (View 1) peer-adjusted positive/negative outliers and (View 2) demographically-adjusted trends over time — using a single shared, deterministic, outcome-configurable analysis engine.

How to read this doc. Sections 1–6 are the first-principles spec (what this should be, independent of any existing code). Section 7 evaluates the existing prototype against that spec and proposes an integrated workflow. Section 8 is the build sequence. Section 9 lists the decisions I need from you before Claude Code starts. Appendices give concrete schemas Claude Code can implement directly.

1. Product definition (first principles)

1.1 The two questions, in plain language

Every outcome gets two analysis views, and only these two:

• View 1 — Outliers vs. similar peers. "Among schools serving demographically similar students under a similar admissions structure, which schools score notably higher or lower than that profile would predict?"
• View 2 — Trends not explained by demographics. "Which schools are improving or declining over time in a way that is not explained by changes in their student population?"

1.2 What these views are NOT (framing guardrails — load-bearing)

This is a civic-transparency tool, and mislabeling a school as a "negative outlier" has real reputational cost. The framing must stay descriptive:

• An outlier flag means "performs above/below what its student composition and admission type would statistically predict." It is not a causal claim that the school is good/bad, and not a ranking of school quality.
• Demographic variables are statistical controls to enable fair comparison, never judgments about students.
• The platform shows data with uncertainty, not verdicts. Small-N schools, suppressed cells, and noisy single-year blips must be visibly down-weighted, not silently flagged.
• Echoing the Civic Intelligence brief: publish methodology and error/coverage rates openly, maintain editorial neutrality ("data, not analysis"), and keep a "Not affiliated with NYCDOE/NYSED" disclaimer site-wide.

1.3 The 8 outcomes, by grade band

Two outcomes need an explicit operational definition before build: #8 Course access (which courses count, and how to index them) and #3 CCR (which published CCR metric, and which graduation cohort). See §9.

1.4 Unit of analysis

• Grain: school × year. Primary key: DBN (District-Borough-Number) + canonical year string YYYY-YY.
• A DBN crosswalk tracks identity changes (opens/closes/merges/renumbering). Trend analysis runs only on DBNs with enough comparable years (§3.2).

2. Data model & canonical entities

2.1 Covariates (the "controls") — and a gap to close

The outcome source list you provided does not include a covariate source. We need two:

1. Sociodemographic covariates — from the NYC School Demographic Snapshot (InfoHub → School Quality → Demographic Data). Fields: % economically disadvantaged / Economic Need Index (ENI), % ELL/MLL, % students with disabilities (SWD), race/ethnicity composition (% Black, Hispanic, Asian, White), % students in temporary housing, total enrollment. (Note: NYC's poverty proxy is ENI + % economic need, not California-style FRPM; pick one poverty measure as canonical — see §9 decision 9.)
2. Admission / structure metadata — admission method comes from the HS Directory (method1) and MS Directory (admissionsmethod_prog1) directory fields; elementary defaults to Zoned unless the school is G&T. Charter status and school-type flags (D75 special-ed, D79/transfer) from the school master list. (Source #9 — the HS Directory xlsx — therefore does double duty: it is both the admission-method source for HS and the source for outcome #8 course access.)

2.2 Covariate sets are partly band-specific

A base sociodemographic set is shared across all outcomes (ENI/poverty, %ELL, %SWD, race composition, enrollment). On top of that, structural covariates differ by band:

• ES: G&T flag (no admissions screening otherwise).
• MS: admission method (admissionsmethod_prog1), where applicable.
• HS: admission method (method1) — the dominant predictor; buckets like Specialized/exam, Screened, Ed-Opt, Zoned, Unscreened/Lottery, Audition.

Define these in config/covariates.yaml keyed by band so the engine stays shared (§Appendix B).

2.3 Outcome catalog with known comparability breaks

These hazards drive config/comparability.yaml (§3.2, Appendix C). They are the single biggest reason a naive "year-over-year change" view is wrong.

3. Statistical methodology — the shared engine (the crux)

Goal: one fitting routine and one flagging routine, parameterised per outcome via config. No bespoke per-outcome statistics in v1.

3.1 View 1 — peer-adjusted outlier detection (model-residual approach)

Why regression residuals rather than peer-group deciles. Defining ~40-school peer groups and flagging top/bottom decile (the prototype's v1 approach) is intuitive but: (a) the flag depends on arbitrary group construction (K, distance weights); (b) it yields a rank, not a continuous effect size with uncertainty; and (c) it doesn't down-weight schools whose extreme position is just small-N noise. A regression that conditions on the same demographic + admission variables does the same job continuously and deterministically, and the same model serves every outcome by swapping the response variable. Recommendation: model residual is primary; keep peer-group context as a complementary display + robustness cross-check (§7).

For each (outcome, year):

1. Eligibility filter. Drop suppressed cells; drop school types flagged non-comparable for this outcome (config, e.g., D75/D79/transfer for graduation); require N ≥ min_n (config).

2. Fit weighted least squares of the outcome y on the band's covariate set, weights = N (tested/respondents/cohort size), within that single year (cross-section):

   • Admission handling (config): covariate (categorical term — default) or stratify (separate model per admission bucket when a bucket is large enough). With demographic + admission terms, the regression already encodes "similar peers."
   • Functional form (config): identity (OLS, default) or empirical_logit for bounded proportions near 0/1. OLS-on-rate with robust SEs is the transparent default; logit is offered for robustness and is usually immaterial unless rates approach the bounds.
   • Optional prior_achievement covariate (config) converts the residual to a growth-like measure — see §9 decision 3.

3. Residual + uncertainty. Raw residual r_i = y_i − ŷ_i. Standardize accounting for both model misfit and the school's own sampling error:

   z_i = r_i / sqrt( s^2 + se_i^2 )
   s    = model RMSE (residual SD, the unexplained between-school spread)
   se_i = school's sampling SE  (for a proportion: sqrt( p_i (1 - p_i) / N_i ))
   

   This is the key fairness move: a small-N school needs a much larger raw gap to clear the threshold, so we don't flag noise.

4. Flag (direction-aware). Positive outlier if z_i ≥ z_threshold, negative if z_i ≤ −z_threshold (config, e.g., 2.0). Every outcome carries desirable_direction (high/low) so that for chronic absenteeism and bullying — where high = bad — a "positive outlier" still means "doing well." This single field prevents the most common interpretation bug.

5. Stability. Report per-year flags plus a persistence summary ("flagged in ≥k of last m years") so single-year blips are visibly distinguished from durable outliers.

6. Outputs per school-year: ŷ, r, z, percentile-within-peers (for display), flag, N, suppression flag, pointer to model diagnostics.

3.2 View 2 — trend not explained by demographics (residual-trend, reusing §3.1)

Core move: run the trend on the demographic-adjusted residual from §3.1, expressed in within-year standardized units (z). This simultaneously (a) nets out demographic change — a rising residual means the school is gaining relative to demographically similar schools, not because its intake changed; and (b) sidesteps the 2023 test-scale break and citywide COVID shifts, because z measures relative standing each year rather than absolute proficiency/rate.

For each (outcome, school):

1. Build the residual series {(year, z_iy)} across comparable years only, per config/comparability.yaml. Trends are computed within a regime, never across a break:
   • Tests: Regime A ≈ 2013–2019, gap 2020–2021 excluded, Regime B ≈ 2023→. 2022 handled per config.
   • Chronic absenteeism: pre-2020 vs 2021/22→ regimes.
   • Survey outcomes: instrument-stable window(s) defined after verification.
2. Fit trend. OLS of z_iy on year within a regime → slope β (z-units/yr) with SE; require min_years_within_regime (config, e.g., ≥4) and N ≥ min_n each year.
3. Flag improving/declining. |β| ≥ slope_threshold AND statistically distinguishable from 0 (slope-SE t-test or robust criterion) AND not driven by a single endpoint (report a leave-one-year-out sensitivity). Direction-aware via desirable_direction.
4. Outputs per school-outcome: slope (z/yr; and native-unit slope within regime for readability), SE, p, n_years, regime label, flag, and the sparkline series.

Documented alternative (v2 rigor option): a school fixed-effects panel model on the raw outcome with time-varying demographics and a school×time slope. More rigorous, less transparent; the residual-slope default is the shared, explainable v1. Note it in METHODOLOGY.md.

3.3 Why this is genuinely "shared + configurable + deterministic"

• Both views call the same fit_year() routine; the trend view just iterates it over years and fits a slope on the residuals.
• Everything that differs by outcome lives in config/outcomes.yaml (response variable, direction, form, covariate set, thresholds, comparability regime, excluded school types). Adding an outcome = adding a config block + a parser, not new statistics.
• Determinism: OLS/WLS is closed-form (no seeds needed); any randomized step (e.g., a KNN peer display) is seeded; the analysis stage makes no network calls and records input content-hashes in its outputs; golden/snapshot tests assert byte-identical re-runs (§5).

3.4 Limitations to publish (overclaim guardrails)

State these plainly on the methodology page:

• Residuals are associational, not causal; omitted variables (e.g., incoming achievement unless modeled, neighborhood, peer effects, selection) can drive them.
• Status models without prior-achievement controls partly reward favorable intake even after demographic adjustment; growth-style controls (if enabled) change interpretation.
• Survey outcomes carry response-rate bias (low-response schools) on top of sampling error — surface response rate alongside survey residuals.
• Cross-regime comparisons are reported only in standardized-position terms, never as raw proficiency/rate deltas.

3.5 Validation strategy (per outcome, before it ships)

• Face/known-case checks: a curated list of schools whose standing is independently knowable as a sanity test (not as ground truth).
• Diagnostics: model R², residual distribution/heteroskedasticity note, leverage/influence check.
• Convergence check: does the model-residual outlier set overlap sensibly with (i) the peer-group-decile method and (ii) NYCENET comparison-group standing where available? Large divergence = investigate.
• Sensitivity: vary z_threshold, weighting on/off, admission covariate-vs-stratify, and confirm flags are not knife-edge.

4. Pipeline architecture (deterministic, staged, verifiable)

Governing principle (your standing rule): source/research files are read-only; all transformation is via reproducible, documented pipeline scripts that read source files and generate the website JSON; never edit source data from the website side. The stages below enforce that.

Stage 0  Ingest        download → /data/raw (read-only) + provenance manifest + checksums
Stage 1  Parse         raw → /data/interim tidy tables (parquet), one parser per source
Stage 2  Assemble      interim → /data/processed/analysis_table.parquet (join covariates, apply DBN crosswalk)
Stage 3  Analyze       analysis_table + config → outlier & trend results (shared engines)
Stage 4  Verify (QA)   automated checks + per-outcome human sign-off (outcome cards)
Stage 5  Export        results → /data/outputs/website/*.json (versioned)
Stage 6  Site          Next.js static site reads the JSON (no computation in the front-end)

Each stage is an independently runnable CLI step (make ingest, make analyze OUTCOME=…, etc.), emits a log + a small report artifact, and has explicit acceptance criteria (§5.2) that gate the next stage.

4.1 Determinism & reproducibility requirements

• Pin all dependencies with a lockfile (uv.lock/requirements.txt with hashes); record tool versions in each run's report.
• Analysis stage: no network, fixed seeds for any stochastic display step, input content-hashes written into outputs.
• Golden tests: committed expected outputs for the pilot outcome; CI fails on any drift. Re-running Stage 3–5 from the same raw inputs must reproduce identical JSON.
• Provenance: every raw file carries {source_url, retrieved_at, http_status, sha256, bytes, notes} in /data/raw/<source>/<date>/provenance.json.

4.2 Repo layout

/data
  /raw/<source>/<retrieved_date>/...        # read-only; provenance.json + .sha256
  /interim/<source>/*.parquet               # tidy parsed tables
  /processed/analysis_table.parquet         # canonical school-year analysis dataset
  /outputs/website/<outcome>/*.json         # exported, versioned site data
/pipeline
  /ingest/<source>_download.py              # one per source → raw + manifest
  /parse/<source>_parse.py                  # raw → interim
  /assemble/build_analysis_table.py         # join covariates + crosswalk → processed
  /analysis/engine.py                       # SHARED fit_year(), standardize(), flag()
  /analysis/outliers.py                     # View 1 orchestration
  /analysis/trends.py                       # View 2 orchestration
  /analysis/run_outcome.py                  # one outcome end-to-end
  /verify/checks.py                         # automated stage checks
  /export/to_website_json.py
/config
  outcomes.yaml  covariates.yaml  comparability.yaml  thresholds.yaml
/crosswalk/dbn_crosswalk.csv
/docs
  METHODOLOGY.md  DATA_DICTIONARY.md  PROVENANCE.md
  /outcome_cards/<outcome>.md               # filled per outcome (definition of done)
  /qa_reports/<outcome>__<run_id>.md
/site                                       # Next.js static (or evolve the prototype)
/tests                                      # golden/snapshot + unit
Makefile  (or justfile)

5. Verification & documentation system

"Every step verifiable; methodology fully documented for replication" is the headline requirement — so verification is a first-class subsystem, not an afterthought.

5.1 Artifacts that must exist

• PROVENANCE.md + per-file manifests (URL, date, checksum) — anyone can confirm what was downloaded and when.
• METHODOLOGY.md — human-readable version of §3, plus the published limitations (§3.4).
• DATA_DICTIONARY.md — every field in analysis_table and every JSON field the site consumes.
• Outcome cards (/docs/outcome_cards/<outcome>.md) — per outcome: exact response definition, covariate set, admission handling, functional form, thresholds, comparability regime, excluded school types, data vintage, diagnostics summary, known-case check results, sign-off. An outcome is not "done" until its card is filled and checks pass.
• QA reports — auto-generated per run: coverage matrix (schools × years), % suppressed, join-match rate (outcome rows matched to a demographic row), distribution snapshots, model R², flag counts.
• CHANGELOG.md + data versioning so a published figure can always be traced to a specific data vintage + code commit.

5.2 Stage acceptance criteria (gates)

• Stage 0: all expected source files present; checksums recorded; manifest complete.
• Stage 1: schema validates; row counts within expected band; no duplicate primary keys; year strings normalized.
• Stage 2: join-match rate ≥ threshold (and unmatched rows enumerated, not dropped silently); crosswalk applied; coverage matrix produced.
• Stage 3: model converges; diagnostics within sane ranges; flag counts plausible; standardized residuals finite.
• Stage 4: automated checks green and human sign-off on the outcome card.
• Stage 5: JSON validates against the front-end schema; golden test passes.

6. Website spec (front-end)

The front-end only renders pre-computed JSON — no analysis in the browser (keeps it deterministic and the methodology auditable).

6.1 Information architecture

Home  →  Outcome index  →  Outcome page (tabs: Outliers | Trends)  →  School profile
                                                                   →  Methodology / Transparency
                                                                   →  Data downloads

6.2 Outlier view (per outcome)

• Scatter and/or table of schools; flagged positive/negative outliers highlighted, colored by desirable_direction (good = one color regardless of metric polarity).
• For each flagged school: residual in native units + standardized z, peer context (peer distribution 10/50/90 + drill-down to the peer set), N, suppression badge, "persistent" badge for multi-year flags.
• Uncertainty visible: confidence band on the residual or an explicit "low-N — interpret with caution" flag for schools near the min_n floor.
• Filters: grade band, admission bucket, borough; opt-in toggles for suppressed rows and D75/D79/transfer.

6.3 Trend view (per outcome)

• Per-school sparkline of the demographic-adjusted standardized residual over time; slope + significance; regime boundaries drawn as vertical markers (e.g., the 2020–2022 test break, the COVID absenteeism break) so users never read a spurious jump across a break.
• Improving/declining flags (direction-aware); secondary toggle to view the raw outcome with a demographic overlay for context.

6.4 School profile

One school across all 8 outcomes: each outcome's residual + trend mini-view, the peer set used, the school's demographic snapshot, and the relevant caveats inline.

6.5 Transparency, downloads, tech

• Methodology pages (human-readable + linked machine-readable config + outcome cards + the published error/coverage report).
• Bulk + per-outcome downloads (CSV/Parquet) — serves the "researchers" use case and the Civic Intelligence "infrastructure others build on" goal.
• Tech: Next.js static export reading JSON (matches the prototype and your prior static-site pattern). Charts via the prototype's existing library (Recharts) for consistency. Accessibility and neutral presentation throughout.
• Optional (borrowed from the adjacent project, see §7): expose a small read-only API/MCP endpoint over the same JSON for programmatic access.

7. Evaluation of the existing prototype + integrated workflow

Based on the live methodology page at nyc-edu-data.vercel.app/methodology (the repo mwg1378/nyc-edu-data is private — a code-level audit is a defined step once you have access, §8 Phase 0). For contrast, the adjacent public project jkinberg/nyc-school-explorer is a conversational AI tool (chat-over-data with guardrails, LLM-as-judge scoring, an MCP endpoint, 328 tests) — a different paradigm, but a good source of patterns for verification, responsible framing, and programmatic access.

7.1 What the prototype already does well — keep it

• Peer-context as the core lens ("vs. peers, not citywide averages") — exactly View 1's intent.
• Hard filters (grade band, admission bucket) + demographic similarity — the same covariate spirit as §3.
• Proclivity decile = a working version of the "punching above its profile" narrative.
• Suppression handling (low-N shown, not silently dropped; hidden from outlier lists by default with a toggle) — matches §3.1/§6.2.
• School-universe tagging (charters in; D75/D79/alt tagged out of defaults, opt-in) — matches §6.2.
• Survey rollups (DOE-published + own from raw distributions) and Story ideas (great for the "discoverability" gap the Civic brief flags).
• A machine-readable SPECS.md, year normalization (YYYY-YY), and a sources list that already includes the Demographic Snapshot and NYCENET comparison groups.

7.2 Gaps vs. this spec — enhance

1. Outlier statistic is rank-based, not model-based. The prototype flags top/bottom decile of the peer-group distribution and explicitly defers proportion/low-count handling to "v2." It has no reliability weighting, so small-N schools can land in extreme deciles on noise. → Add the §3.1 regression-residual engine with reliability-weighted standardized residuals; keep the peer-decile as a complementary display and a cross-validation check (§3.5).
2. "Expected outcome" uses only 3 covariates. Proclivity decile = mean percentile rank across % econ-disadvantaged, ELL, SWD — ignoring race/ethnicity composition, size, and admission beyond the hard filter, and it's a separate construct from the outlier computation. → Unify: the regression's covariate set subsumes proclivity; "expected" becomes the model's continuous prediction.
3. Trend view is the biggest gap. "Year-over-year movers" appear to be raw movers; the methodology page describes no demographic-adjustment-over-time and no comparability-regime handling — so it does not yet satisfy View 2, and it is exposed to the 2023 NextGen test break and the COVID absenteeism break. → Add the §3.2 residual-trend engine + comparability.yaml, expressed in within-year standardized units.
4. Determinism/verification unverified. KNN (K=40) + DERIVED admission/tags is reasonable but introduces tuning sensitivity and a derived-source dependency; I can't yet see golden tests, provenance manifests, per-outcome verification cards, or a published error/coverage report. → Add the §5 verification scaffolding and reconcile DERIVED admission/tags against NYCENET comp-groups and the directory method1/admissionsmethod_prog1 fields they already cite.
5. Composites should be built on standardized residuals, not raw rates, for cross-metric comparability and direction-awareness.

7.3 Integrated workflow (evolve, don't rebuild)

1. Keep the prototype's Next.js front-end, IA (Ratings / Story ideas / Methodology), peer-context UX, suppression + universe tagging, and SPECS.md discipline.
2. Insert a Python analysis layer (the §3 shared engines + /config) as the single source of truth that emits the site's JSON, augmenting/replacing the current rating computation. The front-end gains new fields (residual, z, slope, regime, flags) alongside the existing peer-decile fields, so the two methods can be shown side by side during transition.
3. Add comparability.yaml + regime-aware trends, and the verification scaffolding wired into CI.
4. Reconcile admission/tags with NYCENET + directory fields.
5. Sequence: run one pilot outcome through the new layer, validate against the prototype's existing ratings for sanity (§3.5 convergence check), then migrate the rest.

8. Build sequence / milestones

Phase 0 — Foundation (after repo access). Code-level audit of the private prototype; stand up /config schema, repo layout, provenance manifests, and CI with golden tests; lock the §9 decisions.

Phase 1 — Vertical slice (one outcome, end to end). Take a single pilot outcome through Stages 0→6, proving the whole spine before scaling. Recommended pilot: 3rd-grade ELA + Math (ES) — it is the flagship academic metric, has clean structured InfoHub files, and forces us to nail the two hardest features early: the composite and the 2023 comparability regime. (Alternative: Chronic absenteeism, which instead exercises the COVID break and a single non-composite rate — a good second outcome.)

Phase 2 — Validate the pilot. Diagnostics, known-case checks, convergence vs. the prototype + NYCENET, fill the outcome card, publish the methodology page + first error/coverage report.

Phase 3 — Templatize and roll out the rest, one at a time (each gated by its outcome card + checks), grouped by source affinity to share parsers:

• Tests g7 (reuses the g3 loader/engine).
• The three survey outcomes — bullying, teacher/principal trust, would-recommend — share one NYC School Survey loader (verify instrument-stable windows here).
• HS: graduation + CCR (shared HS sources; resolve cohort + CCR-metric decisions).
• Chronic absenteeism (COVID regime).
• Course access (HS) last — needs the operational definition (§9 decision 7) and non-trivial HS Directory parsing; may have limited/No trend.
• Bullying SSEC incident data added as a cross-check layer.

Phase 4 — Site polish + public beta. School profiles, bulk downloads, optional API/MCP, full transparency pages, public error/coverage report.

9. Decisions I need from you (before Claude Code starts)

1. Outlier statistic: model residual primary + peer-decile as cross-check (my recommendation) — confirm?
2. Admission handling: covariate (default) vs. stratify for large screened/specialized HS sets?
3. Prior-achievement controls: status-only for v1 (keeps methodology shared across outcomes; recommended) vs. add growth-style controls where available (grad/CCR/g7)? (Changes interpretation — would be flagged as such.)
4. Thresholds: z_threshold (default 2.0?), persistence window (e.g., ≥2 of last 3 years?), and min_n per outcome family.
5. Comparability regimes: confirm tests = {2013–2019} / exclude {2020,2021} / {2023→}, and how to treat 2022. Survey instrument-stable windows to be set during verification — OK?
6. School universe: include charters (prototype does)? Keep D75/D79/transfer tagged-out of defaults with opt-in (recommended)?
7. Course access (HS) definition: propose an advanced-course-availability index from the HS Directory — components e.g. {# AP courses, calculus offered, advanced science, # world languages, dual-enrollment available}. Confirm/adjust the components and whether it's an index or a set of separate availability flags.
8. CCR metric + graduation cohort: which published CCR index, and 4-year (August) vs 6-year grad rate as primary?
9. Covariate source + poverty measure: confirm Demographic Snapshot as canonical; choose ENI vs. % economically disadvantaged (+ % temporary housing) as the poverty control.
10. Integration target: evolve mwg1378/nyc-edu-data (recommended) vs. a fresh repo? License for the public-data outputs?
11. Site tech: stay Next.js static + JSON (recommended)?
12. Programmatic access: add a read-only API/MCP export over the JSON (borrowing the adjacent project's pattern)?

Appendix A — config/outcomes.yaml (example block)

- id: ela_math_g3
  label: "3rd Grade ELA + Math Proficiency"
  grade_band: ES
  unit: school_year
  sources: [test_results]
  value:
    type: composite                 # mean of components
    components: [g3_ela_pct_l34, g3_math_pct_l34]
    denominator: g3_tested_n
  desirable_direction: high         # high = good  (use "low" for absenteeism/bullying)
  functional_form: identity         # or empirical_logit
  covariates: es_base               # -> covariates.yaml
  admission_handling: covariate     # covariate | stratify | none
  prior_achievement: null
  min_n: 15
  outlier:
    z_threshold: 2.0
    persistence: { window: 3, min_flags: 2 }
  comparability: ela_math_regimes   # -> comparability.yaml
  trend:
    min_years_within_regime: 4
    slope_threshold_z_per_yr: 0.15
  non_comparable_school_types: [D75, D79, transfer]
  notes: "2023 NextGen standards reset; trend within-regime only."

Appendix B — config/covariates.yaml (example)

base_demographic: &base
  - pct_econ_disadvantaged    # or eni  (pick one canonical, see decision 9)
  - pct_ell
  - pct_swd
  - pct_black
  - pct_hispanic
  - pct_asian
  - enrollment_total          # size

es_base:
  covariates: [*base, gt_flag]
ms_base:
  covariates: [*base]
  admission_field: admissionsmethod_prog1
hs_base:
  covariates: [*base]
  admission_field: method1

Appendix C — config/comparability.yaml (example)

ela_math_regimes:
  regimes:
    - { id: A_commoncore, years: [2013, 2014, 2015, 2016, 2017, 2018, 2019] }
    - { id: B_nextgen,    years: [2023, 2024, 2025] }
  excluded_years: [2020, 2021]      # 2020 cancelled; 2021 ~21% participation
  ambiguous_years: [2022]           # old standards, recovery year — handle per policy
  cross_regime_reporting: standardized_only   # never raw proficiency deltas across regimes

chronic_absenteeism_regimes:
  regimes:
    - { id: pre_covid,  years: [2016, 2017, 2018, 2019] }
    - { id: post_covid, years: [2022, 2023, 2024, 2025] }
  excluded_years: [2020, 2021]

Appendix D — Outcome card template (/docs/outcome_cards/<outcome>.md)

# Outcome card: <outcome label>
- Response definition: <exact metric, components, denominator>
- Covariate set: <ref + list>   | Admission handling: <covariate|stratify|none>
- Functional form: <identity|empirical_logit>   | Prior achievement: <none|field>
- Thresholds: z=<>, persistence=<>, min_n=<>
- Comparability regime: <ref + year ranges; excluded/ambiguous years>
- Excluded school types: <list + rationale>
- Data vintage: <source files, retrieved dates, checksums>
- Diagnostics: R²=<>, residual notes=<>, flag counts (pos/neg)=<>
- Known-case checks: <results>
- Convergence vs peer-decile / NYCENET: <results>
- Sensitivity: <threshold/weighting/admission variations>
- Sign-off: <name, date, commit>