Source document

docs/design/04_contribution_plan.md

Served verbatim from the project repository. Internal working document conventions apply: documents may reference file paths, branch names, and findings-ledger anchors from the repo.

04 — Contribution Plan (one shared repo, no fork)

You (kx-jin) are a collaborator on the shared private repo mwg1378/nyc-edu-data, not a forker — changes must drop in cleanly for both co-owners. The model below is settled (per the runbook's locked decisions), captured here as an operating procedure. Conventions are drawn from 01_inventory.md §4.


0. Repo facts that shape the workflow (from the inventory, 01)

  • Default branch main; single branch, no open PRs/issues → clean slate, no in-flight collisions.
  • main is not branch-protected (404 on protection API) — so the discipline below is convention, self-enforced, not gated by GitHub. Treat "no direct commits to main" as a hard rule regardless.
  • No CI (.github/ absent); the only automated gate is the local .githooks/pre-push running vitest run tests/unit/. New gates must be added deliberately (see §4).
  • Repo is private, no license yet (license for public-data outputs is a deferred decision — spec §9.10).

1. Sync first (before every branch)

cd nyc-edu-data
git checkout main && git pull --ff-only        # default branch is main
gh pr list --state open                          # re-check for new in-flight work
git branch -r                                    # confirm no new collaborator branches

(Authenticated as kx-jin via gh — set up in Phase 0/1; HTTPS over the gh credential helper.)

2. Branch per capability — kx/<capability>

One branch + one focused PR per capability from 03_integration_plan.md §C. Never commit directly to main. Suggested branches, in sequence:

OrderBranchMaps toScope
1kx/pipeline-scaffoldC0pipeline/ package, requirements.txt, configs, analysis_table assembly, README fix
2kx/residual-outliersC1residual-z engine + additive DB columns + flagged behind method flag + flip-diff report
3kx/trendsC2within-year z-slope + comparability.yaml + regime-aware trend view
4kx/reconciliationC3verify/checks.py + verify/reconcile.py + manual_csv adapter + QA reports
5kx/provenance-goldenC4source-file checksums/manifests + golden/determinism tests
6kx/composite-residualsC5composites on standardized residuals (after C1)

Keep PRs small and single-purpose; do not bundle capabilities. If a branch grows, split it (e.g. kx/reconciliation-checks then kx/reconciliation-external).

3. Additive + reviewable (the core constraint)

  • Every change additive; feature-flagged where it touches existing logic. The prototype must work at each commit and on main after each merge.
    • New stats land as new nullable DB columns / new tables, never altering comparison_group_percentile or the decile path (lib/queries/schools.ts:166-172).
    • New site behavior behind a flag/toggle, default = current behavior, until the flip-diff is reviewed (getOutliers({ method })).
  • Each PR links the gap analysis — reference the specific dimension in 02_gap_analysis.md and the C-item in 03_integration_plan.md it implements, plus the spec section.
  • Request mwg1378's review on every PR — they co-own the design; this is non-negotiable since the methodology is shared.
  • PR description template: what gap it closes · what's additive vs. flagged · how to run it · what stayed unchanged · screenshots/flip-diff where relevant.

4. Match house style + add gates honestly

  • TypeScript code under scripts/loaders/ or lib/, matching existing idiom (tsx loaders, Prisma access via _lib.ts); lint clean under eslint (eslint.config.mjs); strict TS.
  • Tests under tests/**/*.test.ts (vitest, @→root). Any new TS logic ships with a vitest spec so the pre-push hook covers it (.githooks/pre-push).
  • The Python pipeline (per 03 §B recommendation) introduces a second toolchain with no existing runner. Be explicit with mwg1378 about this; land it with:
    • requirements.txt (pinned) + a make/npm wrapper documented in the README;
    • pytest golden/unit tests for the Python side (the prototype's pre-push hook only runs vitest, so Python tests won't gate automatically);
    • propose adding minimal CI (.github/workflows/) running both lanes — since there is no CI today, this is itself a small, valuable PR (kx/ci or folded into kx/pipeline-scaffold). Until then, document the manual gate (make verify, make test).
  • Commit messages: match the repo's sentence-case, scope-prefixed imperative style (e.g. Add residual-z outlier engine alongside decile); no Conventional-Commits prefixes — the existing history doesn't use them.

5. Socialize before code

  • Share 02_gap_analysis.md + 03_integration_plan.md with mwg1378 and align before opening implementation PRs — especially the stack-fit decision (03 §B) and the deferred inputs (03 §E). These change the work materially and are co-owned.
  • Confirm the locked-but-co-owned points land the same way for both of you: residual-z alongside decile first (decide default from the flip-diff), and the four deferred decisions as configurable inputs.

6. Hygiene

  • No force-pushes to shared branches; rebase only your own un-reviewed feature branches.
  • Never commit data files or secrets — respect and extend .gitignore (raw xlsx/csv, _cache, .env* already covered, .gitignore:9-46); add data/processed/, __pycache__/, and any reference CSVs that contain non-public values. Manual reference CSVs of official, public per-school figures are fine to commit (they're the audit trail); keep them small and documented.
  • Design docs are contributed to the shared repo under docs/design/ (the spec, the review docs 01–05, and the reference scaffold) via a dedicated PR, so both co-owners have the design context in-repo. The local review workspace mechanics and REVIEW_PLAN.md stay local; data files/secrets are never committed (above).
  • Each PR green on the local hook (vitest) and, for Python work, make test/make verify run and pasted into the PR.

One-line summary

Sync → kx/<capability> branch → additive + flagged change matching TS/vitest house style (with an honest second-toolchain + CI conversation for the Python pipeline) → PR linked to the gap analysis → mwg1378 review → merge; socialize the gap analysis, stack-fit, and deferred decisions before any implementation PR.