04 — Contribution Plan (one shared repo, no fork)
You (
kx-jin) are a collaborator on the shared private repomwg1378/nyc-edu-data, not a forker — changes must drop in cleanly for both co-owners. The model below is settled (per the runbook's locked decisions), captured here as an operating procedure. Conventions are drawn from01_inventory.md §4.
0. Repo facts that shape the workflow (from the inventory, 01)
- Default branch
main; single branch, no open PRs/issues → clean slate, no in-flight collisions. mainis not branch-protected (404 on protection API) — so the discipline below is convention, self-enforced, not gated by GitHub. Treat "no direct commits tomain" as a hard rule regardless.- No CI (
.github/absent); the only automated gate is the local.githooks/pre-pushrunningvitest run tests/unit/. New gates must be added deliberately (see §4). - Repo is private, no license yet (license for public-data outputs is a deferred decision — spec §9.10).
1. Sync first (before every branch)
cd nyc-edu-data
git checkout main && git pull --ff-only # default branch is main
gh pr list --state open # re-check for new in-flight work
git branch -r # confirm no new collaborator branches
(Authenticated as kx-jin via gh — set up in Phase 0/1; HTTPS over the gh credential helper.)
2. Branch per capability — kx/<capability>
One branch + one focused PR per capability from 03_integration_plan.md §C. Never
commit directly to main. Suggested branches, in sequence:
| Order | Branch | Maps to | Scope |
|---|---|---|---|
| 1 | kx/pipeline-scaffold | C0 | pipeline/ package, requirements.txt, configs, analysis_table assembly, README fix |
| 2 | kx/residual-outliers | C1 | residual-z engine + additive DB columns + flagged behind method flag + flip-diff report |
| 3 | kx/trends | C2 | within-year z-slope + comparability.yaml + regime-aware trend view |
| 4 | kx/reconciliation | C3 | verify/checks.py + verify/reconcile.py + manual_csv adapter + QA reports |
| 5 | kx/provenance-golden | C4 | source-file checksums/manifests + golden/determinism tests |
| 6 | kx/composite-residuals | C5 | composites on standardized residuals (after C1) |
Keep PRs small and single-purpose; do not bundle capabilities. If a branch grows,
split it (e.g. kx/reconciliation-checks then kx/reconciliation-external).
3. Additive + reviewable (the core constraint)
- Every change additive; feature-flagged where it touches existing logic. The prototype must work at each commit and on
mainafter each merge.- New stats land as new nullable DB columns / new tables, never altering
comparison_group_percentileor the decile path (lib/queries/schools.ts:166-172). - New site behavior behind a flag/toggle, default = current behavior, until the flip-diff is reviewed (
getOutliers({ method })).
- New stats land as new nullable DB columns / new tables, never altering
- Each PR links the gap analysis — reference the specific dimension in
02_gap_analysis.mdand the C-item in03_integration_plan.mdit implements, plus the spec section. - Request mwg1378's review on every PR — they co-own the design; this is non-negotiable since the methodology is shared.
- PR description template: what gap it closes · what's additive vs. flagged · how to run it · what stayed unchanged · screenshots/flip-diff where relevant.
4. Match house style + add gates honestly
- TypeScript code under
scripts/loaders/orlib/, matching existing idiom (tsxloaders, Prisma access via_lib.ts); lint clean undereslint(eslint.config.mjs); strict TS. - Tests under
tests/**/*.test.ts(vitest,@→root). Any new TS logic ships with a vitest spec so the pre-push hook covers it (.githooks/pre-push). - The Python pipeline (per
03 §Brecommendation) introduces a second toolchain with no existing runner. Be explicit with mwg1378 about this; land it with:requirements.txt(pinned) + amake/npm wrapper documented in the README;- pytest golden/unit tests for the Python side (the prototype's pre-push hook only runs vitest, so Python tests won't gate automatically);
- propose adding minimal CI (
.github/workflows/) running both lanes — since there is no CI today, this is itself a small, valuable PR (kx/cior folded intokx/pipeline-scaffold). Until then, document the manual gate (make verify,make test).
- Commit messages: match the repo's sentence-case, scope-prefixed imperative style (e.g.
Add residual-z outlier engine alongside decile); no Conventional-Commits prefixes — the existing history doesn't use them.
5. Socialize before code
- Share
02_gap_analysis.md+03_integration_plan.mdwith mwg1378 and align before opening implementation PRs — especially the stack-fit decision (03 §B) and the deferred inputs (03 §E). These change the work materially and are co-owned. - Confirm the locked-but-co-owned points land the same way for both of you: residual-z alongside decile first (decide default from the flip-diff), and the four deferred decisions as configurable inputs.
6. Hygiene
- No force-pushes to shared branches; rebase only your own un-reviewed feature branches.
- Never commit data files or secrets — respect and extend
.gitignore(raw xlsx/csv,_cache,.env*already covered,.gitignore:9-46); adddata/processed/,__pycache__/, and any reference CSVs that contain non-public values. Manual reference CSVs of official, public per-school figures are fine to commit (they're the audit trail); keep them small and documented. - Design docs are contributed to the shared repo under
docs/design/(the spec, the review docs 01–05, and the reference scaffold) via a dedicated PR, so both co-owners have the design context in-repo. The local review workspace mechanics andREVIEW_PLAN.mdstay local; data files/secrets are never committed (above). - Each PR green on the local hook (
vitest) and, for Python work,make test/make verifyrun and pasted into the PR.
One-line summary
Sync → kx/<capability> branch → additive + flagged change matching TS/vitest house
style (with an honest second-toolchain + CI conversation for the Python pipeline) →
PR linked to the gap analysis → mwg1378 review → merge; socialize the gap analysis,
stack-fit, and deferred decisions before any implementation PR.