# NYSED School-Level Chronic Absenteeism — Ingest Format Notes

Scouted 2026-06-09. All claims below are sourced; spot-checks were run live against
data.nysed.gov (server-rendered report-card pages driven by the same database).

## 1. Bottom line / recommended ingest path

- **Authoritative source:** the NYSED **Report Card Database (SRC)** zips at
  https://data.nysed.gov/downloads.php. Each zip contains **only** a `.accdb`, an identical
  `.mdb`, and a ReadMe PDF — **no CSV/xlsx alternative exists** for chronic absenteeism (the
  only Excel files on the downloads page are assessment-refusal workbooks).
- The chronic absenteeism (CA) tables are **`ACC EM Chronic Absenteeism`** (grades 1–8) and
  **`ACC HS Chronic Absenteeism`** (grades 9–12) inside each SRC database. Table names contain
  spaces.
- **Tooling:** `mdbtools` is **NOT installed** on this machine (`mdb-tables`/`mdb-export` not
  found; nothing was installed during scouting). Ingest requires `brew install mdbtools`, then:
  ```
  mdb-export SRC2025_Group3.mdb "ACC EM Chronic Absenteeism" > em_ca_2025.csv
  mdb-export SRC2025_Group3.mdb "ACC HS Chronic Absenteeism" > hs_ca_2025.csv
  ```
- **Minimal download set for 2018-19 → 2024-25** (each SRC holds the current + prior reporting
  year, see §5): `SRC2019` (YEAR 2018, 2019), `SRC2022` (2021, 2022), `SRC2024` (2023, 2024),
  `SRC2025` (2024, 2025). Four zips, ~1.4 GB total. 2019-20 was never published (COVID).
  Prefer SRC2023 in addition if you want each year from its own-year file (revisions are
  possible between releases; overlapping years allow consistency checks).
- Fallback (no mdbtools): per-school server-rendered HTML at
  `https://data.nysed.gov/essa.php?year=<YYYY>&instid=<instid>&createreport=1&EMchronic=1&HSchronic=1`
  works (verified) but requires an instid list and ~1,800 NYC requests/year, and renders
  suppression as "—". Not recommended except for spot-checks. No JSON API was found on
  data.nysed.gov.

## 2. File inventory per year (URL, size, contents)

All from https://data.nysed.gov/downloads.php. Sizes from `curl -sI` Content-Length on
2026-06-09. Inner contents read from the zip central directory via HTTP range requests
(`remote_zip_ls.py` in this folder).

| School year | Zip URL | Zip size | Inner files (uncompressed) |
|---|---|---|---|
| 2024-25 | https://data.nysed.gov/files/essa/24-25/SRC2025.zip | 367.2 MB | `SRC2025_Group3.accdb` (1.62 GB), `SRC2025_Group3.mdb` (1.62 GB), `SRC2025ReadMe_Group3.pdf` |
| 2023-24 | https://data.nysed.gov/files/essa/23-24/SRC2024.zip | 337.7 MB | `SRC2024_Group5.accdb`, `SRC2024_Group5.mdb` (1.49 GB), `SRC2024ReadMe_Group5.pdf` |
| 2022-23 | https://data.nysed.gov/files/essa/22-23/SRC2023.zip | 332.6 MB | `SRC2023_Group5.accdb/.mdb` (1.46 GB), `2022-23 SRCmdbReadMeGroup5Release.pdf` |
| 2021-22 | https://data.nysed.gov/files/essa/21-22/SRC2022.zip | 307.2 MB | `SRC2022_GroupIV.accdb`, `SRC2022_Group IV.mdb` (1.33 GB), `SRC2022ReadMe_Group IV.pdf` |
| 2020-21 | https://data.nysed.gov/files/essa/20-21/SRC2021.zip | 133.1 MB | `SRC2021_Group II.accdb/.mdb` (589 MB), `SRC2021ReadMe_GroupII.pdf` |
| 2019-20 | https://data.nysed.gov/files/essa/19-20/SRC2020.zip | 15.1 MB | `SRC2020_Post-GroupIII.accdb/.mdb` (82 MB), ReadMe — **no CA tables at all** |
| 2018-19 | https://data.nysed.gov/files/essa/18-19/SRC2019.zip | 305.2 MB | `SRC2019_Post-GroupIV.accdb/.mdb` (1.48 GB), `SRC2019ReadMe_Post-GroupIV.pdf` |

- 2017-18 also exists: https://data.nysed.gov/files/essa/17-18/SRC2018.zip (and SRC2019's CA
  tables also carry YEAR=2018 rows).
- **2024-25 (SRC2025) exists and was released** (ReadMe last updated 2026-04-30).
- ReadMe PDFs for 2019–2025 were extracted (via range requests, without downloading the mdb)
  and saved in this folder as `SRC20NNReadMe.pdf` + converted `.txt`. These PDFs are the
  official layout documentation for every table.
- The **Student and Educator Database (STUDED)**, e.g.
  https://data.nysed.gov/files/studed/24-25/STUDED2025.zip (11.3 MB, also Access-only),
  contains "student attendance" = average daily attendance, **not** chronic absenteeism.

## 3. Table schemas and key columns

Source: `SRC2024ReadMe_Group5.pdf` (extracted from SRC2024.zip) and `SRC2025ReadMe_Group3.pdf`
(extracted from SRC2025.zip), pages "Table: ACC EM/HS Chronic Absenteeism".

### `ACC EM Chronic Absenteeism` / `ACC HS Chronic Absenteeism` (SRC2022–SRC2024 layout)

All columns are Access **Text** (use numeric casts on ingest; NYSED says values are
presentation-formatted).

| Column | Meaning |
|---|---|
| `INSTITUTION_ID` | 12-digit NYSED institution id, stable over time (not in SRC2019, which has only ENTITY_CD) |
| `ENTITY_CD` | **12-digit BEDS code** (changes if school moves/redistricts) — the join key for the repo's `nysed_beds_code` |
| `ENTITY_NAME` | entity name |
| `YEAR` | reporting year: `2024` = 2023-24 SY (each SRC holds current + prior year, e.g. SRC2024 has 2023 and 2024) |
| `SUBJECT` | `38_CA` (EM table) / `HS_CA` (HS table) |
| `SUBGROUP_NAME` | text subgroup name (see §6) |
| `ENROLLMENT` | students (gr 1–8 EM / 9–12 HS, incl. ungraded age-equivalent) enrolled ≥10 instructional days and present ≥1 day |
| `ABSENT_COUNT` | of those, number absent ≥10% of enrolled days |
| `ABSENT_RATE` | (ABSENT_COUNT / ENROLLMENT) × 100 |
| `LEVEL` | accountability level 1–4 or `s` (dropped from CA tables in SRC2025) |
| `OVERRIDE` | `Y` ⇒ values suppressed (`s`) by Office of Accountability |
| `DATA_REP_FLAG` | `Y` reported / `N` did not report attendance data / null = no enrollment. **Key NYC caveat — see §4 (Success Academy)** |
| `PARTIAL_DATA_FLAG` | district/state rows only: `Y` = some schools missing ⇒ partial aggregate; null = subgroup below accountability minimum-N (<30 in 2024; <20 in 2025) |
| `COUNT_ZERO_NON_DISPLAY_FLAG` | (new 2024) `Y` = zero-enrollment subgroup row kept for researchers; other fields `s` |

Variants by era:
- **SRC2019 (YEAR 2018/2019):** no `INSTITUTION_ID`; extra goal-tracking columns
  `STATE_BASELINE, SCHOOL_BASELINE, MIP, STATE_MIP, LONG_TERM_GOAL, EXCEED_LT_GOAL,
  MET_SH_TARGET, MET_AG_TARGET, END_GOAL` (source: SRC2019ReadMe pp. 11ff).
- **SRC2021 (YEAR 2021 only):** core columns + `DATA_REP_FLAG`, `PARTIAL_DATA_FLAG`; no
  LEVEL/MIP (accountability waived; CA "reported for informational purposes" — SRC2021ReadMe
  front matter).
- **SRC2025 (YEAR 2024/2025):** same core but **no `LEVEL`/`OVERRIDE`** — CA "was not used as
  an indicator in 2025 … still reported in NYS School Report Cards for information and
  improvement purposes" (SRC2025ReadMe p. 24).

### Suppression markers
- `s` = suppressed (accountability suppression rules; in 2023 `s` in ENROLLMENT "equates to 0").
- Website renders suppressed/small-N cells as `—`; the database `s` is authoritative.
- Rate columns are formatted text (e.g. `48.6%`-style on the site; verify exact mdb format on
  first export). Recompute rate from counts where both are present.
- Assessment-style "fewer than 5 tested" suppression does not apply to CA; CA follows the
  accountability rules doc linked from each ReadMe ("Understanding the New York State
  Accountability System under ESSA…", e.g.
  https://www.nysed.gov/sites/default/files/programs/accountability/24-25-sy-haw.pdf).

## 4. Entity coverage (NYC spot-checks, all verified live)

Included in SRC CA tables: state (`ENTITY_CD = 111111111111`), counties, N/RC groups, public
school districts, public schools, **and charter schools** (ReadMe front matter, all years).

| Entity | instid | BEDS (`ENTITY_CD`) | CA data check |
|---|---|---|---|
| PS 15 Roberto Clemente (DBN 01M015) | 800000048517 | `310100010015` | 2024 EM CA: 142 enrolled / 69 / 48.6% ✓; 2019 also present (with MIP-era columns) — https://data.nysed.gov/essa.php?year=2024&instid=800000048517&createreport=1&EMchronic=1 |
| KIPP Academy CS (Bronx charter) | 800000034168 | `320700860820` | 2024 EM CA: 769 / 434 / 56.4% ✓ — charters fully covered when they report |
| Success Academy CS – Harlem 1 (DBN 84M351) | 800000059316 | `310300860897` | 2022: 24.3% ✓; 2023: 25.9% ✓; **2024 and 2025: all `—`, "Level 1* … data were not reported"** (`DATA_REP_FLAG = N`). Expect this across the SA network from 2023-24 on — quantify on ingest. https://data.nysed.gov/essa.php?year=2024&instid=800000059316&createreport=1&EMchronic=1 |
| PS Q811 (D75, DBN 75Q811) | 800000042164 | `307500014811` | D75 schools exist as institutions; check CA rows on ingest |
| PS Q4 (D75, DBN 75Q004) | 800000042188 | `307500014004` | confirms D75 BEDS pattern |
| NYC Special Schools – District 75 | 800000057444 | `307500010000` | district aggregate |
| NYC Public Schools (citywide) | 7889678368 | `300000010000` | citywide aggregate |
| NYC Geographic Districts #1–32 | e.g. 800000048532 (D1) | `31xx000000xx`-style, digits 9–12 = 0000 | district aggregates exist per CSD |
| **D79** (Pathways to Graduation etc.) | — | — | **Not found** in data.nysed.gov school or district lists (lists.php pages, searched "PATHWAYS TO GRADUATION", "DISTRICT 79"); D79 programs do not appear to be report-card entities. Confirm by listing `ENTITY_CD LIKE '3079%'` after mdb export. |

## 5. BEDS code ↔ DBN

Official BEDS structure (SRC2025ReadMe "General → ENTITY_CD", p. 4):
digits 1–2 county, 3–4 city/town, 5–6 school-district number, 7–8 district/LEA type,
9–12 building. Special rules: digit 1 = `3` ⇒ NYC, digits 3–4 = community school district;
digits 7–8 = `86` ⇒ charter school; digits 9–12 = `0000` ⇒ district record.

County codes (same source): `30` NYC Central Office, `31` New York (Manhattan), `32` Bronx,
`33` Kings (Brooklyn), `34` Queens, `35` Richmond (Staten Island).

Derivation rules observed (verified examples above and 15K001→`331500010001` in NYC Open
Data wg9x-4ke6):

- **District schools** (DBN `ddBnnn`): BEDS = `[31..35 per borough][dd]0001[0][nnn]` —
  i.e. borough county code + 2-digit CSD + `00` + `01` + 4-digit zero-padded school number.
  Deterministic both directions.
- **D75** (DBN `75Bnnn`): BEDS = `307500 01 [b][nnn]` where `[b]` is a 1-digit borough code
  in the building field (Q = 4 in both checked examples; M/X/K/R values unverified — verify
  against crosswalk on ingest).
- **Charters** (DBN `84Bnnn`): BEDS = `[31..35][CSD of location]00 86 [bldg]`, and the 4-digit
  building number does **NOT** match the DBN number (84M351 → `…0876`-style mismatch:
  `310300860897`). **A crosswalk file is required for charters.**

Crosswalk sources:
- **NYC Open Data "2019-2020 School Locations"** (LCGMS extract), Socrata id `wg9x-4ke6`:
  fields `system_code` (DBN) + `beds`; includes 283 charters; verified
  `84M351 → 310300860897` matches NYSED's profile exactly.
  https://data.cityofnewyork.us/resource/wg9x-4ke6.json?$select=system_code,location_name,beds,managed_by_name
  This is the most recent *tabular* vintage on Socrata; schools opened after 2020 (e.g. newer
  KIPP Bronx II/III, KIPP Beyond) need a fresher source.
- **LCGMS live** (current data): NYC Open Data `3bkj-34v2` ("NYC DOE Public School Location
  Information") is an href-type pointer to DOE's LCGMS (https://www.nycenet.edu/PublicApps/LCGMS.aspx),
  which exports DBN + SED BEDS code for all current DOE + charter locations.
- Fallback for charters: NYSED profile pages expose BEDS per instid
  (`https://data.nysed.gov/profile.php?instid=…`).

Repo tie-in (read-only check, repo untouched): `SPECS.md:54` defines `nysed_beds_code` on the
schools table; `scripts/loaders/dbn-beds-crosswalk.ts` exists but is an **unimplemented stub**
(same for `scripts/loaders/nysed-ssec.ts`); run order in `scripts/loaders/README.md` puts the
crosswalk at step 2.

## 6. Subgroups in the CA tables

Exactly 10, as `SUBGROUP_NAME` text (SRC2024/2025 ReadMe, CA table definitions):
All Students; American Indian or Alaska Native; Asian or Native Hawaiian/Other Pacific
Islander; Black or African American; Hispanic or Latino; Multiracial; White; English Language
Learner(s); Students with Disabilities; Economically Disadvantaged.

**Not present:** gender, homeless, foster care, migrant, parent-in-armed-forces (those appear
in some assessment tables but not in the CA tables). No subgroup *codes* — names only; note
minor label drift ("English Language Learners" pre-2024 vs "English Language Learner" 2024+).

## 7. Years published + definitional changes

| SY | Where | Notes |
|---|---|---|
| 2017-18 | SRC2018; also SRC2019 YEAR=2018 | ESSA CA indicator begins; MIP/goal columns |
| 2018-19 | SRC2019 YEAR=2019 | same layout |
| 2019-20 | **nowhere** | SRC2020 has no CA tables (COVID waiver; ReadMe front matter) |
| 2020-21 | SRC2021 YEAR=2021 only | informational only (waiver); interpret with extreme caution — remote-instruction year |
| 2021-22 | SRC2022 YEAR=2022 (also 2021) | CA indicator resumed |
| 2022-23 | SRC2023 YEAR=2023 (also 2022) | `s` in ENROLLMENT "equates to 0" quirk documented for 2023 |
| 2023-24 | SRC2024 YEAR=2024 (also 2023) | adds `COUNT_ZERO_NON_DISPLAY_FLAG`; ENROLLMENT uses literal 0 |
| 2024-25 | SRC2025 YEAR=2025 (also 2024) | **CA no longer an accountability indicator** — replaced by the new **Attendance Index** (`ACC EM Attendance` / `ACC HS Attendance` tables: ENROLLMENT now ≥30 days, LEVEL1–4_COUNT bands by % days present, `ATTENDANCE_INDEX`). **CA still published** in `ACC EM/HS Chronic Absenteeism`, minus LEVEL/OVERRIDE; accountability minimum-N drops 30→20. Source: SRC2025ReadMe pp. 11–14, 24. |

Downloads-page blurbs corroborate: SRC2024 blurb lists "chronic absenteeism" among
indicators; SRC2025 blurb lists "attendance … progress" instead
(https://data.nysed.gov/downloads.php).

## 8. Open questions for ingest

1. **D75 borough digit map** in BEDS building field (only Q=4 verified) — resolve via
   crosswalk join, not derivation.
2. **D79 absence** — verified absent from site lists; confirm no `3079…` ENTITY_CD rows after
   first mdb export.
3. **Success Academy non-reporting scope** — Harlem 1 stopped reporting in 2023-24; check
   `DATA_REP_FLAG = N` counts across all NYC charters per year and surface as a data-quality
   table.
4. **Exact value formatting in mdb** (`ABSENT_RATE` with/without `%`, `s` literals, nulls) —
   inspect after `mdb-export`; all columns are text.
5. **Prior-year rows vs own-year files** — SRCn carries YEAR n−1 too; decide precedence
   (recommend own-year file wins; diff overlaps as a QA step).
6. **Charter crosswalk freshness** — wg9x-4ke6 is 2019-20 vintage; pull current LCGMS export
   for charters opened ≥2020.

## 9. Artifacts saved in this folder

- `downloads.html` — snapshot of https://data.nysed.gov/downloads.php
- `remote_zip_ls.py`, `remote_zip_extract.py` — list/extract members of the remote zips via
  HTTP range requests (no full download)
- `SRC2019ReadMe.pdf/.txt` … `SRC2025ReadMe_Group3.pdf` / `SRC2025ReadMe.txt` — official
  layout docs for every SRC year 2019–2025 (extracted from the zips; ~0.4–0.6 MB each)
- No mdb/accdb downloaded (mdbtools unavailable; zips are ~300–370 MB each).