GDELT FULL-YEAR AUDIT AND PEN MATCHING PREPARATION

Start with GDELT_Full_Year_Audit.txt. Run python audit_year.py to regenerate
outputs using Python 3 (standard library only). All file paths are relative.
The SQL query was run by the user in BigQuery Sandbox, not by this script.
The optional coverage query has not been executed.

inputs/gdelt_full_year_raw.csv is the original annual export, byte-for-byte.
inputs/gdelt_july_checkpoint.csv is for reconciliation only; do not append it.
inputs/pen_events.csv is the unchanged 3,362-row 2022-2023 PEN baseline from the
prior reproducibility package. Its source workbook used the Sorted by Author &
Title sheet only. Source sheet and row are retained. This ZIP does not replace
the earlier full-workbook source package.
PEN source: https://docs.google.com/spreadsheets/d/1a6v7R7pidO7TIwRZTIh9T6c0--QNNVufcUUrDcz2GJM/edit

outputs/review_queue.csv preserves raw fields and adds decoded titles, duplicate
flags, a lexical prioritization cue and empty reviewer fields. Those fields
are pending, not validated findings. Record IDs are stable keys for review.
Copy review files before adding decisions: rerunning audit_year.py regenerates
outputs and resets reviewer fields. Keep completed reviews separately.
outputs/district_name_candidates.csv contains unverified full-name matches.
State context is text presence, not a verified geographic assignment.
No article text has been fetched by this audit and no NCES IDs were inferred.

Recorded dates are GDELT dates. Title HTML entities are decoded only in derived
fields. Raw URLs are preserved; exact repeats are flagged, not collapsed.
Headline normalization decodes HTML, case-folds and collapses whitespace only.
Equal headlines do not prove articles are identical. Blank headlines are
excluded from headline-group counts rather than treated as one shared story.

USER-APPROVED RESEARCH QUESTION
RQ3. What happened when a district reviewed a challenged book with input from librarians and local residents?

CHECKS BEFORE ANALYSIS
- Each district name will be checked against the NCES directory and assigned one NCES identification number. If the match is uncertain, the record will remain unresolved until it can be checked manually.
- Every restriction will be assigned to a school year. A month will be recorded only when the source provides enough detail; otherwise, the record will show that the date is approximate.
- Exact and possible duplicate records will be listed before any records are combined. Restrictions ordered at the state level will not be merged with actions taken by individual districts.
- The yearly, state and district totals will be recalculated and compared with PEN America’s published figures. Differences will be investigated and recorded.
- Missing information and unavailable sources will be reported for each state and school year.
- A forecasting model will use only information that existed before the period being predicted.

This audit implements structural checks and prepares review queues. It does
not claim all research checks above are completed. No final report, website,
forecast, QGIS map, or validated news-to-PEN outcome analysis was updated.
