Add scheduled FOMC meetings and backfill missing release dates - #5
Merged
Conversation
Upcoming meetings become Scheduled Meeting rows, dropped once a statement or minute exists for that date. Only future dates are emitted, so past content-less entries like notation votes stay out. Reconciliation moves into a pure merge_communications() so it can be tested without files or network.
Scoped to what breaks here: the Fed changing its calendar HTML, and the merge losing, duplicating or reordering rows. Verified by mutation testing, with every remaining test uniquely catching at least one seeded bug.
All are unscheduled meetings whose minutes the Fed appends to the next regular meeting's document rather than publishing separately, so the calendar page carries no release date to scrape. Each date is recovered from the identical-text twin that has one; the two 2020 rows come from the Fed's historical page. Only those 29 cells changed.
vtasca
force-pushed
the
fomc-scheduled-meetings
branch
from
August 17, 2026 20:28
3a9dd2f to
1f44cb7
Compare
vtasca
marked this pull request as ready for review
August 17, 2026 20:29
vtasca
added a commit
that referenced
this pull request
Aug 17, 2026
Add scheduled FOMC meetings and backfill missing release dates
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds upcoming FOMC meetings to the dataset, backfills 29 rows missing a release date, and adds tests. Supersedes
add-scheduled-meetings, which can be deleted once this lands.Scheduled meetings
Upcoming meetings are recorded as a
Scheduled Meetingrow with emptyText. The row is dropped once aStatementorMinuteexists for that date, so entries convert to real content without duplicating. Only future dates are emitted, which excludes past content-less entries like the August 2025 notation vote.Reconciliation moved into a pure
merge_communications()that does no I/O, so it can be tested without files or network.main()now reconciles on every run rather than only when new communications appear: the schedule shifts independently of statements, so the old early return would have frozen the scheduled rows.Release dates
29
Minuterows had noRelease Date. All are unscheduled meetings and conference calls, whose minutes the Fed appends to the next regular meeting's document instead of publishing separately, so the calendar page has no(Released ...)text to scrape. The Fed says as much on its 2020 page for the March 2 meeting: "Minutes: See end of minutes of March 15 meeting".The right date is the one belonging to the document the row holds. The scraper stored that document's full text on every row referencing it, so 27 of 29 were recovered from their identical-text twin, each with a single candidate. The other two (
2020-03-03,2020-03-15) share text only with each other; their document isfomcminutes20200315.htm, released April 08, 2020. Every date was checked against the Fed'sfomchistorical<year>.htmpages.Only those 29 cells changed.
Date,TypeandTextare byte-identical, the row count is unchanged, and no release precedes its meeting.backfill_release_dates.pydocuments the derivation and can be re-run.Tests
8 tests, 0.4s, no network. Scoped to what actually breaks here: the Fed changing its calendar HTML, and the merge losing, duplicating or reordering rows in a dataset published to Kaggle and Hugging Face.
Checked by mutation testing — 14 seeded bugs, all caught, and every test uniquely catches at least one. Two that escaped the first draft are now covered: dedup keyed on
(Date, Type)rather than the whole row, andmain()reconciling unconditionally.Also
Python 3.9 to 3.12 in both workflows, and
.gitignorepicks up Python caches.Live run against federalreserve.gov adds 11 future meetings with no Statement or Minute lost.