data: refresh UFC datasets through 2026-06-27

[komaksym]

Summary

• Refresh UFCStats fight data from the previous cutoff of 2026-03-28 through 2026-06-27.
• Add 146 fights across 12 completed UFC events.
• Increase stats_raw.csv to 8,737 fights.
• Refresh UFCStats profiles for all 286 fighters appearing in the newly added bouts.
• Rebuild the decisive-only and all-bouts processed datasets.
• Rebuild the stats-plus-scorecards table while preserving all existing scorecard values.
• Add a rematch-safe incremental merge utility plus a regression test for the two Sakuraba vs. Silveira bouts at UFC Ultimate Japan.

Updated files

• data/external_data/raw_fighter_details.csv
• data/stats/stats_raw.csv
• data/stats/stats_processed.csv
• data/stats/stats_processed_all_bouts.csv
• data/merged_stats_n_scorecards/merged_stats_n_scorecards.csv
• src/data_processing/stats_update.py
• tests/data_processing/test_stats_update.py

Validation

• The branch is one commit ahead of main and changes only the seven files listed above.
• Exactly 146 net-new fight rows were added; no historical fight identities were removed.
• No historical raw fight values changed.
• The same-event Sakuraba vs. Silveira no-contest and rematch are both preserved.
• No newly added processed row is missing an entire fighter feature block.
• Existing scorecard values are unchanged.
• Both processed datasets reproduce exactly from the committed raw stats and fighter-profile inputs.
• Data-processing test suite: 6 passed.

Scorecard note

Scorecards for the newly added fights were not scraped and OCR-processed in this refresh, so their scorecard-total fields remain empty. Existing scorecard values were preserved exactly.

Breaking changes

None.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[komaksym]

Review verdict: changes required before merge.

1. The deduplication removes a legitimate historical bout.

The base raw dataset contains two distinct Kazushi Sakuraba vs. Marcus Silveira bouts at UFC - Ultimate Japan on 21/12/1997: the first was recorded as no_contest / Overturned, and the rematch was red_win / Submission. The refresh deduplicates on only (red_fighter_name, blue_fighter_name, event_date, event_name), so it collapses those two real fights into one. This is why the scrape adds 146 new identity keys but stats_raw.csv grows by only 145 rows.

Please restore both bouts and use a true bout identifier—preferably the UFCStats fight-detail ID/URL. If that is unavailable, the identity must include a bout discriminator such as method, round, time, or an explicit occurrence index. Fighter pair + event + date is not unique.

2. The refreshed processed datasets are largely missing fighter features.

For the 146 new fights, only 19 rows have the full static fighter-feature block populated for both fighters. In 121 rows, at least one fighter has the entire height/reach/stance/career-stat block missing; in 62 rows, both fighters are entirely missing that block. Consequently, most new rows in stats_processed.csv have blank stance and blank height/reach/career-stat delta features.

Please refresh data/external_data/raw_fighter_details.csv (or the equivalent fighter-details source) before rebuilding stats_processed_all_bouts.csv and stats_processed.csv. Otherwise these files should not be presented as fully refreshed analytical outputs.

What passed the audit:

• all four schemas are unchanged;
• 146 genuinely new bout identities across 12 events were added;
• no other historical raw fight values changed;
• existing scorecard values were preserved exactly;
• output cardinalities are internally consistent;
• both processed files reproduce exactly from the currently committed raw and fighter-details inputs.

[komaksym]

AI-generated response: The earlier review comment on this PR was produced by ChatGPT acting on Maksym's behalf. Future comments I post on this PR will be explicitly labeled as AI-generated.

[komaksym]

AI-generated response: Final re-review completed on the cleaned, one-commit branch.

No remaining blocking findings.

Verified:

• exactly 146 new fights across 12 events, through 2026-06-27;
• raw dataset contains 8,737 fights;
• no historical raw fight rows or values were removed or changed;
• both same-event Sakuraba vs. Silveira bouts are preserved;
• all 286 fighters appearing in the new bouts have profile rows;
• no new processed row is missing an entire fighter-feature block;
• existing scorecard values are unchanged;
• processed outputs are exactly reproducible from the committed inputs;
• data-processing tests pass: 6 passed;
• branch is one commit ahead of main and changes only the seven documented files.

Known, documented limitation: scorecards for newly added fights were not scraped/OCR-processed, so those new scorecard fields remain empty.

PR #11 is ready for human review. It has not been merged.