Skip to content

Speed up multi-state spatial binding - #653

Open
dshkol wants to merge 2 commits into
walkerke:masterfrom
dshkol:agent/spatial-bind
Open

Speed up multi-state spatial binding#653
dshkol wants to merge 2 commits into
walkerke:masterfrom
dshkol:agent/spatial-bind

Conversation

@dshkol

@dshkol dshkol commented Aug 18, 2026

Copy link
Copy Markdown

I've been putting together a continuously-improving agent skill for performance optimizations for R packages, particularly ones like this that have workflows requiring large extracts from APIs and assembling them. We've implemented some really significant speeds up in other packages and I tested against and benchmarked some optimization improvements against tidycensus. This is one PR, but there's a few more I can submit later -- I didn't want to send too many at once.

The change here is a really simple, adding a helper that binds sf objects in one go rather than sequentially. Speed up on large spatial downloads are meaningful, especially when working with tracts.

Benchmarks are against real-data extracts and unit tests are already added.


Summary

  • replace sequential reduce(rbind) accumulation of multi-state sf results with one-pass row binding
  • apply the shared helper to six spatial paths across ACS, decennial Census, and population estimates
  • preserve the existing geometry-type reconciliation and final as_tibble() / st_as_sf() normalization
  • add an exact normalized-sf regression test; keep the official-geometry benchmark harness local

Real-world benchmark

The benchmark uses official Census geometry, including 2023 national cartographic-boundary tracts from tigris. The largest workload combines those tract geometries with actual 2023 ACS 5-year B01001 estimates/MOEs for six variables, matching a tidy multi-state get_acs(..., geometry = TRUE) output. Final normalized sf results are checked with identical() before timing.

Workload Features Pieces Output Reps Baseline PR Saved Speedup Allocations, baseline → PR
National counties 3,143 51 3.9 MB 9 0.157 s 0.015 s 0.142 s 10.5x 11.2 → 0.3 MB
National tracts 85,186 56 157.3 MB 7 6.829 s 0.122 s 6.707 s 56.0x 1,124.9 → 13.1 MB
National 6-variable tidy ACS geometry 510,270 52 821.6 MB 3 27.390 s 0.485 s 26.905 s 56.5x 3,298.8 → 54.7 MB

Times are randomized-order medians after warm-up. Allocations are cumulative Rprofmem() totals, not peak memory. Tidy national timing ranges were 27.340–27.525 seconds for baseline and 0.482–0.485 seconds for this PR.

Cold national geometry download took 4.250 seconds, and capture of the 52 state/DC/Puerto Rico ACS fixtures took 147.426 seconds. Those network measurements are reported separately and are not counted as code speedup because live tigris caching and API transfer vary independently.

The county workload saves only 0.142 seconds; this change is motivated by the 6.7–26.9 second absolute savings at national tract scale.

The benchmark harness, fixture cache, and full reproduction notes are retained locally and intentionally excluded from git.

Validation

  • devtools::test(): 84 passed, 0 failed, 0 warnings
  • devtools::check(error_on = "warning"): 0 errors, 0 warnings, 0 notes
  • final row/column order, values, geometry, CRS, and sf attributes are byte-for-byte identical at every benchmark scale

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant