feat(import): add --minimize-reads for type-scoped state loading

[riyazsh]

What this PR adds

This PR adds --minimize-reads to the datadog-sync import subcommand. The option, its validation, and the underlying loading-strategy code already exist (introduced for sync in #544 / #545) — only the click decorator was missing on import.

When enabled, import scopes the bucket state-load to the requested resource types (type-scoped) or to filter-extracted exact IDs (ID-targeted), instead of scanning every file under both source and destination prefixes.

The problem this solves

State.__init__ calls load_state(Origin.ALL), which reads every file under both prefixes (utils/state.py:77, utils/storage/gcs_bucket.py:56-84). On reasonably-populated buckets the state-load dominates per-run wall-clock regardless of how many records the run actually fetches — the boot-time read of prior runs' files becomes the bottleneck.

Observed shape on a ~22k-file source bucket, importing one resource type at a time:

Resource type	Records imported	Wall-clock	Per-record cost
authn_mappings	3	41 min	825 sec/record (~99.97% state-load overhead)
dashboard_lists	381	42 min	6.6 sec/record
dashboards	14,000	53 min	230 ms/record

Three records, 41 minutes — nearly all of it the unscoped bucket scan. --minimize-reads fixes this for sync already; this PR makes the same fix reachable from import, which is the entry point most automation uses for incremental refreshes.

How to enable

datadog-sync import --minimize-reads --resource-per-file --resources=<types> ...

Constraints enforced in utils/configuration.py:412-417 (unchanged in this PR):

• requires --resource-per-file
• requires --resources

--cleanup is not registered on the import command (it lives in _diffs_options, applied only to sync/diffs), so the --minimize-reads + --cleanup rule cannot be violated through the import CLI. Click rejects import --cleanup=... at parse time. Locked in by test_import_does_not_accept_cleanup_option.

What does not change

• sync command behaviour
• utils/configuration.py validation logic
• State constructor or storage layer
• Default behaviour for import — the flag is opt-in (default=False)

Code summary

• commands/_import.py: add --minimize-reads @option(...) decorator; import option and CustomOptionClass.
• commands/sync.py: drop now-stale "Only available on the sync command." line from help text.
• tests/unit/test_minimize_reads_type_scoped.py: flip the pre-existing negative test that asserted import rejected the flag; add positive --help check, two validation-mirror tests, and a click-level guard for --cleanup on import.

Test plan

• pytest tests/unit/ — 500 passed
• --minimize-reads listed in datadog-sync import --help
• import --minimize-reads without --resource-per-file → UsageError
• import --minimize-reads without --resources → UsageError
• import --minimize-reads --cleanup=Force → click "no such option" (parse-time rejection)
• sync --help still renders correctly after help-text edit
• CI green across macOS / Linux x86 / Linux arm / Windows

Rollout

Flag is opt-in (default=False). Reverting the commit removes the option from import --help but doesn't break any caller that wasn't using it. No on-disk or wire-format effect.

Out of scope

• migrate subcommand still lacks --minimize-reads. Adding it requires confirming that the apply phase's existing minimize-reads handling covers the migrate flow. Follow-up.
• Parallelising _list_and_load (the per-type file walk is still sequential). Separate effort.

[riyazsh]

Self-review notes

Posting these before external review so reviewers can focus on what I missed.

Reachability

Validation at configuration.py:412-419 is command-agnostic. New reachability from import:

• --minimize-reads requires --resource-per-file → reachable (common_options). Pinned by test_import_minimize_reads_requires_resource_per_file.
• --minimize-reads requires --resources → reachable (common_options). Pinned by test_import_minimize_reads_requires_resources_flag.
• --minimize-reads cannot be combined with --cleanup → unreachable from import, intentionally. --cleanup lives in _diffs_options, not applied to import. Click rejects import --cleanup=... at parse time. Pinned by test_import_does_not_accept_cleanup_option.

No dead branches added.

Tests

No new fakes/mocks. CliRunner is the same harness existing sync-side tests use. 500 unit tests pass.

Follow-ups (not in this PR)

• --minimize-reads scopes the load but doesn't parallelise within a type. _list_and_load (gcs_bucket.py:68-84 + S3/Azure) is still serial. Worth a parallelisation pass — likely with the existing --max-concurrent-reads knob.
• migrate doesn't yet expose --minimize-reads. Adding it needs maintainer alignment on whether migrate's apply phase needs the destination state pre-loaded or whether the lazy-loader at state.py:149 is sufficient.
• For import specifically, load_state(Origin.ALL) still loads destination state, which import never reads (resources_handler.py:376-435 only touches state.source). With --minimize-reads that's a smaller dead-weight scan than before, but not zero.

Tightened in-PR

• test_import_does_not_accept_cleanup_option now asserts against the Error: line specifically (commit 0ddc8abf) so a future click message reword doesn't silently mask a regression that accepts --cleanup on import.
• Dropped the Must not be combined with --cleanup line from import's --minimize-reads help (commit 36b6ec31) — it was technically true but implied import accepts --cleanup, which it doesn't.