FSDP2: handle footgun as a rejection case in pytorch issue 180666 (the mark of the beast)

[bghira]

This pull request adds important validation logic to prevent incompatible configurations when using FSDP v2 with CPU parameter offload, and extends test coverage to ensure these checks work as intended. The main focus is to avoid runtime errors caused by using certain optimizer options that are not supported together.

Validation logic for FSDP v2 CPU offload:

• Added checks in _normalize_input_args (cmd_args.py) and _load_fsdp_plugin (trainer.py) to raise errors if FSDP v2 CPU parameter offload is enabled together with either post-accumulate gradient hook based optimizers (e.g., optimizer_release_gradients with "optimi" optimizers) or the torchao optimizer CPU offload method. This prevents unsupported configurations and provides clear error messages to users. [1] [2]

Test coverage improvements:

• Added new tests in test_fsdp_cmd_args.py to verify that invalid argument combinations are rejected and valid ones are allowed.
• Added new tests in test_trainer.py to ensure that the trainer correctly raises errors for incompatible FSDP v2 CPU offload and optimizer settings.

These changes improve the robustness of configuration handling and help users avoid hard-to-debug runtime failures.