Skip to content

[aws] Enable Identity Federation for the CloudWatch policy template - #20525

Closed
seanrathier wants to merge 4 commits into
elastic:mainfrom
seanrathier:aws/cloudwatch-identity-federation
Closed

[aws] Enable Identity Federation for the CloudWatch policy template#20525
seanrathier wants to merge 4 commits into
elastic:mainfrom
seanrathier:aws/cloudwatch-identity-federation

Conversation

@seanrathier

@seanrathier seanrathier commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Removes hide_in_var_group_options: credential_type: [identity_federation] from the aws-cloudwatch and aws/metrics inputs in the cloudwatch policy template, making them visible in the Identity Federation credential group
  • Adds external_id: {{external_id}} to cloudwatch_logs/aws-cloudwatch.yml.hbs and cloudwatch_metrics/stream.yml.hbs, enabling role assumption with an External ID
  • Part of ingest-dev#8802 — enabling Identity Federation across agentless-eligible aws package streams

Note: Cloud Connectors automatic token exchange (use_cloud_connectors) is not yet supported by the aws-cloudwatch and aws/metrics Beats input types. This PR enables the role-assumption path (user provides role_arn + external_id); the Cloud Connectors path will follow in a future Beats release.

Paired cloudbeat PR: elastic/cloudbeat#7622 (adds logs:DescribeLogGroups, logs:FilterLogEvents, cloudwatch:ListMetrics, cloudwatch:GetMetricData to the federated IAM role)

Depends on: #20437 (Config), #20439 (Inspector) — carries federation infrastructure from those PRs; hold merge until both predecessors land

E2E test plan

What this run uniquely proves: role assumption with external_id works for both aws-cloudwatch and aws/metrics input types when using the Identity Federation credential group.

  1. Create the federated IAM role via the paired cloudbeat CFT (or the deployed cloudformation-federated-identity-aws-9.4.0.yml)
  2. In Fleet, create a CloudWatch policy with credential type = Identity Federation; enter the role ARN and external ID from the CFT output
  3. Enable the cloudwatch_logs data stream; confirm logs appear in logs-aws.cloudwatch_logs-*
  4. Enable the cloudwatch_metrics data stream; confirm metrics appear in metrics-aws.cloudwatch_metrics-*
  5. Verify no DEGRADED status in Fleet for either data stream
  6. Regression: Create a second policy with Direct Access Keys; confirm it still collects data (gate removal must not break non-federation paths)

seanrathier and others added 4 commits July 31, 2026 16:53
Migrate the Inspector HTTPJSON template from hand-rolled SigV4 signing
to the input's auth.aws configuration. The template previously built
the Authorization header in Go template expressions — twice, because
the body changes between pagination pages — and never set
X-Amz-Security-Token, which silently broke temporary credentials.
With auth.aws the input signs every request (including re-signed
pagination pages) at the transport layer.

- Add the full auth.aws block including the use_cloud_connectors hook
  driven by supports_identity_federation (same pattern as GuardDuty)
- Remove the manual X-Amz-Date and Authorization transforms and the
  pagination re-signing steps
- Remove the hide_in_var_group_options gate from the inspector policy
  template, making Identity Federation selectable on the agentless path
- Fixes session-token auth and adds Role ARN / shared-credential
  support, none of which the hand-rolled signing handled

Part of elastic/ingest-dev#8802. The paired cloudbeat PR mirrors the
stream's single API call (inspector2:ListFindings) into the static
fallback template.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Removes the identity_federation hide gate from the aws-cloudwatch and
aws/metrics inputs in the cloudwatch policy template, and adds
external_id support to both stream templates. This enables role
assumption with an External ID when using the Identity Federation
credential group for cloudwatch_logs and cloudwatch_metrics.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Changelog link mismatch — expected https://github.com/elastic/integrations/pull/20525 in the following file(s):

  • packages/aws/changelog.yml

Tip

If expected, add the changelog-link-check:skip label to skip this check. Or, if an issue link was intended, use .../issues/<n> instead.

View Buildkite build
@seanrathier

@infra-vault-gh-plugin-prod

infra-vault-gh-plugin-prod Bot commented Aug 4, 2026

Copy link
Copy Markdown

💔 Build Failed

Failed CI Steps

History

@seanrathier

Copy link
Copy Markdown
Contributor Author

Closing in favour of a single combined PR covering all remaining gated policy templates.

@seanrathier seanrathier closed this Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant