Support sequential regional stack applies by yuhuyoyo · Pull Request #181 · DataDog/oracle-cloud-integration

yuhuyoyo · 2026-08-27T15:15:08Z

Summary

add apply_regional_stacks_sequentially to Quick Start so deployments can opt out of concurrent OCI Resource Manager child jobs
expose the setting as a boolean under Quick Start's Advanced configuration options
preserve parallel regional applies by default
share the per-region stack create/update/apply behavior between parallel and sequential execution
scope the option to datadog-integration; direct Terraform onboarding does not create Resource Manager child jobs and is unaffected by this limit

Test plan

Notes

The test tenancy had subscribed regions in Sydney, Toronto, Frankfurt, and Ashburn.
Direct Terraform onboarding intentionally does not expose this variable because the reported concurrency threshold is specific to Resource Manager jobs.
KMS resources in pending deletion remain subject to OCI's deletion window; the final test-created vault, key, and secret were scheduled for deletion by the stack Destroy job.

🤖 Generated with Claude Code

Allow Quick Start deployments to avoid OCI concurrent Resource Manager job limits while preserving parallel applies by default. Co-Authored-By: Claude <noreply@anthropic.com>

github-actions · 2026-08-27T15:15:21Z

Version bumped: 1.2.3 → 1.2.4 (patch auto-bump)

Override if needed:

Breaking change → manually set VERSION and datadog-terraform-onboarding/VERSION to bump the major digit.
New feature → manually bump the minor digit.
Minor bug fix → no change needed; the patch bump above is correct.

datadog-datadog-us1-prod

The shared script sends invalid defined-tag data to every regional child stack. The Quick Start schema also cannot set the new sequential option.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 9d8d653 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

datadog-datadog-us1-prod · 2026-08-27T15:52:08Z

+  --arg home_region "$HOME_REGION" \
+  --arg region_key "$REGION_KEY" \
+  --arg subnet_ocid "$SUBNET_OCID" \
+  --arg defined_tags "$DEFINED_TAGS" \


Remove the extra defined-tags encoding

Every regional child-stack apply can fail when OCI resources receive a string instead of a tag map.

Assertion details

Input: Any regional stack apply, including the default empty defined-tags value.

Expected: The child stack must receive defined_tags as one JSON-encoded map string.

Actual: The caller passes a JSON-encoded string. jq encodes it again. The child stack then decodes it to a string instead of a map.

Suggested change

--arg defined_tags "$DEFINED_TAGS" \

--argjson defined_tags "$DEFINED_TAGS" \

Was this helpful? React 👍 or 👎
_{🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

we might want to test the json encoding for this since I remember the behavior was weird when I first added it

Tested — the encoding is correct now. Verified with a live apply using defined_tags="datadog.env:test": the plan showed "datadog.env" = "test" on resources and the apply succeeded. The earlier weirdness was the double jsonencode(jsonencode(...)) which was fixed in 18017fa; the current single-encode + jq --arg path is right.

datadog-datadog-us1-prod · 2026-08-27T15:52:08Z

+  STACK_ID=$(oci resource-manager stack create --compartment-id "$COMPARTMENT_ID" --display-name "$STACK_NAME" \
+    --config-source ./modules/regional-stacks/dd_regional_stack.zip --variables "$VARIABLES_JSON" \
+    --terraform-version 1.5.x \
+    $DEFINED_TAGS_FLAG \


Pass defined tags as separate shell arguments

Quick Start deployments with defined tags can fail when the script creates, updates, or applies a regional stack.

Assertion details

Input: A Quick Start deployment with one or more defined tags.

Expected: Pass --defined-tags and its JSON value as separate shell arguments. Apply this handling to stack create, stack update, and apply-job creation.

Actual: The caller builds the flag and JSON as one shell value. Shell expansion removes or retains quote characters as data, so OCI receives invalid JSON.

Was this helpful? React 👍 or 👎
_{🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

datadog-datadog-us1-prod · 2026-08-27T15:52:08Z

  default     = ""
 }

+variable "apply_regional_stacks_sequentially" {


Expose the sequential option in Quick Start

Users with low concurrent-job limits cannot enable the sequential mode through the intended Quick Start flow.

Assertion details

Input: A user starts or reapplies the integration through OCI Quick Start and needs sequential regional jobs.

Expected: Declare the option in datadog-integration/schema.yaml and add it to a suitable variable group.

Actual: The Terraform variable exists, but schema.yaml does not declare it or show it in a variable group. Quick Start therefore keeps the false default.

Was this helpful? React 👍 or 👎
_{🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

datadog-datadog-us1-prod

A failed non-final regional apply does not stop the sequential loop. A later successful region can make Terraform report success and register an incomplete integration.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 18017fa · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

datadog-datadog-us1-prod · 2026-08-27T16:33:49Z

+      FAILURE=$(echo "$REGIONS_JSON" | jq -r --arg region "$REGION" '.[$region].failure')
+      REGION_KEY=$(echo "$REGIONS_JSON" | jq -r --arg region "$REGION" '.[$region].region_key')
+      SUBNET_OCID=$(echo "$REGIONS_JSON" | jq -r --arg region "$REGION" '.[$region].subnet_ocid')
+      bash ./create_apply_regional_stack.sh "$REGION" "$FAILURE" "$REGION_KEY" "$SUBNET_OCID" '${module.compartment.id}' '${terraform_data.stack_digest.id}' '${var.tenancy_ocid}' '${var.datadog_site}' '${module.kms[0].api_key_secret_id}' '${local.home_region_name}' '${jsonencode(local.defined_tags)}' '${var.enable_regional_vaults}' '${jsonencode(local.compartment_defined_tags)}'


Stop after a failed sequential regional apply

Terraform can register the integration as successful while regional infrastructure is missing.

Assertion details

Input: Set apply_regional_stacks_sequentially to true. Let a non-final regional apply fail. Let a later region succeed or exit as unsupported.

Expected: The provisioner must stop and return a nonzero status when any regional apply fails.

Actual: The loop continues after the script returns a nonzero status. A later region can return zero and become the status of the complete provisioner command.

Suggested change

bash ./create_apply_regional_stack.sh "$REGION" "$FAILURE" "$REGION_KEY" "$SUBNET_OCID" '${module.compartment.id}' '${terraform_data.stack_digest.id}' '${var.tenancy_ocid}' '${var.datadog_site}' '${module.kms[0].api_key_secret_id}' '${local.home_region_name}' '${jsonencode(local.defined_tags)}' '${var.enable_regional_vaults}' '${jsonencode(local.compartment_defined_tags)}'

bash ./create_apply_regional_stack.sh "$REGION" "$FAILURE" "$REGION_KEY" "$SUBNET_OCID" '${module.compartment.id}' '${terraform_data.stack_digest.id}' '${var.tenancy_ocid}' '${var.datadog_site}' '${module.kms[0].api_key_secret_id}' '${local.home_region_name}' '${jsonencode(local.defined_tags)}' '${var.enable_regional_vaults}' '${jsonencode(local.compartment_defined_tags)}' || exit $?

Was this helpful? React 👍 or 👎
_{🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

Agreed with this

Fixed in 1492a2e — the loop now exits with a nonzero status on the first failed regional apply.

EamonBrady1 · 2026-08-27T16:49:07Z


+variable "apply_regional_stacks_sequentially" {
+  type        = bool
+  description = "Apply regional Resource Manager stacks sequentially to support tenancies with a low concurrent job limit. This setting only affects the Quick Start deployment; it has no effect when using the Terraform onboarding module directly."


Nit: Add that this should not be used unless the customer has stack quotas preventing parallel computation

Done in 1492a2e — updated the schema.yaml description to: "Do not enable this unless the tenancy has a stack quota that prevents parallel Resource Manager jobs."

EamonBrady1

Approving to unblock the customer, but I think we definitely need to make it exit on a failed region since they are silently swallowed now, and I think we should also verify the defined_tags parsing with the new argument, since I know the behavior is weird with trying to pass it as a string versus a json list

yuhuyoyo · 2026-08-27T19:44:09Z

Thanks Eamon. Both addressed in 1492a2e:

Exit on failed region: the sequential loop now aborts with a nonzero status on the first failed regional apply (|| { echo "ERROR: ..." >&2; exit 1; }), so a later succeeding region can no longer mask the failure.
defined_tags parsing: verified the JSON encoding end-to-end with a live Quick Start apply (defined_tags="datadog.env:test"). The child module's var.defined_tags is type=string and does jsondecode(), so the parent passes a single jsonencode() and the script uses jq --arg (string) — the child receives {"datadog.env":"test"} and decodes it to the map correctly. The plan output confirmed "datadog.env" = "test" applied to resources, and the apply succeeded across all four subscribed regions sequentially. (The autotest bot's P0 suggestion to switch to --argjson would have broken this by passing an object to a string-typed variable.)

Also tightened the schema.yaml description per your nit: "Do not enable this unless the tenancy has a stack quota that prevents parallel Resource Manager jobs."

The concurrency limit applies to Resource Manager child jobs, so omit the option from direct Terraform onboarding where it would have no effect. Co-Authored-By: Claude <noreply@anthropic.com>

Make the Resource Manager concurrency workaround selectable from the Quick Start advanced options. Co-Authored-By: Claude <noreply@anthropic.com>

Pass child tags with a single JSON encoding and preserve OCI defined-tag JSON as one quoted CLI argument across regional stack operations. Co-Authored-By: Claude <noreply@anthropic.com>

The sequential regional-stack loop continued after a failed region's apply, letting a later succeeding region mask the failure and report overall success with an incomplete integration. Exit the loop with a nonzero status on the first failure so the parent apply fails loudly. Also clarify the schema.yaml description per review feedback: the sequential option should only be used when a tenancy stack quota prevents parallel Resource Manager jobs. Co-Authored-By: Claude <noreply@anthropic.com>

fix: support sequential regional stack applies

4f571d6

Allow Quick Start deployments to avoid OCI concurrent Resource Manager job limits while preserving parallel applies by default. Co-Authored-By: Claude <noreply@anthropic.com>

datadog-datadog-us1-prod Bot reviewed Aug 27, 2026

View reviewed changes

yuhuyoyo marked this pull request as ready for review August 27, 2026 16:30

yuhuyoyo requested a review from a team as a code owner August 27, 2026 16:30

yuhuyoyo requested review from EamonBrady1 and griffin-michalak August 27, 2026 16:30

datadog-datadog-us1-prod Bot reviewed Aug 27, 2026

View reviewed changes

EamonBrady1 reviewed Aug 27, 2026

View reviewed changes

EamonBrady1 approved these changes Aug 27, 2026

View reviewed changes

griffin-michalak approved these changes Aug 27, 2026

View reviewed changes

gh-worker-dd-devflow-36fce6 Bot added the mergequeue-status: waiting label Aug 27, 2026

github-actions Bot and others added 5 commits August 27, 2026 18:24

chore: bump version 1.2.3 -> 1.2.4

d62a9c9

fix: scope sequential applies to quick start

58c41a5

The concurrency limit applies to Resource Manager child jobs, so omit the option from direct Terraform onboarding where it would have no effect. Co-Authored-By: Claude <noreply@anthropic.com>

feat: expose sequential applies in quick start

06f88a4

Make the Resource Manager concurrency workaround selectable from the Quick Start advanced options. Co-Authored-By: Claude <noreply@anthropic.com>

fix: preserve regional stack defined tags

2cfc2cb

Pass child tags with a single JSON encoding and preserve OCI defined-tag JSON as one quoted CLI argument across regional stack operations. Co-Authored-By: Claude <noreply@anthropic.com>

yuhuyoyo force-pushed the clouds-8671-sequential-regional-applies branch from 1492a2e to 57e2e8d Compare August 27, 2026 22:24

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: waiting mergequeue-status: queued labels Aug 27, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit 2b66ce3 into master Aug 27, 2026
19 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the clouds-8671-sequential-regional-applies branch August 27, 2026 22:26

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: done and removed mergequeue-status: in_progress labels Aug 27, 2026

	--arg defined_tags "$DEFINED_TAGS" \
	--argjson defined_tags "$DEFINED_TAGS" \

	bash ./create_apply_regional_stack.sh "$REGION" "$FAILURE" "$REGION_KEY" "$SUBNET_OCID" '${module.compartment.id}' '${terraform_data.stack_digest.id}' '${var.tenancy_ocid}' '${var.datadog_site}' '${module.kms[0].api_key_secret_id}' '${local.home_region_name}' '${jsonencode(local.defined_tags)}' '${var.enable_regional_vaults}' '${jsonencode(local.compartment_defined_tags)}'
	bash ./create_apply_regional_stack.sh "$REGION" "$FAILURE" "$REGION_KEY" "$SUBNET_OCID" '${module.compartment.id}' '${terraform_data.stack_digest.id}' '${var.tenancy_ocid}' '${var.datadog_site}' '${module.kms[0].api_key_secret_id}' '${local.home_region_name}' '${jsonencode(local.defined_tags)}' '${var.enable_regional_vaults}' '${jsonencode(local.compartment_defined_tags)}' \|\| exit $?

Uh oh!

Conversation

yuhuyoyo commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Notes

Uh oh!

github-actions Bot commented Aug 27, 2026

Uh oh!

datadog-datadog-us1-prod Bot left a comment

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-us1-prod Bot Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

EamonBrady1 Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

yuhuyoyo Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-us1-prod Bot Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-us1-prod Bot Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-us1-prod Bot left a comment

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-us1-prod Bot Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

EamonBrady1 Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

yuhuyoyo Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

EamonBrady1 Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

yuhuyoyo Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

EamonBrady1 left a comment

Choose a reason for hiding this comment

Uh oh!

yuhuyoyo commented Aug 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

yuhuyoyo commented Aug 27, 2026 •

edited

Loading