ECI-2165: add CUSTOM_DD_SITE for custom intake endpoints - #185
Open
yuhuyoyo wants to merge 4 commits into
Open
Conversation
Introduce client.IntakeURL to centralize intake URL construction across
the events, logs, and metrics forwarders. When CUSTOM_DD_SITE is set, it
provides the full host base following the (dot→dash) prefix, keeping the
whole host under a single wildcard certificate (e.g. *.mrf.datadoghq.com)
so no per-customer certificate is needed. When unset, behavior is
unchanged (standard {prefix}.{DD_SITE}{path} form).
Co-Authored-By: Claude Code <noreply@anthropic.com>
yuhuyoyo
changed the title
yuhuyoyo
marked this pull request as ready for review
September 4, 2026 19:32
Add a custom_datadog_site variable (default "") to both the datadog-terraform-onboarding and datadog-integration roots, threading it through to the regional-stacks function config as CUSTOM_DD_SITE. In the integration root the value flows through the create_apply_regional_stack.sh shell arg and Resource Manager variables JSON. Also surface it as an advanced option in the ORM schema.yaml so it only appears when "Use advanced options" is enabled. Empty by default, so existing deployments are unaffected. Co-Authored-By: Claude Code <noreply@anthropic.com>
Contributor
|
Version bumped: Override if needed:
|
Address code-review findings on the CUSTOM_DD_SITE change: - backfillBucketName matched strings.Contains on the full URL, so a CUSTOM_DD_SITE whose host contained "ocimetrics"/"cloudchanges"/"logs" could misroute 5xx payloads to the wrong backfill bucket. Match on the API path (/api/v2/...) instead, which is stable and host-independent. - IntakeURL now sanitizes CUSTOM_DD_SITE: trims whitespace and strips any pasted "https://"/"http://" scheme and trailing slashes so a host base misconfiguration produces a well-formed URL rather than a malformed one. - Align the custom_datadog_site variable description across all four variables.tf files (the two module-level copies were missing the wildcard-certificate rationale). Add TestBackfillBucketName and extend TestIntakeURL with sanitization cases. Co-Authored-By: Claude Code <noreply@anthropic.com>
rheei
reviewed
Sep 4, 2026
| This is the "Site Parameter" in the following <a href="https://docs.datadoghq.com/getting_started/site/#access-the-datadog-site" target="_blank">table from the Datadog documentation</a> . | ||
| required: true | ||
|
|
||
| custom_datadog_site: |
Contributor
There was a problem hiding this comment.
should this option go in the advanced config section if it is uncommon
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
CUSTOM_DD_SITEenvironment variable that lets the OCI forwarders send to custom intake endpoints without provisioning a per-customer TLS certificate.Introduces
client.IntakeURL(prefix, path, site)to centralize intake URL construction across the events, logs, and metrics forwarders (previously each handler inlined its ownfmt.Sprintf).Why
The certificate covering custom intake endpoints has its SAN at a wildcard level (e.g.
*.mrf.datadoghq.com). To stay under that single cert for any customer, the host must use dashes instead of dots to separate the prefix labels — e.g.http-intake-logs-customerA.mrf.datadoghq.comrather thanhttp-intake.logs.customerA.mrf.datadoghq.com. This avoids creating a certificate per customer.How it works
IntakeURLbuilds the intake URL for a given host prefix and API path:CUSTOM_DD_SITEunset (default, unchanged behavior):e.g.
https://http-intake.logs.datadoghq.com/api/v2/logsCUSTOM_DD_SITEset — provides the full host base following the (dot→dash) prefix:e.g. with
CUSTOM_DD_SITE=customerA.mrf.datadoghq.com:https://cloudplatform-intake-customerA.mrf.datadoghq.com/api/v2/cloudchangeshttps://http-intake-logs-customerA.mrf.datadoghq.com/api/v2/logshttps://ocimetrics-intake-customerA.mrf.datadoghq.com/api/v2/ocimetricsThe base domain is not hardcoded —
CUSTOM_DD_SITEcarries the full base, so it can be repointed (e.g. offmrf.datadoghq.com) with no code change.Changes
lib/client/client.go— newIntakeURLhelper.events-forwarder/internal/handler/handler.go— useIntakeURL.logs-forwarder/internal/handler/handler.go— useIntakeURL.metrics-forwarder/internal/handler/handler.go— useIntakeURL.lib/client/client_test.go—TestIntakeURLcovering default and custom-site forms.Verification
go build ./...passes forlib,events-forwarder,logs-forwarder,metrics-forwarder.TestIntakeURLpasses (5 subtests).backfillBucketName) is unaffected — it matches on path substrings (cloudchanges/logs/ocimetrics), which are preserved.Out of scope / follow-ups
CUSTOM_DD_SITEinto the function config (datadog-terraform-onboarding/datadog-integrationlocals.tf) is not included in this PR. The forwarders read the env var directly, so it can be set on the function config out-of-band until the Terraform variable is added.🤖 Generated with Claude Code