Add GCP telemetry integration#238
Conversation
arseniycodes
force-pushed
the
ash/gcp-integration
branch
from
908b821 to
9523789
Compare
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review completed against the latest diff
Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.
Re-trigger cubic
arseniycodes
force-pushed
the
ash/gcp-integration
branch
from
9523789 to
ef659cc
Compare
|
@cubic-dev-ai review this PR against the latest head, including the fixes for the prior findings. |
@arseniycodes I can't start this review because your workspace has reached its monthly review limit. cubic has reviewed 81,271 of the 80,000 allowed lines of code this month. Reviews resume on 3 August 2026 (in 20 days). Enable flex capacity to cover overages automatically and resume reviews now. Learn how flex capacity works. To help optimise your usage, you can tune cubic to get the most out of your usage limits:
|
|
@codex review pls |
arseniycodes
force-pushed
the
ash/gcp-integration
branch
from
ef659cc to
315ce3c
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ef659ccb4c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: de8c23c25c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 783f79393e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f07724eda4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| eq(schema.gcpConnections.projectId, row.projectId), | ||
| ne(schema.gcpConnections.id, row.id), | ||
| eq(schema.gcpConnections.status, "connected"), | ||
| isNull(schema.gcpConnections.revokedAt), | ||
| ), |
There was a problem hiding this comment.
When two OAuth callbacks for different new GCP connections on the same Superlog project overlap, each request only deprovisions the superseded row it observed before provisioning, but this transaction revokes every other connected row at commit time. If callback B commits after callback A, B marks A revoked without ever calling gateway.deprovision for A's sink/topic/subscription, leaving live Google resources with no active DB row. Restrict this update to the specific superseded connection that was actually cleaned up, or serialize connection completion per project.
Useful? React with 👍 / 👎.
| body: JSON.stringify({ | ||
| policy: removeMember(projectPolicy, "roles/monitoring.viewer", readerMember), |
There was a problem hiding this comment.
If the same customer GCP project is connected to another Superlog project, that later connection will see the shared reader service account already present and store monitoringViewerGrantCreated=false. When this older connection is replaced or cleaned up, this branch still removes the shared roles/monitoring.viewer member from the customer project, so the other active connection's metrics pulls start failing even though its DB row remains connected. Check for other non-revoked connections to the same gcpProjectId before removing the IAM member, or make the grant ownership per connection.
Useful? React with 👍 / 👎.
| .object({ | ||
| otlp: z.object({ traces: z.boolean(), logs: z.boolean(), metrics: z.boolean() }).strict(), | ||
| aws: z.object({ logs: z.boolean(), metrics: z.boolean() }).strict(), | ||
| gcp: z.object({ logs: z.boolean(), metrics: z.boolean() }).strict(), |
There was a problem hiding this comment.
During a rolling deploy or for users with a cached pre-GCP web bundle, the ingest-filter PUT still sends the full previous state without a gcp key. Because this schema is strict and now requires gcp, those otherwise valid saves return 400 and no filter changes are persisted until the client reloads. Treat a missing gcp section as enabled-by-default to keep existing clients compatible with the new source.
Useful? React with 👍 / 👎.
1 participant
Summary
0098database migration for GCP connection stateCost behavior
Pub/Sub and Monitoring API quota are billed to the integration project. The persisted monthly reservation prevents concurrent workers from starting metric reads beyond the configured cap.
Validation
Summary by cubic
Add Google Cloud telemetry integration: connect a GCP project via OAuth, stream logs via authenticated Pub/Sub push, and pull curated Cloud Monitoring metrics under a monthly cap. Adds Settings UI, hardened provisioning and safe replacement, and fixes cleanup provenance and metric checkpoints. Creates a new
gcp_connectionstable.New Features
@superlog/gcp-auth, enforces a per-connection monthly returned-series cap (default 100M), and improves metric conversion (typed values and distributions); quota is billed to the integration project./gcp/pull/metrics.gcplogs/metrics.gcpConnectenabled only when required env vars are set.Migration
0104and0105to create and extendgcp_connections.GCP_OAUTH_CLIENT_ID,GCP_OAUTH_CLIENT_SECRET,GCP_OAUTH_REDIRECT_URI,GCP_INTEGRATION_PROJECT_ID,GCP_READER_SERVICE_ACCOUNT_EMAIL,GCP_PUBSUB_PUSH_SERVICE_ACCOUNT_EMAIL,GCP_PUBSUB_PUSH_ENDPOINT,GCP_PUBSUB_PUSH_AUDIENCE,GCP_WORKLOAD_IDENTITY_CONFIG(when not on GCP).GCP_PUBSUB_PUSH_SERVICE_ACCOUNT_EMAIL,GCP_PUBSUB_PUSH_AUDIENCE.GCP_INTEGRATION_PROJECT_ID,GCP_METRICS_INTAKE_URL,GCP_METRICS_MONTHLY_SERIES_LIMIT(optional),GCP_WORKLOAD_IDENTITY_CONFIG(when not on GCP).google-auth-libraryadded toapi,proxy, andworker; new workspace package@superlog/gcp-authused byapiandworker.Written for commit f07724e. Summary will update on new commits.