From d996714b9bc2b9903080b2df7ec07a6ade2fc4cb Mon Sep 17 00:00:00 2001 From: Dmitriy Rusov Date: Tue, 18 Aug 2026 11:47:36 +0200 Subject: [PATCH 1/4] docs(monitoring): document Query History export tier requirement and GCS auth (CUB-3900) (#11577) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs(monitoring): document Query History export tier requirement and GCS auth (CUB-3900) Query History export is gated on the deployment's Monitoring Integrations Tier being set to Medium, and below that tier it fails completely silently — no events reach any sink, including a console sink, and nothing is logged. That was undocumented and cost a four-week support thread. - Document the tier requirement and the silent failure on the monitoring integrations page, next to the existing plan-availability note, and cross-reference it from the Query History export recipe. - Add a Google Cloud Storage guide covering authentication: credentials_base64, the /etc/credentials/.json mount that credentials_path must point at, and the lowercase-and-dashes sink-name constraint. - Disable the healthcheck in the GCS example, as the S3 example already does, with the reason: the gcp_cloud_storage healthcheck HEADs the bucket root and needs storage.objects.list, which roles/storage.objectCreator does not grant. * docs(monitoring): clarify GCS credentials ownership from review (CUB-3900) - Split the authentication table by which side consumes each option: credentials_base64 is handled by Cube and removed before the config reaches Vector, which is why it is absent from Vector's own sink reference; credentials_path is Vector's. A reader cross-checking the linked reference could otherwise have dropped the option the setup depends on. - Paraphrase the healthcheck failure instead of quoting an exact log string, and note that with the healthcheck disabled the sink always reports healthy, so the bucket is the thing to check. - Make "Exporting Query History" an H3 under Configuration, matching "Exporting logs". - Add the missing trailing newline to the monitoring integrations page. * docs(monitoring): drop credentials_path from GCS examples (CUB-3900) credentials_base64 is the only supported way to supply GCS credentials, so documenting credentials_path alongside it implied a second, configurable knob that is not part of the supported surface. - Remove credentials_path from the example and from the authentication section. - State that credentials_base64 is the only supported mechanism, and that the credentials file is named after the sink — which is what makes the sink-name constraint load-bearing. - Note that the environment variable name must start with CUBE_CLOUD_MONITORING_, since only variables with that prefix are exposed to the Vector agent. * docs(monitoring): tighten GCS authentication wording from review (CUB-3900) - Restore the Kubernetes-object-name reason for the sink-name constraint. The previous rewrite justified it with "the credentials file is named after the sink", which does not explain why underscores fail — filenames accept them. - Drop the claim that there is "no path to configure". Removing credentials_path reflects the supported surface, but asserting that no path needs configuring states a mechanism that is not verifiable from this repo. - Say "referencing the environment variable" rather than "set to the name of" it: the value is a $-prefixed reference, and the literal name would be base64-decoded as a string. - Drop the duplicated credentials_base64 code block; the example above it already shows the option. --- .../monitoring-integrations/gcs.mdx | 107 ++++++++++++++++++ .../monitoring-integrations/index.mdx | 20 +++- .../admin/monitoring/query-history-export.mdx | 9 ++ docs-mintlify/docs.json | 1 + 4 files changed, 136 insertions(+), 1 deletion(-) create mode 100644 docs-mintlify/admin/monitoring/monitoring-integrations/gcs.mdx diff --git a/docs-mintlify/admin/monitoring/monitoring-integrations/gcs.mdx b/docs-mintlify/admin/monitoring/monitoring-integrations/gcs.mdx new file mode 100644 index 0000000000000..ec47c760e85db --- /dev/null +++ b/docs-mintlify/admin/monitoring/monitoring-integrations/gcs.mdx @@ -0,0 +1,107 @@ +--- +title: Integration with Google Cloud Storage +sidebarTitle: Google Cloud Storage +description: Google Cloud Storage is a popular object storage system. This guide demonstrates how to set up Cube to export logs to Google Cloud Storage. +--- + +[Google Cloud Storage](https://cloud.google.com/storage) is a popular object +storage system. This guide demonstrates how to set up Cube to export logs to +Google Cloud Storage. + +## Configuration + +First, enable [monitoring integrations][ref-monitoring-integrations] in Cube. + +### Exporting logs + +To export logs to Google Cloud Storage, start by creating a bucket for Cube logs +and a service account that can write to it. + +Then, put the service account key in an environment variable under **Settings → +Environment variables**, base64-encoded. The name must start with +`CUBE_CLOUD_MONITORING_` — only variables with that prefix are available to the +Vector agent: + +```bash +CUBE_CLOUD_MONITORING_GCS_CREDENTIALS=eyJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsIC4uLn0= +``` + +Finally, configure the [`gcp_cloud_storage`](https://vector.dev/docs/reference/configuration/sinks/gcp_cloud_storage/) +sink in your [`vector.toml` configuration file][ref-monitoring-integrations-conf]. + +Example configuration: + +```toml +[sinks.gcp-cloud-storage] +type = "gcp_cloud_storage" +inputs = [ + "cubejs-server", + "refresh-scheduler", + "warmup-job", + "cubestore" +] +bucket = "your-gcs-bucket-name" +compression = "gzip" +credentials_base64 = "$CUBE_CLOUD_MONITORING_GCS_CREDENTIALS" + +[sinks.gcp-cloud-storage.encoding] +codec = "json" + +[sinks.gcp-cloud-storage.healthcheck] +enabled = false +``` + +Commit the configuration for Vector, it should take effect in a minute. Then, +navigate to your bucket and watch the logs coming. + +### Authentication + +Authenticate with the `credentials_base64` option, referencing the environment +variable that holds the base64-encoded service account key, as in the example above. +This is the only supported way to give the sink its credentials. + +`credentials_base64` is specific to Cube — it does not appear in [Vector's own sink +reference][vector-docs-sinks-gcs]. Cube decodes the key and provides it to the Vector +agent as a file named after the sink. + + + +The sink name becomes the name of a Kubernetes object that carries the credentials +file, so it must be lowercase alphanumeric characters and dashes only — **no +underscores**. A sink named `query_history_gcs` will not get a credentials file; name +it `query-history-gcs` instead. + + + +### Healthcheck + +The example above sets `healthcheck.enabled = false`. Vector's +`gcp_cloud_storage` healthcheck sends a `HEAD` request to the bucket root, which +requires the `storage.objects.list` permission — write access alone, such as +`roles/storage.objectCreator`, does not grant it. Without this, the sink fails to +start with a forbidden-healthcheck error even though it could write objects +successfully. + +With the healthcheck disabled, the sink always starts and reports healthy, so check +the bucket itself to confirm that data is arriving. + +### Exporting Query History + +Add the `query-history` input to the sink to bring [Query History +export][ref-query-history-export] data to the same bucket. + + + +Query History export additionally requires the **Monitoring Integrations Tier** of +your deployment to be set to **Medium (Up to 50 GB/mo)**. On a lower tier it fails +silently — the sink reports healthy and the bucket stays empty, with no error +logged anywhere. + + + + +[ref-monitoring-integrations]: /admin/monitoring/monitoring-integrations +[ref-monitoring-integrations-conf]: /admin/monitoring/monitoring-integrations#configuration +[ref-query-history-export]: /admin/monitoring/monitoring-integrations#query-history-export +[vector-docs-sinks-gcs]: + https://vector.dev/docs/reference/configuration/sinks/gcp_cloud_storage/ diff --git a/docs-mintlify/admin/monitoring/monitoring-integrations/index.mdx b/docs-mintlify/admin/monitoring/monitoring-integrations/index.mdx index 12e94e6a502a9..4750db6e66546 100644 --- a/docs-mintlify/admin/monitoring/monitoring-integrations/index.mdx +++ b/docs-mintlify/admin/monitoring/monitoring-integrations/index.mdx @@ -54,6 +54,9 @@ following guides and configuration examples to get tool-specific instructions: Export logs and metrics to Datadog. + + Archive logs to a Google Cloud Storage bucket. + Export logs and metrics to Grafana Cloud. @@ -291,6 +294,20 @@ available on the [Enterprise plan](https://cube.dev/pricing). + + +Query History export also requires the **Monitoring Integrations Tier** of your +deployment to be set to **Medium (Up to 50 GB/mo)**. You can find it under +**Settings → Monitoring Integrations**; deployments default to **X-Small (Up to 10 +GB/mo)**. + +On **X-Small** or **Small**, Query History export fails silently: no events reach +any sink, including a `console` sink, and no error is logged. If the tier is not +available in your deployment settings, ask your Cube contact or the [Cube support +team][ref-support] to set it. + + +