[pull] master from cube-js:master

docs-mintlify/docs/explore-analyze/charts/configuration/small-multiples.mdx

@@ -23,7 +23,7 @@ These options appear once a **Split by** dimension is chosen.
 
 | Option | What it does |
 |---|---|
-| **Grid** | Columns × rows, up to 5 × 4. Both are preselected from the number of distinct values in the split dimension, so a four-value dimension opens as a 2 × 2 grid. |
+| **Grid** | Columns × rows, up to 5 × 5. Both are preselected from the number of distinct values in the split dimension, so a four-value dimension opens as a 2 × 2 grid. |
 | **Axis scales** | Whether every panel is drawn against the same scale (**Shared**) or each scales to its own data (**Independent**). |
 | **Sort panels by** | Orders the panels by the dimension's own values (**Value**) or by a measure (**Measure**). |
 | **Sort order** | **Ascending** or **Descending**. |
@@ -38,7 +38,7 @@ Switch to **Independent** when the question is about the *shape* of each series
 
 ### How many panels are drawn
 
-The grid bounds the render: a chart split into 3 × 2 draws at most six panels, so a high-cardinality dimension can never produce hundreds of unreadable ones. The largest grid is 5 × 4, or twenty panels.
+The grid bounds the render: a chart split into 3 × 2 draws at most six panels, so a high-cardinality dimension can never produce hundreds of unreadable ones. The largest grid is 5 × 5, or twenty-five panels.
 
 When a dimension has more values than the grid has tiles, the chart draws the first ones in the current sort order. The underlying query is unaffected, so no data is lost — a bigger grid simply shows more of it.
 

docs-mintlify/embedding/multitenancy.mdx

@@ -199,9 +199,10 @@ cube(`products`, {
 
 ### Running in Production
 
-Each unique id generated by `contextToAppId` or `contextToOrchestratorId` will
-generate a dedicated set of resources, including data model compile cache, SQL
-compile cache, query queues, in-memory result caching, etc. Depending on your
+Each unique id generated by `contextToAppId` will generate a dedicated data model
+compile cache and SQL compile cache, and each unique id generated by
+`contextToOrchestratorId` will generate a dedicated query orchestrator with its
+own database connections, query queues, and in-memory result cache. Depending on your
 data model complexity and usage patterns, those resources can have a pretty
 sizable memory footprint ranging from single-digit MBs on the lower end and
 dozens of MBs on the higher end. So you should make sure Node VM has enough
@@ -250,6 +251,40 @@ module.exports = {
 }
 ```
 
+## Same DB Instance with per Tenant Data Model
+
+If [`context_to_app_id`][ref-config-ctx-to-appid] is the only option you define,
+Cube compiles and caches a separate data model for each app id it returns. This
+is what enables [`COMPILE_CONTEXT`][ref-cube-security-ctx] and per-tenant
+[`repository_factory`][ref-config-repofactory], so each tenant can get a
+different data model built from the same or different data model files.
+
+**cube.js:**
+
+```javascript
+module.exports = {
+  contextToAppId: ({ securityContext }) =>
+    `CUBE_APP_${securityContext.tenantId}`
+}
+```
+
+<Warning>
+
+`context_to_app_id` scopes multitenancy to data model compilation only.
+Everything below the data model remains shared across all tenants: database
+connections, query queues, the in-memory query results cache, and
+pre-aggregations, including their tables in the pre-aggregations schema. So
+tenants whose data models generate the same SQL will share queue entries, cached
+results, and pre-aggregation tables.
+
+To isolate those resources per tenant as well, add
+[`pre_aggregations_schema`][ref-config-preagg-schema] for per-tenant
+pre-aggregation tables and [`context_to_orchestrator_id`][ref-config-ctx-to-orch-id]
+for per-tenant connections, queues, and results cache, as shown in the sections
+below.
+
+</Warning>
+
 ## Multiple DB Instances with Same Data Model
 
 Let's consider an example where we store data for different users in different
@@ -280,10 +315,11 @@ select the database, based on the `appId` and `userId`:
 <Warning>
 
 The App ID (the result of [`contextToAppId`][ref-config-ctx-to-appid]) is used
-as a caching key for various in-memory structures like data model compilation
-results, connection pool. The Orchestrator ID (the result of
+as a caching key for the compiled data model and other data model-related
+in-memory structures. The Orchestrator ID (the result of
 [`contextToOrchestratorId`][ref-config-ctx-to-orch-id]) is used as a caching key
-for database connections, execution queues and pre-aggregation table caches. Not
+for the query orchestrator, which holds database connections and their pools,
+execution queues, query results cache, and pre-aggregation table caches. Not
 declaring these properties will result in unexpected caching issues such as the
 data model or data of one tenant being used for another.
 

docs-mintlify/recipes/configuration/custom-data-model-per-tenant.mdx

@@ -23,6 +23,16 @@ from security contexts.
 - [`scheduled_refresh_contexts`][ref-scheduled-refresh-contexts] to provide
 a list of security contexts.
 
+<Note>
+
+`context_to_app_id` makes the data model per-tenant, which is all this recipe
+needs. Querying and caching (database connections, query queues, and
+pre-aggregations) stay shared between tenants unless you also configure
+[`context_to_orchestrator_id`][ref-context-to-orchestrator-id] and
+[`pre_aggregations_schema`][ref-pre-aggregations-schema].
+
+</Note>
+
 Put the following code into your `cube.py` or `cube.js` [configuration
 file][ref-config-files]:
 
@@ -525,6 +535,8 @@ code that fetches data model files for each tenant.
 [ref-multitenancy]: /embedding/multitenancy
 [ref-scheduled-refresh-contexts]: /reference/configuration/config#scheduled_refresh_contexts
 [ref-context-to-app-id]: /reference/configuration/config#context_to_app_id
+[ref-context-to-orchestrator-id]: /reference/configuration/config#context_to_orchestrator_id
+[ref-pre-aggregations-schema]: /reference/configuration/config#pre_aggregations_schema
 [ref-config-files]: /admin/connect-to-data#cubepy-and-cubejs-files
 [ref-mls]: /docs/data-modeling/access-control/member-level-security
 [ref-cubes-public]: /reference/data-modeling/cube#public

docs-mintlify/recipes/configuration/multiple-sources-same-schema.mdx

@@ -18,18 +18,27 @@ same data model.
 
 To enable multitenancy, use the
 [`contextToAppId`](/reference/configuration/config#context_to_app_id) function to
-provide distinct identifiers for each tenant. Also, implement the
+provide distinct identifiers for each tenant's data model and the
+[`contextToOrchestratorId`](/reference/configuration/config#context_to_orchestrator_id)
+function to give each tenant its own database connections and query queues.
+Without the latter, all tenants would share a single connection to whichever
+data source was resolved first. Also, implement the
 [`driverFactory`](/reference/configuration/config#driver_factory) function where
 you can select a data source based on the tenant name.
 [JSON Web Token](/docs/data-modeling/access-control) includes information about the tenant name in
 the `tenant` property of the `securityContext`.
 
 ```javascript
 module.exports = {
-  // Provides distinct identifiers for each tenant which are used as caching keys
+  // Provides a distinct identifier for each tenant's compiled data model
   contextToAppId: ({ securityContext }) =>
     `CUBE_APP_${securityContext.tenant}`,
 
+  // Provides a distinct identifier for each tenant's query orchestrator, so
+  // that every tenant gets its own database connections and query queues
+  contextToOrchestratorId: ({ securityContext }) =>
+    `CUBE_APP_${securityContext.tenant}`,
+
   // Selects the database connection configuration based on the tenant name
   driverFactory: ({ securityContext }) => {
     if (!securityContext.tenant) {

docs-mintlify/reference/configuration/config.mdx

@@ -51,9 +51,27 @@ or when a more flexible setup is needed.
 
 It's a [multitenancy][ref-multitenancy] option.
 
-`context_to_app_id` is a function to determine an app id which is used as
-caching key for various in-memory structures like data model compilation
-results, etc.
+`context_to_app_id` is a function to determine an app id which is used as the
+caching key for the compiled data model and other data model-related in-memory
+structures. Each distinct app id gets its own data model compilation, which is
+what enables [`COMPILE_CONTEXT`][ref-compile-context] and per-tenant
+[`repository_factory`][self-repofactory].
+
+<Warning>
+
+`context_to_app_id` enables multitenancy for data model compilation only. On its
+own, it leaves querying and caching shared across app ids. To isolate those per
+tenant as well:
+
+- [`pre_aggregations_schema`][self-pre-aggregations-schema] gives each tenant its
+  own pre-aggregation tables. It is resolved per app id, so a schema name derived
+  from the security context requires `context_to_app_id` to be defined as well.
+- [`context_to_orchestrator_id`][self-orchestrator-id] gives each tenant its own
+  database connections, query queues, and query results cache.
+
+See [multitenancy][ref-multitenancy] for details.
+
+</Warning>
 
 Called on each request.
 
@@ -633,6 +651,16 @@ be used exclusively by Cube and shall not be shared with any application.
 
 Called once per [`app_id`][self-opts-ctx-to-appid].
 
+<Warning>
+
+Because this option is evaluated once per app id, deriving the schema name from
+the security context requires [`context_to_app_id`][self-opts-ctx-to-appid] to be
+defined as well. Otherwise all tenants share a single app id, and every one of
+them ends up pinned to the schema name resolved for whichever tenant triggered
+the first data model compilation.
+
+</Warning>
+
 <CodeGroup>
 
 ```python title="Python"
@@ -1527,6 +1555,7 @@ module.exports = {
 [ref-rest-scopes]: /reference/core-data-apis/rest-api#api-scopes
 [ref-config-options]: /admin/connect-to-data#configuration-options
 [self-orchestrator-id]: #context_to_orchestrator_id
+[ref-compile-context]: /reference/data-modeling/context-variables#compile_context
 [ref-multiple-data-sources]: /admin/connect-to-data/multiple-data-sources
 [ref-websockets]: /recipes/core-data-api/real-time-data-fetch
 [ref-matching-preaggs]: /docs/pre-aggregations/matching-pre-aggregations

docs-mintlify/reference/core-data-apis/sql-api/security.mdx

@@ -51,8 +51,8 @@ configuration file:
 
 ```javascripttitle="cube.js"
 module.exports = {
-  // Create a new appId for each team, this prevents teams from seeing each
-  // other's data
+  // Create a new appId for each team so that a separate data model is
+  // compiled for each of them and `COMPILE_CONTEXT` can be used below
   // https://cube.dev/docs/product/configuration/reference/config#context_to_app_id
   contextToAppId: ({ securityContext }) => {
     return securityContext.team

examples/recipes/multiple-data-sources/cube.js

@@ -1,10 +1,17 @@
 const PostgresDriver = require('@cubejs-backend/postgres-driver');
 
 module.exports = {
-  // Provides distinct identifiers for each tenant which are used as caching keys
+  // Provides a distinct identifier for each tenant's compiled data model
   contextToAppId: ({ securityContext }) =>
     `CUBEJS_APP_${securityContext.tenant}`,
 
+  // Provides a distinct identifier for each tenant's query orchestrator, so
+  // that every tenant gets its own database connections and query queues.
+  // Without this, all tenants would share the driver created for whichever
+  // tenant connected first.
+  contextToOrchestratorId: ({ securityContext }) =>
+    `CUBEJS_APP_${securityContext.tenant}`,
+
   // Selects the database connection configuration based on the tenant name
   driverFactory: ({ securityContext }) => {
 

rust/cubesql/cubesql/src/compile/test/snapshots/cubesql__compile__test__test_introspection__domo_column_inspection_query.snap

@@ -0,0 +1,25 @@
+---
+source: cubesql/src/compile/test/test_introspection.rs
+expression: "execute_query(r#\"\n            SELECT\n              a.attname AS column_name,\n              format_type(a.atttypid, NULL) AS data_type,\n              a.atttypid = ANY ('{int8,numeric,bool}'::regtype[]) AS is_matched,\n              a.atttypid::regtype = ANY ('{int8,numeric,bool}'::regtype[]) AS is_matched_as_regtype,\n              CASE\n                WHEN a.atttypid = ANY ('{int,int8,int2}'::regtype[]) THEN NULL\n                ELSE CASE\n                  WHEN a.atttypmod = -1 THEN NULL\n                  ELSE (a.atttypmod - 4) >> 16\n                END\n              END AS numeric_precision,\n              CASE\n                WHEN a.atttypid = ANY ('{int,int8,int2}'::regtype[]) THEN NULL\n                ELSE CASE\n                  WHEN a.atttypmod = -1 THEN NULL\n                  ELSE (a.atttypmod - 4) & 65535\n                END\n              END AS numeric_scale\n            FROM pg_catalog.pg_attribute a\n              JOIN pg_catalog.pg_class c ON a.attrelid = c.oid\n              JOIN pg_catalog.pg_namespace n ON c.relnamespace = n.oid\n            WHERE c.relname = 'KibanaSampleDataEcommerce'\n              AND n.nspname = 'public'\n              AND a.attnum > 0\n              AND NOT a.attisdropped\n              AND c.relkind IN ('r', 'v', 'm')\n            ORDER BY a.attnum\n            \"#.to_string(),\nDatabaseProtocol::PostgreSQL).await?"
+---
++--------------------+-----------------------------+------------+-----------------------+-------------------+---------------+
+| column_name        | data_type                   | is_matched | is_matched_as_regtype | numeric_precision | numeric_scale |
++--------------------+-----------------------------+------------+-----------------------+-------------------+---------------+
+| count              | bigint                      | true       | true                  | NULL              | NULL          |
+| maxPrice           | numeric                     | true       | true                  | NULL              | NULL          |
+| sumPrice           | numeric                     | true       | true                  | NULL              | NULL          |
+| minPrice           | numeric                     | true       | true                  | NULL              | NULL          |
+| avgPrice           | numeric                     | true       | true                  | NULL              | NULL          |
+| countDistinct      | bigint                      | true       | true                  | NULL              | NULL          |
+| id                 | numeric                     | true       | true                  | NULL              | NULL          |
+| order_date         | timestamp without time zone | false      | false                 | NULL              | NULL          |
+| last_mod           | timestamp without time zone | false      | false                 | NULL              | NULL          |
+| customer_gender    | text                        | false      | false                 | NULL              | NULL          |
+| notes              | text                        | false      | false                 | NULL              | NULL          |
+| taxful_total_price | numeric                     | true       | true                  | NULL              | NULL          |
+| has_subscription   | boolean                     | true       | true                  | NULL              | NULL          |
+| is_male            | boolean                     | true       | true                  | NULL              | NULL          |
+| is_female          | boolean                     | true       | true                  | NULL              | NULL          |
+| __user             | text                        | false      | false                 | NULL              | NULL          |
+| __cubeJoinField    | text                        | false      | false                 | NULL              | NULL          |
++--------------------+-----------------------------+------------+-----------------------+-------------------+---------------+

rust/cubesql/cubesql/src/compile/test/snapshots/cubesql__compile__test__test_introspection__empty_regtype_array_comparison.snap

@@ -0,0 +1,9 @@
+---
+source: cubesql/src/compile/test/test_introspection.rs
+expression: "execute_query(r#\"\n            SELECT\n              a.atttypid = ANY ('{}'::regtype[]) AS is_matched,\n              a.atttypid::regtype = ANY ('{}'::regtype[]) AS is_matched_as_regtype,\n              a.atttypid::regtype = ANY (ARRAY[]) AS is_matched_in_untyped_array\n            FROM pg_catalog.pg_attribute a\n              JOIN pg_catalog.pg_class c ON a.attrelid = c.oid\n            WHERE c.relname = 'KibanaSampleDataEcommerce'\n              AND a.attnum = 1\n            \"#.to_string(),\nDatabaseProtocol::PostgreSQL).await?"
+---
++------------+-----------------------+-----------------------------+
+| is_matched | is_matched_as_regtype | is_matched_in_untyped_array |
++------------+-----------------------+-----------------------------+
+| false      | false                 | false                       |
++------------+-----------------------+-----------------------------+

rust/cubesql/cubesql/src/compile/test/test_introspection.rs

@@ -3437,3 +3437,81 @@ async fn test_pg_user_mapping() -> Result<(), CubeError> {
 
     Ok(())
 }
+
+/// Domo's live Postgres connection issues this to inspect the columns of a table picked in
+/// its UI. It tests `atttypid` against a set of types spelled as a `regtype[]` literal.
+///
+/// The `is_matched` columns are not part of Domo's query; they project the type test itself
+/// so that the snapshot pins which OIDs a `regtype[]` literal resolves to. Every row here has
+/// `atttypmod = -1`, so Domo's own columns are `NULL` either way.
+#[tokio::test]
+async fn domo_column_inspection_query() -> Result<(), CubeError> {
+    insta::assert_snapshot!(
+        "domo_column_inspection_query",
+        execute_query(
+            r#"
+            SELECT
+              a.attname AS column_name,
+              format_type(a.atttypid, NULL) AS data_type,
+              a.atttypid = ANY ('{int8,numeric,bool}'::regtype[]) AS is_matched,
+              a.atttypid::regtype = ANY ('{int8,numeric,bool}'::regtype[]) AS is_matched_as_regtype,
+              CASE
+                WHEN a.atttypid = ANY ('{int,int8,int2}'::regtype[]) THEN NULL
+                ELSE CASE
+                  WHEN a.atttypmod = -1 THEN NULL
+                  ELSE (a.atttypmod - 4) >> 16
+                END
+              END AS numeric_precision,
+              CASE
+                WHEN a.atttypid = ANY ('{int,int8,int2}'::regtype[]) THEN NULL
+                ELSE CASE
+                  WHEN a.atttypmod = -1 THEN NULL
+                  ELSE (a.atttypmod - 4) & 65535
+                END
+              END AS numeric_scale
+            FROM pg_catalog.pg_attribute a
+              JOIN pg_catalog.pg_class c ON a.attrelid = c.oid
+              JOIN pg_catalog.pg_namespace n ON c.relnamespace = n.oid
+            WHERE c.relname = 'KibanaSampleDataEcommerce'
+              AND n.nspname = 'public'
+              AND a.attnum > 0
+              AND NOT a.attisdropped
+              AND c.relkind IN ('r', 'v', 'm')
+            ORDER BY a.attnum
+            "#
+            .to_string(),
+            DatabaseProtocol::PostgreSQL
+        )
+        .await?
+    );
+
+    Ok(())
+}
+
+/// An empty `regtype[]` literal is rewritten to an empty `ARRAY[]`, which has to plan and
+/// compare false, as Postgres answers for an empty array. Pins both spellings: the column keeps
+/// its OID when the literal resolves, and renders as a type name when there is no regtype to
+/// key off, as with the untyped `ARRAY[]`.
+#[tokio::test]
+async fn empty_regtype_array_comparison() -> Result<(), CubeError> {
+    insta::assert_snapshot!(
+        "empty_regtype_array_comparison",
+        execute_query(
+            r#"
+            SELECT
+              a.atttypid = ANY ('{}'::regtype[]) AS is_matched,
+              a.atttypid::regtype = ANY ('{}'::regtype[]) AS is_matched_as_regtype,
+              a.atttypid::regtype = ANY (ARRAY[]) AS is_matched_in_untyped_array
+            FROM pg_catalog.pg_attribute a
+              JOIN pg_catalog.pg_class c ON a.attrelid = c.oid
+            WHERE c.relname = 'KibanaSampleDataEcommerce'
+              AND a.attnum = 1
+            "#
+            .to_string(),
+            DatabaseProtocol::PostgreSQL
+        )
+        .await?
+    );
+
+    Ok(())
+}