diff --git a/docs-mintlify/docs/explore-analyze/workbooks/python-analysis.mdx b/docs-mintlify/docs/explore-analyze/workbooks/python-analysis.mdx index dbbb09db70176..74daa648ae53f 100644 --- a/docs-mintlify/docs/explore-analyze/workbooks/python-analysis.mdx +++ b/docs-mintlify/docs/explore-analyze/workbooks/python-analysis.mdx @@ -3,6 +3,14 @@ title: Python analysis description: Attach a Python script to a report to run forecasting, regression, cohort, and other analysis that SQL can't express, and save the result as a re-runnable report. --- + + +Python analysis is currently in preview, and the user experience and the script +contract may still change. Reach out to the [Cube support +team](/admin/account-billing/support) to activate this feature for your account. + + + A report can carry an attached **Python script** that transforms the report's SQL result. The report's chart then renders the script's **output** instead of the raw SQL rows. This turns an analysis that would otherwise scroll away in a chat @@ -38,24 +46,24 @@ SQL report, that is usually correct behavior. -### In a workbook +### From the toolbar -Open the tab menu and choose **Add Python**. Cube seeds a starter script and -reveals the code panel. **Remove Python** detaches the script, after which the tab -behaves like any SQL-backed report again. +Workbooks and [Explore](/docs/explore-analyze/explore) share the same flow. -Attaching Python clears any existing SQL result: a python-backed report renders its -last Python run, and a freshly attached script has none until you press **Run**. +Click **Python** in the toolbar to open the Python panel, then **Add script** to +attach one. Cube seeds a starter script and opens it on the **Script** tab. +Opening the panel does not attach anything by itself — only **Add script** does. -### In Explore +**Remove**, in the panel header, detaches the script, after which the report +behaves like any SQL-backed one again. -Use the **Add Python** button in the header, with the same **Remove Python** -counterpart. +Attaching Python clears any existing SQL result: a python-backed report renders its +last Python run, and a freshly attached script has none until you press **Run**. -This is only available on a **saved** [exploration](/docs/explore-analyze/explore). -On an unsaved one the button is disabled with the tooltip *"Save exploration before -adding Python."* — **Run** executes server-persisted code, so the analysis needs a -saved report to live on. +Attaching Python in Explore is only available on a **saved** exploration. On an +unsaved one the **Python** button is disabled with the tooltip *"Save the +exploration to add Python"* — **Run** executes server-persisted code, so the +analysis needs a saved report to live on. ## Writing the script @@ -68,7 +76,7 @@ The script runs in a sandbox against a fixed contract: A top-level dict or object is rejected — flatten any nested structure into one array of uniform rows. -{/* TODO: screenshot — the Add Python tab menu entry */} +{/* TODO: screenshot — the Python panel's Add script button */} ## Python environment @@ -100,7 +108,8 @@ adding packages to the environment is coming. ## Running and refreshing -The code panel is editable in place, with line numbers. +The panel's **Script** tab is editable in place, with line numbers. **Reset** +restores the starter template. - **Edits do not run anything.** They save with the report, and the rendered result keeps showing the previous run. @@ -109,10 +118,14 @@ The code panel is editable in place, with line numbers. after the last run. Run to refresh the saved result."* - **Run** executes the stored script in the sandbox and persists the refreshed result. +- The **Output** tab shows what the last run printed — the script's stdout and + stderr, so `print()` is how you inspect intermediate values. Both are captured up + to the cap in [Limits](#limits), so a chatty script gets truncated. - The **input SQL panel is read-only** on a python report: that SQL is the - sandbox's input, not what gets charted. -- **A failed run keeps the previous chart.** The error and the script's - stdout/stderr surface in the UI while the last good result stays rendered. + sandbox's input, not what gets charted. It still offers the **Semantic SQL** and + **Generated SQL** tabs, both derived from that input query. +- **A failed run keeps the previous chart.** The error surfaces alongside the last + successful result, which stays rendered. **Run is the only way the saved result changes.** Opening the report, reloading the page, or viewing a dashboard never re-runs anything on its own. diff --git a/docs-mintlify/docs/explore-analyze/workbooks/querying-data.mdx b/docs-mintlify/docs/explore-analyze/workbooks/querying-data.mdx index a6139f2fdd90b..bc25e05b7a59c 100644 --- a/docs-mintlify/docs/explore-analyze/workbooks/querying-data.mdx +++ b/docs-mintlify/docs/explore-analyze/workbooks/querying-data.mdx @@ -241,7 +241,7 @@ the math. ## Inspecting queries -Every query in the Semantic Query tab can be inspected as code. Open the SQL panel via the **SQL** button in the top-right toolbar to see the query that the workbook generates, switch between representations, and copy it for use elsewhere. +Every query in the Semantic Query tab can be inspected as code. Open the SQL panel via the **SQL** button in the toolbar, next to the Results and Chart tabs, to see the query that the workbook generates, switch between representations, and copy it for use elsewhere. diff --git a/rust/cubestore/Cargo.lock b/rust/cubestore/Cargo.lock index 664af2863965c..fc9954196b596 100644 --- a/rust/cubestore/Cargo.lock +++ b/rust/cubestore/Cargo.lock @@ -95,17 +95,6 @@ dependencies = [ "subtle", ] -[[package]] -name = "ahash" -version = "0.7.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "43bb833f0bf979d8475d38fbf09ed3b8a55e1885fe93ad3f93239fc6a4f17b98" -dependencies = [ - "getrandom 0.2.14", - "once_cell", - "version_check", -] - [[package]] name = "ahash" version = "0.8.11" @@ -254,7 +243,7 @@ name = "arrow-array" version = "54.2.1" source = "git+https://github.com/cube-js/arrow-rs.git?branch=cube-46.0.1#008ff5b21afa64ec7dfd6c2c41526fff4b37db7b" dependencies = [ - "ahash 0.8.11", + "ahash", "arrow-buffer", "arrow-data", "arrow-schema", @@ -390,7 +379,7 @@ name = "arrow-select" version = "54.2.1" source = "git+https://github.com/cube-js/arrow-rs.git?branch=cube-46.0.1#008ff5b21afa64ec7dfd6c2c41526fff4b37db7b" dependencies = [ - "ahash 0.8.11", + "ahash", "arrow-array", "arrow-buffer", "arrow-data", @@ -1853,7 +1842,7 @@ name = "datafusion-common" version = "46.0.1" source = "git+https://github.com/cube-js/arrow-datafusion?branch=cube-46.0.1#ea204976c12c057d7c315ae13040264258b23131" dependencies = [ - "ahash 0.8.11", + "ahash", "arrow", "arrow-ipc", "base64 0.22.1", @@ -2003,7 +1992,7 @@ name = "datafusion-functions-aggregate" version = "46.0.1" source = "git+https://github.com/cube-js/arrow-datafusion?branch=cube-46.0.1#ea204976c12c057d7c315ae13040264258b23131" dependencies = [ - "ahash 0.8.11", + "ahash", "arrow", "datafusion-common", "datafusion-doc", @@ -2023,7 +2012,7 @@ name = "datafusion-functions-aggregate-common" version = "46.0.1" source = "git+https://github.com/cube-js/arrow-datafusion?branch=cube-46.0.1#ea204976c12c057d7c315ae13040264258b23131" dependencies = [ - "ahash 0.8.11", + "ahash", "arrow", "datafusion-common", "datafusion-expr-common", @@ -2123,7 +2112,7 @@ name = "datafusion-physical-expr" version = "46.0.1" source = "git+https://github.com/cube-js/arrow-datafusion?branch=cube-46.0.1#ea204976c12c057d7c315ae13040264258b23131" dependencies = [ - "ahash 0.8.11", + "ahash", "arrow", "datafusion-common", "datafusion-expr", @@ -2144,7 +2133,7 @@ name = "datafusion-physical-expr-common" version = "46.0.1" source = "git+https://github.com/cube-js/arrow-datafusion?branch=cube-46.0.1#ea204976c12c057d7c315ae13040264258b23131" dependencies = [ - "ahash 0.8.11", + "ahash", "arrow", "datafusion-common", "datafusion-expr-common", @@ -2175,7 +2164,7 @@ name = "datafusion-physical-plan" version = "46.0.1" source = "git+https://github.com/cube-js/arrow-datafusion?branch=cube-46.0.1#ea204976c12c057d7c315ae13040264258b23131" dependencies = [ - "ahash 0.8.11", + "ahash", "arrow", "arrow-ord", "arrow-schema", @@ -2592,6 +2581,12 @@ version = "1.0.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1" +[[package]] +name = "foldhash" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb" + [[package]] name = "foreign-types" version = "0.3.2" @@ -2900,22 +2895,13 @@ dependencies = [ "num-traits 0.2.19", ] -[[package]] -name = "hashbrown" -version = "0.11.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ab5ef0d4909ef3724cc8cce6ccc8572c5c817592e9285f5464f8e86f8bd3726e" -dependencies = [ - "ahash 0.7.4", -] - [[package]] name = "hashbrown" version = "0.14.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" dependencies = [ - "ahash 0.8.11", + "ahash", "allocator-api2", ] @@ -2925,6 +2911,17 @@ version = "0.15.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5971ac85611da7067dbfcabef3c70ebb5606018acd9e2a3903a0da507521e0d5" +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" +dependencies = [ + "allocator-api2", + "equivalent", + "foldhash", +] + [[package]] name = "headers" version = "0.3.4" @@ -3772,11 +3769,11 @@ dependencies = [ [[package]] name = "lru" -version = "0.6.6" +version = "0.18.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7ea2d928b485416e8908cff2d97d621db22b27f7b3b6729e438bcf42c671ba91" +checksum = "5d2f2f9b4ba7e6b24d95e7e899329d35be83bcded72c8540cdd5368932d1d90a" dependencies = [ - "hashbrown 0.11.2", + "hashbrown 0.17.1", ] [[package]] @@ -4626,7 +4623,7 @@ version = "54.2.1" source = "git+https://github.com/cube-js/arrow-rs.git?branch=cube-46.0.1#008ff5b21afa64ec7dfd6c2c41526fff4b37db7b" dependencies = [ "aes-gcm", - "ahash 0.8.11", + "ahash", "arrow-array", "arrow-buffer", "arrow-cast", @@ -4977,7 +4974,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e9552f850d5f0964a4e4d0bf306459ac29323ddfbae05e35a7c0d35cb0803cc5" dependencies = [ "anyhow", - "itertools 0.10.1", + "itertools 0.13.0", "proc-macro2", "quote", "syn 2.0.87", @@ -6815,8 +6812,8 @@ version = "1.6.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "97fee6b57c6a41524a810daee9286c02d7752c4253064d0b05472833a438f675" dependencies = [ - "cfg-if 0.1.10", - "rand 0.7.3", + "cfg-if 1.0.0", + "rand 0.8.5", "static_assertions", ] diff --git a/rust/cubestore/cubestore/Cargo.toml b/rust/cubestore/cubestore/Cargo.toml index 8e0907d4fe0ee..7a98f838889e9 100644 --- a/rust/cubestore/cubestore/Cargo.toml +++ b/rust/cubestore/cubestore/Cargo.toml @@ -92,7 +92,7 @@ opentelemetry-otlp = { version = "0.26.0", default-features = false, features = "trace", "metrics", "logs", "http-proto", "http-json", "reqwest-client", "tokio" ] } opentelemetry-http = { version = "0.26.0", features = ["reqwest"] } -lru = "0.6.5" +lru = "0.18.2" moka = { version = "0.10.1", features = ["future"] } ctor = "0.1.20" json = "0.12.4" diff --git a/rust/cubestore/cubestore/src/sql/cache.rs b/rust/cubestore/cubestore/src/sql/cache.rs index 6e441c7ee076e..d0616d19b48ea 100644 --- a/rust/cubestore/cubestore/src/sql/cache.rs +++ b/rust/cubestore/cubestore/src/sql/cache.rs @@ -9,6 +9,7 @@ use futures::Future; use log::trace; use moka::future::{Cache, ConcurrentCacheExt, Iter}; use std::collections::{HashMap, HashSet}; +use std::num::NonZeroUsize; use std::sync::Arc; use std::time::{Duration, Instant}; use tokio::sync::{watch, Mutex}; @@ -120,7 +121,10 @@ impl SqlResultCache { }); Self { - queue_cache: Mutex::new(lru::LruCache::new(queue_cache_max_capacity as usize)), + // `LruCache::new` takes NonZeroUsize since lru 0.9; a configured 0 would panic. + queue_cache: Mutex::new(lru::LruCache::new( + NonZeroUsize::new(queue_cache_max_capacity as usize).unwrap_or(NonZeroUsize::MIN), + )), result_cache: cache_builder .max_capacity(capacity_bytes) .weigher(sql_result_cache_sizeof) @@ -420,6 +424,14 @@ mod tests { use std::sync::Arc; use std::time::Duration; + /// A configured capacity of 0 must not panic on startup. + #[test] + fn queue_cache_capacity_zero_does_not_panic() { + let cache = SqlResultCache::new(1 << 20, Some(120), 0, None); + + assert_eq!(cache.queue_cache.blocking_lock().cap().get(), 1); + } + #[tokio::test] async fn simple() -> Result<(), CubeError> { let cache = Arc::new(SqlResultCache::new(1 << 20, Some(120), 1000, None));