diff --git a/README.md b/README.md index 499d361..0d06628 100644 --- a/README.md +++ b/README.md @@ -92,9 +92,10 @@ Console compatibility slice: Axum and Tauri, with exact locked-classpath startup and forced-read-only metadata sessions; and - an original-frontend compatibility layer for native MySQL connection testing, - datasource CRUD/tree, database/schema/table discovery, and synchronous table - preview over the historical `{success,data,errorCode,errorMessage}` envelope; - and + datasource CRUD/tree, database/schema/table discovery, compact table lists, + columns, indexes, keys, views, functions, procedures, triggers, and synchronous + table preview over the historical `{success,data,errorCode,errorMessage}` + envelope, including the original DDL-list `total` field; and - durable SQLite-backed Community Console create/get/list/update/delete, including SQL text, datasource/database/schema binding, saved status, and open-tab state across process restarts; and @@ -127,6 +128,14 @@ typed Console SELECT, row truncation, active-query cancellation, retained paging, and proof after every operation that Java remained dormant. The complete repository `make verify` gate and the explicit real-MySQL `native-mysql-integration` target also passed after the final compatibility fix. +The metadata parity increment adds a real MySQL fixture with a foreign key, +composite index, view, function, procedure, and trigger; its Core product test, +Axum queries, and desktop dispatch contracts pass while Java remains dormant. +The legacy boundary matches paged table searches against names or comments, +ignores `searchKey` on Community's complete-list endpoints, validates metadata +`pageSize` in `1..=100000`, returns binding failures in the HTTP 200 JSON +envelope, and preserves `defaultValue: null` separately from an empty-string +default. Stage 6 is complete. Web and desktop own the product runtime and publish its owner-only local endpoint; CLI and MCP attach to that host and never contact @@ -164,10 +173,10 @@ build a row-limited SELECT without opening JDBC, then Rust validates that SQL an executes it through the existing forced-read-only query and retained-result path. The fixed 149-JAR classpath keeps H2 and MySQL; PostgreSQL and other dialects do not block the MySQL preview. MySQL writes, Agent, CLI, and MCP -conformance remain outside this small read-only milestone. The current MySQL -connection, database/schema/table, preview, and supported Console SELECT routes +conformance remain outside this read-only milestone. The current MySQL +connection, object metadata, preview, and supported Console SELECT routes dispatch to `mysql_async` before Java lease acquisition; Community parser, -formatter, completion, builders, and advanced metadata remain Java-backed. +formatter, completion, and builders remain Java-backed. The first Console compatibility slice adds SQLite migration 3 for saved Consoles and the historical `/api/operation/saved/*` plus @@ -205,7 +214,9 @@ React / TypeScript CLI / MCP client See [`docs/architecture.md`](docs/architecture.md) for ownership, [`docs/protocol.md`](docs/protocol.md) for the implemented 1.0 process contract, and [`docs/driver-packs.md`](docs/driver-packs.md) for the local manifest and -startup contract. +startup contract. [`docs/mysql-community-parity.md`](docs/mysql-community-parity.md) +is the source-locked acceptance contract for matching the original Community +frontend's complete MySQL feature surface. ## Build diff --git a/apps/chat2db-web/src/legacy.rs b/apps/chat2db-web/src/legacy.rs index 70cb056..75d208d 100644 --- a/apps/chat2db-web/src/legacy.rs +++ b/apps/chat2db-web/src/legacy.rs @@ -12,13 +12,21 @@ use std::{ use axum::{ Json, Router, extract::{Query, State}, + http::StatusCode, + middleware, + response::{IntoResponse, Response}, routing::{get, post}, }; use chat2db_contract::{ - ApiError, ColumnNullability, CommunityTable, Datasource, DatasourceConnection, - DatasourceConnectionProperty, DatasourceSecretChange, JdbcDriver, JdbcValue, JdbcValueType, - ListCommunityDatabasesRequest, ListCommunitySchemasRequest, ListCommunityTablesRequest, - OperationEvent, QueryAccepted, QueryLimits, ResultColumn, ResultMetadata, ResultPageRequest, + ApiError, ColumnNullability, CommunityFunction, CommunityProcedure, CommunityTable, + CommunityTableColumn, CommunityTableIndex, CommunityTableIndexColumn, CommunityTrigger, + Datasource, DatasourceConnection, DatasourceConnectionProperty, DatasourceSecretChange, + GetCommunityFunctionRequest, GetCommunityProcedureRequest, GetCommunityTriggerRequest, + JdbcDriver, JdbcValue, JdbcValueType, ListCommunityColumnsRequest, + ListCommunityDatabasesRequest, ListCommunityFunctionsRequest, ListCommunityIndexesRequest, + ListCommunityProceduresRequest, ListCommunitySchemasRequest, ListCommunityTablesRequest, + ListCommunityTriggersRequest, ListCommunityViewsRequest, OperationEvent, QueryAccepted, + QueryLimits, ResultColumn, ResultMetadata, ResultPageRequest, StartCommunityTablePreviewRequest, StartQueryRequest, UpdateDatasourceRequest, }; use chat2db_core::{AppError, Application}; @@ -32,6 +40,7 @@ const DEFAULT_PAGE_NO: u32 = 1; const DEFAULT_PAGE_SIZE: u32 = 20; const DEFAULT_PREVIEW_PAGE_SIZE: u32 = 200; const DEFAULT_SQL_PAGE_SIZE: u32 = 200; +const MAX_METADATA_PAGE_SIZE: u32 = 100_000; const MAX_PREVIEW_ROWS: u32 = 1_000; const MAX_SQL_ROWS: u32 = 10_000; const RESULT_PAGE_MAX_BYTES: u64 = 8 * 1024 * 1024; @@ -68,6 +77,41 @@ impl LegacyEnvelope { } } +/// Historical list wrapper used by metadata screens that request the complete +/// response instead of only its `data` field. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyCountedEnvelope { + pub success: bool, + pub data: Option>, + pub total: Option, + pub error_code: Option, + pub error_message: Option, +} + +impl LegacyCountedEnvelope { + fn success(data: Vec) -> Self { + let total = data.len(); + Self { + success: true, + data: Some(data), + total: Some(total), + error_code: None, + error_message: None, + } + } + + fn failure(error: LegacyFailure) -> Self { + Self { + success: false, + data: None, + total: None, + error_code: Some(error.code), + error_message: Some(error.message), + } + } +} + /// Safe failure projected into the historical response wrapper. #[derive(Debug, Clone, PartialEq, Eq)] pub struct LegacyFailure { @@ -275,6 +319,111 @@ pub struct LegacyTable { pub update_time: String, } +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacySimpleTable { + pub name: String, + pub comment: String, + pub table_type: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyColumn { + pub old_name: Option, + pub name: String, + pub table_name: String, + pub column_type: String, + pub data_type: Option, + pub default_value: Option, + pub auto_increment: Option, + pub comment: String, + pub primary_key: Option, + pub schema_name: String, + pub database_name: String, + pub type_name: Option, + pub column_size: Option, + pub buffer_length: Option, + pub decimal_digits: Option, + pub num_prec_radix: Option, + pub nullable_int: Option, + pub sql_data_type: Option, + pub sql_datetime_sub: Option, + pub char_octet_length: Option, + pub ordinal_position: Option, + pub nullable: Option, + pub generated_column: Option, + pub extent: String, + pub edit_status: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyIndexColumn { + pub index_name: String, + pub table_name: String, + #[serde(rename = "type")] + pub index_type: String, + pub comment: String, + pub column_name: String, + pub ordinal_position: Option, + pub collation: String, + pub schema_name: String, + pub database_name: String, + pub non_unique: Option, + pub index_qualifier: String, + pub asc_or_desc: String, + pub cardinality: Option, + pub pages: Option, + pub filter_condition: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyIndex { + pub columns: Option, + pub name: String, + #[serde(rename = "type")] + pub index_type: String, + pub comment: String, + pub column_list: Vec, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyFunction { + pub database_name: String, + pub schema_name: String, + pub function_name: String, + pub remarks: String, + pub function_type: Option, + pub specific_name: String, + pub function_body: String, + pub function_template: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyProcedure { + pub database_name: String, + pub schema_name: String, + pub procedure_name: String, + pub remarks: String, + pub procedure_type: Option, + pub specific_name: String, + pub procedure_body: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyTrigger { + pub database_name: String, + pub schema_name: String, + pub trigger_name: String, + pub event_manipulation: String, + pub trigger_body: String, +} + #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] pub struct LegacyResultHeader { @@ -493,6 +642,62 @@ pub struct LegacyTableListQuery { pub search_key: String, } +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyTableDetailQuery { + pub data_source_id: LegacyIdentifier, + #[serde(default)] + pub database_name: String, + #[serde(default)] + pub schema_name: String, + #[serde(default)] + pub database_type: String, + #[serde(default)] + pub table_name: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyFunctionDetailQuery { + pub data_source_id: LegacyIdentifier, + #[serde(default)] + pub database_name: String, + #[serde(default)] + pub schema_name: String, + #[serde(default)] + pub database_type: String, + #[serde(default)] + pub function_name: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyProcedureDetailQuery { + pub data_source_id: LegacyIdentifier, + #[serde(default)] + pub database_name: String, + #[serde(default)] + pub schema_name: String, + #[serde(default)] + pub database_type: String, + #[serde(default)] + pub procedure_name: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LegacyTriggerDetailQuery { + pub data_source_id: LegacyIdentifier, + #[serde(default)] + pub database_name: String, + #[serde(default)] + pub schema_name: String, + #[serde(default)] + pub database_type: String, + #[serde(default)] + pub trigger_name: String, +} + #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] pub struct LegacyTablePreviewRequest { @@ -960,6 +1165,7 @@ pub(crate) async fn list_tables( application: &Application, query: &LegacyTableListQuery, ) -> LegacyResult> { + validate_metadata_page(query)?; let datasource_id = query.data_source_id.as_string(); let database_type = resolve_database_type(application, &datasource_id, &query.database_type).await?; @@ -978,13 +1184,269 @@ pub(crate) async fn list_tables( .into_iter() .map(table_response) .collect(); - if !query.search_key.trim().is_empty() { - let needle = query.search_key.to_lowercase(); - items.retain(|item| item.name.to_lowercase().contains(&needle)); - } + items.retain(|item| table_matches_search(item, &query.search_key)); Ok(paginate(items, query.page_no, query.page_size)) } +/// Lists the compact table projection used by autocomplete and table pickers. +pub(crate) async fn list_simple_tables( + application: &Application, + query: &LegacyTableListQuery, +) -> LegacyResult> { + validate_metadata_page(query)?; + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + Ok(application + .list_community_tables(ListCommunityTablesRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + table_name_pattern: String::new(), + }) + .await? + .items + .into_iter() + .map(simple_table_response) + .collect()) +} + +/// Lists table or view columns in the historical `ColumnResponse` shape. +pub(crate) async fn list_columns( + application: &Application, + query: &LegacyTableDetailQuery, +) -> LegacyResult> { + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + Ok(application + .list_community_columns(ListCommunityColumnsRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + table_name: query.table_name.clone(), + }) + .await? + .items + .into_iter() + .map(column_response) + .collect()) +} + +/// Lists table indexes in the historical `IndexResponse` shape. +pub(crate) async fn list_indexes( + application: &Application, + query: &LegacyTableDetailQuery, +) -> LegacyResult> { + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + Ok(application + .list_community_indexes(ListCommunityIndexesRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + table_name: query.table_name.clone(), + }) + .await? + .items + .into_iter() + .map(index_response) + .collect()) +} + +/// Community's historical key endpoint is an alias of its index metadata. +pub(crate) async fn list_keys( + application: &Application, + query: &LegacyTableDetailQuery, +) -> LegacyResult> { + list_indexes(application, query).await +} + +/// Lists views in the same page wrapper used by the retained tree. +pub(crate) async fn list_views( + application: &Application, + query: &LegacyTableListQuery, +) -> LegacyResult> { + validate_metadata_page(query)?; + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + let items = application + .list_community_views(ListCommunityViewsRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + view_name_pattern: String::new(), + }) + .await? + .items + .into_iter() + .map(table_response) + .collect(); + Ok(full_page(items)) +} + +/// Reads one view, including its DDL, through the exact Core detail path. +pub(crate) async fn get_view( + application: &Application, + query: &LegacyTableDetailQuery, +) -> LegacyResult { + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + Ok(table_response( + application + .get_community_view(ListCommunityViewsRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + view_name_pattern: query.table_name.clone(), + }) + .await?, + )) +} + +/// Lists stored functions in the historical paged metadata shape. +pub(crate) async fn list_functions( + application: &Application, + query: &LegacyTableListQuery, +) -> LegacyResult> { + validate_metadata_page(query)?; + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + let items = application + .list_community_functions(ListCommunityFunctionsRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + }) + .await? + .items + .into_iter() + .map(function_response) + .collect(); + Ok(full_page(items)) +} + +/// Reads one stored function in the historical metadata shape. +pub(crate) async fn get_function( + application: &Application, + query: &LegacyFunctionDetailQuery, +) -> LegacyResult { + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + Ok(function_response( + application + .get_community_function(GetCommunityFunctionRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + function_name: query.function_name.clone(), + }) + .await?, + )) +} + +/// Lists stored procedures in the historical paged metadata shape. +pub(crate) async fn list_procedures( + application: &Application, + query: &LegacyTableListQuery, +) -> LegacyResult> { + validate_metadata_page(query)?; + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + let items = application + .list_community_procedures(ListCommunityProceduresRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + }) + .await? + .items + .into_iter() + .map(procedure_response) + .collect(); + Ok(full_page(items)) +} + +/// Reads one stored procedure in the historical metadata shape. +pub(crate) async fn get_procedure( + application: &Application, + query: &LegacyProcedureDetailQuery, +) -> LegacyResult { + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + Ok(procedure_response( + application + .get_community_procedure(GetCommunityProcedureRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + procedure_name: query.procedure_name.clone(), + }) + .await?, + )) +} + +/// Lists triggers in the historical paged metadata shape. +pub(crate) async fn list_triggers( + application: &Application, + query: &LegacyTableListQuery, +) -> LegacyResult> { + validate_metadata_page(query)?; + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + let items = application + .list_community_triggers(ListCommunityTriggersRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + }) + .await? + .items + .into_iter() + .map(trigger_response) + .collect(); + Ok(full_page(items)) +} + +/// Reads one trigger in the historical metadata shape. +pub(crate) async fn get_trigger( + application: &Application, + query: &LegacyTriggerDetailQuery, +) -> LegacyResult { + let datasource_id = query.data_source_id.as_string(); + let database_type = + resolve_database_type(application, &datasource_id, &query.database_type).await?; + Ok(trigger_response( + application + .get_community_trigger(GetCommunityTriggerRequest { + datasource_id, + database_type, + database_name: query.database_name.clone(), + schema_name: query.schema_name.clone(), + trigger_name: query.trigger_name.clone(), + }) + .await?, + )) +} + /// Runs a table preview through the generated-SQL, forced-read-only Core path /// and waits for its retained result so the old synchronous frontend can use it. #[allow(clippy::too_many_lines)] @@ -1552,6 +2014,119 @@ fn table_response(table: CommunityTable) -> LegacyTable { } } +fn simple_table_response(table: CommunityTable) -> LegacySimpleTable { + LegacySimpleTable { + name: table.name, + comment: table.comment, + // The original service built `SimpleTable` without setting this field. + table_type: None, + } +} + +fn column_response(column: CommunityTableColumn) -> LegacyColumn { + let old_name = Some(column.name.clone()); + LegacyColumn { + old_name, + name: column.name, + table_name: column.table_name, + column_type: column.column_type, + data_type: column.data_type, + default_value: column.default_value, + auto_increment: column.auto_increment, + comment: column.comment, + primary_key: column.primary_key, + schema_name: column.schema_name, + database_name: column.database_name, + type_name: None, + column_size: column.column_size, + buffer_length: column.buffer_length, + decimal_digits: column.decimal_digits, + num_prec_radix: column.num_prec_radix, + nullable_int: None, + sql_data_type: column.sql_data_type, + sql_datetime_sub: column.sql_datetime_sub, + char_octet_length: column.char_octet_length, + ordinal_position: column.ordinal_position, + nullable: column.nullable, + generated_column: column.generated_column, + extent: column.extent, + edit_status: None, + } +} + +fn index_response(index: CommunityTableIndex) -> LegacyIndex { + LegacyIndex { + columns: None, + name: index.name, + index_type: index.index_type, + comment: index.comment, + column_list: index + .columns + .into_iter() + .map(index_column_response) + .collect(), + } +} + +fn index_column_response(column: CommunityTableIndexColumn) -> LegacyIndexColumn { + LegacyIndexColumn { + index_name: column.index_name, + table_name: column.table_name, + index_type: column.column_type, + comment: column.comment, + column_name: column.column_name, + ordinal_position: column.ordinal_position, + collation: column.collation, + schema_name: column.schema_name, + database_name: column.database_name, + non_unique: column.non_unique, + index_qualifier: column.index_qualifier, + asc_or_desc: column.sort_order, + cardinality: decimal_i64(column.cardinality.as_deref()), + pages: decimal_i64(column.pages.as_deref()), + filter_condition: column.filter_condition, + } +} + +fn function_response(function: CommunityFunction) -> LegacyFunction { + LegacyFunction { + database_name: function.database_name, + schema_name: function.schema_name, + function_name: function.name, + remarks: function.remarks, + function_type: function.function_type, + specific_name: function.specific_name, + function_body: function.body, + function_template: function.template, + } +} + +fn procedure_response(procedure: CommunityProcedure) -> LegacyProcedure { + LegacyProcedure { + database_name: procedure.database_name, + schema_name: procedure.schema_name, + procedure_name: procedure.name, + remarks: procedure.remarks, + procedure_type: procedure.procedure_type, + specific_name: procedure.specific_name, + procedure_body: procedure.body, + } +} + +fn trigger_response(trigger: CommunityTrigger) -> LegacyTrigger { + LegacyTrigger { + database_name: trigger.database_name, + schema_name: trigger.schema_name, + trigger_name: trigger.name, + event_manipulation: trigger.event_manipulation, + trigger_body: trigger.body, + } +} + +fn decimal_i64(value: Option<&str>) -> Option { + value.and_then(|value| value.parse().ok()) +} + fn result_header(column: &ResultColumn) -> LegacyResultHeader { LegacyResultHeader { data_type: legacy_data_type(column.value_type).to_owned(), @@ -1809,6 +2384,36 @@ fn paginate(items: Vec, page_no: u32, page_size: u32) -> LegacyPage { } } +fn table_matches_search(table: &LegacyTable, search_key: &str) -> bool { + if search_key.trim().is_empty() { + return true; + } + let search_key = search_key.to_lowercase(); + table.name.to_lowercase().contains(&search_key) + || (!table.comment.trim().is_empty() && table.comment.to_lowercase().contains(&search_key)) +} + +fn validate_metadata_page(query: &LegacyTableListQuery) -> LegacyResult<()> { + if !(1..=MAX_METADATA_PAGE_SIZE).contains(&query.page_size) { + return Err(LegacyFailure::invalid( + "invalid_legacy_request", + "pageSize must be between 1 and 100000", + )); + } + Ok(()) +} + +fn full_page(items: Vec) -> LegacyPage { + let total = items.len(); + LegacyPage { + data: items, + page_no: 1, + page_size: u32::try_from(total).unwrap_or(u32::MAX), + total, + has_next_page: false, + } +} + /// Dispatches a historical Community request without depending on Axum. /// /// Tauri IPC can pass its `requestUrl`, `method`, and `message` fields here and @@ -1824,6 +2429,7 @@ pub async fn dispatch( .next() .unwrap_or(request.request_url.as_str()); let method = request.method.to_ascii_lowercase(); + let counted_response = LEGACY_COUNTED_PATHS.contains(&path); let result = match (method.as_str(), path) { ("get", "/api/system") => Ok(serde_json::json!({ "systemUuid": "chat2db-rust-community" @@ -1911,6 +2517,75 @@ pub async fn dispatch( Ok(query) => serialized(list_tables(application, &query).await), Err(error) => Err(error), }, + ("get", "/api/rdb/table/table_list") => { + match decode::(request.message) { + Ok(query) => serialized(list_simple_tables(application, &query).await), + Err(error) => Err(error), + } + } + ( + "get", + "/api/rdb/table/column_list" | "/api/rdb/ddl/column_list" | "/api/rdb/view/column_list", + ) => match decode::(request.message) { + Ok(query) => serialized(list_columns(application, &query).await), + Err(error) => Err(error), + }, + ("get", "/api/rdb/table/index_list" | "/api/rdb/ddl/index_list") => { + match decode::(request.message) { + Ok(query) => serialized(list_indexes(application, &query).await), + Err(error) => Err(error), + } + } + ("get", "/api/rdb/table/key_list" | "/api/rdb/ddl/key_list") => { + match decode::(request.message) { + Ok(query) => serialized(list_keys(application, &query).await), + Err(error) => Err(error), + } + } + ("get", "/api/rdb/view/list") => match decode::(request.message) { + Ok(query) => serialized(list_views(application, &query).await), + Err(error) => Err(error), + }, + ("get", "/api/rdb/view/detail") => { + match decode::(request.message) { + Ok(query) => serialized(get_view(application, &query).await), + Err(error) => Err(error), + } + } + ("get", "/api/rdb/function/list") => { + match decode::(request.message) { + Ok(query) => serialized(list_functions(application, &query).await), + Err(error) => Err(error), + } + } + ("get", "/api/rdb/function/detail") => { + match decode::(request.message) { + Ok(query) => serialized(get_function(application, &query).await), + Err(error) => Err(error), + } + } + ("get", "/api/rdb/procedure/list") => { + match decode::(request.message) { + Ok(query) => serialized(list_procedures(application, &query).await), + Err(error) => Err(error), + } + } + ("get", "/api/rdb/procedure/detail") => { + match decode::(request.message) { + Ok(query) => serialized(get_procedure(application, &query).await), + Err(error) => Err(error), + } + } + ("get", "/api/rdb/trigger/list") => match decode::(request.message) { + Ok(query) => serialized(list_triggers(application, &query).await), + Err(error) => Err(error), + }, + ("get", "/api/rdb/trigger/detail") => { + match decode::(request.message) { + Ok(query) => serialized(get_trigger(application, &query).await), + Err(error) => Err(error), + } + } ("post" | "put", "/api/rdb/dml/execute_table") => { match decode::(request.message) { Ok(body) => serialized(preview_table(application, &body).await), @@ -1932,7 +2607,11 @@ pub async fn dispatch( "The requested route does not exist", )), }; - envelope_value(result) + if counted_response { + counted_envelope_value(result) + } else { + envelope_value(result) + } } const LEGACY_PATHS: &[&str] = &[ @@ -1952,10 +2631,33 @@ const LEGACY_PATHS: &[&str] = &[ "/api/rdb/database/list", "/api/rdb/schema/list", "/api/rdb/table/list", + "/api/rdb/table/table_list", + "/api/rdb/table/column_list", + "/api/rdb/table/index_list", + "/api/rdb/table/key_list", + "/api/rdb/ddl/column_list", + "/api/rdb/ddl/index_list", + "/api/rdb/ddl/key_list", + "/api/rdb/view/list", + "/api/rdb/view/column_list", + "/api/rdb/view/detail", + "/api/rdb/function/list", + "/api/rdb/function/detail", + "/api/rdb/procedure/list", + "/api/rdb/procedure/detail", + "/api/rdb/trigger/list", + "/api/rdb/trigger/detail", "/api/rdb/dml/execute", "/api/rdb/dml/execute_table", ]; +const LEGACY_COUNTED_PATHS: &[&str] = &[ + "/api/rdb/ddl/column_list", + "/api/rdb/ddl/index_list", + "/api/rdb/ddl/key_list", + "/api/rdb/view/column_list", +]; + fn decode(message: serde_json::Value) -> LegacyResult { serde_json::from_value(message).map_err(|_| { LegacyFailure::invalid( @@ -1991,6 +2693,26 @@ fn envelope_value(result: LegacyResult) -> serde_json::Value }) } +fn counted_envelope_value(result: LegacyResult) -> serde_json::Value { + let envelope: LegacyCountedEnvelope = match result { + Ok(serde_json::Value::Array(data)) => LegacyCountedEnvelope::success(data), + Ok(_) => LegacyCountedEnvelope::failure(LegacyFailure { + code: "internal_error".to_owned(), + message: "The operation could not be completed".to_owned(), + }), + Err(error) => LegacyCountedEnvelope::failure(error), + }; + serde_json::to_value(envelope).unwrap_or_else(|_| { + serde_json::json!({ + "success": false, + "data": null, + "total": null, + "errorCode": "internal_error", + "errorMessage": "The operation could not be completed" + }) + }) +} + pub(crate) fn routes() -> Router { Router::new() .route("/api/system", get(system_handler)) @@ -2036,6 +2758,22 @@ pub(crate) fn routes() -> Router { .route("/api/rdb/database/list", get(database_list_handler)) .route("/api/rdb/schema/list", get(schema_list_handler)) .route("/api/rdb/table/list", get(table_list_handler)) + .route("/api/rdb/table/table_list", get(simple_table_list_handler)) + .route("/api/rdb/table/column_list", get(table_column_list_handler)) + .route("/api/rdb/table/index_list", get(table_index_list_handler)) + .route("/api/rdb/table/key_list", get(table_key_list_handler)) + .route("/api/rdb/ddl/column_list", get(ddl_column_list_handler)) + .route("/api/rdb/ddl/index_list", get(ddl_index_list_handler)) + .route("/api/rdb/ddl/key_list", get(ddl_key_list_handler)) + .route("/api/rdb/view/list", get(view_list_handler)) + .route("/api/rdb/view/column_list", get(view_column_list_handler)) + .route("/api/rdb/view/detail", get(view_detail_handler)) + .route("/api/rdb/function/list", get(function_list_handler)) + .route("/api/rdb/function/detail", get(function_detail_handler)) + .route("/api/rdb/procedure/list", get(procedure_list_handler)) + .route("/api/rdb/procedure/detail", get(procedure_detail_handler)) + .route("/api/rdb/trigger/list", get(trigger_list_handler)) + .route("/api/rdb/trigger/detail", get(trigger_detail_handler)) .route( "/api/rdb/dml/execute", post(sql_execute_handler).put(sql_execute_handler), @@ -2044,6 +2782,7 @@ pub(crate) fn routes() -> Router { "/api/rdb/dml/execute_table", post(table_preview_handler).put(table_preview_handler), ) + .layer(middleware::map_response(legacy_bad_request_envelope)) } fn envelope(result: LegacyResult) -> Json> { @@ -2053,6 +2792,27 @@ fn envelope(result: LegacyResult) -> Json> { }) } +async fn legacy_bad_request_envelope(response: Response) -> Response { + if response.status() != StatusCode::BAD_REQUEST { + return response; + } + ( + StatusCode::OK, + envelope::(Err(LegacyFailure::invalid( + "invalid_legacy_request", + "The Community request query is invalid", + ))), + ) + .into_response() +} + +fn counted_envelope(result: LegacyResult>) -> Json> { + Json(match result { + Ok(data) => LegacyCountedEnvelope::success(data), + Err(error) => LegacyCountedEnvelope::failure(error), + }) +} + async fn system_handler() -> Json> { envelope(Ok(serde_json::json!({ "systemUuid": "chat2db-rust-community" @@ -2180,6 +2940,118 @@ async fn table_list_handler( envelope(list_tables(&application, &query).await) } +async fn simple_table_list_handler( + State(application): State, + Query(query): Query, +) -> Json>> { + envelope(list_simple_tables(&application, &query).await) +} + +async fn table_column_list_handler( + State(application): State, + Query(query): Query, +) -> Json>> { + envelope(list_columns(&application, &query).await) +} + +async fn table_index_list_handler( + State(application): State, + Query(query): Query, +) -> Json>> { + envelope(list_indexes(&application, &query).await) +} + +async fn table_key_list_handler( + State(application): State, + Query(query): Query, +) -> Json>> { + envelope(list_keys(&application, &query).await) +} + +async fn ddl_column_list_handler( + State(application): State, + Query(query): Query, +) -> Json> { + counted_envelope(list_columns(&application, &query).await) +} + +async fn ddl_index_list_handler( + State(application): State, + Query(query): Query, +) -> Json> { + counted_envelope(list_indexes(&application, &query).await) +} + +async fn ddl_key_list_handler( + State(application): State, + Query(query): Query, +) -> Json> { + counted_envelope(list_keys(&application, &query).await) +} + +async fn view_list_handler( + State(application): State, + Query(query): Query, +) -> Json>> { + envelope(list_views(&application, &query).await) +} + +async fn view_column_list_handler( + State(application): State, + Query(query): Query, +) -> Json> { + counted_envelope(list_columns(&application, &query).await) +} + +async fn view_detail_handler( + State(application): State, + Query(query): Query, +) -> Json> { + envelope(get_view(&application, &query).await) +} + +async fn function_list_handler( + State(application): State, + Query(query): Query, +) -> Json>> { + envelope(list_functions(&application, &query).await) +} + +async fn function_detail_handler( + State(application): State, + Query(query): Query, +) -> Json> { + envelope(get_function(&application, &query).await) +} + +async fn procedure_list_handler( + State(application): State, + Query(query): Query, +) -> Json>> { + envelope(list_procedures(&application, &query).await) +} + +async fn procedure_detail_handler( + State(application): State, + Query(query): Query, +) -> Json> { + envelope(get_procedure(&application, &query).await) +} + +async fn trigger_list_handler( + State(application): State, + Query(query): Query, +) -> Json>> { + envelope(list_triggers(&application, &query).await) +} + +async fn trigger_detail_handler( + State(application): State, + Query(query): Query, +) -> Json> { + envelope(get_trigger(&application, &query).await) +} + async fn table_preview_handler( State(application): State, Json(request): Json, @@ -2193,3 +3065,380 @@ async fn sql_execute_handler( ) -> Json>> { envelope(execute_sql(&application, &request).await) } + +#[cfg(test)] +mod tests { + use axum::{ + body::Body, + http::{Request, StatusCode}, + }; + use http_body_util::BodyExt as _; + use tower::ServiceExt as _; + + use super::*; + + const REQUIRED_METADATA_PATHS: &[&str] = &[ + "/api/rdb/table/table_list", + "/api/rdb/table/column_list", + "/api/rdb/table/index_list", + "/api/rdb/table/key_list", + "/api/rdb/ddl/column_list", + "/api/rdb/ddl/index_list", + "/api/rdb/ddl/key_list", + "/api/rdb/view/list", + "/api/rdb/view/column_list", + "/api/rdb/view/detail", + "/api/rdb/function/list", + "/api/rdb/function/detail", + "/api/rdb/procedure/list", + "/api/rdb/procedure/detail", + "/api/rdb/trigger/list", + "/api/rdb/trigger/detail", + ]; + + fn metadata_message(path: &str) -> serde_json::Value { + let mut message = serde_json::json!({ + "dataSourceId": 1, + "databaseName": "inventory", + "schemaName": "" + }); + let object = message + .as_object_mut() + .expect("metadata message must be an object"); + match path { + "/api/rdb/table/table_list" + | "/api/rdb/view/list" + | "/api/rdb/function/list" + | "/api/rdb/procedure/list" + | "/api/rdb/trigger/list" => { + object.insert("pageNo".to_owned(), serde_json::json!(1)); + object.insert("pageSize".to_owned(), serde_json::json!(20)); + object.insert("searchKey".to_owned(), serde_json::json!("")); + } + "/api/rdb/function/detail" => { + object.insert( + "functionName".to_owned(), + serde_json::json!("double_amount"), + ); + } + "/api/rdb/procedure/detail" => { + object.insert("procedureName".to_owned(), serde_json::json!("count_items")); + } + "/api/rdb/trigger/detail" => { + object.insert( + "triggerName".to_owned(), + serde_json::json!("items_trim_label"), + ); + } + _ => { + object.insert("tableName".to_owned(), serde_json::json!("items")); + } + } + message + } + + fn metadata_query(path: &str) -> String { + let object_name = match path { + "/api/rdb/function/detail" => "&functionName=double_amount", + "/api/rdb/procedure/detail" => "&procedureName=count_items", + "/api/rdb/trigger/detail" => "&triggerName=items_trim_label", + "/api/rdb/table/table_list" + | "/api/rdb/view/list" + | "/api/rdb/function/list" + | "/api/rdb/procedure/list" + | "/api/rdb/trigger/list" => "&pageNo=1&pageSize=20&searchKey=", + _ => "&tableName=items", + }; + format!("{path}?dataSourceId=1&databaseName=inventory&schemaName={object_name}") + } + + #[test] + fn counted_metadata_envelope_keeps_total_at_the_top_level() { + let body = counted_envelope_value(Ok(serde_json::json!([ + { "name": "id" }, + { "name": "created_at" } + ]))); + + assert_eq!(body["success"], true); + assert_eq!(body["total"], 2); + assert_eq!(body["data"][0]["name"], "id"); + assert!(body["data"].get("total").is_none()); + } + + #[test] + fn metadata_projections_use_the_historical_frontend_field_names() { + let column = column_response(CommunityTableColumn { + database_name: "catalog".to_owned(), + schema_name: "public".to_owned(), + table_name: "users".to_owned(), + name: "id".to_owned(), + column_type: "BIGINT".to_owned(), + data_type: Some(-5), + primary_key: Some(true), + nullable: Some(0), + ..CommunityTableColumn::default() + }); + let column = serde_json::to_value(column).expect("column projection must serialize"); + assert_eq!(column["oldName"], "id"); + assert_eq!(column["tableName"], "users"); + assert_eq!(column["columnType"], "BIGINT"); + assert_eq!(column["primaryKey"], true); + assert_eq!(column["nullable"], 0); + assert!(column["defaultValue"].is_null()); + + let empty_default = column_response(CommunityTableColumn { + default_value: Some(String::new()), + ..CommunityTableColumn::default() + }); + let empty_default = + serde_json::to_value(empty_default).expect("empty default must serialize"); + assert_eq!(empty_default["defaultValue"], ""); + + let index = index_response(CommunityTableIndex { + name: "PRIMARY".to_owned(), + index_type: "BTREE".to_owned(), + columns: vec![CommunityTableIndexColumn { + index_name: "PRIMARY".to_owned(), + table_name: "users".to_owned(), + column_name: "id".to_owned(), + cardinality: Some("42".to_owned()), + sort_order: "A".to_owned(), + ..CommunityTableIndexColumn::default() + }], + ..CommunityTableIndex::default() + }); + let index = serde_json::to_value(index).expect("index projection must serialize"); + assert_eq!(index["type"], "BTREE"); + assert_eq!(index["columnList"][0]["columnName"], "id"); + assert_eq!(index["columnList"][0]["cardinality"], 42); + assert_eq!(index["columnList"][0]["ascOrDesc"], "A"); + + let function = function_response(CommunityFunction { + name: "calculate_total".to_owned(), + body: "RETURN 1".to_owned(), + template: "CREATE FUNCTION calculate_total".to_owned(), + ..CommunityFunction::default() + }); + let function = serde_json::to_value(function).expect("function projection must serialize"); + assert_eq!(function["functionName"], "calculate_total"); + assert_eq!(function["functionBody"], "RETURN 1"); + assert_eq!( + function["functionTemplate"], + "CREATE FUNCTION calculate_total" + ); + + let view = table_response(CommunityTable { + name: "active_users".to_owned(), + table_type: "VIEW".to_owned(), + ddl: "CREATE VIEW active_users AS SELECT 1".to_owned(), + ..CommunityTable::default() + }); + let view = serde_json::to_value(view).expect("view projection must serialize"); + assert_eq!(view["name"], "active_users"); + assert_eq!(view["tableType"], "VIEW"); + assert_eq!(view["ddl"], "CREATE VIEW active_users AS SELECT 1"); + + let procedure = procedure_response(CommunityProcedure { + name: "refresh_users".to_owned(), + body: "CREATE PROCEDURE refresh_users() SELECT 1".to_owned(), + ..CommunityProcedure::default() + }); + let procedure = + serde_json::to_value(procedure).expect("procedure projection must serialize"); + assert_eq!(procedure["procedureName"], "refresh_users"); + assert_eq!( + procedure["procedureBody"], + "CREATE PROCEDURE refresh_users() SELECT 1" + ); + + let trigger = trigger_response(CommunityTrigger { + name: "users_before_insert".to_owned(), + event_manipulation: "INSERT".to_owned(), + body: "CREATE TRIGGER users_before_insert".to_owned(), + ..CommunityTrigger::default() + }); + let trigger = serde_json::to_value(trigger).expect("trigger projection must serialize"); + assert_eq!(trigger["triggerName"], "users_before_insert"); + assert_eq!(trigger["eventManipulation"], "INSERT"); + assert_eq!(trigger["triggerBody"], "CREATE TRIGGER users_before_insert"); + + let page = full_page(vec!["one", "two", "three"]); + assert_eq!(page.page_no, 1); + assert_eq!(page.page_size, 3); + assert_eq!(page.total, 3); + assert!(!page.has_next_page); + } + + #[test] + fn table_search_matches_community_name_and_comment_behavior() { + let table = LegacyTable { + name: "orders".to_owned(), + comment: "Invoice archive".to_owned(), + table_type: "TABLE".to_owned(), + pinned: false, + ddl: String::new(), + engine: String::new(), + charset: String::new(), + collate: String::new(), + increment_value: None, + partition: String::new(), + tablespace: String::new(), + rows: None, + data_length: None, + create_time: String::new(), + update_time: String::new(), + }; + + assert!(table_matches_search(&table, "ORDER")); + assert!(table_matches_search(&table, "invoice")); + assert!(table_matches_search(&table, " ")); + assert!(!table_matches_search(&table, "customer")); + } + + #[test] + fn required_metadata_paths_are_known_to_desktop_dispatch() { + let unique: HashSet<&str> = REQUIRED_METADATA_PATHS.iter().copied().collect(); + assert_eq!(unique.len(), REQUIRED_METADATA_PATHS.len()); + for path in REQUIRED_METADATA_PATHS { + assert!(LEGACY_PATHS.contains(path), "missing dispatch path: {path}"); + } + for path in LEGACY_COUNTED_PATHS { + assert!(REQUIRED_METADATA_PATHS.contains(path)); + } + } + + #[tokio::test] + async fn required_metadata_paths_reach_their_desktop_dispatch_branches() { + for path in REQUIRED_METADATA_PATHS { + let response = dispatch( + &Application::new(), + LegacyDispatchRequest { + request_url: (*path).to_owned(), + method: "get".to_owned(), + message: serde_json::Value::Null, + }, + ) + .await; + assert_eq!( + response["errorCode"], "invalid_legacy_request", + "missing desktop dispatch branch: {path}" + ); + if LEGACY_COUNTED_PATHS.contains(path) { + assert!(response.get("total").is_some()); + } + } + } + + #[tokio::test] + async fn required_metadata_paths_accept_community_dispatch_fields() { + let application = Application::new(); + for path in REQUIRED_METADATA_PATHS { + let response = dispatch( + &application, + LegacyDispatchRequest { + request_url: (*path).to_owned(), + method: "get".to_owned(), + message: metadata_message(path), + }, + ) + .await; + assert_eq!(response["success"], false, "unexpected success: {path}"); + assert_ne!( + response["errorCode"], "invalid_legacy_request", + "Community fields did not decode: {path}" + ); + assert_ne!( + response["errorCode"], "route_not_found", + "dispatch route is missing: {path}" + ); + if LEGACY_COUNTED_PATHS.contains(path) { + assert!(response.get("total").is_some(), "missing total: {path}"); + } + } + } + + #[tokio::test] + async fn required_metadata_paths_accept_community_axum_query_fields() { + let router = routes().with_state(Application::new()); + for path in REQUIRED_METADATA_PATHS { + let response = router + .clone() + .oneshot( + Request::get(metadata_query(path)) + .body(Body::empty()) + .expect("request must build"), + ) + .await + .expect("router must respond"); + assert_eq!(response.status(), StatusCode::OK, "invalid route: {path}"); + let body = response + .into_body() + .collect() + .await + .expect("response body must collect") + .to_bytes(); + let body: serde_json::Value = + serde_json::from_slice(&body).expect("response body must be JSON"); + assert_eq!(body["success"], false, "unexpected success: {path}"); + assert!(body.get("errorCode").is_some(), "missing envelope: {path}"); + if LEGACY_COUNTED_PATHS.contains(path) { + assert!(body.get("total").is_some(), "missing total: {path}"); + } + } + } + + #[tokio::test] + async fn invalid_axum_query_uses_the_community_json_envelope() { + let response = routes() + .with_state(Application::new()) + .oneshot( + Request::get("/api/rdb/table/table_list?pageNo=1&pageSize=20") + .body(Body::empty()) + .expect("request must build"), + ) + .await + .expect("router must respond"); + + assert_eq!(response.status(), StatusCode::OK); + let body = response + .into_body() + .collect() + .await + .expect("response body must collect") + .to_bytes(); + let body: serde_json::Value = + serde_json::from_slice(&body).expect("response body must be JSON"); + assert_eq!(body["success"], false); + assert_eq!(body["data"], serde_json::Value::Null); + assert_eq!(body["errorCode"], "invalid_legacy_request"); + } + + #[tokio::test] + async fn metadata_page_size_matches_the_community_range() { + let application = Application::new(); + for page_size in [0, MAX_METADATA_PAGE_SIZE + 1] { + let response = dispatch( + &application, + LegacyDispatchRequest { + request_url: "/api/rdb/view/list".to_owned(), + method: "get".to_owned(), + message: serde_json::json!({ + "dataSourceId": 1, + "databaseName": "inventory", + "schemaName": "", + "pageNo": 1, + "pageSize": page_size, + "searchKey": "ignored" + }), + }, + ) + .await; + assert_eq!(response["success"], false); + assert_eq!(response["errorCode"], "invalid_legacy_request"); + assert_eq!( + response["errorMessage"], + "pageSize must be between 1 and 100000" + ); + } + } +} diff --git a/apps/frontend/src/generated/contract.ts b/apps/frontend/src/generated/contract.ts index ef8a59b..97bfb51 100644 --- a/apps/frontend/src/generated/contract.ts +++ b/apps/frontend/src/generated/contract.ts @@ -1673,7 +1673,7 @@ export interface components { /** Format: int32 */ decimalDigits?: number | null; defaultConstraintName: string; - defaultValue: string; + defaultValue?: string | null; extent: string; generatedColumn?: boolean | null; /** Format: int32 */ diff --git a/contracts/openapi/chat2db-v1.json b/contracts/openapi/chat2db-v1.json index 4df5f9a..04b5221 100644 --- a/contracts/openapi/chat2db-v1.json +++ b/contracts/openapi/chat2db-v1.json @@ -6510,7 +6510,6 @@ "tableName", "name", "columnType", - "defaultValue", "comment", "primaryKeyName", "primaryKeyOrder", @@ -6581,7 +6580,10 @@ "type": "string" }, "defaultValue": { - "type": "string" + "type": [ + "string", + "null" + ] }, "extent": { "type": "string" diff --git a/crates/chat2db-contract/src/community.rs b/crates/chat2db-contract/src/community.rs index 9e45943..27b93d6 100644 --- a/crates/chat2db-contract/src/community.rs +++ b/crates/chat2db-contract/src/community.rs @@ -215,7 +215,7 @@ pub struct CommunityTableColumn { pub name: String, pub column_type: String, pub data_type: Option, - pub default_value: String, + pub default_value: Option, pub auto_increment: Option, pub comment: String, pub primary_key: Option, @@ -1915,7 +1915,7 @@ mod tests { name: "id".to_owned(), column_type: "BIGINT".to_owned(), data_type: Some(-5), - default_value: "NEXT VALUE FOR seq_items".to_owned(), + default_value: Some("NEXT VALUE FOR seq_items".to_owned()), auto_increment: Some(true), comment: "Primary identifier".to_owned(), primary_key: Some(true), diff --git a/crates/chat2db-core/src/community.rs b/crates/chat2db-core/src/community.rs index 8e24fee..56b8b90 100644 --- a/crates/chat2db-core/src/community.rs +++ b/crates/chat2db-core/src/community.rs @@ -248,6 +248,16 @@ impl Application { &self, request: ListCommunityColumnsRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_columns( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &request.table_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let ListCommunityColumnsRequest { @@ -292,6 +302,16 @@ impl Application { &self, request: ListCommunityIndexesRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_indexes( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &request.table_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let ListCommunityIndexesRequest { @@ -336,6 +356,16 @@ impl Application { &self, request: ListCommunityViewsRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_views( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &request.view_name_pattern, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let ListCommunityViewsRequest { @@ -371,6 +401,39 @@ impl Application { .await } + /// Reads one view in the Community table projection. + /// + /// # Errors + /// + /// Returns datasource, storage, engine, metadata, or not-found errors. + pub async fn get_community_view( + &self, + request: ListCommunityViewsRequest, + ) -> Result { + let view_name = request.view_name_pattern.clone(); + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::get_view( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &view_name, + ) + .await; + } + self.list_community_views(request) + .await? + .items + .into_iter() + .find(|view| view.name.eq_ignore_ascii_case(&view_name)) + .ok_or_else(|| { + AppError::invalid( + "community_view_not_found", + "The requested Community view does not exist", + ) + }) + } + /// Lists foreign keys imported by one table using a forced read-only session. /// /// # Errors @@ -380,6 +443,15 @@ impl Application { &self, request: ListCommunityTableKeysRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_imported_keys( + self, + &request.datasource_id, + &request.database_name, + &request.table_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let ListCommunityTableKeysRequest { @@ -424,6 +496,15 @@ impl Application { &self, request: ListCommunityTableKeysRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_exported_keys( + self, + &request.datasource_id, + &request.database_name, + &request.table_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let ListCommunityTableKeysRequest { @@ -468,6 +549,16 @@ impl Application { &self, request: ListCommunityTableKeysRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_primary_keys( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &request.table_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let ListCommunityTableKeysRequest { @@ -512,6 +603,15 @@ impl Application { &self, request: ListCommunityFunctionsRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_functions( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let ListCommunityFunctionsRequest { @@ -548,6 +648,16 @@ impl Application { &self, request: GetCommunityFunctionRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::get_function( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &request.function_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let GetCommunityFunctionRequest { @@ -590,6 +700,16 @@ impl Application { &self, request: GetCommunityFunctionRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_function_parameters( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &request.function_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let GetCommunityFunctionRequest { @@ -637,6 +757,15 @@ impl Application { &self, request: ListCommunityProceduresRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_procedures( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let ListCommunityProceduresRequest { @@ -673,6 +802,16 @@ impl Application { &self, request: GetCommunityProcedureRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::get_procedure( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &request.procedure_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let GetCommunityProcedureRequest { @@ -715,6 +854,16 @@ impl Application { &self, request: GetCommunityProcedureRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_procedure_parameters( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &request.procedure_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let GetCommunityProcedureRequest { @@ -762,6 +911,15 @@ impl Application { &self, request: ListCommunityTriggersRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::list_triggers( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let ListCommunityTriggersRequest { @@ -798,6 +956,16 @@ impl Application { &self, request: GetCommunityTriggerRequest, ) -> Result { + if native_mysql::is_mysql_database_type(&request.database_type) { + return native_mysql::get_trigger( + self, + &request.datasource_id, + &request.database_name, + &request.schema_name, + &request.trigger_name, + ) + .await; + } let storage = self.require_storage()?; let engine = self.require_community_engine().await?; let GetCommunityTriggerRequest { @@ -1269,7 +1437,7 @@ fn community_table_column(column: BridgeCommunityTableColumn) -> CommunityTableC name: column.name, column_type: column.column_type, data_type: column.data_type, - default_value: column.default_value, + default_value: Some(column.default_value), auto_increment: column.auto_increment, comment: column.comment, primary_key: column.primary_key, @@ -2244,7 +2412,7 @@ mod tests { name: "id".to_owned(), column_type: "BIGINT".to_owned(), data_type: Some(-5), - default_value: "NEXT VALUE FOR seq_items".to_owned(), + default_value: Some("NEXT VALUE FOR seq_items".to_owned()), auto_increment: Some(true), comment: "Primary identifier".to_owned(), primary_key: Some(true), diff --git a/crates/chat2db-core/src/native_mysql.rs b/crates/chat2db-core/src/native_mysql.rs index 47010f2..23de413 100644 --- a/crates/chat2db-core/src/native_mysql.rs +++ b/crates/chat2db-core/src/native_mysql.rs @@ -1,7 +1,13 @@ use chat2db_contract::{ - ApiError, CommunityDatabase, CommunityDatabaseList, CommunitySchemaList, CommunityTable, - CommunityTableList, CommunityTablePreviewAccepted, DatasourceConnection, QueryLimits, - ResultMetadata, StartCommunityTablePreviewRequest, StartQueryRequest, + ApiError, CommunityDatabase, CommunityDatabaseList, CommunityForeignKey, + CommunityForeignKeyList, CommunityFunction, CommunityFunctionList, CommunityFunctionParameter, + CommunityFunctionParameterList, CommunityPrimaryKey, CommunityPrimaryKeyList, + CommunityProcedure, CommunityProcedureList, CommunityProcedureParameter, + CommunityProcedureParameterList, CommunitySchemaList, CommunityTable, CommunityTableColumn, + CommunityTableColumnList, CommunityTableIndex, CommunityTableIndexColumn, + CommunityTableIndexList, CommunityTableList, CommunityTablePreviewAccepted, CommunityTrigger, + CommunityTriggerList, CommunityViewList, DatasourceConnection, QueryLimits, ResultMetadata, + StartCommunityTablePreviewRequest, StartQueryRequest, }; use chat2db_engine_protocol::wire; use chat2db_java_bridge::QueryOptions; @@ -9,10 +15,10 @@ use chat2db_storage::Storage; use mysql_async::{ Column, Conn, Error as MysqlError, Opts, OptsBuilder, Row, SslOpts, Value, consts::{ColumnFlags, ColumnType}, - prelude::Queryable, + prelude::{FromValue, Queryable}, }; use prost::Message; -use std::time::Duration; +use std::{future::Future, time::Duration}; use tokio::sync::watch; use url::Url; @@ -28,6 +34,7 @@ const JDBC_MYSQL_SCHEME: &str = "jdbc:mysql://"; const CONNECT_TIMEOUT: Duration = Duration::from_secs(15); const DISCONNECT_TIMEOUT: Duration = Duration::from_secs(5); const CONTROL_TIMEOUT: Duration = Duration::from_secs(5); +const METADATA_TIMEOUT: Duration = Duration::from_secs(30); const DEFAULT_BATCH_ROWS: u32 = 256; const DEFAULT_BATCH_BYTES: u32 = 256 * 1024; const DEFAULT_RESULT_BYTES: u64 = wire::JdbcResultByteLimit::DefaultResultBytes as u64; @@ -51,6 +58,61 @@ type TableRow = ( Option, Option, ); +type ColumnRow = ( + String, + String, + Option, + String, + String, + String, + String, + i32, + Option, + String, + Option, + Option, +); +type IndexRow = ( + String, + String, + u8, + String, + String, + i32, + Option, + Option, + Option, + Option, + String, + String, +); +type ForeignKeyRow = ( + String, + String, + String, + String, + String, + String, + i32, + String, + String, + String, + Option, +); +type RoutineParameterRow = ( + String, + String, + i32, + Option, + Option, + String, + String, + Option, + Option, + Option, + Option, + Option, +); #[derive(Debug, PartialEq, Eq)] enum SqlToken { @@ -75,25 +137,23 @@ pub(crate) async fn list_databases( ) -> Result { let resolved = resolve_native_connection(application, datasource_id).await?; let mut conn = open_connection(&resolved.connection).await?; - let result = conn - .query::<(String, String, String), _>( - "SELECT SCHEMA_NAME, DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME \ + let result = metadata_query(conn.query::<(String, String, String), _>( + "SELECT SCHEMA_NAME, DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME \ FROM information_schema.SCHEMATA ORDER BY SCHEMA_NAME", - ) - .await - .map(|rows| CommunityDatabaseList { - items: rows - .into_iter() - .map(|(name, charset, collation)| CommunityDatabase { - system: is_system_database(&name), - name, - charset, - collation, - ..CommunityDatabase::default() - }) - .collect(), - }) - .map_err(mysql_query_error); + )) + .await + .map(|rows| CommunityDatabaseList { + items: rows + .into_iter() + .map(|(name, charset, collation)| CommunityDatabase { + system: is_system_database(&name), + name, + charset, + collation, + ..CommunityDatabase::default() + }) + .collect(), + }); finish_connection(conn, result).await } @@ -127,50 +187,572 @@ pub(crate) async fn list_tables( DATE_FORMAT(CREATE_TIME, '%Y-%m-%dT%H:%i:%s'), \ DATE_FORMAT(UPDATE_TIME, '%Y-%m-%dT%H:%i:%s') \ FROM information_schema.TABLES \ - WHERE TABLE_SCHEMA = ? AND (? = '' OR TABLE_NAME LIKE ?) \ + WHERE TABLE_SCHEMA = ? \ + AND TABLE_TYPE IN ('BASE TABLE', 'SYSTEM VIEW') \ + AND (? = '' OR TABLE_NAME LIKE ?) \ ORDER BY TABLE_NAME"; let pattern = table_name_pattern.trim().to_owned(); - let result = conn - .exec::(query, (database_name.to_owned(), pattern.clone(), pattern)) - .await - .map(|rows| CommunityTableList { - items: rows - .into_iter() - .map( - |( - database_name, - name, - table_type, - comment, - engine, - collation, - increment_value, - rows, - data_length, - create_time, - update_time, - )| CommunityTable { - database_name, - name, - table_type: normalize_table_type(&table_type).to_owned(), - comment, - database_type: "MYSQL".to_owned(), - engine, - charset: collation - .split_once('_') - .map_or_else(String::new, |(charset, _)| charset.to_owned()), - collation, - increment_value, - rows, - data_length, - create_time: create_time.unwrap_or_default(), - update_time: update_time.unwrap_or_default(), - ..CommunityTable::default() - }, - ) - .collect(), + let result = metadata_query( + conn.exec::(query, (database_name.to_owned(), pattern.clone(), pattern)), + ) + .await + .map(|rows| CommunityTableList { + items: rows + .into_iter() + .map( + |( + database_name, + name, + table_type, + comment, + engine, + collation, + increment_value, + rows, + data_length, + create_time, + update_time, + )| CommunityTable { + database_name, + name, + table_type: normalize_table_type(&table_type).to_owned(), + comment, + database_type: "MYSQL".to_owned(), + engine, + charset: collation + .split_once('_') + .map_or_else(String::new, |(charset, _)| charset.to_owned()), + collation, + increment_value, + rows, + data_length, + create_time: create_time.unwrap_or_default(), + update_time: update_time.unwrap_or_default(), + ..CommunityTable::default() + }, + ) + .collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn list_columns( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + table_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(table_name, "tableName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT COLUMN_NAME, DATA_TYPE, COLUMN_DEFAULT, COALESCE(EXTRA, ''), \ + COALESCE(COLUMN_COMMENT, ''), COALESCE(COLUMN_KEY, ''), IS_NULLABLE, \ + ORDINAL_POSITION, NUMERIC_SCALE, COLUMN_TYPE, CHARACTER_SET_NAME, \ + COLLATION_NAME \ + FROM information_schema.COLUMNS \ + WHERE TABLE_SCHEMA = ? AND TABLE_NAME = ? ORDER BY ORDINAL_POSITION"; + let result = metadata_query( + conn.exec::(query, (database_name.to_owned(), table_name.to_owned())), + ) + .await + .map(|rows| CommunityTableColumnList { + items: rows + .into_iter() + .map(|row| community_column(database_name, schema_name, table_name, row)) + .collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn list_indexes( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + table_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(table_name, "tableName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT TABLE_SCHEMA, TABLE_NAME, NON_UNIQUE, INDEX_SCHEMA, INDEX_NAME, \ + SEQ_IN_INDEX, COLUMN_NAME, COLLATION, CARDINALITY, SUB_PART, \ + INDEX_TYPE, COALESCE(INDEX_COMMENT, '') \ + FROM information_schema.STATISTICS \ + WHERE TABLE_SCHEMA = ? AND TABLE_NAME = ? \ + ORDER BY INDEX_NAME, SEQ_IN_INDEX"; + let result = metadata_query( + conn.exec::(query, (database_name.to_owned(), table_name.to_owned())), + ) + .await + .map(|rows| CommunityTableIndexList { + items: community_indexes(rows, schema_name), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn list_views( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + view_name_pattern: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE, COALESCE(TABLE_COMMENT, ''), \ + COALESCE(ENGINE, ''), COALESCE(TABLE_COLLATION, ''), \ + CAST(AUTO_INCREMENT AS CHAR), CAST(TABLE_ROWS AS CHAR), \ + CAST(DATA_LENGTH AS CHAR), \ + DATE_FORMAT(CREATE_TIME, '%Y-%m-%dT%H:%i:%s'), \ + DATE_FORMAT(UPDATE_TIME, '%Y-%m-%dT%H:%i:%s') \ + FROM information_schema.TABLES \ + WHERE TABLE_SCHEMA = ? AND TABLE_TYPE = 'VIEW' \ + AND (? = '' OR TABLE_NAME LIKE ?) ORDER BY TABLE_NAME"; + let pattern = view_name_pattern.trim().to_owned(); + let result = metadata_query( + conn.exec::(query, (database_name.to_owned(), pattern.clone(), pattern)), + ) + .await + .map(|rows| CommunityViewList { + items: rows + .into_iter() + .map(|row| community_table(row, schema_name)) + .collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn get_view( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + view_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(view_name, "viewName")?; + let qualified_name = + qualified_identifier(database_name, "databaseName", view_name, "viewName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let result = + metadata_query(conn.query_first::(format!("SHOW CREATE VIEW {qualified_name}"))) + .await + .and_then(|row| { + let row = + row.ok_or_else(|| metadata_not_found("view", database_name, view_name))?; + Ok(CommunityTable { + database_name: database_name.to_owned(), + schema_name: schema_name.to_owned(), + name: view_name.to_owned(), + table_type: "VIEW".to_owned(), + database_type: "MYSQL".to_owned(), + ddl: row_string_at(&row, 1)?, + ..CommunityTable::default() + }) + }); + finish_connection(conn, result).await +} + +pub(crate) async fn list_imported_keys( + application: &Application, + datasource_id: &str, + database_name: &str, + table_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(table_name, "tableName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT kcu.REFERENCED_TABLE_SCHEMA, kcu.REFERENCED_TABLE_NAME, \ + kcu.REFERENCED_COLUMN_NAME, kcu.TABLE_SCHEMA, kcu.TABLE_NAME, \ + kcu.COLUMN_NAME, kcu.ORDINAL_POSITION, rc.UPDATE_RULE, rc.DELETE_RULE, \ + kcu.CONSTRAINT_NAME, rc.UNIQUE_CONSTRAINT_NAME \ + FROM information_schema.KEY_COLUMN_USAGE kcu \ + JOIN information_schema.REFERENTIAL_CONSTRAINTS rc \ + ON rc.CONSTRAINT_SCHEMA = kcu.CONSTRAINT_SCHEMA \ + AND rc.CONSTRAINT_NAME = kcu.CONSTRAINT_NAME \ + AND rc.TABLE_NAME = kcu.TABLE_NAME \ + WHERE kcu.TABLE_SCHEMA = ? AND kcu.TABLE_NAME = ? \ + AND kcu.REFERENCED_TABLE_NAME IS NOT NULL \ + ORDER BY kcu.CONSTRAINT_NAME, kcu.ORDINAL_POSITION"; + let result = metadata_query( + conn.exec::(query, (database_name.to_owned(), table_name.to_owned())), + ) + .await + .map(|rows| CommunityForeignKeyList { + items: rows.into_iter().map(community_foreign_key).collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn list_exported_keys( + application: &Application, + datasource_id: &str, + database_name: &str, + table_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(table_name, "tableName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT kcu.REFERENCED_TABLE_SCHEMA, kcu.REFERENCED_TABLE_NAME, \ + kcu.REFERENCED_COLUMN_NAME, kcu.TABLE_SCHEMA, kcu.TABLE_NAME, \ + kcu.COLUMN_NAME, kcu.ORDINAL_POSITION, rc.UPDATE_RULE, rc.DELETE_RULE, \ + kcu.CONSTRAINT_NAME, rc.UNIQUE_CONSTRAINT_NAME \ + FROM information_schema.KEY_COLUMN_USAGE kcu \ + JOIN information_schema.REFERENTIAL_CONSTRAINTS rc \ + ON rc.CONSTRAINT_SCHEMA = kcu.CONSTRAINT_SCHEMA \ + AND rc.CONSTRAINT_NAME = kcu.CONSTRAINT_NAME \ + AND rc.TABLE_NAME = kcu.TABLE_NAME \ + WHERE kcu.REFERENCED_TABLE_SCHEMA = ? AND kcu.REFERENCED_TABLE_NAME = ? \ + ORDER BY kcu.TABLE_SCHEMA, kcu.TABLE_NAME, kcu.CONSTRAINT_NAME, \ + kcu.ORDINAL_POSITION"; + let result = metadata_query( + conn.exec::(query, (database_name.to_owned(), table_name.to_owned())), + ) + .await + .map(|rows| CommunityForeignKeyList { + items: rows.into_iter().map(community_foreign_key).collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn list_primary_keys( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + table_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(table_name, "tableName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, CONSTRAINT_NAME \ + FROM information_schema.KEY_COLUMN_USAGE \ + WHERE TABLE_SCHEMA = ? AND TABLE_NAME = ? AND CONSTRAINT_NAME = 'PRIMARY' \ + ORDER BY ORDINAL_POSITION"; + let result = metadata_query(conn.exec::<(String, String, String, String), _, _>( + query, + (database_name.to_owned(), table_name.to_owned()), + )) + .await + .map(|rows| CommunityPrimaryKeyList { + items: rows + .into_iter() + .map( + |(database_name, table_name, column_name, name)| CommunityPrimaryKey { + database_name, + schema_name: schema_name.to_owned(), + table_name, + column_name, + name, + }, + ) + .collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn list_functions( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT ROUTINE_SCHEMA, ROUTINE_NAME, SPECIFIC_NAME, \ + COALESCE(ROUTINE_COMMENT, '') \ + FROM information_schema.ROUTINES \ + WHERE ROUTINE_SCHEMA = ? AND ROUTINE_TYPE = 'FUNCTION' \ + ORDER BY ROUTINE_NAME"; + let result = metadata_query( + conn.exec::<(String, String, String, String), _, _>(query, (database_name.to_owned(),)), + ) + .await + .map(|rows| CommunityFunctionList { + items: rows + .into_iter() + .map( + |(database_name, name, specific_name, remarks)| CommunityFunction { + database_name, + schema_name: schema_name.to_owned(), + name, + remarks, + function_type: Some(1), + specific_name, + ..CommunityFunction::default() + }, + ) + .collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn get_function( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + function_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(function_name, "functionName")?; + let qualified_name = + qualified_identifier(database_name, "databaseName", function_name, "functionName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let result = async { + let metadata = metadata_query(conn.exec_first::<(String, String, String, String), _, _>( + "SELECT ROUTINE_SCHEMA, ROUTINE_NAME, SPECIFIC_NAME, \ + COALESCE(ROUTINE_COMMENT, '') FROM information_schema.ROUTINES \ + WHERE ROUTINE_SCHEMA = ? AND ROUTINE_NAME = ? \ + AND ROUTINE_TYPE = 'FUNCTION'", + (database_name.to_owned(), function_name.to_owned()), + )) + .await? + .ok_or_else(|| metadata_not_found("function", database_name, function_name))?; + let row = metadata_query( + conn.query_first::(format!("SHOW CREATE FUNCTION {qualified_name}")), + ) + .await? + .ok_or_else(|| metadata_not_found("function", database_name, function_name))?; + Ok(CommunityFunction { + database_name: metadata.0, + schema_name: schema_name.to_owned(), + name: metadata.1, + remarks: metadata.3, + function_type: Some(1), + specific_name: metadata.2, + body: row_string_at(&row, 2)?, + template: String::new(), }) - .map_err(mysql_query_error); + } + .await; + finish_connection(conn, result).await +} + +pub(crate) async fn list_function_parameters( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + function_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(function_name, "functionName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT SPECIFIC_SCHEMA, SPECIFIC_NAME, ORDINAL_POSITION, PARAMETER_MODE, \ + PARAMETER_NAME, DATA_TYPE, DTD_IDENTIFIER, CHARACTER_MAXIMUM_LENGTH, \ + CHARACTER_OCTET_LENGTH, NUMERIC_PRECISION, NUMERIC_SCALE, \ + DATETIME_PRECISION FROM information_schema.PARAMETERS \ + WHERE SPECIFIC_SCHEMA = ? AND SPECIFIC_NAME = ? \ + AND ROUTINE_TYPE = 'FUNCTION' ORDER BY ORDINAL_POSITION"; + let result = metadata_query(conn.exec::( + query, + (database_name.to_owned(), function_name.to_owned()), + )) + .await + .map(|rows| CommunityFunctionParameterList { + items: rows + .into_iter() + .map(|row| community_function_parameter(row, schema_name)) + .collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn list_procedures( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT ROUTINE_SCHEMA, ROUTINE_NAME, SPECIFIC_NAME, \ + COALESCE(ROUTINE_COMMENT, '') \ + FROM information_schema.ROUTINES \ + WHERE ROUTINE_SCHEMA = ? AND ROUTINE_TYPE = 'PROCEDURE' \ + ORDER BY ROUTINE_NAME"; + let result = metadata_query( + conn.exec::<(String, String, String, String), _, _>(query, (database_name.to_owned(),)), + ) + .await + .map(|rows| CommunityProcedureList { + items: rows + .into_iter() + .map( + |(database_name, name, specific_name, remarks)| CommunityProcedure { + database_name, + schema_name: schema_name.to_owned(), + name, + remarks, + procedure_type: Some(2), + specific_name, + body: String::new(), + }, + ) + .collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn get_procedure( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + procedure_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(procedure_name, "procedureName")?; + let qualified_name = qualified_identifier( + database_name, + "databaseName", + procedure_name, + "procedureName", + )?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let result = async { + let metadata = metadata_query(conn.exec_first::<(String, String, String, String), _, _>( + "SELECT ROUTINE_SCHEMA, ROUTINE_NAME, SPECIFIC_NAME, \ + COALESCE(ROUTINE_COMMENT, '') FROM information_schema.ROUTINES \ + WHERE ROUTINE_SCHEMA = ? AND ROUTINE_NAME = ? \ + AND ROUTINE_TYPE = 'PROCEDURE'", + (database_name.to_owned(), procedure_name.to_owned()), + )) + .await? + .ok_or_else(|| metadata_not_found("procedure", database_name, procedure_name))?; + let row = metadata_query( + conn.query_first::(format!("SHOW CREATE PROCEDURE {qualified_name}")), + ) + .await? + .ok_or_else(|| metadata_not_found("procedure", database_name, procedure_name))?; + Ok(CommunityProcedure { + database_name: metadata.0, + schema_name: schema_name.to_owned(), + name: metadata.1, + remarks: metadata.3, + procedure_type: Some(2), + specific_name: metadata.2, + body: row_string_at(&row, 2)?, + }) + } + .await; + finish_connection(conn, result).await +} + +pub(crate) async fn list_procedure_parameters( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + procedure_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(procedure_name, "procedureName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT SPECIFIC_SCHEMA, SPECIFIC_NAME, ORDINAL_POSITION, PARAMETER_MODE, \ + PARAMETER_NAME, DATA_TYPE, DTD_IDENTIFIER, CHARACTER_MAXIMUM_LENGTH, \ + CHARACTER_OCTET_LENGTH, NUMERIC_PRECISION, NUMERIC_SCALE, \ + DATETIME_PRECISION FROM information_schema.PARAMETERS \ + WHERE SPECIFIC_SCHEMA = ? AND SPECIFIC_NAME = ? \ + AND ROUTINE_TYPE = 'PROCEDURE' ORDER BY ORDINAL_POSITION"; + let result = metadata_query(conn.exec::( + query, + (database_name.to_owned(), procedure_name.to_owned()), + )) + .await + .map(|rows| CommunityProcedureParameterList { + items: rows + .into_iter() + .map(|row| community_procedure_parameter(row, schema_name)) + .collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn list_triggers( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let query = "SELECT TRIGGER_SCHEMA, TRIGGER_NAME, EVENT_MANIPULATION \ + FROM information_schema.TRIGGERS \ + WHERE TRIGGER_SCHEMA = ? ORDER BY TRIGGER_NAME"; + let result = metadata_query( + conn.exec::<(String, String, String), _, _>(query, (database_name.to_owned(),)), + ) + .await + .map(|rows| CommunityTriggerList { + items: rows + .into_iter() + .map( + |(database_name, name, event_manipulation)| CommunityTrigger { + database_name, + schema_name: schema_name.to_owned(), + name, + event_manipulation, + body: String::new(), + }, + ) + .collect(), + }); + finish_connection(conn, result).await +} + +pub(crate) async fn get_trigger( + application: &Application, + datasource_id: &str, + database_name: &str, + schema_name: &str, + trigger_name: &str, +) -> Result { + validate_metadata_identifier(database_name, "databaseName")?; + validate_metadata_identifier(trigger_name, "triggerName")?; + let qualified_name = + qualified_identifier(database_name, "databaseName", trigger_name, "triggerName")?; + let resolved = resolve_native_connection(application, datasource_id).await?; + let mut conn = open_connection(&resolved.connection).await?; + let result = async { + let metadata = metadata_query(conn.exec_first::<(String, String, String), _, _>( + "SELECT TRIGGER_SCHEMA, TRIGGER_NAME, EVENT_MANIPULATION \ + FROM information_schema.TRIGGERS \ + WHERE TRIGGER_SCHEMA = ? AND TRIGGER_NAME = ?", + (database_name.to_owned(), trigger_name.to_owned()), + )) + .await? + .ok_or_else(|| metadata_not_found("trigger", database_name, trigger_name))?; + let row = metadata_query( + conn.query_first::(format!("SHOW CREATE TRIGGER {qualified_name}")), + ) + .await? + .ok_or_else(|| metadata_not_found("trigger", database_name, trigger_name))?; + Ok(CommunityTrigger { + database_name: metadata.0, + schema_name: schema_name.to_owned(), + name: metadata.1, + event_manipulation: metadata.2, + body: row_string_at(&row, 2)?, + }) + } + .await; finish_connection(conn, result).await } @@ -1047,6 +1629,461 @@ fn mysql_display(value: Value) -> String { mysql_text(value).unwrap_or_else(|_| "[unavailable]".to_owned()) } +fn community_table( + ( + database_name, + name, + table_type, + comment, + engine, + collation, + increment_value, + rows, + data_length, + create_time, + update_time, + ): TableRow, + schema_name: &str, +) -> CommunityTable { + CommunityTable { + database_name, + schema_name: schema_name.to_owned(), + name, + table_type: normalize_table_type(&table_type).to_owned(), + comment, + database_type: "MYSQL".to_owned(), + engine, + charset: collation + .split_once('_') + .map_or_else(String::new, |(charset, _)| charset.to_owned()), + collation, + increment_value, + rows, + data_length, + create_time: create_time.unwrap_or_default(), + update_time: update_time.unwrap_or_default(), + ..CommunityTable::default() + } +} + +fn community_column( + database_name: &str, + schema_name: &str, + table_name: &str, + ( + name, + data_type, + default_value, + extra, + comment, + column_key, + is_nullable, + ordinal_position, + numeric_scale, + column_definition, + charset, + collation, + ): ColumnRow, +) -> CommunityTableColumn { + let data_type = data_type.to_ascii_uppercase(); + let (column_size, decimal_digits) = + mysql_column_size(&data_type, &column_definition, numeric_scale); + CommunityTableColumn { + database_name: database_name.to_owned(), + schema_name: schema_name.to_owned(), + table_name: table_name.to_owned(), + name, + column_type: data_type, + default_value, + auto_increment: Some(extra.contains("auto_increment")), + comment, + primary_key: Some(column_key.eq_ignore_ascii_case("PRI")), + column_size, + decimal_digits, + ordinal_position: Some(ordinal_position), + nullable: Some(i32::from(is_nullable.eq_ignore_ascii_case("YES"))), + charset: charset.unwrap_or_default(), + collation: collation.unwrap_or_default(), + on_update_current_timestamp: Some(extra.contains("on update CURRENT_TIMESTAMP")), + ..CommunityTableColumn::default() + } +} + +fn mysql_column_size( + data_type: &str, + column_definition: &str, + numeric_scale: Option, +) -> (Option, Option) { + let mut decimal_digits = Some(numeric_scale.unwrap_or_default()); + let Some(open) = column_definition.find('(') else { + return (None, decimal_digits); + }; + let Some(close_offset) = column_definition[open + 1..].find(')') else { + return (None, decimal_digits); + }; + if matches!(data_type, "ENUM" | "SET") { + return (None, decimal_digits); + } + let size = &column_definition[open + 1..open + 1 + close_offset]; + let mut parts = size.split(',').map(str::trim); + let column_size = parts.next().and_then(|value| value.parse::().ok()); + if let Some(scale) = parts.next().and_then(|value| value.parse::().ok()) { + decimal_digits = Some(scale); + } + (column_size, decimal_digits) +} + +fn community_indexes(rows: Vec, schema_name: &str) -> Vec { + let mut indexes: Vec = Vec::new(); + for ( + database_name, + table_name, + non_unique, + index_schema, + index_name, + ordinal_position, + column_name, + collation, + cardinality, + sub_part, + method, + comment, + ) in rows + { + let non_unique = non_unique != 0; + let column = CommunityTableIndexColumn { + database_name: database_name.clone(), + schema_name: schema_name.to_owned(), + table_name: table_name.clone(), + index_name: index_name.clone(), + column_name: column_name.unwrap_or_default(), + ordinal_position: Some(ordinal_position), + collation: collation.clone().unwrap_or_default(), + non_unique: Some(non_unique), + index_qualifier: index_schema, + sort_order: mysql_index_sort_order(collation.as_deref()), + cardinality: cardinality.map(|value| value.to_string()), + sub_part: sub_part.map(|value| value.to_string()), + ..CommunityTableIndexColumn::default() + }; + if let Some(index) = indexes.iter_mut().find(|index| index.name == index_name) { + index.columns.push(column); + continue; + } + let unique = !non_unique; + indexes.push(CommunityTableIndex { + database_name, + schema_name: schema_name.to_owned(), + table_name, + name: index_name.clone(), + index_type: mysql_index_type(&index_name, unique, &method).to_owned(), + unique: Some(unique), + comment, + columns: vec![column], + method, + ..CommunityTableIndex::default() + }); + } + indexes +} + +fn mysql_index_type(index_name: &str, unique: bool, method: &str) -> &'static str { + if index_name.eq_ignore_ascii_case("PRIMARY") { + "Primary" + } else if unique { + "Unique" + } else if method.eq_ignore_ascii_case("SPATIAL") { + "Spatial" + } else if method.eq_ignore_ascii_case("FULLTEXT") { + "Fulltext" + } else { + "Normal" + } +} + +fn mysql_index_sort_order(collation: Option<&str>) -> String { + match collation { + Some(value) if value.eq_ignore_ascii_case("A") => "ASC".to_owned(), + Some(value) if value.eq_ignore_ascii_case("D") => "DESC".to_owned(), + _ => String::new(), + } +} + +fn community_foreign_key( + ( + primary_table_database, + primary_table_name, + primary_column_name, + foreign_table_database, + foreign_table_name, + foreign_column_name, + key_sequence, + update_rule, + delete_rule, + foreign_key_name, + primary_key_name, + ): ForeignKeyRow, +) -> CommunityForeignKey { + CommunityForeignKey { + primary_table_database, + primary_table_name, + primary_column_name, + foreign_table_database, + foreign_table_name, + foreign_column_name, + key_sequence, + update_rule: mysql_referential_rule(&update_rule), + delete_rule: mysql_referential_rule(&delete_rule), + foreign_key_name, + primary_key_name: primary_key_name.unwrap_or_default(), + deferrability: 7, + ..CommunityForeignKey::default() + } +} + +fn mysql_referential_rule(rule: &str) -> i32 { + match rule.trim().to_ascii_uppercase().as_str() { + "CASCADE" => 0, + "RESTRICT" => 1, + "SET NULL" => 2, + "SET DEFAULT" => 4, + _ => 3, + } +} + +fn community_function_parameter( + row: RoutineParameterRow, + schema_name: &str, +) -> CommunityFunctionParameter { + let RoutineParameterProjection { + database_name, + routine_name, + ordinal_position, + mode, + parameter_name, + data_type, + precision, + length, + scale, + radix, + char_octet_length, + } = routine_parameter_projection(row); + CommunityFunctionParameter { + function_database: database_name, + function_schema: schema_name.to_owned(), + function_name: routine_name.clone(), + column_name: parameter_name, + column_type: Some(mysql_function_column_type( + mode.as_deref(), + ordinal_position, + )), + data_type: Some(mysql_metadata_jdbc_type(&data_type)), + type_name: data_type.to_ascii_uppercase(), + precision, + length, + scale, + radix, + nullable: Some(1), + char_octet_length, + ordinal_position: Some(ordinal_position), + is_nullable: "YES".to_owned(), + specific_name: routine_name, + ..CommunityFunctionParameter::default() + } +} + +fn community_procedure_parameter( + row: RoutineParameterRow, + schema_name: &str, +) -> CommunityProcedureParameter { + let RoutineParameterProjection { + database_name, + routine_name, + ordinal_position, + mode, + parameter_name, + data_type, + precision, + length, + scale, + radix, + char_octet_length, + } = routine_parameter_projection(row); + CommunityProcedureParameter { + procedure_database: database_name, + procedure_schema: schema_name.to_owned(), + procedure_name: routine_name.clone(), + column_name: parameter_name, + column_type: Some(mysql_procedure_column_type(mode.as_deref())), + data_type: Some(mysql_metadata_jdbc_type(&data_type)), + type_name: data_type.to_ascii_uppercase(), + precision, + length, + scale, + radix, + nullable: Some(1), + char_octet_length, + ordinal_position: Some(ordinal_position), + is_nullable: "YES".to_owned(), + specific_name: routine_name, + ..CommunityProcedureParameter::default() + } +} + +struct RoutineParameterProjection { + database_name: String, + routine_name: String, + ordinal_position: i32, + mode: Option, + parameter_name: String, + data_type: String, + precision: Option, + length: Option, + scale: Option, + radix: Option, + char_octet_length: Option, +} + +fn routine_parameter_projection( + ( + database_name, + routine_name, + ordinal_position, + mode, + parameter_name, + data_type, + _column_definition, + character_length, + character_octet_length, + numeric_precision, + numeric_scale, + datetime_precision, + ): RoutineParameterRow, +) -> RoutineParameterProjection { + let temporal_size = mysql_temporal_display_size(&data_type, datetime_precision); + let precision = + optional_metadata_i32_clamped(numeric_precision.or(character_length).or(temporal_size)); + let length = optional_metadata_i32_clamped( + character_octet_length + .or(character_length) + .or(temporal_size), + ); + let scale = numeric_scale.map(|value| i32::try_from(value).unwrap_or(i32::MAX)); + RoutineParameterProjection { + database_name, + routine_name, + ordinal_position, + mode, + parameter_name: parameter_name.unwrap_or_default(), + data_type, + precision, + length, + scale, + radix: Some(10), + char_octet_length: optional_metadata_i32_clamped(character_octet_length), + } +} + +fn optional_metadata_i32_clamped(value: Option) -> Option { + value.map(|value| i32::try_from(value).unwrap_or(i32::MAX)) +} + +fn mysql_temporal_display_size(data_type: &str, fractional_seconds: Option) -> Option { + let fractional = fractional_seconds + .filter(|value| *value > 0) + .map_or(0, |value| u64::from(value) + 1); + match data_type.trim().to_ascii_uppercase().as_str() { + "DATE" => Some(10), + "TIME" => Some(8 + fractional), + "DATETIME" | "TIMESTAMP" => Some(19 + fractional), + "YEAR" => Some(4), + _ => None, + } +} + +fn mysql_function_column_type(mode: Option<&str>, ordinal_position: i32) -> i32 { + match mode.map(str::trim) { + None if ordinal_position == 0 => 4, + Some(value) if value.eq_ignore_ascii_case("IN") => 1, + Some(value) if value.eq_ignore_ascii_case("INOUT") => 2, + Some(value) if value.eq_ignore_ascii_case("OUT") => 3, + _ => 0, + } +} + +fn mysql_procedure_column_type(mode: Option<&str>) -> i32 { + match mode.map(str::trim) { + Some(value) if value.eq_ignore_ascii_case("IN") => 1, + Some(value) if value.eq_ignore_ascii_case("INOUT") => 2, + Some(value) if value.eq_ignore_ascii_case("OUT") => 4, + _ => 0, + } +} + +fn mysql_metadata_jdbc_type(data_type: &str) -> i32 { + match data_type.trim().to_ascii_uppercase().as_str() { + "BIT" => -7, + "TINYINT" => -6, + "SMALLINT" => 5, + "MEDIUMINT" | "INT" | "INTEGER" => 4, + "BIGINT" => -5, + "FLOAT" => 7, + "DOUBLE" | "DOUBLE PRECISION" | "REAL" => 8, + "DECIMAL" | "NUMERIC" => 3, + "DATE" | "YEAR" => 91, + "TIME" => 92, + "DATETIME" | "TIMESTAMP" => 93, + "CHAR" | "ENUM" | "SET" => 1, + "VARCHAR" => 12, + "BINARY" => -2, + "VARBINARY" => -3, + "TINYBLOB" | "BLOB" | "MEDIUMBLOB" | "LONGBLOB" | "GEOMETRY" => -4, + "TINYTEXT" | "TEXT" | "MEDIUMTEXT" | "LONGTEXT" | "JSON" => -1, + "BOOLEAN" | "BOOL" => 16, + _ => 1111, + } +} + +fn row_string_at(row: &Row, index: usize) -> Result { + row_value_at(row, index) +} + +fn row_value_at(row: &Row, index: usize) -> Result { + row.get_opt::(index) + .ok_or_else(result_decode_error)? + .map_err(|_| result_decode_error()) +} + +fn validate_metadata_identifier(value: &str, field: &str) -> Result<(), AppError> { + if value.trim().is_empty() || value.len() > MAX_IDENTIFIER_BYTES || value.contains('\0') { + return Err(AppError::invalid( + "invalid_mysql_metadata_request", + format!("{field} is invalid"), + )); + } + Ok(()) +} + +fn qualified_identifier( + database_name: &str, + database_field: &str, + object_name: &str, + object_field: &str, +) -> Result { + Ok(format!( + "{}.{}", + quote_identifier(database_name, database_field)?, + quote_identifier(object_name, object_field)? + )) +} + +fn metadata_not_found(kind: &str, database_name: &str, object_name: &str) -> AppError { + AppError::invalid( + "mysql_metadata_not_found", + format!("MySQL {kind} {database_name}.{object_name} does not exist"), + ) +} + fn validate_query_options(options: QueryOptions) -> Result<(), AppError> { if options.target_batch_rows > MAX_BATCH_ROWS { return Err(AppError::invalid( @@ -1237,8 +2274,23 @@ async fn open_connection_with_opts(opts: Opts) -> Result { .map_err(mysql_connection_error) } +async fn metadata_query(query: F) -> Result +where + F: Future>, +{ + tokio::time::timeout(METADATA_TIMEOUT, query) + .await + .map_err(|_| { + AppError::unavailable( + "mysql_metadata_timeout", + "The MySQL metadata query did not finish in time", + ) + })? + .map_err(mysql_query_error) +} + async fn finish_connection(conn: Conn, result: Result) -> Result { - let close = conn.disconnect().await.map_err(mysql_connection_error); + let close = disconnect_connection(conn).await; match result { Ok(value) => close.map(|()| value), Err(error) => { @@ -1398,8 +2450,10 @@ mod tests { use chat2db_contract::{DatasourceConnection, DatasourceConnectionProperty}; use super::{ - connection_opts, is_mysql_database_type, is_native_read_candidate, normalize_table_type, - quote_identifier, validate_read_sql, + community_column, community_foreign_key, community_function_parameter, community_indexes, + community_procedure_parameter, connection_opts, is_mysql_database_type, + is_native_read_candidate, normalize_table_type, qualified_identifier, quote_identifier, + validate_read_sql, }; #[test] @@ -1513,12 +2567,198 @@ mod tests { } } + #[test] + fn mysql_column_metadata_preserves_community_projection() { + let column = community_column( + "inventory", + "", + "items", + ( + "amount".to_owned(), + "decimal".to_owned(), + Some("0.00".to_owned()), + "DEFAULT_GENERATED on update CURRENT_TIMESTAMP".to_owned(), + "Money".to_owned(), + "PRI".to_owned(), + "NO".to_owned(), + 2, + Some(2), + "decimal(12,2) unsigned".to_owned(), + None, + None, + ), + ); + + assert_eq!(column.database_name, "inventory"); + assert_eq!(column.table_name, "items"); + assert_eq!(column.column_type, "DECIMAL"); + assert_eq!(column.default_value.as_deref(), Some("0.00")); + assert_eq!(column.column_size, Some(12)); + assert_eq!(column.decimal_digits, Some(2)); + assert_eq!(column.ordinal_position, Some(2)); + assert_eq!(column.nullable, Some(0)); + assert_eq!(column.primary_key, Some(true)); + assert_eq!(column.auto_increment, Some(false)); + assert_eq!(column.on_update_current_timestamp, Some(true)); + } + + #[test] + fn mysql_index_metadata_groups_columns_and_uses_community_types() { + let indexes = community_indexes( + vec![ + ( + "inventory".to_owned(), + "items".to_owned(), + 0, + "inventory".to_owned(), + "PRIMARY".to_owned(), + 1, + Some("id".to_owned()), + Some("A".to_owned()), + Some(2), + None, + "BTREE".to_owned(), + String::new(), + ), + ( + "inventory".to_owned(), + "items".to_owned(), + 1, + "inventory".to_owned(), + "idx_label_amount".to_owned(), + 1, + Some("label".to_owned()), + Some("A".to_owned()), + Some(2), + Some(16), + "BTREE".to_owned(), + "Lookup".to_owned(), + ), + ( + "inventory".to_owned(), + "items".to_owned(), + 1, + "inventory".to_owned(), + "idx_label_amount".to_owned(), + 2, + Some("amount".to_owned()), + Some("D".to_owned()), + Some(2), + None, + "BTREE".to_owned(), + "Lookup".to_owned(), + ), + ], + "", + ); + + assert_eq!(indexes.len(), 2); + assert_eq!(indexes[0].index_type, "Primary"); + assert_eq!(indexes[0].unique, Some(true)); + assert_eq!(indexes[1].index_type, "Normal"); + assert_eq!(indexes[1].columns.len(), 2); + assert_eq!(indexes[1].columns[0].sort_order, "ASC"); + assert_eq!(indexes[1].columns[0].sub_part.as_deref(), Some("16")); + assert_eq!(indexes[1].columns[1].sort_order, "DESC"); + } + + #[test] + fn mysql_relation_and_routine_metadata_match_jdbc_constants() { + let key = community_foreign_key(( + "inventory".to_owned(), + "parent".to_owned(), + "id".to_owned(), + "inventory".to_owned(), + "child".to_owned(), + "parent_id".to_owned(), + 1, + "CASCADE".to_owned(), + "SET NULL".to_owned(), + "fk_child_parent".to_owned(), + Some("PRIMARY".to_owned()), + )); + assert_eq!(key.update_rule, 0); + assert_eq!(key.delete_rule, 2); + assert_eq!(key.deferrability, 7); + + let function_return = community_function_parameter( + ( + "inventory".to_owned(), + "calculate_total".to_owned(), + 0, + None, + None, + "decimal".to_owned(), + "decimal(12,2)".to_owned(), + None, + None, + Some(12), + Some(2), + None, + ), + "", + ); + assert_eq!(function_return.column_type, Some(4)); + assert_eq!(function_return.data_type, Some(3)); + assert_eq!(function_return.precision, Some(12)); + assert_eq!(function_return.scale, Some(2)); + + let procedure_output = community_procedure_parameter( + ( + "inventory".to_owned(), + "load_total".to_owned(), + 1, + Some("OUT".to_owned()), + Some("total".to_owned()), + "int".to_owned(), + "int".to_owned(), + None, + None, + Some(10), + Some(0), + None, + ), + "", + ); + assert_eq!(procedure_output.column_type, Some(4)); + assert_eq!(procedure_output.data_type, Some(4)); + assert_eq!(procedure_output.radix, Some(10)); + + let long_text_input = community_procedure_parameter( + ( + "inventory".to_owned(), + "store_text".to_owned(), + 1, + Some("IN".to_owned()), + Some("content".to_owned()), + "longtext".to_owned(), + "longtext".to_owned(), + Some(4_294_967_295), + Some(4_294_967_295), + None, + None, + None, + ), + "", + ); + assert_eq!(long_text_input.precision, Some(i32::MAX)); + assert_eq!(long_text_input.length, Some(i32::MAX)); + assert_eq!(long_text_input.char_octet_length, Some(i32::MAX)); + assert_eq!(long_text_input.nullable, Some(1)); + assert_eq!(long_text_input.is_nullable, "YES"); + } + #[test] fn preview_identifiers_are_quoted_as_one_mysql_identifier() { assert_eq!( quote_identifier("odd`name", "tableName").expect("identifier should quote"), "`odd``name`" ); + assert_eq!( + qualified_identifier("odd`db", "databaseName", "run`me", "procedureName") + .expect("qualified identifier should quote each component"), + "`odd``db`.`run``me`" + ); assert!(quote_identifier("", "tableName").is_err()); assert!(quote_identifier("bad\0name", "tableName").is_err()); } diff --git a/crates/chat2db-core/tests/native_mysql_product.rs b/crates/chat2db-core/tests/native_mysql_product.rs index 51c690b..c00c483 100644 --- a/crates/chat2db-core/tests/native_mysql_product.rs +++ b/crates/chat2db-core/tests/native_mysql_product.rs @@ -3,10 +3,13 @@ use std::{panic::AssertUnwindSafe, time::Duration}; use base64::{Engine as _, engine::general_purpose::STANDARD}; use chat2db_contract::{ CancelDisposition, ComponentState, CreateDatasourceRequest, DatasourceConnection, - DatasourceConnectionProperty, JdbcValue, ListCommunityDatabasesRequest, - ListCommunitySchemasRequest, ListCommunityTablesRequest, OperationEvent, OperationStatus, - QueryLimits, QueryParameter, ResultMetadata, ResultPageRequest, - StartCommunityTablePreviewRequest, StartQueryRequest, + DatasourceConnectionProperty, GetCommunityFunctionRequest, GetCommunityProcedureRequest, + GetCommunityTriggerRequest, JdbcValue, ListCommunityColumnsRequest, + ListCommunityDatabasesRequest, ListCommunityFunctionsRequest, ListCommunityIndexesRequest, + ListCommunityProceduresRequest, ListCommunitySchemasRequest, ListCommunityTableKeysRequest, + ListCommunityTablesRequest, ListCommunityTriggersRequest, ListCommunityViewsRequest, + OperationEvent, OperationStatus, QueryLimits, QueryParameter, ResultMetadata, + ResultPageRequest, StartCommunityTablePreviewRequest, StartQueryRequest, }; use chat2db_core::{Application, RuntimeConfig, RuntimeHost}; use chat2db_java_bridge::{EngineCommand, EngineConfig}; @@ -170,6 +173,7 @@ async fn verify_native_product(config: &MysqlTestConfig, database_name: &str) { .expect("native MySQL datasource must persist without a JDBC pack"); verify_native_metadata(&application, &datasource.id, database_name).await; + verify_native_object_metadata(&application, &datasource.id, database_name).await; verify_native_preview(&application, &datasource.id, database_name).await; verify_native_console(&application, &datasource.id).await; verify_rejected_native_selects(&application, &datasource.id).await; @@ -233,12 +237,266 @@ async fn verify_native_metadata( .iter() .find(|table| table.name == "items") .expect("fixture table must be visible"); + assert!( + tables + .items + .iter() + .all(|table| table.name != "active_items"), + "views must not leak into the Community table inventory" + ); assert_eq!(table.database_name, database_name); assert_eq!(table.table_type, "TABLE"); assert_eq!(table.engine, "InnoDB"); assert_java_dormant(application); } +#[allow(clippy::too_many_lines)] +async fn verify_native_object_metadata( + application: &Application, + datasource_id: &str, + database_name: &str, +) { + let columns = application + .list_community_columns(ListCommunityColumnsRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + table_name: "items".to_owned(), + }) + .await + .expect("native MySQL columns must list"); + let amount = columns + .items + .iter() + .find(|column| column.name == "amount") + .expect("fixture amount column must be visible"); + assert_eq!(amount.column_type, "DECIMAL"); + assert_eq!(amount.default_value.as_deref(), Some("0.00")); + assert_eq!(amount.column_size, Some(12)); + assert_eq!(amount.decimal_digits, Some(2)); + let label = columns + .items + .iter() + .find(|column| column.name == "label") + .expect("fixture label column must be visible"); + assert_eq!(label.default_value, None); + + let indexes = application + .list_community_indexes(ListCommunityIndexesRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + table_name: "items".to_owned(), + }) + .await + .expect("native MySQL indexes must list"); + assert!( + indexes + .items + .iter() + .any(|index| index.name == "PRIMARY" && index.index_type == "Primary") + ); + let composite = indexes + .items + .iter() + .find(|index| index.name == "idx_items_label_amount") + .expect("fixture composite index must be visible"); + assert_eq!(composite.columns.len(), 2); + assert_eq!(composite.columns[0].column_name, "label"); + assert_eq!(composite.columns[1].column_name, "amount"); + assert_eq!(composite.columns[1].sort_order, "DESC"); + + let table_keys = ListCommunityTableKeysRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + table_name: "items".to_owned(), + }; + let imported = application + .list_community_imported_keys(table_keys.clone()) + .await + .expect("native MySQL imported keys must list"); + assert!(imported.items.iter().any(|key| { + key.foreign_key_name == "fk_items_category" + && key.primary_table_name == "categories" + && key.foreign_table_name == "items" + })); + let primary = application + .list_community_primary_keys(table_keys) + .await + .expect("native MySQL primary keys must list"); + assert!( + primary + .items + .iter() + .any(|key| key.name == "PRIMARY" && key.column_name == "id") + ); + let exported = application + .list_community_exported_keys(ListCommunityTableKeysRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + table_name: "categories".to_owned(), + }) + .await + .expect("native MySQL exported keys must list"); + assert!( + exported + .items + .iter() + .any(|key| key.foreign_key_name == "fk_items_category") + ); + + let view_request = ListCommunityViewsRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + view_name_pattern: "active_items".to_owned(), + }; + let views = application + .list_community_views(view_request.clone()) + .await + .expect("native MySQL views must list"); + assert!(views.items.iter().any(|view| view.name == "active_items")); + let view = application + .get_community_view(view_request) + .await + .expect("native MySQL view detail must load"); + assert_eq!(view.table_type, "VIEW"); + assert!(view.ddl.contains("CREATE")); + assert!(view.ddl.contains("active_items")); + + let functions = application + .list_community_functions(ListCommunityFunctionsRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + }) + .await + .expect("native MySQL functions must list"); + assert!( + functions + .items + .iter() + .any(|function| function.name == "double_amount") + ); + let function = application + .get_community_function(GetCommunityFunctionRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + function_name: "double_amount".to_owned(), + }) + .await + .expect("native MySQL function detail must load"); + assert!(function.body.contains("CREATE")); + assert!(function.body.contains("double_amount")); + let function_parameters = application + .list_community_function_parameters(GetCommunityFunctionRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + function_name: "double_amount".to_owned(), + }) + .await + .expect("native MySQL function parameters must list"); + assert!( + function_parameters + .items + .iter() + .any(|parameter| parameter.ordinal_position == Some(0)) + ); + assert!( + function_parameters + .items + .iter() + .any(|parameter| parameter.column_name == "input_value") + ); + + let procedures = application + .list_community_procedures(ListCommunityProceduresRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + }) + .await + .expect("native MySQL procedures must list"); + assert!( + procedures + .items + .iter() + .any(|procedure| procedure.name == "count_items") + ); + let procedure = application + .get_community_procedure(GetCommunityProcedureRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + procedure_name: "count_items".to_owned(), + }) + .await + .expect("native MySQL procedure detail must load"); + assert!(procedure.body.contains("CREATE")); + assert!(procedure.body.contains("count_items")); + let procedure_parameters = application + .list_community_procedure_parameters(GetCommunityProcedureRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + procedure_name: "count_items".to_owned(), + }) + .await + .expect("native MySQL procedure parameters must list"); + assert!( + procedure_parameters + .items + .iter() + .any(|parameter| parameter.column_name == "item_count" + && parameter.column_type == Some(4)) + ); + + let triggers = application + .list_community_triggers(ListCommunityTriggersRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + }) + .await + .expect("native MySQL triggers must list"); + assert!( + triggers + .items + .iter() + .any(|trigger| trigger.name == "items_trim_label") + ); + let trigger = application + .get_community_trigger(GetCommunityTriggerRequest { + datasource_id: datasource_id.to_owned(), + database_type: MYSQL_DATABASE_TYPE.to_owned(), + database_name: database_name.to_owned(), + schema_name: String::new(), + trigger_name: "items_trim_label".to_owned(), + }) + .await + .expect("native MySQL trigger detail must load"); + assert_eq!(trigger.event_manipulation, "INSERT"); + assert!(trigger.body.contains("CREATE")); + assert!(trigger.body.contains("items_trim_label")); + assert_java_dormant(application); +} + async fn verify_native_preview( application: &Application, datasource_id: &str, @@ -481,23 +739,63 @@ async fn provision_database(config: &MysqlTestConfig, database_name: &str) { )) .await .expect("native MySQL fixture database must create"); + conn.query_drop(format!( + "CREATE TABLE `{database_name}`.`categories` (\ + `id` BIGINT NOT NULL, `name` VARCHAR(128) NOT NULL, PRIMARY KEY (`id`)\ + ) ENGINE=InnoDB" + )) + .await + .expect("native MySQL category fixture must create"); conn.query_drop(format!( "CREATE TABLE `{database_name}`.`items` (\ `id` BIGINT NOT NULL, `label` VARCHAR(128) NOT NULL, \ - `amount` DECIMAL(12,2) NOT NULL, `active` BOOLEAN NOT NULL, \ - `created_at` DATETIME NOT NULL, PRIMARY KEY (`id`)\ + `amount` DECIMAL(12,2) NOT NULL DEFAULT 0.00, `active` BOOLEAN NOT NULL, \ + `created_at` DATETIME NOT NULL, `category_id` BIGINT NOT NULL, \ + PRIMARY KEY (`id`), \ + KEY `idx_items_label_amount` (`label`, `amount` DESC), \ + CONSTRAINT `fk_items_category` FOREIGN KEY (`category_id`) \ + REFERENCES `{database_name}`.`categories` (`id`)\ ) ENGINE=InnoDB" )) .await .expect("native MySQL fixture table must create"); + conn.query_drop(format!( + "INSERT INTO `{database_name}`.`categories` VALUES (1, 'default')" + )) + .await + .expect("native MySQL category fixture row must insert"); conn.query_drop(format!( "INSERT INTO `{database_name}`.`items` VALUES \ - (1, 'mysql-ready', 99.99, TRUE, '2026-07-27 12:34:56'), \ - (2, 'second', 2.50, FALSE, '2026-07-28 01:02:03'), \ - (3, 'third', 3.75, TRUE, '2026-07-29 04:05:06')" + (1, 'mysql-ready', 99.99, TRUE, '2026-07-27 12:34:56', 1), \ + (2, 'second', 2.50, FALSE, '2026-07-28 01:02:03', 1), \ + (3, 'third', 3.75, TRUE, '2026-07-29 04:05:06', 1)" )) .await .expect("native MySQL fixture rows must insert"); + conn.query_drop(format!( + "CREATE VIEW `{database_name}`.`active_items` AS \ + SELECT `id`, `label`, `amount` FROM `{database_name}`.`items` WHERE `active` = TRUE" + )) + .await + .expect("native MySQL fixture view must create"); + conn.query_drop(format!( + "CREATE FUNCTION `{database_name}`.`double_amount`(input_value DECIMAL(12,2)) \ + RETURNS DECIMAL(12,2) DETERMINISTIC RETURN input_value * 2" + )) + .await + .expect("native MySQL fixture function must create"); + conn.query_drop(format!( + "CREATE PROCEDURE `{database_name}`.`count_items`(OUT item_count INT) \ + SELECT COUNT(*) INTO item_count FROM `{database_name}`.`items`" + )) + .await + .expect("native MySQL fixture procedure must create"); + conn.query_drop(format!( + "CREATE TRIGGER `{database_name}`.`items_trim_label` BEFORE INSERT \ + ON `{database_name}`.`items` FOR EACH ROW SET NEW.`label` = TRIM(NEW.`label`)" + )) + .await + .expect("native MySQL fixture trigger must create"); conn.disconnect() .await .expect("native MySQL fixture connection must close"); diff --git a/docs/architecture.md b/docs/architecture.md index 3d8a1d0..9649bc7 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -71,8 +71,8 @@ after preserving the disabled-engine error contract. | Durable state | Rust | SQLite, retained-result files, and a mandatory injected secret-vault contract | | AI agent | Rust | Provider adapters, tool loop, limits, compaction, and cancellation | | MCP and CLI | Rust | Adapters around the same product services and policy | -| Native MySQL product slice | Rust / `mysql_async` | Connection, database/schema/table discovery, preview, supported SELECT, typed streaming, limits, cancellation, and retained results | -| Compatibility databases and advanced MySQL operations | Java 17 | Existing SPI/plugins, JDBC, advanced metadata, builders, parsing, formatting, completion, writes, and transactions | +| Native MySQL product slice | Rust / `mysql_async` | Connection, first-stage read-only object metadata and Community-compatible legacy routes/envelopes, nullable defaults, preview, supported SELECT, typed streaming, limits, cancellation, and retained results | +| Compatibility databases and advanced MySQL operations | Java 17 | Existing SPI/plugins, JDBC, SQL builders, parsing, formatting, completion, writes, transactions, and non-MySQL metadata | | SQL parsing, formatting, and completion | Java 17 | Existing Java ANTLR grammars, parser behavior, formatter behavior, and completion | | Rust-to-Java IPC | Shared Protobuf contract | Length-prefixed frames over private stdin/stdout | diff --git a/docs/mysql-community-parity.md b/docs/mysql-community-parity.md new file mode 100644 index 0000000..6181227 --- /dev/null +++ b/docs/mysql-community-parity.md @@ -0,0 +1,91 @@ +# MySQL Community Parity Contract + +## Status + +- Community baseline: `OtterMind/Chat2DB` `main@3cb8af54cad5bd5caa20bb25f10d9b0e4f01931c`. +- Rust baseline: `OtterMind/Chat2DB-Rust` `main@352838b7d20fad568fd68f5d825e65c56104bd29`. +- Product target: the original Community React frontend running against the Rust Web or Tauri host. +- Runtime-tested now: datasource CRUD/test; database, schema, table, column, + index, foreign-key, primary-key, view, function, procedure, trigger, and + routine-parameter metadata; all first-stage historical metadata routes; + bounded table preview; saved Console CRUD; and one unparameterized read-only MySQL + `SELECT` with paging and cancellation. Native metadata keeps Java dormant. + Paged table name/comment search, complete-list filtering, page-size validation, + HTTP binding-error envelopes, and nullable column defaults match the locked + Community baseline. +- Complete parity: not implemented. + +This file is the acceptance contract for MySQL work. Community frontend routes +and user-visible behavior define parity. A modern Core, Axum, Tauri, Java, or +native MySQL capability does not count as complete until the original frontend +route reaches it and a real MySQL product test covers the behavior. + +## Ownership + +- `mysql_async` owns MySQL connections, metadata, query/update execution, + transactions, cancellation, large values, and data transfer. +- The fixed Community Java compatibility process retains the exact Community + ANTLR parser, formatter, completion engine, and plugin SQL builders where + reproducing their behavior in Rust would create unnecessary divergence. +- Rust remains the only product host. Java has no HTTP port and starts only for + compatibility operations that require it. +- The original Community frontend and its styles remain unchanged. Compatibility + is implemented behind its existing HTTP and `window.javaQuery` contracts. + +## Capability Matrix + +| Area | Community frontend contract | Rust baseline | Required parity | +| --- | --- | --- | --- | +| Runtime bootstrap | `/api/system`, `/api/common/environment`, `/api/jdbc/driver/list` | Implemented | Preserve exact envelopes and immutable driver inventory. | +| Datasource CRUD and test | `/api/connection/datasource/list`, `/datasource`, `/datasource/create`, `/datasource/pre_connect`, `/datasource/update`, `DELETE /datasource` | Implemented | Keep secret-safe persistence and native MySQL connection testing. | +| Datasource lifecycle | `/api/connection/datasource/connect`, `/datasource/close`, `/connection/close`, `/connection/console/connect`, `/datasource/clone` | Not implemented | Match explicit connect/close/clone behavior and frontend refresh semantics. | +| SSH and JDBC driver management | `/api/connection/ssh/pre_connect`, `/api/jdbc/driver/download`, `/upload`, `/save`, `/delete` | Not implemented | Match Community SSH testing and local driver lifecycle without exposing secrets. | +| Datasource import/export and namespaces | converter upload routes, `/api/connection/datasource/import_community`, `/datasource/export`, `/api/namespaces/*` | Not implemented | Support Community, Chat2DB, Navicat, DBeaver, DataGrip, export, grouping, and ordering. | +| Database and schema metadata | `/api/rdb/database/list`, `/database_schema_list`, `/api/rdb/schema/list` | Database/schema list implemented | Match filtering, system flags, charset/collation, comments, and pagination envelopes. | +| Database and schema mutation | database create/modify/delete and `/api/rdb/delete/{database,schema}/{prepare,execute}` | Builder-only modern contracts | Generate previews through Community builders and execute only after the same explicit frontend command. | +| Table inventory and detail | `/api/rdb/table/list`, `/table_list`, `/table_meta`, `/column_list`, `/index_list`, `/key_list`, `/query` | List, compact list, column, index, and key routes implemented with native MySQL metadata; nullable defaults and legacy envelopes match Community; `table_meta` and `/query` remain | Add the remaining table-meta and query projections without regressing the native routes. | +| Table data operations | `/api/rdb/dml/execute_table`, `/execute_update`, `/get_update_sql`, `/copy_update_sql`, `/copy_in_values_sql`, `/count` | Read-only preview only; closed typed DML generation exists behind modern APIs | Support editable rows, insert/update/delete SQL, counts, copy helpers, optimistic predicates, and bounded execution. | +| Table DDL | `/api/rdb/ddl/*`, `/api/rdb/table/modify/sql`, `/delete`, `/truncate`, `/copy`, create/update examples, DDL export | Partial builder infrastructure only | Match create/alter/drop/truncate/copy preview and execution, including columns, indexes, keys, charset, collation, comments, and MySQL types. | +| Views | `/api/rdb/view/list`, `/column_list`, `/detail`, `/query`, `/view_meta`, `/modify/sql`, `/delete`, `/drop` | Native list, column list, and `SHOW CREATE VIEW` detail are mapped to the legacy routes | Add data query, metadata, preview, create/alter, and drop flows. | +| Functions, procedures, and triggers | `/api/rdb/{function,procedure,trigger}/{list,detail}`, `/api/rdb/routine/{preview_invocation,preview_migration,execute_migration}` | Native list/detail and routine-parameter projections are implemented; every original list/detail route is mapped | Add invocation and migration preview/execution flows. | +| Console SELECT | `/api/rdb/dml/execute`, desktop `sql-execute`/`sql-cancel` | One unparameterized SELECT, paging, limits, cancellation | Add parameters, CTEs, all MySQL read statements, multiple result sets, warnings, affected rows, and Community result shapes. | +| Console scripts and writes | `/api/rdb/dml/execute`, `/execute_ddl` | Not implemented natively | Match Community statement splitting, script execution policy, DDL/DML, transaction settlement, cancellation, and per-statement results. | +| Large cell values | `/api/rdb/cell/value`, `/download`, `/download_path` | Not implemented | Preserve bounded previews and explicit full-value download without loading unbounded cells into the WebView. | +| Saved consoles and SQL history | `/api/operation/saved/*`, `/api/operation/log/{create,list}` and detail | Saved Console CRUD implemented; execution history absent | Persist and filter Community-compatible history and keep restart-safe Console state. | +| SQL parser, formatter, validation, completion | `/api/sql/format`, `/valid_select`, `/api/sql_parser/get_keywords`, `/context/{parser,quick_parser,tip,hover}` | Modern parser/validation/formatter/completion contracts implemented through Java; legacy routes absent | Map every original endpoint to the fixed Community implementation with matching UTF-16 offsets and envelopes. | +| Import, export, and tasks | `/api/import/{sql_file,other_file}`, `/api/export/{sql_file,other_file}`, `/api/task/*`, `/api/rdb/dml/export`, table class generation | Not implemented | Add bounded streaming import/export, progress, stop, download, cleanup, and failure recovery. | +| Account administration | `/api/rdb/account/{capability,list,grants,preview,execute}` | Not implemented | Match MySQL users, hosts, authentication, privileges, role/grant previews, execution, and current Community escaping rules. | +| Structure comparison | `/api/diff/sql` | Not implemented | Match Community structure projection and MySQL synchronization SQL without changing shared query parsing behavior. | +| Pins and ER metadata | `/api/pin/table/*`, `/api/er/*` | Not implemented | Persist pinned tables and expose the metadata needed by the existing ER view. | +| AI, CLI, and MCP | Original `/api/ai` UI plus Community CLI/MCP database actions | Rust Agent, owner-only CLI attachment, and read-only MCP exist behind modern contracts | Map the original AI workspace and make MySQL read/write tools pass the same product conformance gates. | + +## Delivery Order + +1. Complete: native read-only object metadata and every matching original + metadata route, with Axum/dispatch contracts and a real MySQL 8.4 product + vertical that proves Java remains dormant. +2. Full Console statement execution, multi-result handling, writes, transactions, + history, cancellation, and large-cell retrieval. +3. Table data editing plus database/schema/table/view DDL preview and execution. +4. Import/export/tasks, datasource lifecycle/SSH/import, account administration, + routines, structure comparison, pins, and ER metadata. +5. Original AI mapping and MySQL conformance for Agent, CLI, and MCP. + +Each stage requires focused unit tests, a real MySQL product vertical with Java +dormancy assertions for native operations, original Web and Tauri contract tests, +the complete repository verification gate, and all GitHub Actions jobs. + +## Source Anchors + +- `third_party/chat2db-community/chat2db-community-client/src/service/connection.ts` +- `third_party/chat2db-community/chat2db-community-client/src/service/sql.ts` +- `third_party/chat2db-community/chat2db-community-client/src/service/executeSql.ts` +- `third_party/chat2db-community/chat2db-community-client/src/service/importExport.ts` +- `third_party/chat2db-community/chat2db-community-client/src/service/accountAdmin.ts` +- `third_party/chat2db-community/chat2db-community-client/src/service/schemaSync.ts` +- `third_party/chat2db-community/chat2db-community-server/chat2db-community-web/src/main/java/ai/chat2db/community/web/api/controller/` +- `third_party/chat2db-community/chat2db-community-server/chat2db-community-plugins/chat2db-community-mysql/src/main/java/ai/chat2db/plugin/mysql/` +- `crates/chat2db-core/src/native_mysql.rs` +- `crates/chat2db-core/tests/native_mysql_product.rs` +- `crates/chat2db-core/src/community.rs` +- `apps/chat2db-web/src/legacy.rs` diff --git a/docs/stages.md b/docs/stages.md index ac134c0..4477eae 100644 --- a/docs/stages.md +++ b/docs/stages.md @@ -319,16 +319,23 @@ execution are not implemented and must not be inferred from the historical endpoint name. The native MySQL follow-up pins upstream `mysql_async 0.37.0` with Rustls and -routes MySQL connection testing, database/schema/table discovery, table preview, -and supported Console SELECT before Java lease acquisition. SELECT executes in -a MySQL read-only transaction and emits the existing typed retained-result wire -messages directly from Core. Parameters, CTE-first SELECT, locking/server-file -variants, and multi-statements fail before Java startup. Row and byte truncation -plus cancellation terminate the active MySQL connection through a separate -bounded control connection. The explicit `native-mysql-integration` target and -MySQL CI job use a deliberately missing Java executable and verify connection, -metadata, two-row preview, typed three-row Console output, one-row truncation, -active `SELECT SLEEP(30)` cancellation, retained paging, and dormant Java health. +routes MySQL connection testing; database/schema/table/column/index/key/view/ +function/procedure/trigger metadata; table preview; and supported Console SELECT +before Java lease acquisition. Original Community HTTP and desktop-dispatch +routes expose the read-only metadata projections, including top-level DDL list +totals and `SHOW CREATE` details. The adapter also preserves Community's paged +table name/comment search, ignored filtering on complete-list endpoints, +metadata page-size validation, HTTP 200 error envelopes, and the distinction +between a null and empty-string column default. +SELECT executes in a MySQL read-only transaction +and emits the existing typed retained-result wire messages directly from Core. +Parameters, CTE-first SELECT, locking/server-file variants, and multi-statements +fail before Java startup. Row and byte truncation plus cancellation terminate the +active MySQL connection through a separate bounded control connection. The +explicit `native-mysql-integration` target and MySQL CI job use a deliberately +missing Java executable and verify connection, first-stage object metadata, two-row +preview, typed three-row Console output, one-row truncation, active +`SELECT SLEEP(30)` cancellation, retained paging, and dormant Java health. Runtime-tested: yes. On 2026-07-29 commits `81301c3`, `4199862`, and `6c74421` passed 144 Core unit tests, strict Core all-target Clippy, formatting, Actionlint,