fix(postgres): treat FULLTEXT and SPATIAL as unreserved identifiers#2319
Open
fmguerreiro wants to merge 64 commits intoapache:mainfrom
Open
fix(postgres): treat FULLTEXT and SPATIAL as unreserved identifiers#2319fmguerreiro wants to merge 64 commits intoapache:mainfrom
fmguerreiro wants to merge 64 commits intoapache:mainfrom
Conversation
…R TABLE operations (#1) PostgreSQL supports FORCE ROW LEVEL SECURITY and NO FORCE ROW LEVEL SECURITY as ALTER TABLE operations. Add parsing support for both variants.
feat: parse EXCLUDE constraints for PostgreSQL
…m-main # Conflicts: # Cargo.toml # src/ast/ddl.rs # src/ast/helpers/stmt_create_table.rs # src/ast/mod.rs # src/ast/spans.rs # src/ast/table_constraints.rs # src/parser/mod.rs # tests/sqlparser_postgres.rs
chore: merge upstream apache/datafusion-sqlparser-rs main
- Add HANDLER and VALIDATOR keywords to the keyword list - Add FdwRoutineClause enum for HANDLER/NO HANDLER and VALIDATOR/NO VALIDATOR clauses - Add CreateForeignDataWrapper struct for CREATE FOREIGN DATA WRAPPER - Add CreateForeignTable struct for CREATE FOREIGN TABLE - Export new types from ast::mod and add Statement variants - Add spans.rs coverage returning Span::empty() for the new variants
- Dispatch on FOREIGN keyword in parse_create, branching on DATA WRAPPER or TABLE - parse_create_foreign_data_wrapper: parses optional HANDLER/NO HANDLER, VALIDATOR/NO VALIDATOR, and OPTIONS clauses - parse_create_foreign_table: parses IF NOT EXISTS, column list via parse_columns, required SERVER name, and optional OPTIONS clause
Round-trip tests via pg().verified_stmt for: - FDW: name-only, HANDLER, NO HANDLER, NO VALIDATOR, combined HANDLER+VALIDATOR+OPTIONS - FOREIGN TABLE: basic columns+SERVER, IF NOT EXISTS, table-level OPTIONS
Add Statement::CreateAggregate, CreateAggregate struct, CreateAggregateOption enum, and AggregateModifyKind enum to represent PostgreSQL CREATE AGGREGATE DDL. Options are stored as a typed enum covering all documented parameters (SFUNC, STYPE, FINALFUNC, PARALLEL, moving-aggregate variants, etc.).
Wire AGGREGATE into the CREATE dispatch (before the or_replace error branch so CREATE OR REPLACE AGGREGATE is accepted). parse_create_aggregate parses the name, argument-type list, and the options block. Each recognised option keyword dispatches to parse_create_aggregate_option which produces the typed CreateAggregateOption variant.
Three tests covering: basic old-style aggregate (SFUNC/STYPE/FINALFUNC/INITCOND), CREATE OR REPLACE with PARALLEL = SAFE, and moving-aggregate options (MSFUNC/MINVFUNC/MSTYPE/MFINALFUNC_EXTRA/MFINALFUNC_MODIFY). All use pg().verified_stmt() to assert parse-then-display round-trips identically.
feat: parse CREATE TEXT SEARCH CONFIGURATION/DICTIONARY/PARSER/TEMPLATE
# Conflicts: # src/ast/ddl.rs # src/ast/mod.rs
# Conflicts: # src/ast/ddl.rs # src/ast/mod.rs # src/parser/mod.rs
feat: parse CREATE AGGREGATE
# Conflicts: # src/ast/ddl.rs # src/ast/mod.rs # tests/sqlparser_postgres.rs
fix(spans): add Span arm for CreateAggregate
Add PublicationTarget enum (AllTables, Tables, TablesInSchema), CreatePublication struct, and CreateSubscription struct to ddl.rs. Add Statement variants, Display arms, and re-exports in mod.rs. Add Span::empty() arms in spans.rs. Add PUBLICATION and SUBSCRIPTION to keywords.rs.
…rFunctionKind with Procedure, add SetTablespace to AlterTableOperation and AlterIndexOperation Adds AST types and Display impls for: - AlterDomain + AlterDomainOperation (ADD/DROP CONSTRAINT, RENAME, OWNER TO, SET SCHEMA/DEFAULT, DROP DEFAULT, VALIDATE CONSTRAINT) - AlterTrigger + AlterTriggerOperation (RENAME TO) - AlterExtension + AlterExtensionOperation (UPDATE TO, SET SCHEMA, OWNER TO, RENAME TO) - AlterFunctionKind::Procedure variant for ALTER PROCEDURE - AlterTableOperation::SetTablespace - AlterIndexOperation::SetTablespace Updates Spanned impls to remain exhaustive.
feat: parse CREATE PUBLICATION and CREATE SUBSCRIPTION
# Conflicts: # src/ast/ddl.rs # tests/sqlparser_postgres.rs
feat: parse ALTER TABLE/INDEX SET TABLESPACE, ALTER DOMAIN, ALTER TRIGGER, ALTER EXTENSION, ALTER PROCEDURE
chore: bump version to 0.60.9
… nodes Add SecurityLabelObjectKind enum (TABLE/COLUMN/DATABASE/DOMAIN/FUNCTION/ ROLE/SCHEMA/SEQUENCE/TYPE/VIEW/MATERIALIZED VIEW) and SecurityLabel struct, CreateUserMapping struct with UserMappingUser enum, and CreateTablespace struct to ddl.rs with standard derives, Display impls, and From<X> for Statement. Add Statement variants, Display arms, re-exports in mod.rs. Add Span::empty() arms in spans.rs. Add LABEL and MAPPING to keywords.rs. Refs: pgmold-115, pgmold-119, pgmold-120
…SPACE Add parse_security_label to parse_statement dispatch on Keyword::SECURITY. Add parse_create_user_mapping dispatched from CREATE USER MAPPING branch (checked before existing CREATE USER fallthrough). Add parse_create_tablespace dispatched from parse_create on Keyword::TABLESPACE. Refs: pgmold-115, pgmold-119, pgmold-120
…REATE TABLESPACE Add 4 tests for SECURITY LABEL covering TABLE, NULL label, ROLE, and SCHEMA targets. Add 4 tests for CREATE USER MAPPING covering basic, IF NOT EXISTS with OPTIONS, CURRENT_USER, and PUBLIC user specifiers. Add 3 tests for CREATE TABLESPACE covering basic, OWNER, and WITH options. Refs: pgmold-115, pgmold-119, pgmold-120
…er, CreateTransform AST nodes Add structs and enums for four PostgreSQL-specific DDL statements: - CreateStatistics with StatisticsKind (NDistinct, Dependencies, Mcv) - CreateAccessMethod with AccessMethodType (Index, Table) - CreateEventTrigger with EventTriggerEvent (DdlCommandStart, DdlCommandEnd, TableRewrite, SqlDrop) - CreateTransform / TransformElement with OR REPLACE support Adds TRANSFORM keyword to keywords.rs. Closes pgmold-103, pgmold-104, pgmold-105, pgmold-106.
…TRANSFORM Add parse_create_statistics, parse_create_access_method, parse_create_event_trigger, and parse_create_transform to the parser. - STATISTICS: name, optional IF NOT EXISTS, optional (kind, ...) list, ON expr-list, FROM table - ACCESS METHOD: name TYPE INDEX|TABLE HANDLER function - EVENT TRIGGER: name ON event [WHEN TAG IN (...)] EXECUTE FUNCTION|PROCEDURE name() - TRANSFORM: [OR REPLACE] FOR type LANGUAGE lang (element, ...) TRANSFORM is dispatched before the or_replace error guard.
…T TRIGGER, TRANSFORM
…le AST nodes Add AST structs and Display impls for four PostgreSQL catalog DDL statements. Each follows the established fork pattern: struct in ddl.rs with Debug/Clone/PartialEq/etc derives, Statement enum variant, Display arm, From impl, and Span::empty() arm in spans.rs. New keywords: ALSO, ASSIGNMENT, CONVERSION, IMPLICIT, INLINE, PROCEDURAL, TRUSTED. Closes pgmold-89, pgmold-91, pgmold-92, pgmold-93.
feat: parse SECURITY LABEL, CREATE USER MAPPING, CREATE TABLESPACE
# Conflicts: # src/ast/ddl.rs # src/ast/mod.rs # src/ast/spans.rs # src/parser/mod.rs # tests/sqlparser_postgres.rs
feat: parse CREATE STATISTICS, CREATE ACCESS METHOD, CREATE EVENT TRIGGER, CREATE TRANSFORM
# Conflicts: # src/ast/ddl.rs # src/ast/mod.rs # src/ast/spans.rs # src/parser/mod.rs # tests/sqlparser_postgres.rs
feat: parse CREATE CAST, CREATE CONVERSION, CREATE LANGUAGE, CREATE RULE
chore: bump version to 0.60.10
PRs #14, #15, and #16 each added new AST types that collide in the same places in src/ast/ddl.rs, src/ast/mod.rs, src/parser/mod.rs, and tests/sqlparser_postgres.rs. Several mid-match-arm and mid-function-body concat resolutions during the three rebase-and-merge passes left fn / impl / match blocks missing their closing braces and left the ddl re-export list with duplicate entries. This restores compile: - Adds missing closing braces for CastFunctionKind, StatisticsKind, CreateRule / CreateStatistics / CreateEventTrigger / CreateTransform / CreateLanguage Display impls and parse fns, plus the CreateTransform From-to-Statement impl. - Dedupes and alphabetizes the `pub use self::ddl::{ ... }` list in src/ast/mod.rs. - Adds missing doc comments on CastFunctionKind fields and RuleEvent variants that #[warn(missing_docs)] requires. Unblocks 0.60.10 publish.
fix: close delimiter gaps left by concat merges
PostgreSQL's pg_dump emits WITH NO DATA on unpopulated materialized views. Add a with_data: Option<bool> field to CreateView and parse the clause after the query body. Bump to 0.60.11.
This was an accidentally-committed gitlink (160000 mode) with no corresponding entry in .gitmodules. It does not break git itself but causes any cargo git-dep consumer to fail cloning the repo: failed to update submodule `.worktrees/fix-pr-2307-ci` no URL configured for submodule '.worktrees/fix-pr-2307-ci' Same class of residue that 837b5a0 on feat/exclude-constraint-upstream cleaned up, but that commit never reached main so the gitlink still ships on every descendant branch.
feat(parser): parse WITH [NO] DATA on CREATE MATERIALIZED VIEW
Add AlterTableOperation::AttachPartitionOf and DetachPartitionOf variants
for PostgreSQL declarative partition DDL:
ALTER TABLE parent ATTACH PARTITION child { FOR VALUES <spec> | DEFAULT }
ALTER TABLE parent DETACH PARTITION child [ CONCURRENTLY | FINALIZE ]
These are distinct from the existing ClickHouse AttachPartition/DetachPartition
variants which use the PART|PARTITION <expr> syntax.
- Add FINALIZE keyword to the keyword list
- Add span impls for the new variants
- Add parser tests covering range, list, hash, default attach and
plain, concurrently, finalize detach
- Bump version to 0.60.12
…h-partition feat: parse ALTER TABLE ATTACH/DETACH PARTITION for PostgreSQL
FULLTEXT and SPATIAL are MySQL-specific table constraint keywords. When parse_optional_table_constraint encountered them in a PostgreSQL context it would consume the token, fall through to the _ arm, and call prev_token() to restore it — but the consume-then-backtrack round-trip left room for a subtle parser-state bug that caused the next call to parse_data_type to see 'fulltext' instead of the actual column type. Fix: add an explicit peek-based guard at the top of parse_optional_table_constraint so that for dialects other than MySQL / Generic the FULLTEXT / SPATIAL tokens are never consumed at all. The caller (parse_columns) then correctly falls through to parse_column_def and treats the word as an ordinary column identifier. Also add a parser unit test covering a tsvector column literally named 'fulltext' (as in the pagila DVD-rental schema) and a GiST index that references the same column by name. Bump version to 0.60.13.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PostgreSQL schemas that use
fulltextas a column name (common withtsvector/trigram search) failed to parse:Error:
SQL parse error: sql parser error: Expected: a data type name, found: 'fulltext'Root Cause
parse_optional_table_constraintconsumed theFULLTEXTtoken before realising it was not a valid constraint keyword for the current dialect, then backtracked withprev_token(). The consume-then-backtrack pattern left the parser in a subtly broken state for non-MySQL/Generic dialects, causing the nextparse_column_def_innercall to see the wrong current token.Fix
Add a peek-based early-return guard in
parse_optional_table_constraintbefore any token is consumed. For dialects other than MySQL/Generic,FULLTEXTandSPATIALare valid identifiers and must not be touched by the constraint parser:Test
Added
parse_fulltext_column_and_index_in_postgresintests/sqlparser_postgres.rscovering:CREATE TABLE film (fulltext TSVECTOR NOT NULL)— column namedfulltextwithtsvectortypeCREATE INDEX film_fulltext_idx ON film USING gist (fulltext)— GiST index on afulltextcolumnVersion
Bumped to
0.60.13.