From a66e5a0174184970f101ccb116de2e7970811a6b Mon Sep 17 00:00:00 2001 From: Aasim Khan Date: Tue, 21 Apr 2026 10:38:42 -0700 Subject: [PATCH] Refactor T-SQL syntax verification guide and add agent documentation - Updated the T-SQL syntax verification guide in `.github/prompts/verify-and-test-tsql-syntax.prompt.md` to streamline instructions and emphasize the use of the canonical skill. - Introduced `AGENTS.md` to provide an overview of agent skills and instruction files, ensuring clarity on usage and versioning. - Created `CLAUDE.md` to establish `AGENTS.md` as the canonical instruction source for Claude workflows, reinforcing the importance of following shared guidelines. --- .../skills/add-feature/SKILL.md | 20 +- .../verify-and-test-tsql-syntax/SKILL.md | 115 +++ .../new-feature-implementation.prompt.md | 528 +------------ .../verify-and-test-tsql-syntax.prompt.md | 700 +----------------- AGENTS.md | 32 + CLAUDE.md | 7 + 6 files changed, 194 insertions(+), 1208 deletions(-) rename {.github => .agents}/skills/add-feature/SKILL.md (94%) create mode 100644 .agents/skills/verify-and-test-tsql-syntax/SKILL.md create mode 100644 AGENTS.md create mode 100644 CLAUDE.md diff --git a/.github/skills/add-feature/SKILL.md b/.agents/skills/add-feature/SKILL.md similarity index 94% rename from .github/skills/add-feature/SKILL.md rename to .agents/skills/add-feature/SKILL.md index ce63bec..2873e72 100644 --- a/.github/skills/add-feature/SKILL.md +++ b/.agents/skills/add-feature/SKILL.md @@ -10,9 +10,9 @@ platforms: "SQL Server, Azure SQL DB, Fabric, Fabric DW, VNext" This skill helps add new T-SQL features to the ScriptDOM parser by: 1. **Interviewing** you about the feature and target platform (SQL Server, Azure SQL DB, Fabric, Fabric DW, VNext) -2. **Determining the latest parser version** from `SqlVersionFlags.cs` (currently TSql170) +2. **Determining the latest parser version** from `SqlVersionFlags.cs` 3. **Routing** you to the appropriate parser: - - SQL Server → version-specific parser (TSql120-170) + - SQL Server → version-specific parser (TSql120-180) - Azure SQL DB/Fabric/VNext → latest parser version - Fabric DW → separate TSqlFabricDW parser 4. **Classifying** the change type (grammar, validation, function, data type, etc.) @@ -33,7 +33,7 @@ Ask the user these questions to understand the feature: - **Azure SQL Database** (uses latest parser version) - **Fabric** (uses latest parser version) - **Fabric DW** (uses separate TSqlFabricDW parser) - - **VNext** (future version, uses latest parser version) + - **VNext** (future version, uses latest parser version unless repo instructions say otherwise) 3. **If SQL Server, which version introduced this feature?** - SQL Server 2014 (TSql120) @@ -42,7 +42,8 @@ Ask the user these questions to understand the feature: - SQL Server 2019 (TSql150) - SQL Server 2022 (TSql160) - SQL Server 2025 (TSql170) - - *(Skip if Azure SQL DB, Fabric, or VNext - these use latest version)* + - SQL Server vNext (TSql180) + - *(Skip if Azure SQL DB, Fabric, or VNext - these use TSql180 by default)* 4. **Do you have example T-SQL syntax or Microsoft documentation links?** - This helps verify the exact syntax requirements @@ -124,7 +125,7 @@ For grammar changes (Types B, C, D, E, G), follow this workflow: ### 3.1 Identify Target Grammar File **First, determine the latest parser version:** -Check `SqlScriptDom/Parser/TSql/SqlVersionFlags.cs` for the highest TSql version enum value (currently TSql170). +Check `SqlScriptDom/Parser/TSql/SqlVersionFlags.cs` for the highest TSql version enum value. **Then select the appropriate grammar file:** @@ -135,9 +136,10 @@ Check `SqlScriptDom/Parser/TSql/SqlVersionFlags.cs` for the highest TSql version - SQL 2019+: TSql150.g - SQL 2022+: TSql160.g - SQL 2025+: TSql170.g +- SQL Server vNext: TSql180.g #### For Azure SQL Database, Fabric, VNext: -- Use the **latest parser version** (e.g., TSql170.g) +- Use the **latest parser version** (currently TSql180.g) - Check `SqlVersionFlags.cs` to confirm the highest version #### For Fabric DW: @@ -174,7 +176,7 @@ If adding new AST nodes: ### 4.2 Add Test Method -#### For SQL Server versions (TSql120-170): +#### For SQL Server versions (TSql120-180): In `Test/SqlDom/OnlySyntaxTests.cs`: ```csharp @@ -191,8 +193,8 @@ Add to the `OnlyFabricDWTestInfos` array in `Test/SqlDom/OnlyFabricDWSyntaxTests ```csharp new ParserTestFabricDW("YourFeatureFabricDW.sql", nErrors80: X, nErrors90: Y, nErrors100: Z, - nErrors110: A, nErrors120: B, nErrors130: C, - nErrors140: D, nErrors150: E, nErrors160: F, nErrors170: G), + nErrors110: A, nErrors120: B, nErrors130: C, + nErrors140: D, nErrors150: E, nErrors160: F, nErrors170: G, nErrors180: H), ``` *(Set expected error counts for each parser version)* diff --git a/.agents/skills/verify-and-test-tsql-syntax/SKILL.md b/.agents/skills/verify-and-test-tsql-syntax/SKILL.md new file mode 100644 index 0000000..fed8121 --- /dev/null +++ b/.agents/skills/verify-and-test-tsql-syntax/SKILL.md @@ -0,0 +1,115 @@ +--- +name: verify-and-test-tsql-syntax +description: Verify whether an exact T-SQL script is already supported by SqlScriptDOM, then add or update the correct parser tests and baselines. Use when a user asks if syntax already works, or when adding regression coverage for a specific script. +argument-hint: "Exact T-SQL script or syntax to verify" +user-invocable: true +--- + +## Overview + +This skill determines whether a specific T-SQL script is already supported and then guides the follow-up testing work. + +Use it for: +- verifying whether a script already parses +- confirming which parser version should support the syntax +- adding or updating regression coverage in the existing ScriptDOM test framework + +Do not use it for broad feature design. If the verification shows missing parser support, route the implementation work to the appropriate repo instruction: +- [bug_fixing.guidelines.instructions.md](../../../.github/instructions/bug_fixing.guidelines.instructions.md) +- [grammar_validation.guidelines.instructions.md](../../../.github/instructions/grammar_validation.guidelines.instructions.md) +- [parser.guidelines.instructions.md](../../../.github/instructions/parser.guidelines.instructions.md) +- [testing.guidelines.instructions.md](../../../.github/instructions/testing.guidelines.instructions.md) + +## Core Rules + +- Always verify the exact T-SQL text first, character for character. +- For first-pass verification, add a debug unit test method to an existing test file such as `Only180SyntaxTests.cs` when the syntax targets vNext or the latest parser. +- Do not create standalone console apps, ad hoc parsers, or extra test projects. +- After verification, add proper test coverage through `TestScripts`, baselines, and the existing syntax test classes. +- Remove any temporary debug-only test methods once the result is confirmed. + +## Workflow + +### 1. Gather the minimum inputs + +Ask for: +- the exact T-SQL script to verify +- the SQL Server version or parser version expected to support it +- the current behavior and the expected behavior + +If the user has not supplied the exact script yet, stop and request it before continuing. + +### 2. Verify the exact script first + +Add a temporary debug unit test method to the appropriate existing test class and test the exact script as provided. + +Use this sequence: + +```bash +dotnet build SqlScriptDom/Microsoft.SqlServer.TransactSql.ScriptDom.csproj -c Debug +dotnet test --filter "DebugExactScriptTest" Test/SqlDom/UTSqlScriptDom.csproj -c Debug +``` + +Verification expectations: +- If parsing succeeds, move directly to comprehensive test coverage. +- If parsing fails, capture the exact parse errors and classify the failure. + +### 3. Classify the gap if verification fails + +Use the failure mode to route the next step: +- Grammar issue: parser does not recognize the syntax at all. +- Validation issue: syntax parses but is rejected as unsupported or invalid in context. +- Predicate recognition issue: identifier-based predicate fails in parentheses. + +When similar syntax already works in another context, prefer validation analysis before changing grammar. + +### 4. Add comprehensive coverage + +After verification, create or update the proper permanent tests using the existing framework: + +- Add the input script under `Test/SqlDom/TestScripts/`. +- Add the matching baseline under `Test/SqlDom/Baselines/`. +- Add the `ParserTest` entry or permanent test method in the correct `OnlySyntaxTests.cs` file. +- Preserve the exact user syntax in the test script before adding broader coverage. + +Determine the parser version using the repo mapping: +- SQL Server 2014 -> TSql120 +- SQL Server 2016 -> TSql130 +- SQL Server 2017 -> TSql140 +- SQL Server 2019 -> TSql150 +- SQL Server 2022 -> TSql160 +- SQL Server 2025 -> TSql170 +- SQL Server vNext/latest -> TSql180 + +### 5. Validate the testing slice + +Run validation in this order: + +```bash +dotnet test --filter "FullyQualifiedName~YourFeatureTest" Test/SqlDom/UTSqlScriptDom.csproj -c Debug +dotnet test Test/SqlDom/UTSqlScriptDom.csproj -c Debug +``` + +If the targeted test fails because of a baseline mismatch: +- use the generated output from the failure log to update the baseline +- rerun the same targeted test +- only then run the full suite + +## File Targets + +Use these repo paths when doing the work: +- Grammar files: `SqlScriptDom/Parser/TSql/TSql*.g` +- AST definition: `SqlScriptDom/Parser/TSql/Ast.xml` +- Validation code: `SqlScriptDom/Parser/TSql/TSql80ParserBaseInternal.cs` +- Test scripts: `Test/SqlDom/TestScripts/` +- Baselines: `Test/SqlDom/Baselines/` +- Syntax tests: `Test/SqlDom/OnlySyntaxTests.cs` + +## Completion Checklist + +- [ ] Verified the exact script first in an existing test file +- [ ] Classified the result as already supported or missing support +- [ ] Added or updated permanent test coverage in the standard test framework +- [ ] Preserved the exact user syntax in the new test script +- [ ] Ran targeted validation for the touched test slice +- [ ] Ran the full `UTSqlScriptDom` suite before finishing \ No newline at end of file diff --git a/.github/prompts/new-feature-implementation.prompt.md b/.github/prompts/new-feature-implementation.prompt.md index dd87fc9..69941ec 100644 --- a/.github/prompts/new-feature-implementation.prompt.md +++ b/.github/prompts/new-feature-implementation.prompt.md @@ -1,518 +1,20 @@ -# New SQL Server Feature Implementation Guide - -This prompt will identify the type of SQL Server feature you want to add to SqlScriptDOM and **automatically implement it** using the appropriate guideline. After feature type identification, it will execute the complete implementation workflow. - -## Feature Type Identification - -Please answer the following questions to determine the best implementation approach: - -### 1. What type of SQL Server feature are you implementing? - -**A) Data Type** - A new SQL Server data type (e.g., VECTOR, GEOMETRY, GEOGRAPHY) -- Example: `DECLARE @embedding AS VECTOR(1536, FLOAT32)` -- Example: `CREATE TABLE tbl (geo_data GEOGRAPHY)` -- **Key indicators**: New type with custom parameters, specialized syntax for type definitions - -**B) Index Type** - A specialized index type with unique syntax (e.g., JSON INDEX, VECTOR INDEX) -- Example: `CREATE JSON INDEX IX_JSON ON table (column) FOR ('$.path1', '$.path2')` -- Example: `CREATE VECTOR INDEX IX_VECTOR ON table (column) WITH (METRIC = 'cosine')` -- **Key indicators**: CREATE [TYPE] INDEX syntax, type-specific clauses or options - -**C) System Function** - A new T-SQL built-in function (e.g., JSON_OBJECT, JSON_ARRAY) -- Example: `SELECT JSON_OBJECT('key1': 'value1', 'key2': 'value2')` -- Example: `RETURN JSON_ARRAY('item1', 'item2', 'item3')` -- **Key indicators**: Function calls in expressions, may need RETURN statement support - -**D) Grammar/Syntax Enhancement** - New operators, statements, or syntax modifications -- Example: Adding new WHERE clause predicates like `REGEXP_LIKE` -- Example: New statement types or operators -- **Key indicators**: Parser doesn't recognize syntax, needs AST updates - -**E) Validation Fix** - Existing syntax fails validation but should work per SQL Server docs -- Example: ALTER TABLE RESUMABLE option works in ALTER INDEX but not ALTER TABLE -- **Key indicators**: "Option 'X' is not valid..." errors, similar syntax works elsewhere - -**F) Parser Predicate Issue** - Identifier-based predicates fail with parentheses -- Example: `WHERE REGEXP_LIKE(...)` works but `WHERE (REGEXP_LIKE(...))` fails -- **Key indicators**: Syntax errors near closing parenthesis with identifier predicates - -### 2. SQL Server Feature Details - -**Feature Name**: _______________ -**SQL Server Version**: _______________ -**Example Syntax**: -```sql --- Provide 2-3 examples of the syntax you want to support -``` - -**Current Behavior**: _______________ -**Expected Behavior**: _______________ - -### 3. Feature Characteristics - -Check all that apply to your feature: - -- [ ] Requires completely new AST node classes -- [ ] Extends existing AST nodes with new members -- [ ] Needs new grammar rules in .g files -- [ ] Requires new keywords/constants -- [ ] Needs specialized script generation logic -- [ ] Has version-specific behavior (SQL Server 2014+, 2022+, etc.) -- [ ] Includes optional syntax elements or clauses -- [ ] Supports collections/lists of parameters -- [ ] Requires new validation logic -- [ ] Needs new index options or statement options - -## AUTO-IMPLEMENTATION TRIGGER - -**To begin automatic implementation, provide your feature details in this exact format:** - -``` -Feature Name: [Your feature name] -SQL Server Version: [SQL Server version] -Exact T-SQL Syntax: -```sql -[Copy the EXACT T-SQL syntax from the user's request here] -``` -Feature Type: [Will be determined from analysis below] -``` - -**The system will then automatically identify the feature type and begin implementation.** - -## Implementation Guidance - -Based on your feature type identification below, the system will automatically execute the appropriate implementation workflow: - -### → Data Type (Answer A) -**Auto-Executes**: [New Data Types Guidelines](../instructions/new_data_types.guidelines.instructions.md) - -**Automatic implementation includes**: -- Creating new `DataTypeReference` AST classes -- Adding specialized parsing rules for custom type syntax -- Implementing parameter handling (dimensions, base types, etc.) -- Script generation for type definitions -- Version-specific type support -- Comprehensive testing across all SQL contexts - -**Best for**: VECTOR, custom CLR types, spatial types, hierarchical types - -### → Index Type (Answer B) -**Auto-Executes**: [New Index Types Guidelines](../instructions/new_index_types.guidelines.instructions.md) - -**Automatic implementation includes**: -- Creating new `IndexStatement` AST classes -- Implementing type-specific index syntax parsing -- Adding specialized clauses (FOR, WITH type-specific options) -- Index option registration and validation -- Script generation for index statements -- Integration with existing index framework - -**Best for**: JSON INDEX, VECTOR INDEX, SPATIAL INDEX, custom index types - -### → System Function (Answer C) -**Auto-Executes**: [Function Guidelines](../instructions/function.guidelines.instructions.md) - -**Automatic implementation includes**: -- Function AST design for new T-SQL functions -- Grammar rules with syntactic predicates for RETURN statement support -- ANTLR v2 lookahead limitations and solutions -- Script generation for function calls -- Comprehensive testing in all expression contexts - -**Best for**: JSON_OBJECT, JSON_ARRAY, AI functions, mathematical functions - -### → Grammar/Syntax Enhancement (Answer D) -**Auto-Executes**: [Bug Fixing Guidelines](../instructions/bug_fixing.guidelines.instructions.md) - -**Automatic implementation includes**: -- Grammar rule modifications and AST updates -- Script generation implementation -- Testing framework integration -- Extending literals to expressions pattern -- Version compatibility considerations - -**Best for**: New operators, statement types, expression enhancements - -### → Validation Fix (Answer E) -**Auto-Executes**: [Validation Fix Guidelines](../instructions/validation_fix.guidelines.instructions.md) - -**Automatic implementation includes**: -- Version-gated validation fixes -- SQL Server version compatibility checks -- Context-specific validation rules -- Testing validation behavior across versions -- No grammar changes needed - -**Best for**: Feature works in one context but not another, version support issues - -### → Parser Predicate Issue (Answer F) -**Auto-Executes**: [Parser Guidelines](../instructions/parser.guidelines.instructions.md) - -**Automatic implementation includes**: -- Identifier-based predicate recognition fixes -- `IsNextRuleBooleanParenthesis()` function updates -- Syntactic vs semantic predicate handling -- Parentheses support in boolean contexts - -**Best for**: Functions work without parentheses but fail with them - -## Grammar Extension Patterns - -For users implementing Grammar/Syntax Enhancement (Option D), here are common patterns: - -### Pattern 1: Extending Literals to Expressions - -#### When to Use -When existing grammar rules only accept literal values but need to support dynamic expressions like parameters, variables, or computed values. - -#### Example Problem -Functions or constructs that currently accept only: -- `IntegerLiteral` (e.g., `TOP_N = 10`) -- `StringLiteral` (e.g., `VALUE = 'literal'`) - -But need to support: -- Parameters: `@parameter` -- Variables: `@variable` -- Column references: `table.column` -- Outer references: `outerref.column` -- Function calls: `FUNCTION(args)` -- Computed expressions: `value + 1` - -#### ⚠️ Critical Warning: Avoid Modifying Shared Grammar Rules - -**DO NOT** modify existing shared grammar rules like `identifierColumnReferenceExpression` that are used throughout the codebase. This can cause unintended side effects and break other functionality. - -**Instead**, create specialized rules for your specific context. - -#### Solution Template - -**Step 1: Update AST Definition (`Ast.xml`)** -```xml - - - - - -``` - -**Step 2: Create Context-Specific Grammar Rule (`TSql*.g`)** -```antlr -// Create a specialized rule for your context -yourContextColumnReferenceExpression returns [ColumnReferenceExpression vResult = this.FragmentFactory.CreateFragment()] -{ - MultiPartIdentifier vMultiPartIdentifier; -} - : - vMultiPartIdentifier=multiPartIdentifier[2] // Allows table.column syntax - { - vResult.ColumnType = ColumnType.Regular; - vResult.MultiPartIdentifier = vMultiPartIdentifier; - } - ; - -// Use the specialized rule in your custom grammar -yourContextParameterRule returns [ScalarExpression vResult] - : vResult=signedInteger - | vResult=variable - | vResult=yourContextColumnReferenceExpression // Context-specific rule - | vResult=expression // Allows computed expressions - ; -``` - -**Step 3: Verify Script Generator** -Most script generators using `GenerateNameEqualsValue()` or similar methods work automatically with `ScalarExpression`. No changes typically needed. - -#### Real-World Example: VECTOR_SEARCH TOP_N - -**Problem**: `VECTOR_SEARCH` TOP_N parameter only accepted integer literals. - -**❌ Wrong Approach**: Modify `identifierColumnReferenceExpression` to use `multiPartIdentifier[2]` -- **Result**: Broke `CreateIndexStatementErrorTest` because other grammar rules started accepting invalid syntax - -**✅ Correct Approach**: Create `vectorSearchColumnReferenceExpression` specialized for VECTOR_SEARCH -- **Result**: VECTOR_SEARCH supports multi-part identifiers without affecting other functionality - -**Final Implementation**: -```antlr -signedIntegerOrVariableOrColumnReference returns [ScalarExpression vResult] - : vResult=signedInteger - | vResult=variable - | vResult=vectorSearchColumnReferenceExpression // VECTOR_SEARCH-specific rule - ; - -vectorSearchColumnReferenceExpression returns [ColumnReferenceExpression vResult = ...] - : - vMultiPartIdentifier=multiPartIdentifier[2] // Allows table.column syntax - { - vResult.ColumnType = ColumnType.Regular; - vResult.MultiPartIdentifier = vMultiPartIdentifier; - } - ; -``` - -**Result**: Now supports dynamic TOP_N values: -```sql --- Parameters -VECTOR_SEARCH(..., TOP_N = @k) AS ann - --- Outer references -VECTOR_SEARCH(..., TOP_N = outerref.max_results) AS ann -``` - -### Pattern 2: Adding New Enum Members - -#### When to Use -When adding new operators, keywords, or options to existing constructs. - -#### Solution Template - -**Step 1: Update Enum in AST (`Ast.xml`)** -```xml - - - - - -``` - -**Step 2: Update Grammar Rule (`TSql*.g`)** -```antlr -// Add new token matching -| tNewValue:Identifier -{ - Match(tNewValue, CodeGenerationSupporter.NewValue); - vResult.EnumProperty = ExistingEnumType.NewValue; -} -``` - -**Step 3: Update Script Generator** -```csharp -// Add mapping in appropriate generator file -private static readonly Dictionary _enumGenerators = - new Dictionary() -{ - { EnumType.ExistingValue1, CodeGenerationSupporter.ExistingValue1 }, - { EnumType.ExistingValue2, CodeGenerationSupporter.ExistingValue2 }, - { EnumType.NewValue, CodeGenerationSupporter.NewValue }, // Add this -}; -``` - -### Pattern 3: Adding New Function or Statement - -#### When to Use -When adding completely new T-SQL functions or statements. - -#### Solution Template - -**Step 1: Define AST Node (`Ast.xml`)** -```xml - - - - -``` - -**Step 2: Add Grammar Rule (`TSql*.g`)** -```antlr -newFunctionCall returns [NewFunctionCall vResult = FragmentFactory.CreateFragment()] -{ - ScalarExpression vParam1; - StringLiteral vParam2; -} - : - tFunction:Identifier LeftParenthesis - { - Match(tFunction, CodeGenerationSupporter.NewFunction); - UpdateTokenInfo(vResult, tFunction); - } - vParam1 = expression - { - vResult.Parameter1 = vParam1; - } - Comma vParam2 = stringLiteral - { - vResult.Parameter2 = vParam2; - } - RightParenthesis - ; -``` - -**Step 3: Integrate with Existing Rules** -Add the new rule to appropriate places in the grammar (e.g., `functionCall`, `primaryExpression`, etc.). - -**Step 4: Create Script Generator** -```csharp -public override void ExplicitVisit(NewFunctionCall node) -{ - GenerateIdentifier(CodeGenerationSupporter.NewFunction); - GenerateSymbol(TSqlTokenType.LeftParenthesis); - GenerateFragmentIfNotNull(node.Parameter1); - GenerateSymbol(TSqlTokenType.Comma); - GenerateFragmentIfNotNull(node.Parameter2); - GenerateSymbol(TSqlTokenType.RightParenthesis); -} -``` - -## Quick Decision Tree - -**Start here** → Does the syntax exist in SQL Server documentation? - -**No** → Use Grammar/Syntax Enhancement (D) - -**Yes** → Does SqlScriptDOM recognize the syntax? - -**No** → What type of syntax? -- Data type declaration → Data Type (A) -- CREATE [TYPE] INDEX → Index Type (B) -- Function call → System Function (C) -- Other syntax → Grammar/Syntax Enhancement (D) - -**Yes** → Does it parse without errors? - -**No** → Parser Predicate Issue (F) - -**Yes** → Does validation reject it incorrectly? - -**Yes** → Validation Fix (E) - -**No** → Review existing implementation or check for edge cases - -## Pre-Implementation Checklist - -Before starting implementation: - -- [ ] Verified feature exists in SQL Server documentation -- [ ] Identified target SQL Server version for the feature -- [ ] Confirmed feature doesn't already exist in SqlScriptDOM -- [ ] Collected comprehensive syntax examples from SQL Server docs -- [ ] Reviewed similar existing implementations in the codebase -- [ ] Selected appropriate guideline based on feature type - -## Testing Strategy - -Regardless of feature type, ensure you: - -- [ ] Create comprehensive test scripts covering all syntax variations -- [ ] Generate proper baseline files with expected formatted output -- [ ] Configure error counts for all SQL Server versions -- [ ] Run full test suite to prevent regressions -- [ ] Test edge cases, quoted identifiers, and schema qualification -- [ ] Verify round-trip parsing (parse → generate → parse) - -## Additional Resources - -- **Main Copilot Instructions**: [copilot-instructions.md](../copilot-instructions.md) -- **Testing Framework Guide**: [Testing Guidelines](../instructions/testing.guidelines.instructions.md) -- **Grammar Extension Patterns**: See Grammar Extension Patterns section above -- **Detailed Grammar Guidelines**: [Grammar Guidelines](../instructions/grammer.guidelines.instructions.md) - +--- +name: new-feature-implementation +description: Copilot wrapper for the canonical add-feature skill used to implement new SqlScriptDOM T-SQL features. +argument-hint: "Feature description or exact T-SQL syntax" +agent: agent --- -**Ready to implement?** Follow the guideline that matches your feature type above. Each guide provides step-by-step instructions, real-world examples, and comprehensive testing strategies. - -## Implementation Workflow - -**IMPORTANT**: After identifying your feature type above, this prompt will automatically begin implementation. Provide the following information to start: - -### Required Information -1. **Feature Name**: _______________ -2. **SQL Server Version**: _______________ -3. **Exact T-SQL Syntax Examples**: -```sql --- Provide the EXACT syntax you want to support (copy-paste from user request) --- Example: SELECT JSON_OBJECTAGG( t.c1 : t.c2 ) FROM (VALUES('key1', 'c'), ('key2', 'b'), ('key3','a')) AS t(c1, c2); -``` -4. **Feature Type** (from analysis above): A, B, C, D, E, or F - -### Automatic Implementation Process - -Once you provide the information above, this prompt will: - -#### Phase 1: Analysis and Verification (Always Done First) -1. **Verify current status** using the exact syntax provided -2. **Search existing codebase** for similar implementations -3. **Identify SQL Server version** and parser target -4. **Create implementation plan** with specific steps -5. **Show the plan** and get confirmation before proceeding - -#### Phase 2: Implementation (Executed Automatically) -Based on feature type identification: - -**For Grammar/Syntax Enhancement (Type D)**: -1. **Update AST definition** (`Ast.xml`) if new nodes needed -2. **Add grammar rules** in appropriate `TSql*.g` files -3. **Create script generators** for new AST nodes -4. **Build and validate** parser compilation -5. **Create comprehensive tests** with exact syntax provided -6. **Generate baseline files** from parser output -7. **Run full test suite** to ensure no regressions - -**For Validation Fix (Type E)**: -1. **Locate validation function** throwing the error -2. **Verify Microsoft documentation** for version support -3. **Apply version-gated validation** (not unconditional rejection) -4. **Create test cases** covering all scenarios -5. **Build and validate** the fix -6. **Run full test suite** to ensure correctness - -**For System Function (Type C)**: -1. **Define AST node structure** for the function -2. **Add grammar rules** with syntactic predicates for RETURN statement support -3. **Create script generator** for the function -4. **Build and test** grammar changes -5. **Create comprehensive test scripts** including RETURN statement usage -6. **Validate full test suite** for regressions - -**For Data Type (Type A)**: -1. **Define AST node** inheriting from `DataTypeReference` -2. **Create specialized parsing rule** for the data type -3. **Integrate with scalar data type rule** -4. **Add string constants** for keywords -5. **Create script generator** -6. **Build and comprehensive test** across all SQL contexts - -**For Index Type (Type B)**: -1. **Define AST node** inheriting from `IndexStatement` -2. **Create specialized parsing rule** for the index type -3. **Integrate with main index grammar** -4. **Add index options** if needed -5. **Create script generator** -6. **Build and comprehensive test** all syntax variations - -**For Parser Predicate Issue (Type F)**: -1. **Locate `IsNextRuleBooleanParenthesis()`** function -2. **Add identifier-based predicate detection** -3. **Build and test** the fix -4. **Create tests** covering parentheses scenarios -5. **Validate** existing functionality - -#### Phase 3: Validation and Documentation -1. **Run complete test suite** (`dotnet test Test/SqlDom/UTSqlScriptDom.csproj -c Debug`) -2. **Verify all tests pass** (expect 1,100+ tests to succeed) -3. **Document changes made** with before/after examples -4. **Provide usage examples** showing the new functionality - -### Starting Implementation - -To begin implementation, provide your feature details using this format: - -``` -Feature Name: [FUNCTION_NAME or FEATURE_NAME] -SQL Server Version: [SQL Server 20XX / TSqlXXX] -Exact T-SQL Syntax: -```sql -[EXACT_COPY_OF_SYNTAX_FROM_USER_REQUEST] -``` -Feature Type: [A/B/C/D/E/F based on analysis above] -``` +Use the [canonical add-feature skill](../../.agents/skills/add-feature/SKILL.md) as the source of truth for this workflow. -**The prompt will then automatically execute the appropriate implementation workflow and start making the necessary code changes.** +- Implement the requested SQL Server feature end to end. +- Start by asking the discovery questions defined in the shared skill. +- Classify the request into the correct feature type before editing code. +- Follow the matching file in `.github/instructions/` for implementation, tests, and validation. +- Treat `TSql180` as the current vNext/latest parser target. +- Use the exact T-SQL syntax supplied by the user when creating tests. -### Implementation Principles +This prompt is a Copilot convenience wrapper around the shared skill. Keep the detailed workflow in the skill, not here. -1. **Always test exact syntax first**: Use the exact T-SQL provided, not simplified versions -2. **Follow established patterns**: Reuse existing patterns from similar implementations -3. **Maintain backward compatibility**: Ensure existing functionality continues to work -4. **Comprehensive testing**: Test all syntax variations, edge cases, and error conditions -5. **Version compatibility**: Consider which SQL Server versions should support the feature -6. **Full regression testing**: Always run the complete test suite before completion \ No newline at end of file +Input: +${input:featureDescription:Describe the feature or paste the exact T-SQL syntax} \ No newline at end of file diff --git a/.github/prompts/verify-and-test-tsql-syntax.prompt.md b/.github/prompts/verify-and-test-tsql-syntax.prompt.md index 95fb51f..c7db92f 100644 --- a/.github/prompts/verify-and-test-tsql-syntax.prompt.md +++ b/.github/prompts/verify-and-test-tsql-syntax.prompt.md @@ -1,692 +1,20 @@ --- -title: How to Verify T-SQL Syntax Support and Add Tests -description: Step-by-step guide to check if a T-SQL syntax is already supported and how to add comprehensive test coverage -tags: [testing, verification, tsql, syntax, parser, baseline] +name: verify-and-test-tsql-syntax +description: Copilot wrapper for the canonical shared skill that verifies exact T-SQL syntax support and adds permanent regression coverage. +argument-hint: "Exact T-SQL script or syntax to verify" +agent: agent --- -# How to Verify T-SQL Syntax Support and Add Tests +Use the [canonical verify-and-test-tsql-syntax skill](../../.agents/skills/verify-and-test-tsql-syntax/SKILL.md) as the source of truth for this workflow. -This guide helps you determine if a T-SQL syntax is already supported by ScriptDOM and shows you how to add proper test coverage. +- Verify the exact T-SQL script first, character for character. +- Use an existing test file for any temporary debug verification. +- If the script already works, add comprehensive permanent tests and baselines. +- If it fails, classify the gap before implementing a fix. +- Treat `TSql180` as the current vNext/latest parser target when the syntax is for vNext or the latest parser. +- Remove temporary debug-only verification code before finishing. -## Step 0: Verify the Exact Script First +This prompt is a Copilot convenience wrapper around the shared skill. Keep the detailed workflow in the skill, not here. -**CRITICAL**: Before doing anything else, test the exact T-SQL script provided to confirm whether it works or fails. - -**IMPORTANT**: For initial verification, you MUST add a debug unit test method directly to an existing test file (like Only170SyntaxTests.cs). This is only for initial verification. Once you confirm the syntax status, you'll follow the proper testing workflow to add comprehensive test coverage. - -### Step 1: Add Debug Unit Test Method - -Add this debug test method to the appropriate test file (e.g., `Test/SqlDom/Only170SyntaxTests.cs`): - -```csharp -[TestMethod] -public void DebugExactScriptTest() -{ - // PUT THE EXACT T-SQL SCRIPT HERE - DO NOT CREATE SEPARATE FILES - string script = @"SELECT Id, - DATEADD(DAY, 1, GETDATE()) -FROM Table1"; - - Console.WriteLine($"Testing exact script: {script}"); - - // Test with the target parser version first (e.g., TSql170) - TSql170Parser parser170 = new TSql170Parser(true); - IList errors170; - - using (StringReader reader = new StringReader(script)) - { - TSqlFragment fragment = parser170.Parse(reader, out errors170); - - Console.WriteLine($"\n=== TSql170 Parser Results ==="); - if (errors170.Count == 0) - { - Console.WriteLine("✅ SUCCESS: Parsed without errors"); - - // Test script generation (round-trip) - Sql170ScriptGenerator generator = new Sql170ScriptGenerator(); - string generatedScript; - generator.GenerateScript(fragment, out generatedScript); - Console.WriteLine($"Generated: {generatedScript}"); - } - else - { - Console.WriteLine($"❌ FAILED: {errors170.Count} parse errors:"); - foreach (var error in errors170) - { - Console.WriteLine($" Line {error.Line}, Col {error.Column}: {error.Message}"); - } - } - } - - // Test with older parser version for comparison (e.g., TSql160) - TSql160Parser parser160 = new TSql160Parser(true); - IList errors160; - - using (StringReader reader = new StringReader(script)) - { - TSqlFragment fragment = parser160.Parse(reader, out errors160); - - Console.WriteLine($"\n=== TSql160 Parser Results ==="); - if (errors160.Count == 0) - { - Console.WriteLine("✅ SUCCESS: Parsed without errors"); - } - else - { - Console.WriteLine($"❌ FAILED: {errors160.Count} parse errors:"); - foreach (var error in errors160) - { - Console.WriteLine($" Line {error.Line}, Col {error.Column}: {error.Message}"); - } - } - } - - // Use Assert.Inconclusive to document current status without failing the test - if (errors170.Count > 0) - { - Assert.Inconclusive($"Script currently fails with {errors170.Count} errors. Needs implementation."); - } - else - { - Assert.Inconclusive("Script already works! Can proceed to add comprehensive test coverage."); - } -} -``` - -### Step 2: Build and Run the Debug Test -### Step 2: Build and Run the Debug Test - -```bash -# 1. Build the parser to ensure it's up to date -dotnet build SqlScriptDom/Microsoft.SqlServer.TransactSql.ScriptDom.csproj -c Debug - -# 2. Run the debug test to see current status -dotnet test --filter "DebugExactScriptTest" Test/SqlDom/UTSqlScriptDom.csproj -c Debug - -# 3. Check the test output for detailed results -# Look for the console output showing parsing results -``` - -### Interpret Results - -- **✅ SUCCESS**: Script works! You can skip to Step 6 to add comprehensive tests -- **❌ FAILURE**: Script fails. Continue with Steps 1-5 to implement the missing functionality - -**Important**: Always test the **exact script provided** character-for-character, including: -- Exact table/column names (e.g., `t.c1`, `t.c2`) -- Exact function syntax (e.g., `JSON_OBJECTAGG( t.c1 : t.c2 )`) -- Complete query context (FROM clause, subqueries, etc.) -- Exact whitespace and formatting as provided - -**Remember**: Only add unit test methods to existing test files. Do not create separate SQL files, program files, or any other external files. - -## Step 1: Determine the SQL Server Version - -First, identify which SQL Server version introduced the syntax you want to test. - -### SQL Server Version Mapping - -| SQL Server Version | Parser Version | Year | Common Name | -|-------------------|----------------|------|-------------| -| SQL Server 2000 | TSql80 | 2000 | SQL Server 2000 | -| SQL Server 2005 | TSql90 | 2005 | SQL Server 2005 | -| SQL Server 2008 | TSql100 | 2008 | SQL Server 2008 | -| SQL Server 2012 | TSql110 | 2012 | SQL Server 2012 | -| SQL Server 2014 | TSql120 | 2014 | SQL Server 2014 | -| SQL Server 2016 | TSql130 | 2016 | SQL Server 2016 | -| SQL Server 2017 | TSql140 | 2017 | SQL Server 2017 | -| SQL Server 2019 | TSql150 | 2019 | SQL Server 2019 | -| SQL Server 2022 | TSql160 | 2022 | SQL Server 2022 | -| SQL Server 2025 | TSql170 | 2025 | SQL Server 2025 | - -### How to Find the Version - -1. **Check Microsoft Documentation**: Look for "Applies to: SQL Server 20XX (XX.x)" -2. **Search Online**: Look for the feature announcement or blog posts -3. **Test in SSMS**: Connect to different SQL Server versions and try the syntax - -**Example**: -- `RESUMABLE = ON` for ALTER TABLE → SQL Server 2022 → **TSql160** -- `MAX_DURATION` for indexes → SQL Server 2014 → **TSql120** -- `VECTOR_SEARCH` function → SQL Server 2025 → **TSql170** - -## Step 2: Check if Syntax is Already Supported - -### Method 1: Search Test Scripts (Fastest) -```bash -# Search for the keyword in test scripts -grep -r "YOUR_KEYWORD" Test/SqlDom/TestScripts/ - -# Example: Check if RESUMABLE is tested for ALTER TABLE -grep -r "RESUMABLE" Test/SqlDom/TestScripts/*.sql - -# Search in specific version test files -grep -r "RESUMABLE" Test/SqlDom/TestScripts/*160.sql -``` - -### Method 2: Search Grammar Files -```bash -# Search in grammar files -grep -r "YOUR_KEYWORD" SqlScriptDom/Parser/TSql/*.g - -# Example: Check if RESUMABLE is in grammar -grep -r "Resumable" SqlScriptDom/Parser/TSql/TSql160.g -``` - -### Method 3: Search AST Definitions -```bash -# Search in AST XML -grep -r "YourFeatureName" SqlScriptDom/Parser/TSql/Ast.xml - -# Example: Check for VECTOR_SEARCH node -grep -r "VectorSearch" SqlScriptDom/Parser/TSql/Ast.xml -``` - -### Method 4: Try Parsing with Test Script -Create a unit test method to verify parsing: - -```csharp -// Add to appropriate test file (e.g., Test/SqlDom/Only170SyntaxTests.cs) -[TestMethod] -public void QuickTestExactSyntax() -{ - string script = @"ALTER TABLE t ADD CONSTRAINT pk PRIMARY KEY (id) WITH (RESUMABLE = ON);"; - - TSql170Parser parser = new TSql170Parser(true); - IList errors; - - using (StringReader reader = new StringReader(script)) - { - TSqlFragment fragment = parser.Parse(reader, out errors); - Console.WriteLine($"Errors: {errors.Count}"); - - // This will show you exactly which parser versions support the syntax - // and what error messages are generated if it fails - } - - Assert.Inconclusive($"Test completed with {errors.Count} errors"); -} -``` - -### Method 5: Test in Existing Test Framework -Add a temporary test method to verify quickly: - -```csharp -// Add to appropriate test file (e.g., Test/SqlDom/Only170SyntaxTests.cs) -[TestMethod] -public void TempTestExactScript() -{ - // Put ONLY your exact script here - do not create external files - string script = @"YOUR_EXACT_SCRIPT_HERE"; - - TSql170Parser parser = new TSql170Parser(true); - IList errors; - - using (StringReader reader = new StringReader(script)) - { - TSqlFragment fragment = parser.Parse(reader, out errors); - Console.WriteLine($"Parse result: {errors.Count} errors"); - foreach (var error in errors) - { - Console.WriteLine($"Error: {error.Message}"); - } - } - - Assert.Inconclusive("Temporary test - remove after verification"); -} -``` - -Then run the test: -```bash -dotnet test --filter "TempTestExactScript" -c Debug -``` - -Remember to remove this temporary test method after verification. - -## Step 3: Create a Test Script - -**CRITICAL**: Your test script MUST include the exact T-SQL statement provided. Don't modify, simplify, or generalize the syntax - test the precise statement given. - -### Test File Naming Convention -Follow the pattern from testing.guidelines.instructions.md: -- Format: `Tests.sql` -- Examples: `JsonFunctionTests160.sql`, `AlterTableResumableTests160.sql` -- Location: `Test/SqlDom/TestScripts/` -- Use version number corresponding to SQL Server version where feature was introduced - -### Test Script Requirements - -1. **Start with the exact script provided** - copy it exactly as given -2. **Add comprehensive coverage** as described in testing guidelines: - - Basic syntax variations - - Function in different contexts (SELECT, WHERE, RETURN statements) - - Edge cases (empty parameters, NULL handling, subqueries) - - Integration contexts (variables, parameters, computed expressions) -3. **Include context** - ensure the exact context (table aliases, subqueries) is tested -4. **Test RETURN statements** - Critical for functions, always test in ALTER FUNCTION RETURN statements - -### Test Script Template - -Follow the comprehensive coverage pattern from testing guidelines: - -```sql --- Test 1: EXACT SCRIPT PROVIDED (REQUIRED - COPY EXACTLY) --- PUT THE EXACT T-SQL STATEMENT HERE WITHOUT ANY MODIFICATIONS --- Example: SELECT JSON_OBJECTAGG( t.c1 : t.c2 ) FROM (VALUES('key1', 'c'), ('key2', 'b'), ('key3','a')) AS t(c1, c2); - --- Test 2: Basic function call (if applicable) -SELECT YOUR_FUNCTION('param1', 'param2'); - --- Test 3: Function in different contexts -SELECT col1, YOUR_FUNCTION('param') AS computed FROM table1; -WHERE YOUR_FUNCTION('param') > 0; - --- Test 4: CRITICAL - Function in RETURN statements (for functions) -ALTER FUNCTION TestYourFunction() -RETURNS NVARCHAR(MAX) -AS -BEGIN - RETURN (YOUR_FUNCTION('test_value')); -END; -GO - --- Test 5: With variables/parameters -SELECT YOUR_FUNCTION(@variable); -SELECT YOUR_FUNCTION(column_name); - --- Test 6: Edge cases -SELECT YOUR_FUNCTION(); -- Empty parameters (if valid) -SELECT YOUR_FUNCTION(NULL, 'test', 123); -- NULL handling -SELECT YOUR_FUNCTION((SELECT nested FROM table)); -- Subqueries -``` - -### Real-World Example: ALTER TABLE RESUMABLE - -**File**: `Test/SqlDom/TestScripts/AlterTableResumableTests160.sql` - -```sql --- Test 1: RESUMABLE with MAX_DURATION (minutes) -ALTER TABLE dbo.MyTable -ADD CONSTRAINT pk_test PRIMARY KEY CLUSTERED (id) -WITH (RESUMABLE = ON, MAX_DURATION = 240 MINUTES); - --- Test 2: RESUMABLE = ON -ALTER TABLE dbo.MyTable -ADD CONSTRAINT pk_test PRIMARY KEY CLUSTERED (id) -WITH (RESUMABLE = ON); - --- Test 3: RESUMABLE = OFF -ALTER TABLE dbo.MyTable -ADD CONSTRAINT pk_test PRIMARY KEY CLUSTERED (id) -WITH (RESUMABLE = OFF); - --- Test 4: UNIQUE constraint with RESUMABLE -ALTER TABLE dbo.MyTable -ADD CONSTRAINT uq_test UNIQUE NONCLUSTERED (name) -WITH (RESUMABLE = ON); -``` - -## Step 4: Configure Test Entry - -Add test configuration to the appropriate `OnlySyntaxTests.cs` file as described in testing guidelines. - -### Test Configuration File Location -- Format: `Only{Version}SyntaxTests.cs` -- Example: `Only160SyntaxTests.cs` (for SQL Server 2022) -- Location: `Test/SqlDom/` -- Add to the `Only{Version}TestInfos` array - -### Test Configuration Template - -Use the simplified approach from testing guidelines: - -```csharp -// Option 1: Simplified - only specify error counts you care about -new ParserTest{Version}("YourFeatureTests{Version}.sql"), // All previous versions default to null (ignored), current version expects 0 errors - -// Option 2: Specify only some previous version error counts -new ParserTest{Version}("YourFeatureTests{Version}.sql", nErrors80: 1, nErrors90: 1), // Only SQL 2000/2005 expect errors - -// Option 3: Full specification (legacy compatibility) -new ParserTest{Version}("YourFeatureTests{Version}.sql", - nErrors80: 1, // SQL Server 2000 - expect error for new syntax - nErrors90: 1, // SQL Server 2005 - expect error for new syntax - nErrors100: 1, // SQL Server 2008 - expect error for new syntax - nErrors110: 1, // SQL Server 2012 - expect error for new syntax - nErrors120: 1, // SQL Server 2014 - expect error for new syntax - nErrors130: 1, // SQL Server 2016 - expect error for new syntax - nErrors140: 1, // SQL Server 2017 - expect error for new syntax - nErrors150: 1 // SQL Server 2019 - expect error for new syntax - // nErrors{Version}: 0 is implicit for current version - expect success -), -``` - -### How to Determine Error Counts - -**Rule**: Count the number of SQL statements that will fail in each version. - -**Example**: If your test file has 4 statements with the new feature: -- Versions that DON'T support it: `nErrors = 4` (all 4 statements fail) -- Version that DOES support it: `nErrors = 0` (all 4 statements pass, implicit default) - -### Real-World Example: ALTER TABLE RESUMABLE - -**File**: `Test/SqlDom/Only160SyntaxTests.cs` - -```csharp -new ParserTest160("AlterTableResumableTests160.sql", - nErrors80: 4, // SQL Server 2000: RESUMABLE not supported (4 errors) - nErrors90: 4, // SQL Server 2005: RESUMABLE not supported (4 errors) - nErrors100: 4, // SQL Server 2008: RESUMABLE not supported (4 errors) - nErrors110: 4, // SQL Server 2012: RESUMABLE not supported (4 errors) - nErrors120: 4, // SQL Server 2014: RESUMABLE not supported (4 errors) - nErrors130: 4, // SQL Server 2016: RESUMABLE not supported (4 errors) - nErrors140: 4, // SQL Server 2017: RESUMABLE not supported (4 errors) - nErrors150: 4 // SQL Server 2019: RESUMABLE not supported (4 errors) - // nErrors160: 0 (implicit) - SQL Server 2022: RESUMABLE supported! (0 errors) -), -``` - -## Step 5: Create Baseline File - -Baseline files contain the expected formatted output after parsing and script generation. - -### Baseline File Location -- Format: `Baselines{Version}/YourTestFile{Version}.sql` -- Example: `Baselines160/AlterTableResumableTests160.sql` -- Location: `Test/SqlDom/` -- **Critical**: Baseline filename MUST exactly match the test script filename - -### Baseline Generation Process - -Follow the testing guidelines process: - -#### Initial Creation: -1. **Create empty or placeholder baseline file first** -2. **Run the test** (it will fail) -3. **Copy "Actual" output** from test failure message -4. **Paste into baseline file** with proper formatting - -```bash -# 1. Create placeholder baseline file -New-Item "Test/SqlDom/Baselines160/YourFeatureTests160.sql" -ItemType File - -# 2. Run the test (will fail initially) -dotnet test --filter "YourFeatureTests160" -c Debug - -# 3. Copy the "Actual" output from test failure into baseline file -# Look for the test failure message showing: -# Expected: -# Actual: - -# 4. Re-run the test (should pass now) -dotnet test --filter "YourFeatureTests160" -c Debug -``` - -#### Option B: Manual Creation - -Create the baseline file with properly formatted SQL: - -```sql --- Baseline follows ScriptDOM formatting rules: --- - Keywords in UPPERCASE --- - Proper indentation --- - Line breaks at appropriate places - -ALTER TABLE dbo.MyTable - ADD CONSTRAINT pk_test PRIMARY KEY CLUSTERED (id) WITH (RESUMABLE = ON, MAX_DURATION = 240 MINUTES); - -ALTER TABLE dbo.MyTable - ADD CONSTRAINT pk_test PRIMARY KEY CLUSTERED (id) WITH (RESUMABLE = ON); - -ALTER TABLE dbo.MyTable - ADD CONSTRAINT pk_test PRIMARY KEY CLUSTERED (id) WITH (RESUMABLE = OFF); - -ALTER TABLE dbo.MyTable - ADD CONSTRAINT uq_test UNIQUE NONCLUSTERED (name) WITH (RESUMABLE = ON); -``` - -### Real-World Example: ALTER TABLE RESUMABLE Baseline - -**File**: `Test/SqlDom/Baselines160/AlterTableResumableTests160.sql` - -```sql -ALTER TABLE dbo.MyTable - ADD CONSTRAINT pk_test PRIMARY KEY CLUSTERED (id) WITH (RESUMABLE = ON, MAX_DURATION = 240 MINUTES); - -ALTER TABLE dbo.MyTable - ADD CONSTRAINT pk_test PRIMARY KEY CLUSTERED (id) WITH (RESUMABLE = ON); - -ALTER TABLE dbo.MyTable - ADD CONSTRAINT pk_test PRIMARY KEY CLUSTERED (id) WITH (RESUMABLE = OFF); - -ALTER TABLE dbo.MyTable - ADD CONSTRAINT uq_test UNIQUE NONCLUSTERED (name) WITH (RESUMABLE = ON); -``` - -## Step 6: Run and Validate Test - -Follow the testing guidelines validation process. - -### Build the Parser -```bash -# Build ScriptDOM library -dotnet build SqlScriptDom/Microsoft.SqlServer.TransactSql.ScriptDom.csproj -c Debug - -# Build test project -dotnet build Test/SqlDom/UTSqlScriptDom.csproj -c Debug -``` - -### Run Your Specific Test -```bash -# Run specific test method -dotnet test Test/SqlDom/UTSqlScriptDom.csproj --filter "FullyQualifiedName~TSql160SyntaxIn160ParserTest" -c Debug - -# Run tests for specific version -dotnet test Test/SqlDom/UTSqlScriptDom.csproj --filter "TestCategory=TSql160" -c Debug - -# Run by test script name filter -dotnet test --filter "YourFeatureTests160" -c Debug - -# Run with verbose output to see details -dotnet test --filter "YourFeatureTests160" -c Debug -v detailed -``` - -### Run Full Test Suite (CRITICAL!) -```bash -# Always run ALL tests before committing -dotnet test Test/SqlDom/UTSqlScriptDom.csproj -c Debug - -# Expected output: -# Test summary: total: 1116, failed: 0, succeeded: 1116, skipped: 0 -``` - -### Interpret Results - -Follow the testing guidelines interpretation: - -- ✅ **Success**: Generated output matches baseline, error counts match expectations -- ❌ **Failure**: Review actual vs expected output, adjust baseline or fix grammar -- ⚠️ **Baseline Mismatch**: Copy correct "Actual" output to baseline file -- ⚠️ **Error Count Mismatch**: Adjust error expectations in test configuration - -### Common Test Results - -✅ **Success**: All tests pass, including your new test -``` -Test summary: total: 1116, failed: 0, succeeded: 1116, skipped: 0 -``` - -❌ **Baseline Mismatch**: Generated output doesn't match baseline -``` -Expected: -Actual: -``` -**Solution**: Copy the "Actual" output to your baseline file (note spacing differences) - -❌ **Error Count Mismatch**: Parse error count differs from expected -``` -TestYourFeature.sql: number of errors after parsing is different from expected. -Expected: 1, Actual: 0 -``` -**Solutions**: -- **If Actual < Expected**: Grammar now supports syntax in older versions → Update error counts -- **If Actual > Expected**: Grammar has issues → Fix grammar or adjust test - -❌ **Parse Errors**: Syntax not recognized -``` -SQL46010: Incorrect syntax near 'YOUR_TOKEN'. at offset 45, line 2, column 15 -``` -**Solutions**: Check grammar rules, verify syntactic predicates, see function guidelines for RETURN statement issues - -## Complete Example Workflow - -### Example: Testing ALTER TABLE RESUMABLE for SQL Server 2022 - -```bash -# Step 0: Test exact script first -echo "ALTER TABLE MyTable ADD CONSTRAINT pk PRIMARY KEY (id) WITH (RESUMABLE = ON);" > temp_test_script.sql -# Add debug test method to Only160SyntaxTests.cs and run to confirm current status - -# Step 1: Determine version -# Research shows: RESUMABLE for ALTER TABLE added in SQL Server 2022 → TSql160 - -# Step 2: Check if already supported -grep -r "RESUMABLE" Test/SqlDom/TestScripts/*.sql -# Result: Found in ALTER INDEX tests, but not ALTER TABLE tests - -# Step 3: Create test script -New-Item "Test/SqlDom/TestScripts/AlterTableResumableTests160.sql" -# Add 4 test cases covering different scenarios - -# Step 4: Add test configuration -# Edit Test/SqlDom/Only160SyntaxTests.cs -# Add: new ParserTest160("AlterTableResumableTests160.sql", nErrors80: 4, ...) - -# Step 5: Create empty baseline -New-Item "Test/SqlDom/Baselines160/AlterTableResumableTests160.sql" - -# Step 6: Build and run test -dotnet build SqlScriptDom/Microsoft.SqlServer.TransactSql.ScriptDom.csproj -c Debug -dotnet test --filter "AlterTableResumableTests160" -c Debug -# Test fails - copy "Actual" output into baseline file - -# Step 7: Re-run test -dotnet test --filter "AlterTableResumableTests160" -c Debug -# Test passes! - -# Step 8: Run full suite -dotnet test Test/SqlDom/UTSqlScriptDom.csproj -c Debug -# All 1120 tests pass! - -# Step 9: Commit changes -git add Test/SqlDom/TestScripts/AlterTableResumableTests160.sql -git add Test/SqlDom/Baselines160/AlterTableResumableTests160.sql -git add Test/SqlDom/Only160SyntaxTests.cs -git commit -m "Add tests for ALTER TABLE RESUMABLE option (SQL Server 2022)" -``` - -## Testing Best Practices - -### 1. Comprehensive Coverage -- ✅ **TEST EXACT SCRIPT PROVIDED** (most critical) -- ✅ Test basic syntax variations -- ✅ Test with multiple options -- ✅ Test different statement variations -- ✅ Test with parameters/variables (if applicable) -- ✅ Test edge cases -- ✅ Test error conditions (if relevant) -- ✅ Test complete context (subqueries, table aliases, etc.) - -### 2. Baseline Accuracy -- ✅ Generate baseline from actual parser output -- ✅ Don't hand-edit baseline formatting -- ✅ Verify baseline matches ScriptDOM formatting conventions -- ✅ Check for proper indentation and line breaks - -### 3. Version-Specific Testing -- ✅ Test only in the version where feature was introduced -- ✅ Verify older versions properly reject the syntax -- ✅ Document version dependencies clearly - -### 4. Regression Prevention -- ✅ Always run full test suite before committing -- ✅ Investigate any unexpected test failures -- ✅ Don't assume your change is isolated - -## Common Pitfalls - -### ❌ Wrong Version Number -**Problem**: Testing in TSql150 when feature is TSql160-only -**Solution**: Verify SQL Server version in Microsoft docs - -### ❌ Incorrect Error Counts -**Problem**: `nErrors80: 2` but test has 4 failing statements -**Solution**: Count all statements that use the new feature - -### ❌ Hand-Edited Baselines -**Problem**: Baseline formatting doesn't match ScriptDOM output -**Solution**: Always copy from actual parser output - -### ❌ Skipping Full Test Suite -**Problem**: Your change breaks existing tests -**Solution**: Run `dotnet test Test/SqlDom/UTSqlScriptDom.csproj -c Debug` - -### ❌ Missing Test Cases -**Problem**: Feature works for basic case but fails with parameters -**Solution**: Add comprehensive test coverage - -### ❌ Not Testing Exact Script -**Problem**: Testing simplified/modified versions instead of the exact script provided -**Solution**: Always include the exact T-SQL statement as provided, character-for-character - -## Troubleshooting - -### Test Fails: "Syntax error near..." -**Diagnosis**: Parser doesn't recognize the syntax -**Solution**: Grammar needs to be updated (see [Bug Fixing Guide](../instructions/bug_fixing.guidelines.instructions.md)) - -### Test Fails: "Option 'X' is not valid..." -**Diagnosis**: Validation logic rejects the syntax -**Solution**: See [Validation Fix Guide](../instructions/validation_fix.guidelines.instructions.md) - -### Test Fails: Baseline mismatch -**Diagnosis**: Generated output differs from baseline -**Solution**: Update baseline with actual output or fix generator - -### Full Suite Fails: Other tests break -**Diagnosis**: Your changes affected shared code -**Solution**: Review your changes, create context-specific rules - -## Quick Reference Commands - -```bash -# Step 0: Add debug unit test method first (NO external files) -# Add DebugExactScriptTest method to appropriate test file with exact script embedded - -# Search for syntax in tests -grep -r "KEYWORD" Test/SqlDom/TestScripts/ - -# Search in grammar -grep -r "KEYWORD" SqlScriptDom/Parser/TSql/*.g - -# Build parser -dotnet build SqlScriptDom/Microsoft.SqlServer.TransactSql.ScriptDom.csproj -c Debug - -# Run specific test -dotnet test --filter "TestName" -c Debug - -# Run full suite -dotnet test Test/SqlDom/UTSqlScriptDom.csproj -c Debug - -# Create test files (only for comprehensive testing, not initial verification) -New-Item "Test/SqlDom/TestScripts/MyTest160.sql" -New-Item "Test/SqlDom/Baselines160/MyTest160.sql" -``` - -## Related Guides - -- [debugging_workflow.guidelines.instructions.md](../instructions/debugging_workflow.guidelines.instructions.md) - How to diagnose issues -- [Validation_fix.guidelines.instructions.md](../instructions/validation_fix.guidelines.instructions.md) - Fix validation errors -- [Bug Fixing Guide](../instructions/bug_fixing.guidelines.instructions.md) - Add new grammar rules -- [copilot-instructions.md](../copilot-instructions.md) - Main project documentation +Input: +${input:tsqlScript:Paste the exact T-SQL script to verify} \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..aea8184 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,32 @@ +# AGENTS.md + +This repository keeps its canonical Copilot instruction files in the `.github` folder and its reusable agent skills in `.agents/skills`. + +Before starting work, read `.github/copilot-instructions.md` for repository-wide guidance. + +Then use the relevant topic-specific instruction files in `.github/instructions/`: +- `adding_new_parser.guidelines.instructions.md` +- `bug_fixing.guidelines.instructions.md` +- `database_option.guidelines.instructions.md` +- `debugging_workflow.guidelines.instructions.md` +- `function.guidelines.instructions.md` +- `grammar_validation.guidelines.instructions.md` +- `grammar.guidelines.instructions.md` +- `new_data_types.guidelines.instructions.md` +- `new_index_types.guidelines.instructions.md` +- `parser.guidelines.instructions.md` +- `testing.guidelines.instructions.md` + +When a reusable skill applies, load it from `.agents/skills/`: +- `add-feature/SKILL.md` +- `verify-and-test-tsql-syntax/SKILL.md` + +Version note: +- Treat `TSql180` as the current vNext/latest parser target in agent workflows unless a more specific repo instruction overrides it. + +Selection rule: +- Use only the instruction file or files that match the task. +- If multiple files apply, follow all of them. +- If there is any conflict, treat `.github/copilot-instructions.md` as the repository-wide baseline and then apply the more specific file from `.github/instructions/` for the current task. + +Do not duplicate, restate, or replace the maintained guidance in this file. This file is only a redirect to the instruction and skill sources above. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..7a18d6b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,7 @@ +@AGENTS.md + +## Claude Code + +- Treat `AGENTS.md` as the canonical shared instruction source for this repository. +- When a reusable workflow applies, consult the matching skill under `.agents/skills/`. +- Treat `TSql180` as the current vNext/latest parser target for shared agent workflows. \ No newline at end of file