Merge next's new commits to main

AGENTS.md

@@ -1,171 +1,65 @@
-# AGENTS.md
+# Agent Guidelines
 
-## 项目概述
+## Repository
 
-**dt-sql-parser** 是一个基于 [ANTLR4](https://github.com/antlr/antlr4) 构建的 SQL 解析器库，面向 **大数据** 领域。它提供以下功能：
+This is a TypeScript SQL parser built with ANTLR4/antlr4ng and antlr4-c3. It supports Flink, Generic, Hive, Impala, MySQL, PostgreSQL, Spark, and Trino SQL.
 
-- SQL 语法校验
-- AST 遍历（Visitor / Listener 模式）
-- 代码补全（基于 [antlr4-c3](https://github.com/mike-lischke/antlr4-c3)）
-- 实体提取（表、列等）
-- SQL 语句拆分
+- `src/grammar/<dialect>`: hand-maintained ANTLR grammars
+- `src/lib/<dialect>`: generated Lexer, Parser, Listener, and Visitor files
+- `src/parser/<dialect>`: dialect-specific parser implementations
+- `src/parser/common`: shared parser base classes and utilities
+- `test/parser/<dialect>`: syntax, completion, splitting, and context-collection tests
+- `benchmark`: performance benchmarks
 
-**支持的 SQL 方言**：MySQL、Flink、Spark、Hive、PostgreSQL、Trino、Impala。
+## Working Rules
 
-## 技术栈
+- Understand the relevant implementation, callers, and tests before editing
+- Make the smallest complete change; avoid unrelated refactors and repository-wide formatting
+- Analysis or review requests are read-only unless the user explicitly asks for changes
+- Preserve existing uncommitted work and do not modify unrelated files
+- Do not upgrade dependencies, change the lockfile, or adjust build tooling unless required
+- Do not bypass type errors with `any`, `@ts-ignore`, or unnecessary type assertions
+- Add concise Chinese comments only when a new method or complex conditional needs rationale; omit trailing punctuation
+- Treat changes under `src/parser/common` as cross-dialect changes and inspect affected callers
+- Do not trade correctness across dialects for completion or parsing performance
 
-| 类别           | 说明                                                  |
-| -------------- | ----------------------------------------------------- |
-| 语言           | TypeScript（目标 ES6，模块 ESNext）                   |
-| 运行环境       | Node.js >= 18                                         |
-| 包管理器       | pnpm 9.7.0                                            |
-| 构建工具       | `tsc`（TypeScript 编译器）                            |
-| 测试框架       | Jest（配合 `@swc/jest` 转换器）                       |
-| 解析器生成     | ANTLR4，通过 `antlr4ng` + `antlr4ng-cli`              |
-| 代码补全       | `antlr4-c3`                                           |
-| 代码格式化     | Prettier、`antlr-format-cli`（用于 `.g4` 文件）       |
-| Git Hooks      | Husky + lint-staged + commitlint                      |
+## Environment and Commands
 
-## 仓库结构
-
-```
-dt-sql-parser/
-├── src/
-│   ├── grammar/          # ANTLR4 .g4 语法文件（每个方言一个子目录）
-│   ├── lib/              # 从 .g4 文件生成的 Lexer/Parser/Listener/Visitor
-│   ├── parser/           # SQL Parser 类实现
-│   │   ├── common/       # 基类（BasicSQL）、工具方法、共享类型
-│   │   ├── mysql/        # MySQL 专属解析器、实体收集器等
-│   │   ├── flink/
-│   │   ├── spark/
-│   │   ├── hive/
-│   │   ├── postgresql/
-│   │   ├── trino/
-│   │   └── impala/
-│   ├── locale/           # 国际化资源
-│   └── index.ts          # 公共 API 导出
-├── test/                 # 单元测试（结构与 src/ 对应）
-│   ├── parser/           # 按方言组织的测试
-│   │   ├── mysql/
-│   │   │   ├── syntax/   # 语法规则测试
-│   │   │   ├── suggestion/ # 代码补全测试
-│   │   │   └── contextCollect/ # 实体收集测试
-│   │   └── ...
-│   └── common/           # 共享测试工具
-├── benchmark/            # 性能基准测试
-├── scripts/              # 构建/发布实用脚本
-├── gen/                  # 生成产物
-├── dist/                 # 编译输出（npm 包）
-├── .husky/               # Git Hook 配置
-├── package.json
-├── tsconfig.json
-├── jest.config.js
-└── CONTRIBUTING.md       # 贡献指南（新增方言步骤）
-```
-
-## 核心开发命令
+- Use Node.js 18 or later, `pnpm`, and Java when generating ANTLR files
+- Use `pnpm install --frozen-lockfile` for normal installs; use plain `pnpm install` only for intentional dependency changes
 
 ```bash
-pnpm install              # 安装依赖
-pnpm antlr4               # 从所有 .g4 文件生成 TS
-pnpm antlr4 --lang mysql  # 为指定方言生成
-pnpm build                # 编译 TypeScript（rm -rf dist && tsc）
-pnpm test                 # 运行 Jest 单元测试
-pnpm benchmark            # 运行性能基准测试
-pnpm check-types          # 对 src/ 和 test/ 进行类型检查
-pnpm format               # 用 Prettier 格式化所有文件
-pnpm format-g4            # 格式化 .g4 语法文件
-pnpm prettier-check       # 检查格式是否符合要求（不修改）
-```
-
-## 架构说明
-
-### Parser 类层级
-
-```
-BasicSQL (src/parser/common/)
-  ├── MySQL
-  ├── FlinkSQL
-  ├── SparkSQL
-  ├── HiveSQL
-  ├── PostgreSQL
-  ├── TrinoSQL
-  └── ImpalaSQL
+pnpm install --frozen-lockfile
+pnpm antlr4 --lang <dialect>
+pnpm test -- <test-file-or-pattern>
+pnpm check-types
+pnpm prettier-check
+pnpm benchmark --lang <dialect>
 ```
 
-每个方言的 Parser 类（例如 `src/parser/mysql/index.ts` 中的 `MySQL`）继承自 `BasicSQL`，并实现以下方法：
-
-- `createLexerFromCharStream()` — 创建 ANTLR4 Lexer
-- `createParserFromTokenStream()` — 创建 ANTLR4 Parser
-- `splitListener` getter — 返回用于语句拆分的 `SQLSplitListener`
-- `createEntityCollector()` — 返回用于上下文/实体提取的 `SQLEntityCollector`
-- `processCandidates()` / `preferredRules()` — 代码补全逻辑（antlr4-c3）
-
-### 每方言模块结构
-
-每个 `src/parser/<dialect>/` 目录包含：
-
-| 文件                                        | 用途                                          |
-| ------------------------------------------- | --------------------------------------------- |
-| `index.ts`                                  | 主解析器类，继承 `BasicSQL`                    |
-| `<dialect>EntityCollector.ts`               | 从 AST 中提取表、列、函数等实体                |
-| `<dialect>SplitListener.ts`                 | 通过分号/AST 拆分多语句 SQL                    |
-| `<dialect>ErrorListener.ts`                 | 自定义语法错误处理                             |
-| `<dialect>SemanticContextCollector.ts`      | 收集语义上下文（如 `isStatementBeginning`）    |
-
-### 语法文件 → 代码生成流程
-
-1. `.g4` 文件存放在 `src/grammar/<dialect>/` 目录
-2. 执行 `pnpm antlr4 [--lang <dialect>]` 生成：
-   - `src/lib/<dialect>/<Dialect>Lexer.ts`
-   - `src/lib/<dialect>/<Dialect>Parser.ts`
-   - `src/lib/<dialect>/<Dialect>ParserListener.ts`
-   - `src/lib/<dialect>/<Dialect>ParserVisitor.ts`
-3. `src/parser/<dialect>/` 中的 Parser 类消费这些生成的文件
-
-### 语法规则约定
-
-- 根规则必须命名为 `program`
-- 支持解析多条 SQL 语句
-- 关键字规则前缀为 `KW_`（例如 `KW_SELECT`）
-- 不区分大小写的方言启用 case-insensitive 选项
-
-## 公共 API（来自 `src/index.ts`）
-
-**类**：`MySQL`、`FlinkSQL`、`SparkSQL`、`HiveSQL`、`PostgreSQL`、`TrinoSQL`、`ImpalaSQL`
-
-**Listener/Visitor 类型**：`MySqlParserListener`、`MySqlParserVisitor` 等（每个方言一对）
-
-**枚举**：`EntityContextType`、`StmtContextType`
-
-**类型**：`CaretPosition`、`Suggestions`、`SyntaxSuggestion`、`WordRange`、`TextSlice`、`SyntaxError`、`ParseError`、`ErrorListener`、`StmtContext`、`EntityContext`、`CommonEntityContext`、`ColumnEntityContext`、`FuncEntityContext`
-
-## 测试规范
+## Grammar and Generated Code
 
-- 测试文件结构与 `src/` 对应，位于 `test/parser/<dialect>/`
-- 子目录：`syntax/`（语法）、`suggestion/`（补全）、`contextCollect/`（实体收集）
-- 使用 Jest 配合 `@swc/jest` 实现快速编译
-- 自定义匹配器定义在 `test/matchers.ts`
-- 运行命令：`pnpm test`
+- Never edit files under `src/lib` manually
+- After changing `src/grammar/<dialect>`, run `pnpm antlr4 --lang <dialect>` and regenerate only that dialect
+- The generator may log an error without a nonzero exit code. Require `Compile <dialect> succeeded!`, then inspect the generated files and diff
+- Prefer official SQL documentation or official grammar implementations
+- Keep the `KW_` prefix for keyword lexer rules
+- Add or update the corresponding dialect tests for grammar behavior changes
 
-## 新增 SQL 方言（步骤概要）
+## Verification
 
-1. 在 `src/grammar/<name>/` 下添加 `.g4` 文件（PascalCase 命名，根规则 = `program`，关键字 = `KW_*`）
-2. 执行 `pnpm antlr4 --lang <name>` → 生成 `src/lib/<name>/`
-3. 创建 `src/parser/<name>/index.ts` 继承 `BasicSQL`
-4. 在 `test/parser/<name>/` 下添加测试（lexer、visitor、listener、validate）
-5. 实现 `SQLSplitListener` → 添加 `splitListener` getter
-6. 实现代码补全 → `processCandidates` + `preferredRules`，并在 `suggestion/` 下添加测试
-7. 实现 `SQLEntityCollector` + `createEntityCollector()`，并在 `contextCollect/` 下添加测试
-8. 在 `src/parser/index.ts` 和 `src/index.ts` 中导出新类
+- Add a focused regression test for parser behavior changes
+- Run directly related tests for parser or utility changes
+- For shared parser or completion changes, add representative cross-dialect verification
+- Run `pnpm check-types` for TypeScript changes and `pnpm prettier-check` for formatting
+- Establish correctness before running performance benchmarks
+- `pnpm benchmark` without `--lang` is interactive; use the command above for non-interactive cold-start benchmarks
+- Do not run `pnpm benchmark:release` by default. It requires Node.js 21 or later, and its wrapper may reject the version without a nonzero exit code, so confirm that the benchmark process starts
+- Report timeouts or OOM failures with the exact command and scope; reduced inputs or increased heap limits do not prove the original command passes
 
-## AI Agent 注意事项
+## Git and Delivery
 
-- 修改 `.g4` 文件后 **必须** 执行 `pnpm antlr4`，确保 `src/lib/` 中的生成文件保持同步
-- **不要** 手动编辑 `src/lib/` 中的文件 —— 它们是由 `.g4` 自动生成的
-- 语法文件遵循 ANTLR4 约定；关键字规则必须带有 `KW_` 前缀
-- 项目使用 `antlr4ng`（非 Java antlr4 运行时）作为 TypeScript 目标
-- 代码补全依赖 `antlr4-c3`；修改补全逻辑前先了解该库
-- 实体收集器（`SQLEntityCollector`）是实现丰富代码补全的关键 —— 需理解作用域深度和 `isAccessible` 逻辑
-- 位置/范围约定：行号从 1 开始，列号从 1 开始，索引从 0 开始
-- Prettier 格式化通过 husky + lint-staged 在提交时强制执行
+- Unless explicitly requested, do not stage, commit, push, reset, restore, rebase, or create branches
+- Never use destructive commands such as `git reset --hard`
+- Use the repository's Conventional Commit types when asked to create a commit
+- Before completion, inspect the final diff and report the commands actually run, their results, and any verification limits

README-zh_CN.md

@@ -222,7 +222,7 @@ console.log(sqlSlices)
     startLine: 1,
     endLine: 1,
     startColumn: 1,
-    endColumn: 12,
+    endColumn: 13,
     text: 'SHOW TABLES;'
   },
   {
@@ -231,7 +231,7 @@ console.log(sqlSlices)
     startLine: 2,
     endLine: 2,
     startColumn: 1,
-    endColumn: 17,
+    endColumn: 18,
     text: 'SELECT * FROM tb;'
   }
 ]
@@ -250,7 +250,7 @@ console.log(sqlSlices)
 
     const flink = new FlinkSQL();
     const sql = 'CREATE ';
-    const pos = { lineNumber: 1, column: 16 }; // 最后一个位置
+    const pos = { lineNumber: 1, column: sql.length + 1 }; // 最后一个位置
     const keywords = flink.getSuggestionAtCaretPosition(sql, pos)?.keywords;
 
     console.log(keywords);
@@ -297,10 +297,10 @@ console.log(sqlSlices)
           {
             text: 'tb',
             startIndex: 14,
-            stopIndex: 15,
+            endIndex: 15,
             line: 1,
             startColumn: 15,
-            stopColumn: 16
+            endColumn: 17
           }
         ]
       },
@@ -310,10 +310,10 @@ console.log(sqlSlices)
           {
             text: 'tb',
             startIndex: 14,
-            stopIndex: 15,
+            endIndex: 15,
             line: 1,
             startColumn: 15,
-            stopColumn: 16
+            endColumn: 17
           }
         ]
       }

README.md

@@ -225,7 +225,7 @@ console.log(sqlSlices)
     startLine: 1,
     endLine: 1,
     startColumn: 1,
-    endColumn: 12,
+    endColumn: 13,
     text: 'SHOW TABLES;'
   },
   {
@@ -234,7 +234,7 @@ console.log(sqlSlices)
     startLine: 2,
     endLine: 2,
     startColumn: 1,
-    endColumn: 17,
+    endColumn: 18,
     text: 'SELECT * FROM tb;'
   }
 ]
@@ -244,15 +244,15 @@ console.log(sqlSlices)
 ### Code Completion
 Obtaining code completion information at a specified position in SQL.
 
-Call the `getAllEntities` method on the SQL instance, pass the SQL content and the row and column numbers indicating the position where code completion is desired. The following are some additional explanations about [CaretPosition](#caretposition-of-code-completion).
+Call the `getSuggestionAtCaretPosition` method on the SQL instance, pass the SQL content and the row and column numbers indicating the position where code completion is desired. The following are some additional explanations about [CaretPosition](#caretposition-of-code-completion).
 + **keyword candidates list**
 
     ```javascript
     import { FlinkSQL } from 'dt-sql-parser';
 
     const flink = new FlinkSQL();
     const sql = 'CREATE ';
-    const pos = { lineNumber: 1, column: 16 }; // the end position
+    const pos = { lineNumber: 1, column: sql.length + 1 }; // the end position
     const keywords = flink.getSuggestionAtCaretPosition(sql, pos)?.keywords;
 
     console.log(keywords);
@@ -296,10 +296,10 @@ Call the `getAllEntities` method on the SQL instance, pass the SQL content and t
           {
             text: 'tb',
             startIndex: 14,
-            stopIndex: 15,
+            endIndex: 15,
             line: 1,
             startColumn: 15,
-            stopColumn: 16
+            endColumn: 17
           }
         ]
       },
@@ -309,10 +309,10 @@ Call the `getAllEntities` method on the SQL instance, pass the SQL content and t
           {
             text: 'tb',
             startIndex: 14,
-            stopIndex: 15,
+            endIndex: 15,
             line: 1,
             startColumn: 15,
-            stopColumn: 16
+            endColumn: 17
           }
         ]
       }

benchmark_reports/cold_start/flink.benchmark.md

@@ -4,34 +4,34 @@
 FlinkSQL
 
 ### Report Time
-2024/12/18 14:50:08
+2026/9/24 10:22:50
 
 ### Device
-macOS 15.0.1
-(8) arm64 Apple M1 Pro
+macOS 15.7.9
+(8) arm64 Apple M3
 16.00 GB
 
 ### Version
-`nodejs`: v21.6.1
-`dt-sql-parser`: v4.1.0-beta.0
+`nodejs`: v22.23.1
+`dt-sql-parser`: v4.5.1
 `antlr4-c3`: v3.3.7
 `antlr4ng`: v2.0.11
 
 ### Running Mode
 Cold Start
 
 ### Report
-|  Benchmark Name |           Method Name           |SQL Rows|Average Time(ms)| 
-|-----------------|---------------------------------|--------|----------------| 
-| Query Collection|           getAllTokens          |  1015  |       257      | 
-| Query Collection|             validate            |  1015  |       277      | 
-|  Insert Columns |           getAllTokens          |  1001  |       66       | 
-|  Insert Columns |             validate            |  1001  |       67       | 
-|   Create Table  |           getAllTokens          |  1004  |       27       | 
-|   Create Table  |             validate            |  1004  |       28       | 
-|    Split SQL    |       splitSQLByStatement       |   999  |       53       | 
-| Collect Entities|          getAllEntities         |  1056  |       191      | 
-|    Suggestion   |   getSuggestionAtCaretPosition  |  1056  |       185      | 
-|Collect Semantics|getSemanticContextAtCaretPosition|  1015  |       247      | 
+|  Benchmark Name |           Method Name           |SQL Rows|Average Time(ms)| 
+|-----------------|---------------------------------|--------|----------------| 
+| Query Collection|           getAllTokens          |  1015  |       282      | 
+| Query Collection|             validate            |  1015  |       282      | 
+|  Insert Columns |           getAllTokens          |  1001  |       53       | 
+|  Insert Columns |             validate            |  1001  |       58       | 
+|   Create Table  |           getAllTokens          |  1004  |       16       | 
+|   Create Table  |             validate            |  1004  |       17       | 
+|    Split SQL    |       splitSQLByStatement       |   999  |       40       | 
+| Collect Entities|          getAllEntities         |  1056  |       185      | 
+|    Suggestion   |   getSuggestionAtCaretPosition  |  1056  |       226      | 
+|Collect Semantics|getSemanticContextAtCaretPosition|  1015  |       309      |