Conversation
- `Statement()` lost its `<EOF>` validation in 063d244 ("feat: return any `UnsupportedStatement` content"), which replaced `[ <ST_SEMICOLON> ] <EOF>` with `( <ST_SEMICOLON> | <EOF> )`; ever since, input holding more than one statement silently returned a truncated AST instead of failing, so callers such as PageHelper's count SQL parser reuse the full parameter mappings against a partial statement - `ST_SEMICOLON` also matches blank line runs such as `"\n\n\n"`, which makes the silent truncation reachable from ordinary formatted SQL that has no semicolon at all - restore the pre-063d2442 contract with `( <ST_SEMICOLON> )* <EOF>` in both `Statement()` branches, keeping trailing separators and comments legal while rejecting any unconsumed statement - `Statements()` (which kept its trailing `<EOF>`) and `SingleStatement()` are untouched; walking a script statement by statement stays the incremental API's job - rewrite the routine boundary test that asserted the lenient behaviour to drive the parser with `SingleStatement()`, which is how `streamStatements` consumes multi statement scripts
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Statement()— the production behindCCJSqlParserUtil.parse(String),parse(Reader),parse(InputStream)andparseAST()— silently returns a truncated AST when the input holds more than one statement, instead of reporting a parse error.Since
ST_SEMICOLONmatches blank line runs as well as";":this is reachable from ordinary formatted SQL that contains no semicolon at all:
Downstream tools that parse a statement and then reuse the original metadata are broken by this. A concrete case from a production MyBatis stack: PageHelper's
DefaultCountSqlParsercallsnewParser(reader).Statement()to rewrite a paged query into its count query, andExecutorUtil.executeAutoCountkeeps the originalparameterMappings. A mapper whose dynamic SQL renders three consecutive newlines (for example an XML comment block that MyBatis strips, leaving the newlines behind) is truncated at that point, so the count statement ends up with fewer?placeholders than mappings —Parameter index out of range, or, when the dropped tail has no placeholders, a silently wrong row count.Root cause
Regression from
063d2442("feat: return anyUnsupportedStatementcontent", first shipped in jsqlparser-5.0), which replaced the terminator ofStatement()with a choice:Consuming a separator became an alternative to reaching
EOF, so the end-of-input validation was lost.Statements()kept its trailing<EOF>and was unaffected.Fix
src/main/jjtree/net/sf/jsqlparser/parser/JSqlParserCC.jjt, both branches ofStatement():Trailing separators stay legal (any number of them, so
"…;","…\n\n\n"and"… -- done"still parse), while any unconsumed statement is now reported. This restores the contract that 4.9 and earlier had.Compatibility
Statements()andSingleStatement()are untouched. Walking a script statement by statement remains the incremental API's job, which is whatCCJSqlParserUtil.streamStatementsalready uses.UnsupportedStatementsupport added by063d2442is preserved — that alternative is guarded by its ownLOOKAHEADand stays reachable.SingleStatement().Tests
CCJSqlParserUtilTest#testParseRejectsUnconsumedInput(new): rejects newline- and semicolon-separated input throughparse(String),parse(InputStream),parseASTand a directStatement()call; still accepts trailing separators and comments;parseStatementskeeps splitting correctly.SqlRoutineBodyBoundaryTest: the case that asserted the lenient behaviour (directStatementParserLeavesFollowingStatementAvailable) is rewritten to drive the parser withSingleStatement(). This preserves its original intent — verifying that a routine body ends where the following statement begins — and additionally asserts the separators in between.Tests run: 9093, Failures: 0, Errors: 0, Skipped: 25.