Skip to content

[GLUTEN-13476][VL] Fall back regex patterns with character class intersection - #13477

Merged
philo-he merged 1 commit into
apache:mainfrom
LuciferYang:gcfix-regex-intersection
Nov 12, 2026
Merged

philo-he merged 1 commit into
apache:mainfrom
LuciferYang:gcfix-regex-intersection

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

Java's java.util.regex reads && inside a character class as set intersection, while RE2 reads it as two literal & characters. ensureRegexIsCompatible already rejects nested classes such as [a[b]] and [a&&[b]], but the bare form [a&&b] passes, so the native function returns a different result than vanilla Spark. For example, 'a' RLIKE '[a&&d]' is true in Gluten and false in vanilla Spark.

This rejects && inside a character class, so the expression falls back. && outside a class and an escaped \& are still accepted, since both engines read them the same way.

It also fixes the class tracking for a ] right after [^. Java and RE2 both read that ] as a class member, but the check ended the class there, so [^]&&x] and [^][a]] passed it. Now ] right after [ or [^ keeps the class open.

The Bolt copy of ensureRegexIsCompatible gets the same change.

Fixes #13476.

How was this patch tested?

Added regex with character class intersection falls back to VeloxStringFunctionsSuite. It runs rlike with [a&&d], regexp_extract with [a-z&&l-p] and regexp_replace with [^]&&x], compares each result with vanilla Spark and checks that the fallback reason mentions the character class. It also checks that [a&d] with a single & still runs natively. Without this change the test fails on [a&&d]. With only the && check and not the [^] fix it fails on [^]&&x].

The Bolt change is not compiled or tested by CI.

Was this patch authored or co-authored using generative AI tooling?

Yes. Generated-by: Claude Opus.

Copilot AI balanced review requested due to automatic review settings October 27, 2026 15:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@LuciferYang

Copy link
Copy Markdown
Contributor Author

@philo-he Could you take a look? It's in the re2 regex validation.

@philo-he philo-he left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks!

@philo-he
philo-he merged commit b8ffd91 into apache:main Nov 12, 2026
72 checks passed
@LuciferYang

Copy link
Copy Markdown
Contributor Author

Thank you @philo-he

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[VL] rlike and regexp functions return RE2 results for character class intersections like [a&&d]

3 participants