[VL] Fix and enable native UDF cluster suite by kevinwilfong · Pull Request #13530 · apache/gluten

kevinwilfong · 2026-11-03T23:37:17Z

What changes are proposed in this pull request?

The VeloxUdfSuiteCluster test is currently broken in trunk and not enabled in CI.

Fix and enable VeloxUdfSuiteCluster in the dedicated UDF CI job.

Use the test JVM’s classpath for executors instead of the shaded Gluten package jar. This prevents driver/executor class mismatches that cause UDAF serialization failures and removes the gluten.package.jar requirement.
Construct absolute file URIs for driver UDF libraries so the default relative build paths work.
Enable by-name UDF registration, matching the local suite’s configuration.
Replace SkipTest with UDFTest and run the local and cluster suites in separate JVMs to isolate registration state.
Install required modules before the individual suite runs and trigger the UDF job when the suite or workflow changes

How was this patch tested?

Validated locally with rebuilt native libraries, Spark 3.5.5, Scala 2.12, and JDK 17: both suites passed all eight tests, with none skipped.

Verified the CI run on this PR.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: GPT-6.1 Sol

philo-he

Thanks for the fix!

Nit: the commented-out "Run UDF test" step in velox_backend_arm.yml still uses the old single command, which would now run both suites in one JVM if re-enabled. Could we directly remove it in this PR?

https://github.com/kevinwilfong/incubator-gluten/blob/f555bd65c6f9ff4f7ee4a943538e3a5bb6668ff8/.github/workflows/velox_backend_arm.yml#L231-L240

philo-he · 2026-11-09T00:39:00Z

-          -DargLine="-Dspark.test.home=/opt/shims/spark35/spark_home/"
+          $MVN_CMD install -Pspark-3.5 -Pbackends-velox -Pjava-17 -Piceberg -Pdelta -Ppaimon -pl backends-velox -am -DskipTests
+          for suite in VeloxUdfSuiteLocal VeloxUdfSuiteCluster; do
+            $MVN_CMD test -pl backends-velox -Pspark-3.5 -Pbackends-velox -Pjava-17 -Piceberg -Pdelta -Ppaimon -DtagsToExclude=org.apache.gluten.tags.EnhancedFeaturesTest \


Are -Piceberg -Pdelta -Ppaimon needed here? They only add lake dependencies to backends-velox, which VeloxUdfSuite doesn't use. If we drop them, drop them from the install and both test commands.

philo-he

Looks good. I will merge it if there is no other review comment from the community. Thanks!

kevinwilfong · 2026-11-10T20:52:42Z

Thanks for the review!

marin-ma · 2026-11-10T21:18:55Z

-      .set("spark.driver.extraClassPath", glutenJar)
-      .set("spark.executor.extraClassPath", glutenJar)
+      .set("spark.executor.extraClassPath", sys.props("java.class.path"))
+      .set(VeloxConfig.NATIVE_UDF_BYPASS_REGISTRATION.key, "true")


@kevinwilfong Sorry for the delayed review. Is it possible to test the non-bypass configuration as well?

np, will do!

@marin-ma updated

Run cluster executors with the test JVM classpath instead of the shaded package jar. This matches the already-running unshaded driver classes and fixes UDAF serialization failures. Remove gluten.package.jar and the driver classpath setting that cannot change the running JVM. Build absolute driver UDF library URIs so relative cpp.build.dir works. Select the cluster suite with UDFTest and run each suite in a separate JVM. Install required modules once before selecting suites in backends-velox. Set SPARK_SCALA_VERSION=2.12 so executor launch does not require Spark source build directories. Trigger UDF CI for suite, workflow and native changes. Remove unused lake profiles and the obsolete commented-out ARM UDF step. Run both suites with bypass registration disabled and enabled. Retain Hive UDF/UDAF coverage in both modes, verify native-only names are unavailable with bypass disabled, and run the by-name tests with bypass enabled. Remove the forced bypass configuration from the suite implementations. Validated locally November 10 against base d64a51d with matching rebuilt native libraries, Spark3.5.5, Scala2.12 and JDK17. Both suites pass 5 tests with bypass disabled and 8 with it enabled. Each invocation also passes 20 JUnit tests. No tests skipped. Formatter, license and workflow checks passed. Updated Linux GitHub Actions execution is pending. Originally authored with Claude Code; updated and validated with Codex. Generated-by: Codex CLI 0.164.0

kevinwilfong · 2026-11-11T01:28:10Z

Test failure is in GlutenTransformWithStateAvroSuite which I believe is flaky

philo-he · 2026-11-11T08:49:48Z

Test failure is in GlutenTransformWithStateAvroSuite which I believe is flaky

The failure didn't occur after a rerun.

marin-ma · 2026-11-11T09:14:33Z

-    checkAnswer(df, Seq(Row(2L), Row(3L), Row(4L)))
-  }
+  if (bypassRegistration) {
+    test("native udf with a plain name is callable without a hive udf class") {


It would be better to use a helper function testBypassRegistration for these tests rather than put them into a if.

marin-ma · 2026-11-11T09:15:37Z

+    }
+  } else {
+    test("native udf names are not registered when bypass registration is disabled") {
+      val queries = Seq(


And add if (!bypassRegistration) only in this test.

kevinwilfong marked this pull request as draft November 3, 2026 23:37

github-actions Bot added VELOX INFRA labels Nov 3, 2026

kevinwilfong force-pushed the fix-velox-udf-cluster-classpath branch from b918733 to f555bd6 Compare November 4, 2026 00:07

kevinwilfong requested review from FelixYBW and marin-ma November 4, 2026 01:10

kevinwilfong marked this pull request as ready for review November 4, 2026 01:11

kevinwilfong requested a review from zhouyuan November 6, 2026 18:41

philo-he reviewed Nov 9, 2026

View reviewed changes

kevinwilfong force-pushed the fix-velox-udf-cluster-classpath branch from f555bd6 to f72939c Compare November 10, 2026 18:14

philo-he approved these changes Nov 10, 2026

View reviewed changes

marin-ma reviewed Nov 10, 2026

View reviewed changes

kevinwilfong force-pushed the fix-velox-udf-cluster-classpath branch from f72939c to 26510af Compare November 10, 2026 22:21

marin-ma reviewed Nov 11, 2026

View reviewed changes

Navigation Menu

Repository navigation

[VL] Fix and enable native UDF cluster suite - #13530

[VL] Fix and enable native UDF cluster suite#13530
kevinwilfong wants to merge 1 commit into
apache:mainfrom
kevinwilfong:fix-velox-udf-cluster-classpath

kevinwilfong commented Nov 3, 2026 •

edited

Loading

Uh oh!

philo-he left a comment

Uh oh!

philo-he Nov 9, 2026

Uh oh!

philo-he left a comment

Uh oh!

kevinwilfong commented Nov 10, 2026

Uh oh!

marin-ma Nov 10, 2026

Uh oh!

kevinwilfong Nov 10, 2026

Uh oh!

kevinwilfong Nov 11, 2026

Uh oh!

kevinwilfong commented Nov 11, 2026

Uh oh!

philo-he commented Nov 11, 2026

Uh oh!

marin-ma Nov 11, 2026

Uh oh!

marin-ma Nov 11, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

kevinwilfong commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes are proposed in this pull request?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

Uh oh!

philo-he left a comment

Choose a reason for hiding this comment

Uh oh!

philo-he Nov 9, 2026

Choose a reason for hiding this comment

Uh oh!

philo-he left a comment

Choose a reason for hiding this comment

Uh oh!

kevinwilfong commented Nov 10, 2026

Uh oh!

marin-ma Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

kevinwilfong Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

kevinwilfong Nov 11, 2026

Choose a reason for hiding this comment

Uh oh!

kevinwilfong commented Nov 11, 2026

Uh oh!

philo-he commented Nov 11, 2026

Uh oh!

marin-ma Nov 11, 2026

Choose a reason for hiding this comment

Uh oh!

marin-ma Nov 11, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kevinwilfong commented Nov 3, 2026 •

edited

Loading