[felipepessoto]

What changes are proposed in this pull request?

Fixes #13496.

Add mandatory runtime validation to Delta Spark UT so a missing or disabled Gluten configuration cannot silently produce a green baseline run.

• Check the actual Spark context's plugin configuration, installed Gluten SQL rules, and session/thread enablement at test boundaries. Cover shared Spark, Hive, the newer session-test fixture, Java/JUnit, and the known direct factories. Java rules are transient so serializable test suites remain usable in Spark closures.
• Inspect compiled suite inheritance independently of the hooks, and require matching per-test evidence. Run a real native aggregation in every fork, with process-specific ScalaTest report IDs.
• Capture lossless framework result identities through an sbt listener before JUnit XML normalizes whitespace or rewrites suite-name prefixes. Match exact identities to activation evidence; cross-check JUnit suite presence and test counts. Missing, truncated, duplicated, or corrupt execution records and listener errors fail closed.
• Enforce activation checks before baseline/quarantine/seed handling. Missing, malformed, or failed evidence is fatal; canaries alone cannot make an empty upstream run pass.
• Support explicit, reasoned exemptions and restored scoped disabling. Reject ambiguous exemptions that collapse distinct raw names. Upload evidence and report uncovered suites without assuming they are Spark-free.

This is CI-only: no production changes and no Java agent. Fixture/factory checks do not claim coverage of every additional session created through unwrapped paths inside a test; that limitation is documented. Failure baselines and the exemption list are unchanged.

How was this patch tested?

• All 37 Python installer/gate unit tests pass, including lossless whitespace/prefix matching, missing or corrupt listener output, exact duplicates, JUnit count mismatches, framework-specific Java method prefixes, ambiguous exemptions, generated line lengths, and ASF header preservation.
• Applied the installer to Delta 4.4.1 and reran it idempotently: 194 shared-fixture import aliases, six Java suites, six direct-factory suites, and the Hive/session-test hooks.
• Delta's pinned Scalastyle 1.0.0 plugin and configuration pass across all 489 test Scala sources with zero errors and zero warnings. Only Delta's copyright-specific header rule is exempted in copied ASF sources; their original license headers and all other style checks remain intact.
• Compiled the Scala runtime helpers and sbt meta-build helper in a focused real Spark 4.1.0 / Delta 4.4.1 harness under JDK 17, using the existing Gluten bundle from CI run https://github.com/apache/gluten/actions/runs/40756222035.
• The final two-fork native run passed 21 ScalaTest tests plus one Java/JUnit test. Coverage includes five guard unit tests, nine raw-name regressions (space, newline, carriage return, tab, Unicode line separator, suite prefixes, and leading/trailing whitespace), scoped session/thread disabling, stopped-session cleanup, Java/JUnit lifecycle and serialization, and an explicitly exempt vanilla suite. The independent gate confirmed two native-validated JVMs, 14 checked tests, and one exemption.
• Replayed all eight shards from https://github.com/apache/gluten/actions/runs/40981748904 against the new gate; all passed. For this replay, raw identities were reconstructed from saved XML lexical attributes independently of validation evidence. The old run did not contain the new listener; its actual output was exercised separately by the fresh native harness above.
• Tampering with the fresh native evidence independently verified rejection of a missing newline-variant check, an exact duplicate check, missing native proof, and missing execution-listener files.
• Negative controls removing GlutenPlugin or setting spark.gluten.enabled=false both failed the JVM run and independently failed activation validation with exit code 2.
• Python formatting, changed-file Scala formatting, license headers, YAML parsing, Bash syntax, and whitespace checks pass.

Draft / CI follow-up: the complete Delta matrix has not been rerun locally; the fresh native harness and recorded-run replay are not a replacement for the new CI run. The full repository formatter encounters existing Windows symlink stubs; changed Scala files were checked successfully using the repository formatter and Maven wrapper with a scoped file selector.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: GitHub Copilot (gpt-6-astra).

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[felipepessoto]

Closing in favor of #13504, which now tracks a consolidated native-execution evidence and fallback-coverage regression gate. This is supersession, not a merge or a claim that validation is implemented. Keeping the branch and implementation available for reuse, especially the evidence recording and missing-evidence enforcement.