[for-the-juan]

Superseded at reviewer request by #1635 (VGGT) and #1636 (TRELLIS.2). Both replacements independently target main and have passed separate numerical, packaging, lint and single-card BF16 A/B checks. Closing this combined PR; the branch and historical results are retained.

Summary

Add native BF16 inference for VGGT-1B (recon) and TRELLIS.2-4B (i23d, 512 pipeline), using LightX2V's runner registry and registered attention/MM backends.

• VGGT: camera, depth, and point-map heads; ordered parallel CPU image preprocessing with an identical serial reference.
• TRELLIS.2: dense/sparse flow networks, Euler/CFG scheduling, input encoders, structure/shape/texture decoders, and geometry GLB export. Sorted integer neighbor lookup replaces the CPU hash reference without changing sparse-convolution semantics.
• Shared operators: BF16 linear/attention adapters, portable sparse operators, and a tiled deformable-convolution backend. BiRefNet's native MLU path upcasts matrix operands to FP32; the new backend retains FP32 coordinate interpolation but performs its projection GEMMs in BF16.
• Reproduction: pinned public model revisions, frozen input manifests with SHA256 checks, separate dependency pins, a vendor-runtime Docker recipe, and a paired A/B benchmark with operand-dtype auditing.

Model code has no platform branches, vendor imports, global operator patches, or dependency on an external VGGT/TRELLIS checkout. Platform selection remains in LightX2V's existing infrastructure; operator backends and lookup/preprocessing algorithms are configuration choices.

Validation

• 77 tests passed, including 27 new reconstruction tests. New tests also support CPU-only execution.
• All pre-commit hooks passed on the complete PR diff.
• Both Docker dependency recipes built from the vendor runtime; clean-environment model execution has been checked.
• Real MLU model runs pass strict BF16 matrix-operand auditing. FP32 normalization, coordinate interpolation, accumulation, and geometry are intentional.
• Complete frozen suites: VGGT 35 effective pairs; TRELLIS 39 effective pairs; zero failures. All returned tensors were exactly equal between baseline and optimized variants. Both warmup variants passed the BF16 operand audit.
• Final images were tested with networking disabled, no host source mounted, and two diagnostic requests per model. Both passed; all four TRELLIS diagnostic GLBs were byte-identical to their full-suite counterparts.
• GitHub's upstream lint workflow currently reports action_required with no jobs executed; repository maintainers need to handle that workflow approval/state. The equivalent local pre-commit run --all-files passed.

MLU590-M9D results

Single card per process, 80 GB, driver 6.5.42. Negative pipeline change means faster.

Model	Effective pairs	Pipeline p50 baseline / optimized (s)	Median paired pipeline change	Main-call statistic baseline / optimized (s)
VGGT	35	25.106818 / 8.343892	-66.812%	5.101998 / 5.102104
TRELLIS.2	39	39.752405 / 12.215323	-69.384%	0.212628 / 0.212445

VGGT's gain is in CPU image preprocessing; forward time is unchanged. TRELLIS benefits from sparse neighbor construction during decoding, outside its sampler calls. The TRELLIS main-call statistic reduces the 36 sampler calls within each request, then takes the median across requests; it is not whole-model time. Some segment_reduce operations fall back to CPU in this torch_mlu version, and that cost is included.

Measurement and scope

The benchmark follows the recorded A100 statistical protocol: 36 VGGT requests / 40 TRELLIS requests, discard the first pair, at least 31 successful effective pairs, alternating A/B order, synchronized pipeline wall time, and device events for VGGT forward / TRELLIS sample_once (36 calls per request). It saves raw timing records, output comparisons, configuration, environment, and BF16 audit results.

TRELLIS fixes an earlier experimental fallback's kernel-index ordering and previously missed BF16 decoder/deformable-convolution operands. Historical mesh quality and latency numbers from that fallback are not acceptance results for this integration. A/B output consistency and numerical operator references are checked here; a full ground-truth dataset quality evaluation has not been rerun.

CUDA/A100 and MetaX performance are unmeasured. VGGT tracking, TRELLIS cascade/1024/1536 pipelines, texture baking, offload, quantization, and distributed inference are outside this PR. GLB output contains geometry; texture voxel attributes are returned separately.

Weights, public test assets, and the vendor runtime/driver are external prerequisites with their own access and license conditions. No local experiment source or private runtime patch is required. The README documents the public GSO archive and its digest; downloading that whole archive has not been repeated during this validation.

See scripts/bench/reconstruction/README.md for installation, normal inference, exact benchmark boundaries, and reproduction commands.