Unleashing 2K Multi-View Diffusion for 3D Texturing
High-quality texture generation is essential for creating realistic and production-ready 3D assets. Recent multi-view diffusion methods have shown promising results for image-guided 3D texturing, but they are typically constrained to low operating resolutions such as 512 or 768, making it difficult to preserve high-frequency details from high-resolution reference images. Scaling this paradigm to 2048 resolution is computationally prohibitive, as the unified multi-view sequence exceeds 212K tokens and incurs excessive memory and latency.
In this paper, we present UltraTex, an efficient end-to-end framework for high-resolution multi-view diffusion-based 3D texturing. Our key observation is that object-centric multi-view renderings contain two major sources of redundancy: background-induced sequence redundancy and sparse token interactions within the foreground. To address them, we introduce Background Token Dropping, which removes background tokens before the DiT backbone, and Block-Sparse Attention, which reduces attention computation over the retained foreground sequence. To enable efficient foreground-only inference while avoiding reconstruction artifacts, we further design Foreground-Aware VAE Decoding to ensure the quality of the final high-resolution views.
To satisfy the demanding data requirements of 2K-resolution multi-view diffusion training, we construct G-buffer TexVerse, a large-scale, ultra-high-resolution multi-view rendering dataset covering 351,847 BSDF assets (UltraTex training uses a 268,365-asset subset). Extensive experiments show that UltraTex generates visually faithful textures with rich fine-grained details, while substantially improving efficiency, achieving 20.6×–91.1× training speedup and 22.3×–74.6× end-to-end inference speedup over the baseline on common samples in our dataset.
2048 six-view in-context diffusion is a 212,992-token sequence. The bottleneck is not “attention is expensive” in the abstract — it is two object-centric redundancies, plus a decoding trap that appears only if you keep the speedup at test time.
The sparsity is intrinsic to object-centric multi-view rendering, not a poorly chosen crop. Objects sit in a fixed six-view layout, but their silhouettes are irregular and do not fill the enclosing box: chair legs and character limbs leave large empty regions that cropping cannot remove. A vanilla DiT still tokenizes that background and runs every block on it. The non-obvious fact is that the 1D sequence does not have to stay contiguous. A FLUX probe that injects RoPE into noise and then deletes background tokens still denoises a coherent layout — the transformer reads absolute coordinates from RoPE, not from token adjacency. So we drop background tokens before the MM-DiT, keep each survivor’s original RoPE index (and with it the multi-view geometric prior), and spend the entire budget on surface texels.
Shortening the sequence does not finish the job. At 2048, dense attention among the remaining foreground tokens is still a large fraction of step time. Attention maps of a BTD-trained full-attention model are already sparse in both double-stream and single-stream blocks: most foreground–foreground interactions are irrelevant to local texture synthesis. Block-sparse attention with top-k selection on mean-pooled blocks therefore approximates the interactions that actually fire, instead of paying O(N2) on a sequence that BTD already compressed.
Test-time speedup uses the same foreground-only Euler update, so background latents stay at the initial Gaussian. A stock VAE decoder, fed a clean foreground glued to pure noise, leaks that noise across the object and destroys the 2K detail BTD just paid for. The fix is not to denoise the background (that would throw the speedup away). We swap in a canonical in-distribution background latent and lightly fine-tune the decoder with a foreground-restricted objective so it can read the composite without corrupting the surface.
UltraTex produces geometry-aligned 2K multi-view textures that preserve fine-grained appearance detail from the reference image.
Background Token Dropping shortens the sequence; Block-Sparse Attention sparsifies what remains. The two mechanisms compose multiplicatively.
BTD alone delivers 1.4×–53.0× training speedup (1.0×–48.1× inference) and dominates at low foreground ratios where background tokens are the bulk of the sequence. BSA after BTD contributes a steadier 1.7×–2.9× training (1.6×–4.1× inference) and grows with the foreground ratio, complementing BTD when there is less background to drop.
A large-scale, ultra-high-resolution multi-view rendering dataset built on TexVerse. The public release is 351,847 BSDF assets at up to 4096×4096; UltraTex training uses a 268,365-asset subset.
Every asset is rendered under both configurations. They share the same camera intrinsics, object normalization, aspect-ratio-adaptive distance, and sampled lighting — they differ only in viewpoints.
Four side views at azimuths 0°, 90°, 180°, 270° with zero elevation, plus a top and a bottom view. This defines the six-view layout our diffusion model is trained and evaluated on.
Twelve azimuths spaced every 30° at each of three elevations (−40°, −20°, 30°). Not used by UltraTex; released for tasks needing denser coverage, such as novel-view synthesis and sparse-view reconstruction.
@inproceedings{zhang2026ultratex,
author = {Zhang, Yibo and Yuan, Ze and Cao, Nan and Zhang, Li and
Cao, Yan-Pei and Guo, Yuan-Chen and Ma, Rui},
title = {UltraTex: Unleashing 2K Multi-View Diffusion for 3D Texturing},
year = {2026},
isbn = {9798400728426},
publisher = {Association for Computing Machinery},
address = {Kuala Lumpur, Malaysia},
url = {https://doi.org/10.1145/3829340.3842299},
doi = {10.1145/3829340.3842299},
booktitle = {Proceedings of the SIGGRAPH Asia 2026 Conference Papers},
series = {SA Conference Papers '26},
}