One framework for photorealistic rendering across view direction,
time, and beyond — lifting Gaussian and Beta primitives into
higher-dimensional space for real-time, view-dependent, and dynamic novel-view synthesis.
From conditional slicing to clinical volume rendering, one idea at a time, with real results
from each paper.
The big picture
One framework, many dimensions
Static 3D Gaussian Splatting cannot express specular reflections, temporal dynamics, or
view-dependent geometry. N-dimensional splatting solves this by lifting
each primitive into a joint distribution over 3D position and conditioning variables, then
conditionally slicing it at render time to recover a 3D primitive whose mean,
opacity, and covariance vary continuously with view or time. The same primitives now carry
into the clinic: Render-FM predicts them from CT in one forward pass, and
XClipGS cuts them exactly with an interactive clipping plane, and
FactorSplat makes their transfer function editable at inference.
Adds the missing clinical interaction: an exact, closed-form clipping plane
for Gaussian proxies — sharp at the cut, with
40× less leakage than prior clippers.
Makes the transfer function an inference-time input: one checkpoint per scan
takes unseen TF edits without retraining, switching presets in about 1 ms.
◳
Dimensionality
Lift primitives from 3D to 6D (view) and 7D (view + time) so a single primitive can carry view-dependent and dynamic behavior.
◐
Kernel shape
Swap the Gaussian opacity decay for an anisotropic Beta kernel with per-dimension bandwidth control for richer light transport.
⚡
Conditioning
Replace covariance-derived slicing (inversion + regression) with a directly parameterized Cholesky precision — kernel-agnostic and far faster.
NeurIPS 2026
Direct Conditional Parameterization for N-Dimensional Splatting
Zhongpai Gao, Benjamin Planche, Van Nguyen Nguyen, Meng Zheng, Anwesa Choudhuri, Arun Innanje, Terrence Chen, Ziyan Wu
N-DGS (6DGS/7DGS) and UBS share one expensive conditional-slicing step: every forward pass inverts a
covariance block and builds a regression matrix per primitive. dGS and
dBS parameterize what rendering actually needs directly
— a Cholesky precision factor for opacity and a spatially scaled displacement matrix with learnable
per-dimension coupling for position. The change is kernel-agnostic across Gaussian and Beta
kernels, matches or exceeds its covariance-derived counterparts (up to +1.26 dB), and
runs 6.9–7.7× faster at slicing and up to 2.66× faster
end-to-end.
Rong Liu, Zhongpai Gao, Benjamin Planche, Meida Chen, Van Nguyen Nguyen, Meng Zheng, Anwesa Choudhuri, Terrence Chen, Yue Wang, Andrew Feng, Ziyan Wu
UBS generalizes 3D Gaussian Splatting to N-dimensional anisotropic Beta kernels.
Unlike fixed Gaussian primitives, Beta kernels enable controllable dependency modeling across
spatial, angular, and temporal dimensions. The learned Beta parameters
automatically decompose scene properties — without explicit supervision — into
spatial, angular, and temporal components, achieving real-time rendering while remaining
backward-compatible with Gaussian Splatting and outperforming prior methods across static,
view-dependent, and dynamic benchmarks.
Where direction-aware Gaussians began. DDGS renders realistic X-ray DRRs from CT by splitting the radiosity
into isotropic and direction-dependent Gaussians, so scattering effects are learned instead
of simulated at runtime. A radiodensity-aware initialization adapts 3DGS to tomography. It beats prior
Gaussian DRR methods on image accuracy (+1.04 dB on CTPelvic1K) and drives 2D/3D pose
registration with 4× faster DRRs and lower landmark error than DiffDRR.
Where N-dimensional splatting began: lifting Gaussian primitives along view direction (6D) and then time (7D), with an efficient conditional-slicing mechanism.
ICLR 2025
6DGS: Enhanced Direction-Aware Gaussian Splatting for Volumetric Rendering
6DGS extends 3D Gaussian Splatting into 6D space to capture view-dependent effects, jointly
modeling spatial position and view direction. By enhancing color and opacity representations
and leveraging additional directional information, it achieves up to a 15.73 dB PSNR
improvement with 66.5% fewer Gaussian points than 3DGS — while
staying compatible with the established 3DGS framework.
7DGS represents scenes as seven-dimensional Gaussians unifying spatial
position (3D), time (1D), and viewing direction (3D). An efficient conditional slicing
mechanism converts these 7D primitives into view- and time-conditioned 3D Gaussians,
enabling real-time rendering of dynamic scenes with view-dependent effects — surpassing
prior methods by up to 7.36 dB PSNR at 401 FPS.
Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering
Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri, Van Nguyen Nguyen, Terrence Chen, Ziyan Wu
Render-FM brings the 6D representation to medicine. It directly predicts 6D Gaussian Splatting
parameters from CT volumes in 2.8 seconds — a 500× speedup
over per-scan optimization — while maintaining quality. Anatomy-Guided Priming incorporates
segmentation masks and transfer functions as structural priors, so the feedforward model
generalizes to unseen anatomies and supports real-time interactive rendering on standard GPUs
without per-scan optimization.
Two interactions clinicians expect from a volume renderer, added to Gaussian proxies: cutting the volume open with an exact clipping plane, and editing its transfer function at inference.
arXiv
XClipGS: Exact Half-Space Clipping for Medical Volume Gaussian Splatting
XClipGS makes the clinical clipping plane exact: under the affine EWA model, each
half-space-restricted Gaussian's ray integral factorizes in closed form into its ordinary 2D footprint
times a conditional Gaussian CDF — no learned clipping parameters, no auxiliary network, and full
differentiability for clip-aware training of the hidden interior. On eight CT/MRI volumes it attains the
highest PSNR on every volume, renders at 655 FPS, and leaks roughly
40× less past the plane than ClipGS — including on arbitrary-normal planes
without retraining.
FactorSplat makes the transfer function an inference-time input: each Gaussian reads the
region-specific intensity-to-RGBA curves under its footprint, a local lookup applies the authored color and
opacity change, and a shared functional encoder with low-rank per-Gaussian factors learns the residual, while
geometry stays shared across presets. On seven CT and MR scans it beats region-aware VEG by
1.10–1.52 dB mean PSNR on held-out edits, renders at 524 FPS,
and switches presets in 1.17 ms without retraining.
@inproceedings{gao2024ddgs,
title = {DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering},
author = {Gao, Zhongpai and Planche, Benjamin and Zheng, Meng and
Chen, Xiao and Chen, Terrence and Wu, Ziyan},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2024}
}
@inproceedings{gao20246dgs,
title = {6DGS: Enhanced Direction-Aware Gaussian Splatting for Volumetric Rendering},
author = {Gao, Zhongpai and Planche, Benjamin and Zheng, Meng and
Choudhuri, Anwesa and Chen, Terrence and Wu, Ziyan},
booktitle = {International Conference on Learning Representations (ICLR)},
year = {2025}
}
@inproceedings{gao20257dgs,
title = {7DGS: Unified Spatial-Temporal-Angular Gaussian Splatting},
author = {Gao, Zhongpai and Planche, Benjamin and Zheng, Meng and
Choudhuri, Anwesa and Chen, Terrence and Wu, Ziyan},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
pages = {26316--26325},
year = {2025}
}
@inproceedings{liu2025universal,
title = {Universal Beta Splatting},
author = {Liu, Rong and Gao, Zhongpai and Planche, Benjamin and Chen, Meida and
Nguyen, Van Nguyen and Zheng, Meng and Choudhuri, Anwesa and
Chen, Terrence and Wang, Yue and Feng, Andrew and Wu, Ziyan},
booktitle = {International Conference on Learning Representations (ICLR)},
year = {2026}
}
@inproceedings{gao2026renderfm,
title = {Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering},
author = {Gao, Zhongpai and Planche, Benjamin and Zheng, Meng and Choudhuri, Anwesa and
Nguyen, Van Nguyen and Chen, Terrence and Wu, Ziyan},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2026}
}
@inproceedings{gao2026direct,
title = {Direct Conditional Parameterization for N-Dimensional Splatting},
author = {Gao, Zhongpai and Planche, Benjamin and Nguyen, Van Nguyen and Zheng, Meng and
Choudhuri, Anwesa and Innanje, Arun and Chen, Terrence and Wu, Ziyan},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2026}
}
@article{gao2026xclipgs,
title = {XClipGS: Exact Half-Space Clipping for Medical Volume Gaussian Splatting},
author = {Gao, Zhongpai and Planche, Benjamin and Zheng, Meng and Choudhuri, Anwesa and
Zhou, Chaoyi and Chen, Terrence and Wu, Ziyan},
journal = {arXiv preprint arXiv:2608.07760},
eprint = {2608.07760},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
year = {2026}
}
@article{gao2026factorsplat,
title = {FactorSplat: Appearance-Controllable Gaussian Proxies for Medical Volume Rendering},
author = {Gao, Zhongpai and Planche, Benjamin and Zheng, Meng and Choudhuri, Anwesa and
Chen, Terrence and Wu, Ziyan},
journal = {arXiv preprint arXiv:2610.02382},
eprint = {2610.02382},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
year = {2026}
}
