Volumetric rendering of Computed Tomography (CT) scans is crucial for visualizing complex 3D anatomical structures in medical imaging. Current high-fidelity approaches, especially neural rendering techniques, require time-consuming per-scene optimization, limiting clinical applicability due to computational demands and poor generalizability. We propose Render-FM, a novel foundation model for direct, real-time volumetric rendering of CT scans. Render-FM employs an encoder-decoder architecture that directly regresses 6D Gaussian Splatting (6DGS) parameters from CT volumes, eliminating per-scan optimization through large-scale pre-training on diverse medical data. By integrating robust feature extraction with the expressive power of 6DGS, our approach efficiently generates high-quality, real-time interactive 3D visualizations across diverse clinical CT data. Experiments demonstrate that Render-FM achieves visual fidelity comparable or superior to specialized per-scan methods while drastically reducing preparation time from nearly an hour to seconds for a single inference step. This advancement enables seamless integration into real-time surgical planning and diagnostic workflows.
Here, we present the qualitative comparison results of 6DGS, 6DGS + AGP (Ours), Render-FM (Ours), Render-FM (Ours) + FT, and Ground Truth. Note that, this is under sparse-view setting of 20 views for training 6DGS or fine-tuning Render-FM.
@misc{gao2025renderfm,
title={Render-FM: A Foundation Model for Real-time Photorealistic Volumetric Rendering},
author={Zhongpai Gao and Meng Zheng and Benjamin Planche and Anwesa Choudhuri and Terrence Chen and Ziyan Wu},
year={2025},
eprint={2505.17338},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2505.17338},
}