DeferredGS: Decoupled and relightable Gaussian splatting with deferred shading

Wu, Tong, Sun, Jia-Mu, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680, Ma, Yuewen, Kobbelt, Leif and Gao, Lin 2025. DeferredGS: Decoupled and relightable Gaussian splatting with deferred shading. IEEE Transactions on Pattern Analysis and Machine Intelligence 10.1109/tpami.2025.3560933

Preview

PDF - Accepted Post-Print Version Download (29MB) | Preview

Abstract

Reconstructing and editing 3D objects and scenes both play crucial roles in computer graphics and computer vision. Neural radiance fields (NeRFs) can achieve realistic reconstruction and editing results but suffer from inefficiency in rendering. Gaussian splatting significantly accelerates rendering by rasterizing Gaussian ellipsoids. However, Gaussian splatting utilizes a single Spherical Harmonic (SH) function to model both texture and lighting, limiting independent editing capabilities of these components. Recently, attempts have been made to decouple texture and lighting with the Gaussian splatting representation but may fail to produce plausible geometry and decomposition results on reflective scenes. Additionally, the forward shading technique they employ introduces noticeable blending artifacts during relighting, as the geometry attributes of Gaussians are optimized under the original illumination and may not be suitable for novel lighting conditions. To address these issues, we introduce DeferredGS, a method for decoupling and relighting the Gaussian splatting representation using deferred shading. To achieve successful decoupling, we model the illumination with a learnable environment map and define additional attributes such as texture parameters and normal direction on Gaussians, where the normal is distilled from a jointly trained signed distance function. More importantly, we apply deferred shading, resulting in more realistic relighting effects compared to previous methods. Both qualitative and quantitative experiments demonstrate the superior performance of DeferredGS in novel view synthesis and relighting tasks.

Item Type:	Article
Date Type:	Published Online
Status:	In Press
Schools:	Schools > Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers (IEEE)
ISSN:	0162-8828
Date of First Compliant Deposit:	30 April 2025
Date of Acceptance:	27 March 2025
Last Modified:	30 Apr 2025 13:15
URI:	https://orca.cardiff.ac.uk/id/eprint/177906

Actions (repository staff only)

Edit Item