PCCRender: Joint Learning of Point Cloud Compression and Gaussian Splatting Rendering

Abstract

Point cloud is a crucial 3D representation that plays significant role in fields such as VR/AR and digital museums. With the increasing amount of point cloud data, many compression and rendering methods are proposed, which often focus on either 3D or 2D visual quality. However, with the rapid advancements in display technology, there is a growing expectation for improved 2D and 3D visual quality while using lower bandwidth. To address these challenges, we present PCCRender, a novel end-to-end framework that achieves high-quality attribute preservation and rendering capabilities while maintaining low bandwidth requirements. To achieve efficient variable rate compression, we introduce Geometry-Invariant Rate Adjustment (GIRA) module that mitigates the influence of point cloud density during rate adjustment. Additionally, to improve decoding speed, we develop Uneven Four-Group Context Model (UFCM), achieving a trade-off between the accuracy and complexity of entropy parameter prediction. Moreover, we design Voxel to Gaussian Primitive Converter (V2G-Converter) which generates Gaussian primitives from decoded point clouds, enabling differentiable rendering. Our unified optimization framework jointly minimizes bit rate while maximizing both 2D rendered image quality and point cloud attribute fidelity. Experimental results demonstrate that PCCRender achieves state-of-the-art performance. Compared to the GPCC v23 (GS Render) method, our framework achieves 10.58% BD-Rate gain in attribute compression and 1.5 dB BD-PSNR increase in 2D rendered visual quality. These compelling results, combined with support for variable rate and free viewpoint rendering, establish PCCRender as a practical solution for the next generation point cloud applications.