Differentiable Surface Splatting for Point-based Geometry Processing

Abstract

We propose Differentiable Surface Splatting (DSS), a high-fidelity differentiable renderer for point clouds. Gradients for point locations and normals are carefully designed to handle discontinuities of the rendering function. Regularization terms are introduced to ensure uniform distribution of the points on the underlying surface. We demonstrate applications of DSS to inverse rendering for geometry synthesis and denoising, where large scale topological changes, as well as small scale detail modifications, are accurately and robustly handled without requiring explicit connectivity, outperforming state-of-the-art techniques.

Publication
SIGGRAPH Asia

teaser

Shape Synthesis

We can synthesize shapes from multiple 2D images. This process is not constrained by topology changes. synthesis

Point Cloud Filering

Using DSS we can directly apply image-based filters to a point cloud to achieve various geometric effect. filtering

Point Cloud Denoising

We create state-of-the-art point cloud denoising results by marrying our differential renderer with the famous image-to-image translation deep learning framework Pix2Pix. Denoising

Accompanying Video

Acknowledgement

We would like to thank Federico Danieli for the insightful discussion, Philipp Herholz for the timely feedack, Romann Weber for the video voice-over and Derek Liu for the help during the rebuttal. This work was supported in part by gifts from Adobe, Facebook and Snap, Inc.