PI: Zhu LI

Recent advancement in the 3D capturing technology has enabled an enormous growth in applications that employ point cloud. These applications include autonomous driving, robotics, remote sensing, AR/VR, computer-aided design, objects/environment reconstruction etc. 3D point clouds are acquired through two approaches: 3D scanners and image-based 3D reconstructions. However, both methods introduce noise. 3D scanners suffer from measurement noise especially around sharp edges and corners of an object. Image based reconstruction introduce noise as it often fails to manage matching ambiguities. The presence of noise in point cloud deforms the underlying manifold structure and impairs subsequent geometry processing and computations such as classification, segmentation, registration, compression etc. Thus it is required to denoise it before further geometry processing. We propose to tackle denoising problem using deep learning into two folds: outlier removal and denoising. We used two stage deep learning pipeline where first stage acts as a binary classifier that classifies 3d points either as outlier or non-outlier. The second stage receives non-outlier and noisy points from the first stage and learns the underlying manifold to produce residual noise from the reference(true) surface. The denoised point cloud is reconstructed by subtracting the predicted shift/noise from the noisy point cloud. Currently, we have employed PointCNN(originally designed for classification/segmentation)as base architecture for both the stages and modified slightly to adapt to the aforementioned purposes. In fact, other architectures (e.g. PointNet++) or new model inspired from the existing architecture with better capability of manifold learnings can be used which will be explored during further research. These architecture will not only receive neighborhood points but will utilize other relevant information such as curvature too. We further employ our architecture for V-PCC dynamic point cloud artifact removal.