Welcome To WUJNS
武汉大学学报 英文版 | Wuhan University Journal of Natural Sciences
Wan Fang
CNKI
CSCD
Wuhan University
Latest Article
3D Object Detection Incorporating Instance Segmentation and Image Restoration
Time:2019-8-28  
HUANG Bo1,2, HUANG Man1, GAO Yongbin1 , YU Yuxin3, JIANG Xiaoyan1, ZHANG Juan1
1. Department of Electrical and Electronic Engineering, Shanghai University of Engineering Science, Shanghai 201620, China; 2. Collaborative Innovation Center for Economics Crime Investigation and Prevention Technology, Nanchang 330103, Jiangxi, China; 3. School of Economics and Finance, Shanghai International Studies University, Shanghai 210620, China
Abstract:
Nowadays, 3D object detection, which uses the color and depth information to find object localization in the 3D world and estimate their physical size and pose, is one of the most important 3D perception tasks in the field of computer vision. In order to solve the problem of mixed segmentation results when multiple instances appear in one frustum in the F-PointNet method and in the occlusion that leads to the loss of depth information, a 3D object detection approach based on instance segmentation and image restoration is proposed in this paper. Firstly, instance segmentation with Mask R-CNN on an RGB image is used to avoid mixed segmentation results. Secondly, for the detected occluded objects, we remove the occluding object first in the depth map and then restore the empty pixel region by utilizing the Criminisi Algorithm to recover the missing depth information of the object. The experimental results show that the proposed method improves the average precision score compared with the F-PointNet method.
Key words:image processing 3D object detection instance segmentation depth information image restoration
CLC number: TP 314
References:
[1]	Hoiem D, Efros A A, Hebert M. Geometric context from a single image [C] // IEE International Conference on Com-puter Vision 2005. Washington D C: IEEE, 2005,1:654-661.
[2]	Hoiem D, Efros A A, Hebert M. Putting objects in perspec-tive[J]. International Journal Computer Vision, 2008, 80(1): 3-15.
[3]	Malisiewicz T, Gupta A, Efros A A. Ensemble of exemplar- SVMs for object detection and beyond [C]// IEE International Conference on Computer Vision. Washington D C: IEEE, 2011, 1(2): 6.
[4]	Gupta S, Girshick R, Arbeláez P, et al. Learning rich features from RGB-D images for object detection and segmentation [C]// 13th European Conference on Computer Vision. Berlin: Springer-Verlag, 2014, 8695: 345-360. 
[5]	Deng Z, Latecki L J. Amodal detection of 3D objects: Infer-ring 3D bounding boxed from 2D ones in RGB-depth images [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2017: 1063- 6919.
[6]	Song S, Xiao J. Deep sliding shapes for amodal 3D object detection in RGB-D images[J]. Computer Science, 2015, 139(2): 808-816.
[7]	Qi C R, Liu W, Wu C, et al. Frustum pointnets for 3D object detection from RGB-D data [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D C:IEEE, 2018: 918-927.
[8]	He K, Gkioxari G, Dollar P, et al. Mask R-CNN [C]// Pro-ceedings of the IEEE International Conference on Computer Vision.  Washington D C: IEEE, 2017: 2961- 2969.
[9]	Chen X, Kundu K, Zhang Z, et al. Monocular 3D object detection for autonomous driving[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni-tion. Washington D C:IEEE, 2016, 1: 2147-2156.
[10]	Girshick R, Donahue J, Darrell T, et al. Rich feature hierar-chies for accurate object detection and semantic segmenta-tion[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2014: 580- 587.
[11]	Chen X, Ma H, Wan J, et al. Multi-view 3D object detection network for autonomous driving [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni-tion. Washington D C: IEEE, 2017: 1907-1915.
[12]	Lin D, Fidler S, Urtasun R. Holistic scene understanding for 3D object detection with RGB-D cameras [C]// Proceedings of the IEEE International Conference on Computer Vision. Washington D C: IEEE, 2013, 1: 1417-1424.
[13]	Kim B S, Xu S, Savarese S. Accurate localization of 3D objects from RGB-D data using segmentation hypotheses [C]// Proceedings of the IEEE Conference on Computer Vi-sion and Pattern Recognition. Washington D C: IEEE, 2013, 1: 3182-3189.
[14]	Ren Z, Sudderth E B. Three-dimensional object detection and layout prediction using clouds of oriented gradients [C] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2016: 1525-1533.
[15]	Criminisi A, Pérez P, Toyama K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212.
[16]	Qi C R, Su H, Mo K, et al. Pointnet: Deep learning on point sets for 3D classification and segmentation[J]. Proc Com-puter Vision and Pattern Recognition (CVPR), 2017, 1(2): 4.
[17]	Song S, Xiao J. Sliding shapes for 3D object detection in depth images [C]// 2014 European Conference on Computer Vision. Washington D C: IEEE, 2014: 634-651.
[18]	Chen X, Kundu K, Zhu Y, et al. 3D object proposals for accurate object class detection [C]// Advances in Neural In-formation Processing Systems. Massachusetts: MIT Press, 2015: 424-432.
[19]	Li B. 3D fully convolutional network for vehicle detection in point cloud [C]//Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on IEEE. Washington D C: IEEE, 2017: 1513-1518.
[20]	Qi C R, Yi L, Su H, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems. Massachusetts: MIT Press, 2017: 5099-5108.
Welcome To WUJNS

HOME | Aim and Scope | Editoral Board | Current Issue | Back Issue | Subscribe | Crosscheck | Polishing | Contact us Copyright © 1997-2019 All right reserved