Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

HeteroFusion: Dense scene reconstruction integrating multi-sensors

Yang, Sheng, Li, Beichen, Liu, Minghua, Lai, Yukun ORCID:, Kobbelt, Leif and Hu, Shi-Min ORCID: 2019. HeteroFusion: Dense scene reconstruction integrating multi-sensors. IEEE Transactions on Visualization and Computer Graphics 28 (3) , pp. 181-197. 10.1109/TVCG.2019.2919619

[thumbnail of HeteroFusion-Final.pdf]
PDF - Accepted Post-Print Version
Download (17MB) | Preview


We present a real-time approach that integrates multiple sensors for dense reconstruction of 3D indoor scenes. Existing algorithms are mainly based on a single RGBD camera and require continuous scanning on areas with sufficient geometric details. Failing to do so can lead to tracking loss due to the lack of frame registration hints. Inspired by the fact that utilizing multiple sensors can combine their strengths to form a more robust and accurate implementation, we incorporate multiple types of sensors, which are prevalently equipped in modern robots, including a 2D range sensor, an IMU, and wheel encoders to reinforce the tracking process and obtain better mesh construction. Specifically, we develop a feasible 2D TSDF volume representation for integrating and ray-casting laser frames, leading to a unified cost function in the pose estimation stage. Besides, for validating these estimated poses in the loop-closure optimization process, we train a classifier according to those features extracted from heterogeneous sensors and the registration progress. To evaluate our method on challenging robotic scanning scenarios, we assembled a scanning platform for acquiring real-world scans. We further simulated synthetic scans based on high-fidelity synthetic scenes for quantitative evaluation. Extensive experimental results demonstrate that our system is capable of robustly acquiring dense reconstructions and outperforms state-of-the-art systems.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
ISSN: 1077-2626
Date of First Compliant Deposit: 26 May 2019
Date of Acceptance: 12 May 2019
Last Modified: 07 Nov 2023 05:23

Citation Data

Cited 7 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item Edit Item


Downloads per month over past year

View more statistics