Sparse Depth Enhanced Direct Thermal-Infrared SLAM beyond the Visible Spectrum

In this letter, we propose a thermal-infrared simultaneous localization and mapping (SLAM) system enhanced by sparse depth measurements from light detection ranging (LiDAR). Thermal-infrared cameras are relatively robust against fog, smoke, and dynamic lighting conditions compared to RGB cameras operating under the visible spectrum. Exploiting thermal-infrared cameras for motion estimation and mapping is highly appealing. However, utilizing a thermal-infrared camera directly in existing vision-based methods is difficult because of the modality difference. This letter proposes a method to use sparse depth measurement for 6-DOF motion estimation via direct tracking under 14-bit raw measurement from the thermal camera. We also refine the local accuracy and include a loop closure to maintain global consistency. The experimental results demonstrate that the system is not only robust under various lighting conditions such as day and night, but it also overcomes the scale problem of monocular cameras.