Deep Learning based Human Detection using Thermal-RGB Data Fusion for Safe Automotive Guided-Driving

Yun Heuijee, Daejin Park

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Every year, the number of drivers increases, resulting in a corresponding increase in traffic fatalities. In Korea, pedestrian accidents constituted 35.5% of all traffic accidents for the last 2 years, with the number of child accidents rising annually. Autonomous vehicles currently rely on a lidar, which is insufficient in preventing accidents as it only recognizes obstacles that are far away. To mitigate these accidents, we propose selective thermal imaging data to identify people beyond the limited field of view. First, RGB camera image data for object recognition is performed. When vehicles or obstacles are present, the optional use of thermal data is applied. Thermal data can only identify a person, and it is used to prevent unforeseen incidents. The RGB images are divided into thirds and each section is assessed for obstacles, prioritizing the area with the most obstacles for integration with thermal data. Using the described algorithm, the level of accuracy increased by 2.07 times, from 40.43% to 83.91%. Additionally, experiments performed on a personal computer demonstrate that the algorithm is capable of functioning in real-time at a rate of 2.7 frames per second, utilizing 175.95 megabytes of memory at 0.36 seconds per image. When executed on a lightweight board such as the Jetson Nano, the algorithm runs at a rate of 0.75 frames per second, utilizing 140.08 megabytes of memory, at 1.33 seconds per image.

Original languageEnglish
Title of host publication2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages593-598
Number of pages6
ISBN (Electronic)9798350304367
DOIs
StatePublished - 2024
Event2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2024 - Biarritz, France
Duration: 11 Mar 202415 Mar 2024

Publication series

Name2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2024

Conference

Conference2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2024
Country/TerritoryFrance
CityBiarritz
Period11/03/2415/03/24

Keywords

  • ADAS(Advanced Driver Assistance System)
  • Deep Learning
  • RGB image data
  • Thermal data

Fingerprint

Dive into the research topics of 'Deep Learning based Human Detection using Thermal-RGB Data Fusion for Safe Automotive Guided-Driving'. Together they form a unique fingerprint.

Cite this