TY - GEN
T1 - Deep Learning based Human Detection using Thermal-RGB Data Fusion for Safe Automotive Guided-Driving
AU - Heuijee, Yun
AU - Park, Daejin
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Every year, the number of drivers increases, resulting in a corresponding increase in traffic fatalities. In Korea, pedestrian accidents constituted 35.5% of all traffic accidents for the last 2 years, with the number of child accidents rising annually. Autonomous vehicles currently rely on a lidar, which is insufficient in preventing accidents as it only recognizes obstacles that are far away. To mitigate these accidents, we propose selective thermal imaging data to identify people beyond the limited field of view. First, RGB camera image data for object recognition is performed. When vehicles or obstacles are present, the optional use of thermal data is applied. Thermal data can only identify a person, and it is used to prevent unforeseen incidents. The RGB images are divided into thirds and each section is assessed for obstacles, prioritizing the area with the most obstacles for integration with thermal data. Using the described algorithm, the level of accuracy increased by 2.07 times, from 40.43% to 83.91%. Additionally, experiments performed on a personal computer demonstrate that the algorithm is capable of functioning in real-time at a rate of 2.7 frames per second, utilizing 175.95 megabytes of memory at 0.36 seconds per image. When executed on a lightweight board such as the Jetson Nano, the algorithm runs at a rate of 0.75 frames per second, utilizing 140.08 megabytes of memory, at 1.33 seconds per image.
AB - Every year, the number of drivers increases, resulting in a corresponding increase in traffic fatalities. In Korea, pedestrian accidents constituted 35.5% of all traffic accidents for the last 2 years, with the number of child accidents rising annually. Autonomous vehicles currently rely on a lidar, which is insufficient in preventing accidents as it only recognizes obstacles that are far away. To mitigate these accidents, we propose selective thermal imaging data to identify people beyond the limited field of view. First, RGB camera image data for object recognition is performed. When vehicles or obstacles are present, the optional use of thermal data is applied. Thermal data can only identify a person, and it is used to prevent unforeseen incidents. The RGB images are divided into thirds and each section is assessed for obstacles, prioritizing the area with the most obstacles for integration with thermal data. Using the described algorithm, the level of accuracy increased by 2.07 times, from 40.43% to 83.91%. Additionally, experiments performed on a personal computer demonstrate that the algorithm is capable of functioning in real-time at a rate of 2.7 frames per second, utilizing 175.95 megabytes of memory at 0.36 seconds per image. When executed on a lightweight board such as the Jetson Nano, the algorithm runs at a rate of 0.75 frames per second, utilizing 140.08 megabytes of memory, at 1.33 seconds per image.
KW - ADAS(Advanced Driver Assistance System)
KW - Deep Learning
KW - RGB image data
KW - Thermal data
UR - http://www.scopus.com/inward/record.url?scp=85192465015&partnerID=8YFLogxK
U2 - 10.1109/PerComWorkshops59983.2024.10503400
DO - 10.1109/PerComWorkshops59983.2024.10503400
M3 - Conference contribution
AN - SCOPUS:85192465015
T3 - 2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2024
SP - 593
EP - 598
BT - 2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2024
Y2 - 11 March 2024 through 15 March 2024
ER -