TY - GEN
T1 - Comparison of Knowledge Distillation and Binarized Neural Networks for Human Activity Recognition Using Radar Data
AU - Kakuba, Samuel
AU - Colaco, Savina Jassica
AU - Kim, Jung Hwan
AU - Yoon, Young Jin
AU - Han, Dong Seog
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.
PY - 2025
Y1 - 2025
N2 - Though deep learning models exhibit good performance, it’s usually challenging to deploy them in low-resource devices and embedded platforms for human-to-machine interaction that are often faced with challenges of limited computational resources. In this paper, we carried out comparative experiments to ascertain the robustness of binarized neural networks (BNNs) and deep learning models that use knowledge distillation in relation to their performance on the University of Glasgow radar data for activity recognition. We evaluated the performance of three ImageNet models and used the best deep learning model among them as a teacher to aid learning for a low-parameterized model (student) in a knowledge distillation paradigm. We also evaluated the performance of BNNs on the same dataset. While both approaches exhibit comparable performance, the student model size and loss values are far smaller than the BNN model. However, the BNN model exhibits a better performance in terms of accuracy and confusion ratio which makes the choice of these deep learning approaches for low-resource devices a trade-off between accuracy and model size.
AB - Though deep learning models exhibit good performance, it’s usually challenging to deploy them in low-resource devices and embedded platforms for human-to-machine interaction that are often faced with challenges of limited computational resources. In this paper, we carried out comparative experiments to ascertain the robustness of binarized neural networks (BNNs) and deep learning models that use knowledge distillation in relation to their performance on the University of Glasgow radar data for activity recognition. We evaluated the performance of three ImageNet models and used the best deep learning model among them as a teacher to aid learning for a low-parameterized model (student) in a knowledge distillation paradigm. We also evaluated the performance of BNNs on the same dataset. While both approaches exhibit comparable performance, the student model size and loss values are far smaller than the BNN model. However, the BNN model exhibits a better performance in terms of accuracy and confusion ratio which makes the choice of these deep learning approaches for low-resource devices a trade-off between accuracy and model size.
KW - Activity recognition
KW - Binarized neural network
KW - Knowledge distillation
KW - Radar data
UR - https://www.scopus.com/pages/publications/105003908304
U2 - 10.1007/978-981-97-8764-7_22
DO - 10.1007/978-981-97-8764-7_22
M3 - Conference contribution
AN - SCOPUS:105003908304
SN - 9789819787630
T3 - Smart Innovation, Systems and Technologies
SP - 233
EP - 240
BT - Advances in Intelligent Information Hiding and Multimedia Signal Processing, Volume 1 - Proceeding of the 19th International Conference on IIH-MSP in conjunction with 11th International Conference on Orange Technology, Applications and Tools
A2 - Tseng, Shih-Pang
A2 - Paul, Anand
A2 - Pan, Jeng-Shyang
A2 - Favorskaya, Margarita
PB - Springer Science and Business Media Deutschland GmbH
T2 - 19th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2023, in conjunction with the 11th International Conference on Orange Technology, Applications, and Tools, ICOT 2023
Y2 - 5 December 2023 through 7 December 2023
ER -