Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network

Ah Hyun Yuh; Soon Ju Kang

doi:10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics53846.2021.00027

Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network

Ah Hyun Yuh
, Soon Ju Kang

Kyungpook National University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

9 Scopus citations

Abstract

Over the past years, increasing number of IoT sensors played important role in developing ambient assisted living (AAL) technologies such as elderly home care system by predicting activity of daily livings (ADLs). One way to develop smarter home care services with unobtrusive sensors in ubiquitous forms is using sound. This paper suggests a methodology to detect different sound events generated by residents based on real-life audio data. We propose a guide all the way from installing wireless microphone networks to recording, annotating, and preprocessing audios. Then we extract audio features and design deep learning classifier to classifying sound events. Finally, we deploy classifier on real-life scenarios to implement sound event detection in real-time. We evaluated 2D convolutional classifier with 16 sound events, achieving 95.55 % training accuracy, 94.64 % validation accuracy, 96.40 % recall score, and 94.93 % F1-score.

Original language	English
Title of host publication	Proceedings - IEEE Congress on Cybermatics
Subtitle of host publication	2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021
Editors	James Zheng, Xiao Liu, Tom Hao Luan, Prem Prakash Jayaraman, Haipeng Dai, Karan Mitra, Kai Qin, Rajiv Ranjan, Sheng Wen
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	83-88
Number of pages	6
ISBN (Electronic)	9781665417624
DOIs	https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics53846.2021.00027
State	Published - 2021
Event	2021 IEEE Congress on Cybermatics: 14th IEEE International Conferences on Internet of Things, iThings 2021, 17th IEEE International Conference on Green Computing and Communications, GreenCom 2021, 2021 IEEE International Conference on Cyber Physical and Social Computing, CPSCom 2021 and 7th IEEE International Conference on Smart Data, SmartData 2021 - Virtual, Melbourne, Australia Duration: 6 Dec 2021 → 8 Dec 2021

Publication series

Name	Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021

Conference

Conference	2021 IEEE Congress on Cybermatics: 14th IEEE International Conferences on Internet of Things, iThings 2021, 17th IEEE International Conference on Green Computing and Communications, GreenCom 2021, 2021 IEEE International Conference on Cyber Physical and Social Computing, CPSCom 2021 and 7th IEEE International Conference on Smart Data, SmartData 2021
Country/Territory	Australia
City	Virtual, Melbourne
Period	6/12/21 → 8/12/21

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 7 Affordable and Clean Energy

Keywords

Activity of Daily Living
Audio Signal Pro-cessing
Deep Learning
Real Time System
Sound Event Classification

Access to Document

10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics53846.2021.00027

Cite this

Yuh, A. H., & Kang, S. J. (2021). Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network. In J. Zheng, X. Liu, T. H. Luan, P. P. Jayaraman, H. Dai, K. Mitra, K. Qin, R. Ranjan, & S. Wen (Eds.), Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021 (pp. 83-88). (Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics53846.2021.00027

Yuh, Ah Hyun ; Kang, Soon Ju. / Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network. Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021. editor / James Zheng ; Xiao Liu ; Tom Hao Luan ; Prem Prakash Jayaraman ; Haipeng Dai ; Karan Mitra ; Kai Qin ; Rajiv Ranjan ; Sheng Wen. Institute of Electrical and Electronics Engineers Inc., 2021. pp. 83-88 (Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021).

@inproceedings{203d620e2523429b83939156033ae335,

title = "Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network",

abstract = "Over the past years, increasing number of IoT sensors played important role in developing ambient assisted living (AAL) technologies such as elderly home care system by predicting activity of daily livings (ADLs). One way to develop smarter home care services with unobtrusive sensors in ubiquitous forms is using sound. This paper suggests a methodology to detect different sound events generated by residents based on real-life audio data. We propose a guide all the way from installing wireless microphone networks to recording, annotating, and preprocessing audios. Then we extract audio features and design deep learning classifier to classifying sound events. Finally, we deploy classifier on real-life scenarios to implement sound event detection in real-time. We evaluated 2D convolutional classifier with 16 sound events, achieving 95.55 \% training accuracy, 94.64 \% validation accuracy, 96.40 \% recall score, and 94.93 \% F1-score.",

keywords = "Activity of Daily Living, Audio Signal Pro-cessing, Deep Learning, Real Time System, Sound Event Classification",

author = "Yuh, \{Ah Hyun\} and Kang, \{Soon Ju\}",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 2021 IEEE Congress on Cybermatics: 14th IEEE International Conferences on Internet of Things, iThings 2021, 17th IEEE International Conference on Green Computing and Communications, GreenCom 2021, 2021 IEEE International Conference on Cyber Physical and Social Computing, CPSCom 2021 and 7th IEEE International Conference on Smart Data, SmartData 2021 ; Conference date: 06-12-2021 Through 08-12-2021",

year = "2021",

doi = "10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics53846.2021.00027",

language = "English",

series = "Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "83--88",

editor = "James Zheng and Xiao Liu and Luan, \{Tom Hao\} and Jayaraman, \{Prem Prakash\} and Haipeng Dai and Karan Mitra and Kai Qin and Rajiv Ranjan and Sheng Wen",

booktitle = "Proceedings - IEEE Congress on Cybermatics",

address = "United States",

}

Yuh, AH & Kang, SJ 2021, Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network. in J Zheng, X Liu, TH Luan, PP Jayaraman, H Dai, K Mitra, K Qin, R Ranjan & S Wen (eds), Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021. Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021, Institute of Electrical and Electronics Engineers Inc., pp. 83-88, 2021 IEEE Congress on Cybermatics: 14th IEEE International Conferences on Internet of Things, iThings 2021, 17th IEEE International Conference on Green Computing and Communications, GreenCom 2021, 2021 IEEE International Conference on Cyber Physical and Social Computing, CPSCom 2021 and 7th IEEE International Conference on Smart Data, SmartData 2021, Virtual, Melbourne, Australia, 6/12/21. https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics53846.2021.00027

Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network. / Yuh, Ah Hyun; Kang, Soon Ju.
Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021. ed. / James Zheng; Xiao Liu; Tom Hao Luan; Prem Prakash Jayaraman; Haipeng Dai; Karan Mitra; Kai Qin; Rajiv Ranjan; Sheng Wen. Institute of Electrical and Electronics Engineers Inc., 2021. p. 83-88 (Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network

AU - Yuh, Ah Hyun

AU - Kang, Soon Ju

PY - 2021

Y1 - 2021

N2 - Over the past years, increasing number of IoT sensors played important role in developing ambient assisted living (AAL) technologies such as elderly home care system by predicting activity of daily livings (ADLs). One way to develop smarter home care services with unobtrusive sensors in ubiquitous forms is using sound. This paper suggests a methodology to detect different sound events generated by residents based on real-life audio data. We propose a guide all the way from installing wireless microphone networks to recording, annotating, and preprocessing audios. Then we extract audio features and design deep learning classifier to classifying sound events. Finally, we deploy classifier on real-life scenarios to implement sound event detection in real-time. We evaluated 2D convolutional classifier with 16 sound events, achieving 95.55 % training accuracy, 94.64 % validation accuracy, 96.40 % recall score, and 94.93 % F1-score.

AB - Over the past years, increasing number of IoT sensors played important role in developing ambient assisted living (AAL) technologies such as elderly home care system by predicting activity of daily livings (ADLs). One way to develop smarter home care services with unobtrusive sensors in ubiquitous forms is using sound. This paper suggests a methodology to detect different sound events generated by residents based on real-life audio data. We propose a guide all the way from installing wireless microphone networks to recording, annotating, and preprocessing audios. Then we extract audio features and design deep learning classifier to classifying sound events. Finally, we deploy classifier on real-life scenarios to implement sound event detection in real-time. We evaluated 2D convolutional classifier with 16 sound events, achieving 95.55 % training accuracy, 94.64 % validation accuracy, 96.40 % recall score, and 94.93 % F1-score.

KW - Activity of Daily Living

KW - Audio Signal Pro-cessing

KW - Deep Learning

KW - Real Time System

KW - Sound Event Classification

UR - https://www.scopus.com/pages/publications/85127432348

U2 - 10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics53846.2021.00027

DO - 10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics53846.2021.00027

M3 - Conference contribution

AN - SCOPUS:85127432348

T3 - Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021

SP - 83

EP - 88

BT - Proceedings - IEEE Congress on Cybermatics

A2 - Zheng, James

A2 - Liu, Xiao

A2 - Luan, Tom Hao

A2 - Jayaraman, Prem Prakash

A2 - Dai, Haipeng

A2 - Mitra, Karan

A2 - Qin, Kai

A2 - Ranjan, Rajiv

A2 - Wen, Sheng

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2021 IEEE Congress on Cybermatics: 14th IEEE International Conferences on Internet of Things, iThings 2021, 17th IEEE International Conference on Green Computing and Communications, GreenCom 2021, 2021 IEEE International Conference on Cyber Physical and Social Computing, CPSCom 2021 and 7th IEEE International Conference on Smart Data, SmartData 2021

Y2 - 6 December 2021 through 8 December 2021

ER -

Yuh AH, Kang SJ. Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network. In Zheng J, Liu X, Luan TH, Jayaraman PP, Dai H, Mitra K, Qin K, Ranjan R, Wen S, editors, Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021. Institute of Electrical and Electronics Engineers Inc. 2021. p. 83-88. (Proceedings - IEEE Congress on Cybermatics: 2021 IEEE International Conferences on Internet of Things, iThings 2021, IEEE Green Computing and Communications, GreenCom 2021, IEEE Cyber, Physical and Social Computing, CPSCom 2021 and IEEE Smart Data, SmartData 2021). doi: 10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics53846.2021.00027

Real-Time Sound Event Classification for Human Activity of Daily Living using Deep Neural Network

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this