TY - GEN
T1 - Real-time Semantic Full-Body Haptic Feedback Converted from Sound for Virtual Reality Gameplay
AU - Yun, Gyeore
AU - Choi, Seungmoon
N1 - Publisher Copyright:
© 2025 Copyright held by the owner/author(s).
PY - 2025/4/26
Y1 - 2025/4/26
N2 - We present a multisensory virtual reality (VR) system that enables users to experience concurrent visual, auditory, and haptic feedback, featuring semantic classification of events from sound, sound-to-haptic conversion, and full-body haptic effects. This concept is applied to enhance the user experience of virtual reality (VR) gameplay. The system utilizes a Long-Short-Term Memory (LSTM) model to classify game sounds and detect key events such as gunfire, explosions, and hits. These events are translated into full-body haptic patterns through a haptic suit, providing users with realistic and immersive haptic experiences. The system operates with low latency, ensuring the seamless synchrony between sound and haptic feedback. Evaluations through user studies demonstrate significant improvements in user experience compared to traditional sound-to-haptic methods, emphasizing the importance of accurate sound classification and well-designed haptic effects.
AB - We present a multisensory virtual reality (VR) system that enables users to experience concurrent visual, auditory, and haptic feedback, featuring semantic classification of events from sound, sound-to-haptic conversion, and full-body haptic effects. This concept is applied to enhance the user experience of virtual reality (VR) gameplay. The system utilizes a Long-Short-Term Memory (LSTM) model to classify game sounds and detect key events such as gunfire, explosions, and hits. These events are translated into full-body haptic patterns through a haptic suit, providing users with realistic and immersive haptic experiences. The system operates with low latency, ensuring the seamless synchrony between sound and haptic feedback. Evaluations through user studies demonstrate significant improvements in user experience compared to traditional sound-to-haptic methods, emphasizing the importance of accurate sound classification and well-designed haptic effects.
KW - automatic generation
KW - full body
KW - game
KW - haptic suit
KW - real time
KW - semantic sound classification
KW - sound-haptic conversion
KW - user experience
UR - https://www.scopus.com/pages/publications/105005745276
U2 - 10.1145/3706598.3713355
DO - 10.1145/3706598.3713355
M3 - Conference contribution
AN - SCOPUS:105005745276
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI 2025 - Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
PB - Association for Computing Machinery
T2 - 2025 CHI Conference on Human Factors in Computing Systems, CHI 2025
Y2 - 26 April 2025 through 1 May 2025
ER -