TY - GEN
T1 - Generating Real-Time, Selective, and Multimodal Haptic Effects from Sound for Gaming Experience Enhancement
AU - Yun, Gyeore
AU - Mun, Minjae
AU - Lee, Jungeun
AU - Kim, Dong Geun
AU - Tan, Hong Z.
AU - Choi, Seungmoon
N1 - Publisher Copyright:
© 2023 Owner/Author.
PY - 2023/4/19
Y1 - 2023/4/19
N2 - We propose an algorithm that generates a vibration, an impact, or a vibration+impact haptic effect by processing a sound signal in real time. Our algorithm is selective in that it matches the most appropriate type of haptic effects to the sound using a machine-learning classifier (random forest) that is built on expert-labeled datasets. Our algorithm is tailored to enhance user experiences for video game play, and we present two examples for the RPG (role-playing game) and FPS (first-person shooter) genres. We demonstrate the effectiveness of our algorithm by a user study in comparison to other state-of-the-art (SOTA) methods for the same cross-modal conversion. Our system elicits better multisensory user experiences than the SOTA algorithms for both game genres.
AB - We propose an algorithm that generates a vibration, an impact, or a vibration+impact haptic effect by processing a sound signal in real time. Our algorithm is selective in that it matches the most appropriate type of haptic effects to the sound using a machine-learning classifier (random forest) that is built on expert-labeled datasets. Our algorithm is tailored to enhance user experiences for video game play, and we present two examples for the RPG (role-playing game) and FPS (first-person shooter) genres. We demonstrate the effectiveness of our algorithm by a user study in comparison to other state-of-the-art (SOTA) methods for the same cross-modal conversion. Our system elicits better multisensory user experiences than the SOTA algorithms for both game genres.
KW - audio-haptic conversion
KW - automatic generation
KW - game
KW - multimodal haptic effects
KW - sound-haptic conversion
UR - https://www.scopus.com/pages/publications/85160023023
U2 - 10.1145/3544548.3580787
DO - 10.1145/3544548.3580787
M3 - Conference contribution
AN - SCOPUS:85160023023
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
PB - Association for Computing Machinery
T2 - 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023
Y2 - 23 April 2023 through 28 April 2023
ER -