Learning 2D Human Poses for Better 3D Lifting via Multi-model 3D-Guidance

Sanghyeon Lee, Yoonho Hwang, Jong Taek Lee

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Recent advancements in 2D pose detectors have significantly improved 3D human pose estimation via the 2D-to-3D lifting approach. Despite these advancements, a substantial accuracy gap remains between using ground-truth 2D poses and detected 2D poses for 3D lifting. However, most methods focus solely on enhancing the 3D lifting network, using 2D pose detectors optimized for 2D accuracy without any refinement to better serve the 3D lifting process. To address this limitation, we propose a novel 3D-guided training method that leverages 3D loss to improve 2D pose estimation. Additionally, we introduce a multi-model training method to ensure robust generalization across various 3D lifting networks. Extensive experiments with three 2D pose detectors and four 3D lifting networks demonstrate our method’s effectiveness. Our method achieves an average improvement of 4.6% in MPJPE on Human3.6M and 16.8% on Panoptic, enhancing 2D poses for accurate 3D lifting. The code is available at https://github.com/knu-vis/L2D-Pose.

Original languageEnglish
Title of host publicationComputer Vision – ACCV 2024 - 17th Asian Conference on Computer Vision, Proceedings
EditorsMinsu Cho, Ivan Laptev, Du Tran, Angela Yao, Hongbin Zha
PublisherSpringer Science and Business Media Deutschland GmbH
Pages185-202
Number of pages18
ISBN (Print)9789819608843
DOIs
StatePublished - 2025
Event17th Asian Conference on Computer Vision, ACCV 2024 - Hanoi, Viet Nam
Duration: 8 Dec 202412 Dec 2024

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume15472 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference17th Asian Conference on Computer Vision, ACCV 2024
Country/TerritoryViet Nam
CityHanoi
Period8/12/2412/12/24

Keywords

  • Human pose estimation
  • Training strategy

Fingerprint

Dive into the research topics of 'Learning 2D Human Poses for Better 3D Lifting via Multi-model 3D-Guidance'. Together they form a unique fingerprint.

Cite this