Relative Pose Estimation between Image Object and ShapeNet CAD Model for Automatic 4-DoF Annotation

Soon Yong Park, Chang Min Son, Won Jae Jeong, Sieun Park

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Estimating the three-dimensional (3D) pose of real objects using only a single RGB image is an interesting and difficult topic. This study proposes a new pipeline to estimate and represent the pose of an object in an RGB image only with the 4-DoF annotation to a matching CAD model. The proposed method retrieves CAD candidates from the ShapeNet dataset and utilizes the pose-constrained 2D renderings of the candidates to find the best matching CAD model. The pose estimation pipeline consists of several steps of learned networks followed by image similarity measurements. First, from a single RGB image, the category and the object region are determined and segmented. Second, the 3-DoF rotational pose of the object is estimated by a learned pose-contrast network only using the segmented object region. Thus, 2D rendering images of CAD candidates are generated based on the rotational pose result. Finally, an image similarity measurement is performed to find the best matching CAD model and to determine the 1-DoF focal length of the camera to align the model with the object. Conventional pose estimation methods employ the 9-DoF pose parameters due to the unknown scale of both image object and CAD model. However, this study shows that only 4-DoF annotation parameters between real object and CAD model is enough to facilitates the projection of the CAD model to the RGB space for image-graphic applications such as Extended Reality. In the experiments, performance of the proposed method is analyzed by using ground truth and comparing with a triplet-loss learning method.

Original languageEnglish
Article number693
JournalApplied Sciences (Switzerland)
Volume13
Issue number2
DOIs
StatePublished - Jan 2023

Keywords

  • 4-DoF annotation
  • CAD retrieval
  • ShapeNet
  • extended reality
  • image similarity
  • pose estimation

Fingerprint

Dive into the research topics of 'Relative Pose Estimation between Image Object and ShapeNet CAD Model for Automatic 4-DoF Annotation'. Together they form a unique fingerprint.

Cite this