Human-oriented video retargeting via object detection and patch decision

In this study, we suggest a novel video retargeting approach by considering the essential factors of a video: main object and movement thereof. Such two factors have been considered including the region of interest (ROI) for target object. Experimentally, we set the main object as human for storing the interaction object and movement in each sequential frame. Our method aims to preserve the ROI to the maximum extent possible over retargeting constraints for the target resolution. With a view to preserving the original main object, we rely on an object detection model to identify human-oriented objects; subsequently, we conduct a decision-making process to determine the suitability of our scheme. Upon the application of the proposed method, video frames are split into many patches and then generated with a precise target resolution using a video super-resolution model. The results of retargeting the frame images are compared against quality assessment metrics. The PSNR, SSIM, MS-SSIM, LPIPS, BMPRI, BRISQUE, PIQE and NIQE were used. We perform comparative experiments to confirm that the proposed approach can maintain the original ratio of important objects and the content of the video. We experimentally demonstrate that the proposed approach could enhance video resolution while ensuring visually pleasing quality and original important object.