Automatic video completion using spatio-temporal matching of occluding and occluded objects

We propose an automatic video completion technique to restore occluded moving objects in a video sequence. In the video sequence, where two objects cross each other in the presence of camera motion, we automatically remove the occluding object and restore the occluded object using spatial and temporal matching techniques. First, the background motions in the video sequence are registered to a common image coordinate system to extract moving objects. Secondly, spatio-temporal volumes of both occluding and occluded objects are generated by aligning the refined centroid of the objects. Thirdly, a temporal search technique based on voxel matching separates and removes the occluding object from the occluded object. Finally, image holes left on the occluded object are filled by using a spatial search technique. Experimental results using video sequences of real objects show that the proposed technique generates new video sequences after completing occluded moving objects.