Deep neural network concepts for background subtraction: A systematic review and comparative evaluation

Conventional neural networks have been demonstrated to be a powerful framework for background subtraction in video acquired by static cameras. Indeed, the well-known Self-Organizing Background Subtraction (SOBS) method and its variants based on neural networks have long been the leading methods on the large-scale CDnet 2012 dataset during a long time. Convolutional neural networks, which are used in deep learning, have been recently and excessively employed for background initialization, foreground detection, and deep learned features. The top background subtraction methods currently used in CDnet 2014 are based on deep neural networks, and have demonstrated a large performance improvement in comparison to conventional unsupervised approaches based on multi-feature or multi-cue strategies. Furthermore, since the seminal work of Braham and Van Droogenbroeck in 2016, a large number of studies on convolutional neural networks applied to background subtraction have been published, and a continual gain of performance has been achieved. In this context, we provide the first review of deep neural network concepts in background subtraction for novices and experts in order to analyze this success and to provide further directions. To do so, we first surveyed the background initialization and background subtraction methods based on deep neural networks concepts, and also deep learned features. We then discuss the adequacy of deep neural networks for the task of background subtraction. Finally, experimental results are presented for the CDnet 2014 dataset.