Multi-scale synergy approach for real-time semantic segmentation

In deep convolution neural network based models for semantic segmentation, diverse receptive fields improve the performance by capturing disparate context information. Multiscale inference is good for both thin and large objects. However, the final result is not optimal through averaging or Max pooling combination. In this paper, we propose an approach to take advantage of multi-scale predictions. Our uncertain-pixels part discovers the worse prediction of a low scale and chooses the complement from a high scale. The final output is effectively merged from two scales. We validate our proposed model with a series of experiments on different datasets. The results achieve the accuracy and speed for real-time semantic segmentation. On Cityscapes dataset, our network achieves 76.3 % mIoU at 50 FPS, and on Mapillary, 42.6 % mIoU.