Error bounds for ReLU networks with depth and width parameters

Neural networks have shown high successful performance in a wide range of tasks, but further studies are needed to improve its performance. We construct the specific neural network architecture with a local connection which is universal approximator, and analyze its approximation error. This locally connected networks has higher application than one with the full connection because the locally connected network can be used to explain diverse neural networks such as CNNs. Our error estimate depends on two parameters: one controlling the depth of the hidden layer, and the other, the width of the hidden layers.