Exploiting Output Activation Sparsity Using Bit-Separable Multiplier in CNN Accelerator

This paper demonstrates a bit-separable multiplier (BSM) in CNN accelerators to leverage output activation sparsity. BSM improves computational efficiency by predicting the output of activation functions with sparsity, such as ReLU, using only the upper bits of the weight and skipping unnecessary lower-bit computations. In particular, using BSM resulted in a 25% improvement in inference speed with only a 0.07% accuracy drop across various CNN models. Proposed CNN accelerator was fabricated using a commercial 130nm process and achieved 3 TOPS/W. For MobileNet, the proposed architecture improved processing speed by 26% and power consumption by 28%. BSM presents a novel approach to exploiting output activation sparsity.