Sparse convolutional neural network acceleration with lossless input feature map compression for resource-constrained systems

Jisu Kwon, Joonho Kong, Arslan Munir

Research output: Contribution to journalArticlepeer-review

8 Scopus citations

Abstract

Many recent research efforts have exploited data sparsity for the acceleration of convolutional neural network (CNN) inferences. However, the effects of data transfer between main memory and the CNN accelerator have been largely overlooked. In this work, the authors propose a CNN acceleration technique that leverages hardware/software co-design and exploits the sparsity in input feature maps (IFMs). On the software side, the authors' technique employs a novel lossless compression scheme for IFMs, which are sent to the hardware accelerator via direct memory access. On the hardware side, the authors' technique uses a CNN inference accelerator that performs convolutional layer operations with their compressed data format. With several design optimization techniques, the authors have implemented their technique in a field-programmable gate array (FPGA) system-on-chip platform and evaluated their technique for six different convolutional layers in SqueezeNet. Results reveal that the authors' technique improves the performance by 1.1×–22.6× while reducing energy consumption by 47.7%–97.4% as compared to the CPU-based execution. Furthermore, results indicate that the IFM size and transfer latency are reduced by 34.0%–85.2% and 4.4%–75.7%, respectively, compared to the case without data compression. In addition, the authors' hardware accelerator shows better performance per hardware resource with less than or comparable power consumption to the state-of-the-art FPGA-based designs.

Original languageEnglish
Pages (from-to)29-43
Number of pages15
JournalIET Computers and Digital Techniques
Volume16
Issue number1
DOIs
StatePublished - Jan 2022

Keywords

  • accelerator
  • compression
  • convolutional neural networks
  • field programmable gate array
  • input sparsity

Fingerprint

Dive into the research topics of 'Sparse convolutional neural network acceleration with lossless input feature map compression for resource-constrained systems'. Together they form a unique fingerprint.

Cite this