HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT

Kwangho Lee; Joonho Kong; Arslan Munir

doi:10.1109/ICDS50568.2020.9268757

HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT

Kwangho Lee
, Joonho Kong
, Arslan Munir

School of Electronics Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

9 Scopus citations

Abstract

Cognitive Internet of things (IoT) is a novel paradigm that outfits the contemporary IoT with a 'brain' to impart high-level intelligence. Convolutional neural networks (CNNs) are an integral part of cognitive IoT that support inference and decision-making. In this paper, we demonstrate a resource-efficient hardware/software (HW/SW) co-design of a CNN architecture for cognitive IoT. We only offload image-tocolumn (im2col) and general matrix multiply (GEMM), which are the most time-and energy-consuming part of convolution layer operations, to the field-programmable gate array (FPGA)-based accelerator. We also exploit the parallelism in the operations of convolution layers to efficiently hide a non-negligible portion of execution time required for bias and activation. Experimental results demonstrate the resource, performance, and energy efficiency of our HW/SW co-design. Results indicate a speedup of 1.3X ∼ 2.0X and energy reduction of 19.4%∼ 44.3% as compared to using only a general-purpose processor.

Original language	English
Title of host publication	4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020
Editors	Youness Oubenaalla, El Habib Nfaoui, Jaouad Joumhidi, Chakir Loqman, Jamal Riffi, Robert Kozma, Mohammed Mestari, Cesari Alippi
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728180847
DOIs	https://doi.org/10.1109/ICDS50568.2020.9268757
State	Published - 21 Oct 2020
Event	4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020 - Virtual, Fez, Morocco Duration: 21 Oct 2020 → 23 Oct 2020

Publication series

Name	4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020

Conference

Conference	4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020
Country/Territory	Morocco
City	Virtual, Fez
Period	21/10/20 → 23/10/20

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 7 Affordable and Clean Energy

Keywords

cognitive engine
Convolutional neural networks
cost efficiency
hardware/software co-design
Internet of things

Access to Document

10.1109/ICDS50568.2020.9268757

Cite this

Lee, K., Kong, J., & Munir, A. (2020). HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT. In Y. Oubenaalla, E. H. Nfaoui, J. Joumhidi, C. Loqman, J. Riffi, R. Kozma, M. Mestari, & C. Alippi (Eds.), 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020 Article 9268757 (4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDS50568.2020.9268757

Lee, Kwangho ; Kong, Joonho ; Munir, Arslan. / HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT. 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020. editor / Youness Oubenaalla ; El Habib Nfaoui ; Jaouad Joumhidi ; Chakir Loqman ; Jamal Riffi ; Robert Kozma ; Mohammed Mestari ; Cesari Alippi. Institute of Electrical and Electronics Engineers Inc., 2020. (4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020).

@inproceedings{452c6f4dbca3480587326fb4ed7f99c2,

title = "HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT",

abstract = "Cognitive Internet of things (IoT) is a novel paradigm that outfits the contemporary IoT with a 'brain' to impart high-level intelligence. Convolutional neural networks (CNNs) are an integral part of cognitive IoT that support inference and decision-making. In this paper, we demonstrate a resource-efficient hardware/software (HW/SW) co-design of a CNN architecture for cognitive IoT. We only offload image-tocolumn (im2col) and general matrix multiply (GEMM), which are the most time-and energy-consuming part of convolution layer operations, to the field-programmable gate array (FPGA)-based accelerator. We also exploit the parallelism in the operations of convolution layers to efficiently hide a non-negligible portion of execution time required for bias and activation. Experimental results demonstrate the resource, performance, and energy efficiency of our HW/SW co-design. Results indicate a speedup of 1.3X ∼ 2.0X and energy reduction of 19.4\%∼ 44.3\% as compared to using only a general-purpose processor.",

keywords = "cognitive engine, Convolutional neural networks, cost efficiency, hardware/software co-design, Internet of things",

author = "Kwangho Lee and Joonho Kong and Arslan Munir",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020 ; Conference date: 21-10-2020 Through 23-10-2020",

year = "2020",

month = oct,

day = "21",

doi = "10.1109/ICDS50568.2020.9268757",

language = "English",

series = "4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

editor = "Youness Oubenaalla and Nfaoui, \{El Habib\} and Jaouad Joumhidi and Chakir Loqman and Jamal Riffi and Robert Kozma and Mohammed Mestari and Cesari Alippi",

booktitle = "4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020",

address = "United States",

}

Lee, K, Kong, J & Munir, A 2020, HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT. in Y Oubenaalla, EH Nfaoui, J Joumhidi, C Loqman, J Riffi, R Kozma, M Mestari & C Alippi (eds), 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020., 9268757, 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020, Institute of Electrical and Electronics Engineers Inc., 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020, Virtual, Fez, Morocco, 21/10/20. https://doi.org/10.1109/ICDS50568.2020.9268757

HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT. / Lee, Kwangho; Kong, Joonho; Munir, Arslan.
4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020. ed. / Youness Oubenaalla; El Habib Nfaoui; Jaouad Joumhidi; Chakir Loqman; Jamal Riffi; Robert Kozma; Mohammed Mestari; Cesari Alippi. Institute of Electrical and Electronics Engineers Inc., 2020. 9268757 (4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT

AU - Lee, Kwangho

AU - Kong, Joonho

AU - Munir, Arslan

PY - 2020/10/21

Y1 - 2020/10/21

N2 - Cognitive Internet of things (IoT) is a novel paradigm that outfits the contemporary IoT with a 'brain' to impart high-level intelligence. Convolutional neural networks (CNNs) are an integral part of cognitive IoT that support inference and decision-making. In this paper, we demonstrate a resource-efficient hardware/software (HW/SW) co-design of a CNN architecture for cognitive IoT. We only offload image-tocolumn (im2col) and general matrix multiply (GEMM), which are the most time-and energy-consuming part of convolution layer operations, to the field-programmable gate array (FPGA)-based accelerator. We also exploit the parallelism in the operations of convolution layers to efficiently hide a non-negligible portion of execution time required for bias and activation. Experimental results demonstrate the resource, performance, and energy efficiency of our HW/SW co-design. Results indicate a speedup of 1.3X ∼ 2.0X and energy reduction of 19.4%∼ 44.3% as compared to using only a general-purpose processor.

AB - Cognitive Internet of things (IoT) is a novel paradigm that outfits the contemporary IoT with a 'brain' to impart high-level intelligence. Convolutional neural networks (CNNs) are an integral part of cognitive IoT that support inference and decision-making. In this paper, we demonstrate a resource-efficient hardware/software (HW/SW) co-design of a CNN architecture for cognitive IoT. We only offload image-tocolumn (im2col) and general matrix multiply (GEMM), which are the most time-and energy-consuming part of convolution layer operations, to the field-programmable gate array (FPGA)-based accelerator. We also exploit the parallelism in the operations of convolution layers to efficiently hide a non-negligible portion of execution time required for bias and activation. Experimental results demonstrate the resource, performance, and energy efficiency of our HW/SW co-design. Results indicate a speedup of 1.3X ∼ 2.0X and energy reduction of 19.4%∼ 44.3% as compared to using only a general-purpose processor.

KW - cognitive engine

KW - Convolutional neural networks

KW - cost efficiency

KW - hardware/software co-design

KW - Internet of things

UR - https://www.scopus.com/pages/publications/85097326411

U2 - 10.1109/ICDS50568.2020.9268757

DO - 10.1109/ICDS50568.2020.9268757

M3 - Conference contribution

AN - SCOPUS:85097326411

T3 - 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020

BT - 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020

A2 - Oubenaalla, Youness

A2 - Nfaoui, El Habib

A2 - Joumhidi, Jaouad

A2 - Loqman, Chakir

A2 - Riffi, Jamal

A2 - Kozma, Robert

A2 - Mestari, Mohammed

A2 - Alippi, Cesari

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020

Y2 - 21 October 2020 through 23 October 2020

ER -

Lee K, Kong J, Munir A. HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT. In Oubenaalla Y, Nfaoui EH, Joumhidi J, Loqman C, Riffi J, Kozma R, Mestari M, Alippi C, editors, 4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020. Institute of Electrical and Electronics Engineers Inc. 2020. 9268757. (4th International Conference on Intelligent Computing in Data Sciences, ICDS 2020). doi: 10.1109/ICDS50568.2020.9268757

HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this