Precision Exploration of Floating-Point Arithmetic for Spiking Neural Networks

Myeongjin Kwak; Hyoju Seo; Yongtae Kim

doi:10.1109/ISOCC53507.2021.9614005

Precision Exploration of Floating-Point Arithmetic for Spiking Neural Networks

Myeongjin Kwak, Hyoju Seo, Yongtae Kim

Kyungpook National University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

In this paper, we explore the precision of various floating-point representations for energy-efficient spiking neural network (SNNs). The IEEE 754 based 32-bit single-precision floating-point and reduced precision floating-point formats are applied to the leaky integrate-And-fire (LIF) neuron of the SNN to investigate the impact the reduced precision on the accuracy performance. When adopted in an unsupervised two-layer SNN for the MNIST digit recognition application, the 16-bit floating-point formats can be used in training and inference of the SNN without any classification performance degradation. Additionally, our experimental result reveals that the floating-point format with 4-bit exponent and 6-bit mantissa is enough for the SNN training and inference and offers great area, power, and energy reductions compared to the others.

Original language	English
Title of host publication	Proceedings - International SoC Design Conference 2021, ISOCC 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	71-72
Number of pages	2
ISBN (Electronic)	9781665401746
DOIs	https://doi.org/10.1109/ISOCC53507.2021.9614005
State	Published - 2021
Event	18th International System-on-Chip Design Conference, ISOCC 2021 - Jeju Island, Korea, Republic of Duration: 6 Oct 2021 → 9 Oct 2021

Publication series

Name	Proceedings - International SoC Design Conference 2021, ISOCC 2021

Conference

Conference	18th International System-on-Chip Design Conference, ISOCC 2021
Country/Territory	Korea, Republic of
City	Jeju Island
Period	6/10/21 → 9/10/21

Keywords

floating-point adder
floating-point representation
neuromorphic computing
precision
spiking neural network (SNN)

Access to Document

10.1109/ISOCC53507.2021.9614005

Cite this

Kwak, M., Seo, H., & Kim, Y. (2021). Precision Exploration of Floating-Point Arithmetic for Spiking Neural Networks. In Proceedings - International SoC Design Conference 2021, ISOCC 2021 (pp. 71-72). (Proceedings - International SoC Design Conference 2021, ISOCC 2021). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISOCC53507.2021.9614005

@inproceedings{5746f5d5f13d47038b6f2824a147ff58,

title = "Precision Exploration of Floating-Point Arithmetic for Spiking Neural Networks",

abstract = "In this paper, we explore the precision of various floating-point representations for energy-efficient spiking neural network (SNNs). The IEEE 754 based 32-bit single-precision floating-point and reduced precision floating-point formats are applied to the leaky integrate-And-fire (LIF) neuron of the SNN to investigate the impact the reduced precision on the accuracy performance. When adopted in an unsupervised two-layer SNN for the MNIST digit recognition application, the 16-bit floating-point formats can be used in training and inference of the SNN without any classification performance degradation. Additionally, our experimental result reveals that the floating-point format with 4-bit exponent and 6-bit mantissa is enough for the SNN training and inference and offers great area, power, and energy reductions compared to the others.",

keywords = "floating-point adder, floating-point representation, neuromorphic computing, precision, spiking neural network (SNN)",

author = "Myeongjin Kwak and Hyoju Seo and Yongtae Kim",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 18th International System-on-Chip Design Conference, ISOCC 2021 ; Conference date: 06-10-2021 Through 09-10-2021",

year = "2021",

doi = "10.1109/ISOCC53507.2021.9614005",

language = "English",

series = "Proceedings - International SoC Design Conference 2021, ISOCC 2021",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "71--72",

booktitle = "Proceedings - International SoC Design Conference 2021, ISOCC 2021",

address = "United States",

}

Kwak, M, Seo, H & Kim, Y 2021, Precision Exploration of Floating-Point Arithmetic for Spiking Neural Networks. in Proceedings - International SoC Design Conference 2021, ISOCC 2021. Proceedings - International SoC Design Conference 2021, ISOCC 2021, Institute of Electrical and Electronics Engineers Inc., pp. 71-72, 18th International System-on-Chip Design Conference, ISOCC 2021, Jeju Island, Korea, Republic of, 6/10/21. https://doi.org/10.1109/ISOCC53507.2021.9614005

Precision Exploration of Floating-Point Arithmetic for Spiking Neural Networks. / Kwak, Myeongjin; Seo, Hyoju; Kim, Yongtae.
Proceedings - International SoC Design Conference 2021, ISOCC 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 71-72 (Proceedings - International SoC Design Conference 2021, ISOCC 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Precision Exploration of Floating-Point Arithmetic for Spiking Neural Networks

AU - Kwak, Myeongjin

AU - Seo, Hyoju

AU - Kim, Yongtae

PY - 2021

Y1 - 2021

N2 - In this paper, we explore the precision of various floating-point representations for energy-efficient spiking neural network (SNNs). The IEEE 754 based 32-bit single-precision floating-point and reduced precision floating-point formats are applied to the leaky integrate-And-fire (LIF) neuron of the SNN to investigate the impact the reduced precision on the accuracy performance. When adopted in an unsupervised two-layer SNN for the MNIST digit recognition application, the 16-bit floating-point formats can be used in training and inference of the SNN without any classification performance degradation. Additionally, our experimental result reveals that the floating-point format with 4-bit exponent and 6-bit mantissa is enough for the SNN training and inference and offers great area, power, and energy reductions compared to the others.

AB - In this paper, we explore the precision of various floating-point representations for energy-efficient spiking neural network (SNNs). The IEEE 754 based 32-bit single-precision floating-point and reduced precision floating-point formats are applied to the leaky integrate-And-fire (LIF) neuron of the SNN to investigate the impact the reduced precision on the accuracy performance. When adopted in an unsupervised two-layer SNN for the MNIST digit recognition application, the 16-bit floating-point formats can be used in training and inference of the SNN without any classification performance degradation. Additionally, our experimental result reveals that the floating-point format with 4-bit exponent and 6-bit mantissa is enough for the SNN training and inference and offers great area, power, and energy reductions compared to the others.

KW - floating-point adder

KW - floating-point representation

KW - neuromorphic computing

KW - precision

KW - spiking neural network (SNN)

UR - http://www.scopus.com/inward/record.url?scp=85123342092&partnerID=8YFLogxK

U2 - 10.1109/ISOCC53507.2021.9614005

DO - 10.1109/ISOCC53507.2021.9614005

M3 - Conference contribution

AN - SCOPUS:85123342092

T3 - Proceedings - International SoC Design Conference 2021, ISOCC 2021

SP - 71

EP - 72

BT - Proceedings - International SoC Design Conference 2021, ISOCC 2021

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 18th International System-on-Chip Design Conference, ISOCC 2021

Y2 - 6 October 2021 through 9 October 2021

ER -

Precision Exploration of Floating-Point Arithmetic for Spiking Neural Networks

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this