Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness

Recently, an explicit control of weight sparsity level between the layers in the deep neural network has been proposed and gainfully been utilized to resting-state fMRI (rfMRI) data. However, the reliability of the weight sparsity control scheme via the percentage of non-zero weights (PNZ) was not systematically evaluated in term of the convergence property of the sparsity levels across various scenarios of parameter changes (i.e. learning rates and initial weights). Thus, the primary aim of this study is to systematically evaluate the reliability of the PNZ based sparsity control scheme. In addition, the Hoyer's sparseness (HSP) based on the ratio of L1-and L2-norms was adopted as an alternative option to measure the weight sparsity level. To this end, the whole-brain functional connectivity of the rfMRI data from the Human Connectome Project was used as input of the autoencoder (AE) with the sparsity control scheme via either the PNZ or HSP. Then, the convergence to reach a target sparsity level and converged sparsity levels from the PNZ and HSP based schemes were compared. The presented methods and findings will benefit the training of the (stacked-) AE and/or deep neural network with the weight sparsity control scheme to ease the curse-of-dimensionality issue of very highdimensional neuroimaging data with limited available samples.