Facial reconstruction

Search LJMU Research Online

Browse Repository | Browse E-Theses

Synaptic 1/f noise injection for overfitting suppression in hardware neural networks

Du, Y, Shao, W, Chai, Z, Zhao, H, Diao, Q, Gao, Y, Yuan, X, Wang, Q, Li, T, Zhang, WD, Zhang, JF and Min, T (2022) Synaptic 1/f noise injection for overfitting suppression in hardware neural networks. Neuromorphic Computing and Engineering, 2.

[img]
Preview
Text
Synaptic 1 f noise injection for overfitting suppression in hardware neural networks.pdf - Published Version
Available under License Creative Commons Attribution.

Download (3MB) | Preview
Open Access URL: https://doi.org/10.1088/2634-4386/ac6d05 (Published version)

Abstract

Overfitting is a common and critical challenge for neural networks trained with limited dataset. The conventional solution is software-based regularization algorithms such as Gaussian noise injection. Semiconductor noise, such as 1/f noise, in artificial neuron/synapse devices, which is often regarded as undesirable disturbance to the hardware neural networks (HNNs), could also play a useful role in suppressing overfitting, but that is as yet unexplored. In this work, we proposed the idea of using 1/f noise injection to suppress overfitting in different neural networks, and demonstrated that: (i) 1/f noise could suppress the overfitting in Multilayer Perceptron (MLP) and long short-term memory (LSTM); (ii) 1/f noise and Gaussian noise performs similarly for the MLP but differently for the LSTM; (iii) The superior performance of 1/f noise on LSTM can be attributed to its intrinsic long range dependence. This work reveals that 1/f noise, which is common in semiconductor devices, can be a useful solution to suppress the overfitting in HNNs, and more importantly, further evidents that the imperfectness of semiconductor devices is a rich mine of solutions to boost the development of brain-inspired hardware technologies in the AI era.

Item Type: Article
Subjects: T Technology > TA Engineering (General). Civil engineering (General)
Divisions: Engineering
Publisher: IOP Publishing
SWORD Depositor: A Symplectic
Date Deposited: 10 May 2022 11:56
Last Modified: 12 Aug 2022 10:30
DOI or ID number: 10.1088/2634-4386/ac6d05
URI: https://researchonline.ljmu.ac.uk/id/eprint/16797
View Item View Item