Facial reconstruction

Search LJMU Research Online

Browse Repository | Browse E-Theses

Synaptic 1/f noise injection for overfitting suppression in hardware neural networks

Du, Y, Shao, W, Chai, Z, Zhao, H, Diao, Q, Gao, Y, Yuan, X, Wang, Q, Li, T, Zhang, WD, Zhang, JF and Min, T Synaptic 1/f noise injection for overfitting suppression in hardware neural networks. Neuromorphic Computing and Engineering. (Accepted)

[img]
Preview
Text
Du et al_2022_Neuromorph._Comput._Eng._10.1088_2634-4386_ac6d05.pdf - Accepted Version
Available under License Creative Commons Attribution.

Download (955kB) | Preview

Abstract

Overfitting is a common and critical challenge for neural networks trained with limited dataset. The conventional solution is software-based regularization algorithms such as Gaussian noise injection. Semiconductor noise, such as 1/f noise, in artificial neuron/synapse devices, which is often regarded as undesirable disturbance to the hardware neural networks (HNNs), could also play a useful role in suppressing overfitting, but that is as yet unexplored. In this work, we proposed the idea of using 1/f noise injection to suppress overfitting in different neural networks, and demonstrated that: (i) 1/f noise could suppress the overfitting in Multilayer Perceptron (MLP) and long short-term memory (LSTM); (ii) 1/f noise and Gaussian noise performs similarly for the MLP but differently for the LSTM; (iii) The superior performance of 1/f noise on LSTM can be attributed to its intrinsic long range dependence. This work reveals that 1/f noise, which is common in semiconductor devices, can be a useful solution to suppress the overfitting in HNNs, and more importantly, further evidents that the imperfectness of semiconductor devices is a rich mine of solutions to boost the development of brain-inspired hardware technologies in the AI era.

Item Type: Article
Subjects: T Technology > TA Engineering (General). Civil engineering (General)
Divisions: Engineering
Publisher: IOP Publishing
SWORD Depositor: A Symplectic
Date Deposited: 10 May 2022 11:56
Last Modified: 10 May 2022 12:00
DOI or Identification number: 10.1088/2634-4386/ac6d05
URI: https://researchonline.ljmu.ac.uk/id/eprint/16797

Actions (login required)

View Item View Item