Freitas, P (2023) Simulation and programming strategies to mitigate device non-idealities in memristor based neuromorphic systems. Doctoral thesis, Liverpool John Moores University.

Preview

Text 2023freitasphd.pdf - Accepted Version Available under License Creative Commons Attribution Non-commercial. Download (18MB) | Preview

Abstract

Since its inception, resistive random access memory (RRAM) has widely been regarded as a promising technology, not only for its potential to revolutionize non-volatile data storage by bridging the speed gap between traditional solid state drives (SSD) and dynamic random access memory (DRAM), but also for the promise it brings to in-memory and neuromorphic computing.
Despite the potential, the design process of RRAM neuromorphic arrays still finds itself in its infancy, as reliability (retention, endurance, programming linearity) and variability (read-to-read, cycle-to-cycle and device-to-device) issues remain major hurdles for the mainstream implementation of these systems.
One of the fundamental stages of neuromorphic design is the simulation stage. In this thesis, a simulation framework for evaluating the impact of RRAM non-idealities on NNs, that emphasizes flexibility and experimentation in NN topology and RRAM programming conditions is coded in MATLAB, making full use of its various toolboxes.
Using these tools as the groundwork, various RRAM non-idealities are comprehensively measured and their impact on both inference and training accuracy of a pattern recognition system based on the MNIST handwritten digits dataset are simulated.
In the inference front, variability originated from different sources (read-to-read and programming-to-programming) are statistically evaluated and modelled for two different device types: filamentary and non-filamentary. Based on these results, the impact of various variability sources on inference are simulated and compared, showing much more pronounced variability in the filamentary device compared to its non-filamentary counterpart. The staged programming scheme is introduced as a method to improve linearity and reduce programming variability, leading to negligible accuracy loss in non-filamentary devices. Random telegraph noise (RTN) remains the major source of read variability in both devices. These results can be explained by the difference in switching mechanisms of both devices.
In training, non-idealities such as conductance stepping and cycle-to-cycle variability are characterized and their impact on the training of NNs based on backpropagation are independently evaluated. Analysing the change of weight distributions during training reveals the different impacts on the SET and RESET processes. Based on these findings, a new selective programming strategy is introduced for the suppression of non-idealities impact on accuracy. Furthermore, the impact of these methods are analysed between different NN topologies, including traditional multi-layer perceptron (MLP) and convolutional neural network (CNN) configurations.
Finally, the new dynamic weight range rescaling methodology is introduced as a way of not only alleviating the constraints imposed in hardware due to the limited conductance range of RRAM in training, but also as way of increasing the flexibility of RRAM based deep synaptic layers to different sets of data.

Item Type:	Thesis (Doctoral)
Uncontrolled Keywords:	RRAM; resistive switching; neural network; non-ideality; variability; pattern recognition; analog; training
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:	Engineering
SWORD Depositor:	A Symplectic
Date Deposited:	12 Oct 2023 13:24
Last Modified:	12 Oct 2023 13:24
DOI or ID number:	10.24377/LJMU.t.00021672
Supervisors:	Zhang, W
URI:	https://researchonline.ljmu.ac.uk/id/eprint/21672

View Item