Project topics and materials
Get Complete Project Material File(s) Now! »
Articial neural networks
Articial neural networks have been studied since the 40s. Some of the proposed arti- cial neural networks build on biological ndings and therefore, have a certain degree of biological realism (or plausibility). Such theoretical models may help to understand the human brain and, for instance, allow developing new methods to treat neurological disorders. In other cases, the objective is to construct highly parallel information-processing systems, somewhat inspired from biological neural networks. Therefore, abstract models are proposed where biological plausibility is not the issue in hand. Such biologically inspired information-processing systems are more eective than classical algorithms in a certain group of applications.
Since articial neural networks form a broad research eld, we present a selection of a few models. Firstly, we introduce the historically important model (McCulloch-Pitts model). Next, we describe three models that have dierent objectives: 1) memorize (Hopeld Neural Networks), 2) model and compute (Spiking Neural Networks), 3) learn (Deep learning).
McCulloch-Pitts model
Some of the earliest research comes from McCulloch and Pitts resulting in the proposal of McCulloch-Pitts networks (Pitts and McCulloch, 1947). It is assumed that the synapses are represented with connections between the neurons and that they are unidirectional, i.e. they transmit signals in a predetermined direction. For McCulloch-Pitts networks there is no limit on the number of connections going out from a neuron. This property is called the unlimited fan-in. Figure 2.3 depicts a generic neuron model. The operation of a neuron consists in two functions: an integration function g and an activation function a. The integration function g reduces the q signals v received at the input of the neuron to a single value. This single value is used as an argument to the activation function a that computes the output of the neuron. In McCulloch-Pitts networks the connections transmit only ones or zeros, that is they are unweighted. The connections are of excita- tory or inhibitory type. To distinguish the type of connections, excitatory connections are labeled with v and inhibitory connections with . The integration function g is the sum of the input signals: g(v) = Xq i=1 vi.
Hopeld Neural Networks – memorize information
Since the neocortex operation relies on associations between concepts, articial neural networks for associative memory are also proposed. The concept of associative memory is introduced in Section 1.7. The most prominent model in this category is proposed by Hopeld in (Hopeld, 1982). Throughout this document, we consider that neuroinspired associative memories store messages (also called patterns) that they are later capable of retrieving given a suciently large part of their content. The message definition is detailed in the following section. Hopeld Neural Networks (HNNs) consist of neurons that are all interconnected with each other except being connected to itself. The connections are weighted and their values are restricted to integers. Each neuron in the network of n neurons has its index. Supposing that the set of messages to store contains M messages m1;m2; :::;mM the weight between neurons i and j is obtained as follows: Therefore, the messages are projected onto the connection weights. HNNs can store messages of length n.
Deep learning – learn information
Currently, in many domains (e.g. vision) state-of-the-art methods for learning are based on deep learning. Deep learning allowes adapting weights of neural networks with multiple layers, typically ten hidden layers (hidden layer is a layer that is used neither as input nor output of the network). Networks made of multiple hidden layers prove to be more ecient than shallow networks (with a single hidden layer) (Bengio, 2009). Moreover, a network with a depth insucient for the targeted task requires more neurons than a network with a depth matched to the problem. In addition, some studies show that the human brain processes information through multiple stages of transformation and representation, i.e. a type of deep architecture (Serre et al., 2007).
Table of contents :
Abstract
Acknowledgements
Contents
List of Figures
List of Tables
Abbreviations
Symbols
Introduction
Context and motivation
Objective
Contribution
Report organization
1 MPSoC power management
1.1 Introduction
1.2 MPSoC architecture
1.2.1 Generalities and denitions
1.2.2 Communication schemes for MPSoCs
1.2.3 Dividing MPSoC on Voltage/Frequency Islands
1.3 Power management on MPSoC
1.4 State-of-the-art of power management decision units
1.4.1 Low-level decision units
1.4.2 High-level decision units
1.5 MPSoC power model and optimization formulation
1.6 Game theory for power management on MPSoC
1.7 CAM-SRAM associative memory for power management on MPSoC
1.7.1 Generalities and denitions
1.7.2 CAM-SRAM as a decision unit
1.8 Conclusion
2 Introduction to neural networks and networks of neural cliques
2.1 Introduction
2.2 Biological neural networks
2.3 Articial neural networks
2.3.1 McCulloch-Pitts model
2.3.2 Hopeld Neural Networks – memorize information
2.3.3 Spiking Neural Networks – model biological networks and compute
2.3.4 Deep learning – learn information
2.4 Networks of neural cliques
2.4.1 Message denition
2.4.2 Network structure
2.4.3 Message storing procedure
2.4.4 Message retrieval procedure
2.4.5 Density and error probability denitions
2.4.6 Neural cliques as associative memory
2.4.7 Network dimensioning guidelines
2.5 Conclusion
3 Non-uniformly distributed data in networks of neural cliques
3.1 Introduction
3.2 Non-uniform distribution problem positioning
3.3 Strategies to store non-uniform data
3.3.1 Random clusters
3.3.2 Random bits
3.3.3 Using compression codes
3.3.4 Performance comparison
3.4 Twin neurons for ecient real-world data distribution in networks of neural cliques
3.4.1 Introducing twin neurons
3.4.2 Theoretical analysis
3.4.3 Performance comparison
3.4.3.1 Comments on Human coding technique
3.4.3.2 Comparison
3.4.4 Inuence of distribution’s standard deviation
3.5 Real-world data in two practical applications
3.5.1 MPSoC power management for LTE receiver
3.5.1.1 LTE receiver implemented on MAGALI platform
3.5.1.2 Network of neural cliques used as power management unit
3.5.1.3 Simulation results
3.5.2 Dynamic management of PVT variations
3.5.2.1 Introduction
3.5.2.2 Multiprobe sensor for PVT variations
3.5.2.3 Network of neural cliques used as dynamic management unit
3.5.2.4 Network of neural cliques dimensions
3.5.2.5 Simulation results
3.6 Conclusion
4 Hardware neural cliques in practical applications
4.1 Introduction
4.2 Analog and digital ASIC implementation
4.2.1 Analog circuit
4.2.2 Digital circuit
4.2.3 Comparison
4.3 Hardware 3D considerations
4.3.1 General introduction to 3D neural networks
4.3.2 3D technology
4.3.3 3D neural cliques
4.3.4 Methodology
4.3.5 Simulation model
4.3.6 General study results
4.3.7 Case study simulation results
4.4 MPSoC power management: comparison with game theory decision unit
4.4.1 Generic neural cliques structure
4.4.2 General comparison with game theory decision unit
4.4.3 MPSoC power management for MC-CDMA transmitter . .
4.4.3.1 MC-CDMA transmitter implemented on FAUST platform
4.4.3.2 Network of neural cliques used as power management unit
4.4.3.3 Energy gains
4.5 MPSoC power management: comparison with CAM-SRAM associative memory
4.5.1 Neural cliques-based associative memory – implementation complexity
4.5.2 CAM-based associative memory – implementation complexity
4.5.3 Implementation complexity comparison
4.5.4 LTE receiver implemented on MAGALI platform
4.5.4.1 Dimensions of CAM and SRAM
4.5.4.2 Dimensions of neural cliques
4.5.4.3 Simulation results
4.6 Conclusion
Conclusion and perspectives
Contribution and conclusion
Perspectives
Implementation
Applications
A Process variability in neural cliques analog circuits
B Programming the synapses
List of Publications
Bibliography
