Margin learning in spiking neural networks

by Rafael Brune
Doctoral thesis
Date of Examination:2017-12-15
Date of issue:2018-11-29
Advisor:Dr. Robert Gütig
Referee:Prof. Dr. Theo Geisel
Referee:Dr. Robert Gütig
Persistent Address: http://dx.doi.org/10.53846/goediss-7151

 

 

Files in this item

Name:RBrune-Dissertation-Publish.pdf
Size:7.94Mb
Format:PDF
Description:Dissertation

The following license files are associated with this item:

Abstract

English

The ability to learn, generalize and reliably detect features embedded in continuous sensory input streams is a crucial function of the central nervous system. Sensory neurons process input from thousands of synapses and respond to short features embedded in the input spike stream. Although supervised synaptic learning rules that allow neurons to learn and detect spatio-temporal structures in spike patterns have been developed and studied, it is unclear how neurons can learn to generalize when only a limited set of training examples embedded in high-dimensional input patterns are available. Current learning rules rely on the availability of many training patterns. The neurons generalization performance to previously unseen feature variations suffers from overfitting when the number of its synapses is too high and hence limiting their usefulness when studying neural processing of high-dimensional spatio-temporal input streams. We introduce a novel definition of margin for spiking neuron mod- els and a learning rule that extends the multi-spike tempotron with methods to increase said margin during training. We discover that this margin learning ensures high generalization ability even when only a small set of training patterns are available and the number of synapses is high. Using features embedded in Poisson patterns we demonstrate the improvement in performance even under noise. By successfully applying the introduced margin learning rules to human speech recognition tasks we show their potential for studying neural processing of high-dimensional inputs in spiking sensory neurons.
Keywords: supervised learning; neuron model; margin learning; sensory processing; spiking neuron
 
Statistik