Synaptic energy drives the information processing mechanisms in spiking  neural networks

Karim El Laithy; Martin Bogdan

doi:10.3934/mbe.2014.11.233

Article Contents

2014, Volume 11, Issue 2: 233-256. Doi: 10.3934/mbe.2014.11.233 open access

This issue Previous Article On a spike train probability model with interacting neural units Next Article Stability and optimal control for some classes of tritrophic systems

Synaptic energy drives the information processing mechanisms in spiking neural networks

Karim El Laithy^1, and
Martin Bogdan^1,

1.
Faculty of Mathematics and Computer Science, Dept. of Computer Engineering, Leipzig University

Received: October 31, 2012

Revised: February 28, 2013

Published: September 2013

Abstract Related Papers Cited by

Abstract

Flow of energy and free energy minimization underpins almost every aspect of naturally occurring physical mechanisms. Inspired by this fact this work establishes an energy-based framework that spans the multi-scale range of biological neural systems and integrates synaptic dynamic, synchronous spiking activity and neural states into one consistent working paradigm. Following a bottom-up approach, a hypothetical energy function is proposed for dynamic synaptic models based on the theoretical thermodynamic principles and the Hopfield networks. We show that a synapse exposes stable operating points in terms of its excitatory postsynaptic potential as a function of its synaptic strength. We postulate that synapses in a network operating at these stable points can drive this network to an internal state of synchronous firing. The presented analysis is related to the widely investigated temporal coherent activities (cell assemblies) over a certain range of time scales (binding-by-synchrony). This introduces a novel explanation of the observed (poly)synchronous activities within networks regarding the synaptic (coupling) functionality. On a network level the transitions from one firing scheme to the other express discrete sets of neural states. The neural states exist as long as the network sustains the internal synaptic energy.

Keywords:

Mathematics Subject Classification: Primary: 58F15, 58F17; Secondary: 53C35.

Citation:

Related Papers

Cited by

References

[1]	A. B. Barrett, G. O. Billings, R. G. M. Morris and M. C. W. van Rossum, State based model of long-term potentiation and synaptic tagging and capture, PLoS Comput. Biol., 5 (2009), e1000259, 12 pp.doi: 10.1371/journal.pcbi.1000259.
[2]	K. El-laithy, Towards a Brain-Inspired Information Processing System: Modeling and Analysis of Synaptic Dynamics, Ph.D thesis, Leipzig University, 2011.
[3]	K. El-Laithy and M. Bogdan, Synchrony state generation in artificial neural networks with stochastic synapses, in Artificial Neural Networks - ICANN 2009, Lecture Notes in Computer Science, 5768, Springer, 2009, 181-190.
[4]	K. El-Laithy and M. Bogdan, Predicting spike-timing of a thalamic neuron using a stochastic synaptic model, in ESANN Proceedings, 2010, 357-362.
[5]	K. El-Laithy and M. Bogdan, A hypothetical free synaptic energy function and related states of synchrony, in Artificial Neural Networks and Machine Learning - ICANN 2011, Lecture Notes in Computer Science, 6792, Springer, 2011, 40-47.
[6]	K. El-Laithy and M. Bogdan, On the capacity of transient internal states in liquid-state machines, in Artificial Neural Networks and Machine Learning - ICANN 2011, Lecture Notes in Computer Science, 6792, Springer, 2011, 56-63.
[7]	K. El-laithy and M. Bogdan, Synchrony state generation: An approach using stochastic synapses, J. of Artificial Intelligence and Soft Computing Research, 1 (2011), 17-26.
[8]	K. El-Laithy and M. Bogdan, Temporal finite-state machines: A novel framework for the general class of dynamic networks, in ICONIP 2012, LNCS, 7664, Springer, 2012, 425-434.
[9]	C. Eliasmith and C. H. Anderson, Neural Engineering. Computation, Representation, and Dynamics in Neurobiological Systems, Computational Neuroscience, A Bradford Book, MIT Press, Cambridge, MA, 2003.
[10]	A. K. Engel, P. Fries, P. Knig, M. Brecht and W. Singer, Temporal binding, binocular rivalry, and consciousness, Consciousness and Cognition, 8 (1999), 128-151.
[11]	A. K. Engel and W. Singer, Temporal binding and the neural correlates of sensory awareness, Trends in Cognitive Sciences, 5 (2001), 16-25.
[12]	K. Friston, The free-energy principle: A rough guide to the brain?, Trends in Cognitive Sciences, 13 (2009), 293-301.
[13]	K. Friston, The free-energy principle: A unified brain theory Nature Reviews Neuroscience, 11 (2010), 127-138.
[14]	H. M. Fuchs, Neural Networks with Dynamic Synapses, Ph.D thesis, Institute of Theoretical Computer Science, Austria, 1998.
[15]	J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Nat. Acad. Sci. U.S.A., 79 (1982), 2554-2558.doi: 10.1073/pnas.79.8.2554.
[16]	Y. Ikegaya, G. Aaron, R. Cossart, D. Aronov, I. Lampl, D. Ferster and R. Yuste, Synfire chains and cortical songs: Temporal modules of cortical activity, Science, 304 (2004), 559-564.doi: 10.1126/science.1093173.
[17]	E. Izhikevich, Which model to use for cortical spiking neurons? IEEE Transactions on Neural Networks, 15 (2004), 1063-1070.doi: 10.1109/TNN.2004.832719.
[18]	E. M. Izhikevich, Polychronization: Computation with spikes, Neural Comput., 18 (2006), 245-282.doi: 10.1162/089976606775093882.
[19]	A. Levina, J. M. Herrmann and T. Geisel, Phase transitions towards criticality in a neural system with adaptive interactions, Physical Review Letters, 102 (2009), 118110, 4 pp.doi: 10.1103/PhysRevLett.102.118110.
[20]	J. M. Montgomery and D. V. Madison, State-dependent heterogeneity in synaptic depression between pyramidal cell pairs, Neuron, 33 (2002), 765-777.doi: 10.1016/S0896-6273(02)00606-2.
[21]	J. M. Montgomery and D. V. Madison, Discrete synaptic states define a major mechanism of synapse plasticity, Trends in Neurosciences, 27 (2004), 744-750.doi: 10.1016/j.tins.2004.10.006.
[22]	A. Morrison, M. Diesmann and W. Gerstner, Phenomenological models of synaptic plasticity based on spike timing, Biol. Cybernet., 98 (2008), 459-478.doi: 10.1007/s00422-008-0233-1.
[23]	P. S. Neelakanta and D. DeGroff, Neural Network Modeling: Statistical Mechanics and Cybernetics Perspectives, CRC Press, 1994.
[24]	A. Revonsuo and J. Newman, Binding and consciousness, Consciousness and Cognition, 8 (1999), 123-127.doi: 10.1006/ccog.1999.0393.
[25]	C. Sarasola, A. d'Anjou, F. J. Torrealdea and M. Graña, Minimization of the energy flow in the synchronization of nonidentical chaotic systems, Phys. Rev. E, 72 (2005), 026223, 6 pp.doi: 10.1103/PhysRevE.72.026223.
[26]	A. P. Shon and R. P. N. Rao, Temporal Sequence Learning with Dynamic Synapses, Technical report, UW CSE, 2003.
[27]	W. Singer, Understanding the brain, EMBO Reports, 8 (2007), S16-S19.doi: 10.1038/sj.embor.7400994.
[28]	S. Stringer, E. Rolls and T. Trappenberg, Self-organizing continuous attractor network models of hippocampal spatial view cells, Neurobiology of Learning and Memory, 83 (2005), 79-92.doi: 10.1016/j.nlm.2004.08.003.
[29]	F. J. Torrealdea, A. d'Anjou, M. Graña and C. Sarasola, Energy aspects of the synchronization of model neurons, Physical Review E, 74 (2006), 011905, 6 pp.doi: 10.1103/PhysRevE.74.011905.
[30]	C. von der Malsburg, The what and why of binding: The modeler's perspective, Neuron, 24 (1999), 95-104.