Statistical Field Theory for Neural Networks

ISBN-13: 9783030464431 / Angielski / Miękka / 2020 / 203 str.

David Dahmen

Spis treści

I. Introduction

II. Probabilities, moments, cumulants

A. Probabilities, observables, and moments

B. Transformation of random variables

C. Cumulants

D. Connection between moments and cumulants

III. Gaussian distribution and Wick’s theorem

A. Gaussian distribution

B. Moment and cumulant generating function of a Gaussian

C. Wick’s theorem

D. Graphical representation: Feynman diagrams

E. Appendix: Self-adjoint operators

F. Appendix: Normalization of a Gaussian

IV. Perturbation expansion

A. General case

B. Special case of a Gaussian solvable theory

C. Example: Example: “phi^3 + phi^4” theory

D. External sources

E. Cancellation of vacuum diagrams

F. Equivalence of graphical rules for n-point correlation and n-th moment

G. Example: “phi^3 + phi^4” theory

V. Linked cluster theorem

A. General proof of the linked cluster theorem

B. Dependence on j - external sources - two complimentary views

C. Example: Connected diagrams of the “phi^3 + phi^4” theory

VI. Functional preliminaries

A. Functional derivative

1. Product rule

2. Chain rule

3. Special case of the chain rule: Fourier transform

B. Functional Taylor series

VII. Functional formulation of stochastic differential equations

A. Onsager-Machlup path integral*

B. Martin-Siggia-Rose-De Dominicis-Janssen (MSRDJ) path integral

C. Moment generating functional

D. Response function in the MSRDJ formalism

VIII. Ornstein-Uhlenbeck process: The free Gaussian theory

A. Definition

B. Propagators in time domain

C. Propagators in Fourier domain

IX. Perturbation theory for stochastic differential equations

A. Vanishing moments of response fields

B. Vanishing response loops

C. Feynman rules for SDEs in time domain and frequency domain

D. Diagrams with more than a single external leg

E. Appendix: Unitary Fourier transform

X. Dynamic mean-field theory for random networks

A. Definition of the model and generating functional

B. Property of self-averaging

C. Average over the quenched disorder

D. Stationary statistics: Self-consistent autocorrelation of as motion of a particle in a potential

E. Transition to chaos

F. Assessing chaos by a pair of identical systems

G. Schrödinger equation for the maximum Lyapunov exponent

H. Condition for transition to chaos

XI. Vertex generating function

A. Motivating example for the expansion around a non-vanishing mean value

B. Legendre transform and definition of the vertex generating function Gamma

C. Perturbation expansion of Gamma

D. Generalized one-line irreducibility

E. Example

F. Vertex functions in the Gaussian case

G. Example: Vertex functions of the “phi^3 + phi^4”-theory

H. Appendix: Explicit cancellation until second order

I. Appendix: Convexity of W

J. Appendix: Legendre transform of a Gaussian

XII. Application: TAP approximation

Inverse problem

XIII. Expansion of cumulants into tree diagrams of vertex functions

A. Self-energy or mass operator Sigma

XIV. Loopwise expansion of the effective action - Tree level

A. Counting the number of loops

B. Loopwise expansion of the effective action - Higher numbers of loops

C. Example: phi^3 + phi^4-theory

D. Appendix: Equivalence of loopwise expansion and infinite resummation

E. Appendix: Interpretation of Gamma as effective action

F. Loopwise expansion of self-consistency equation

XV. Loopwise expansion in the MSRDJ formalism

A. Intuitive approach

B. Loopwise corrections to the effective equation of motion

C. Corrections to the self-energy and self-consistency

D. Self-energy correction to the full propagator

E. Self-consistent one-loop

F. Appendix: Solution by Fokker-Planck equation

XVI. Nomenclature

Acknowledgments

References

O autorze

Moritz Helias is group leader at the Jülich Research Centre and assistant professor in the department of physics of the RWTH Aachen University, Germany. He obtained his diploma in theoretical solid state physics at the University of Hamburg and his PhD in computational neuroscience at the University of Freiburg, Germany. Post-doctoral positions in RIKEN Wako-Shi, Japan and Jülich Research Center followed. His main research interests are neuronal network dynamics and function, and their quantitative analysis with tools from statistical physics and field theory.

David Dahmen is a post-doctoral researcher in the Institute of Neuroscience and Medicine at the Jülich Research Centre, Germany. He obtained his Master's degree in physics from RWTH Aachen University, Germany, working on effective field theory approaches to particle physics. Afterwards he moved to the field of computational neuroscience, where he received his PhD in 2017. His research comprises modeling, analysis and simulation of recurrent neuronal networks with special focus on development and knowledge transfer of mathematical tools and simulation concepts. His main interests are field-theoretic methods for random neural networks, correlations in recurrent networks, and modeling of the local field potential.

Blurb

This book presents a self-contained introduction to techniques from field theory applied to stochastic and collective dynamics in neuronal networks. These powerful analytical techniques, which are well established in other fields of physics, are the basis of current developments and offer solutions to pressing open problems in theoretical neuroscience and also machine learning. They enable a systematic and quantitative understanding of the dynamics in recurrent and stochastic neuronal networks.

This book is intended for physicists, mathematicians, and computer scientists and it is designed for self-study by researchers who want to enter the field or as the main text for a one semester course at advanced undergraduate or graduate level. The theoretical concepts presented in this book are systematically developed from the very beginning, which only requires basic knowledge of analysis and linear algebra.