The Maximum Entropy Method

ISBN-13: 9783642644849 / Angielski / Miękka / 2011 / 327 str.

Nailong Wu

Opis

Forty years ago, in 1957, the Principle of Maximum Entropy was first intro duced by Jaynes into the field of statistical mechanics. Since that seminal publication, this principle has been adopted in many areas of science and technology beyond its initial application. It is now found in spectral analysis, image restoration and a number of branches ofmathematics and physics, and has become better known as the Maximum Entropy Method (MEM). Today MEM is a powerful means to deal with ill-posed problems, and much research work is devoted to it. My own research in the area ofMEM started in 1980, when I was a grad uate student in the Department of Electrical Engineering at the University of Sydney, Australia. This research work was the basis of my Ph.D. the sis, The Maximum Entropy Method and Its Application in Radio Astronomy, completed in 1985. As well as continuing my research in MEM after graduation, I taught a course of the same name at the Graduate School, Chinese Academy of Sciences, Beijingfrom 1987to 1990. Delivering the course was theimpetus for developing a structured approach to the understanding of MEM and writing hundreds of pages of lecture notes."

Spis treści

1. Introduction.- 1.1 What is the Maximum Entropy Method.- 1.2 Definition of Entropy.- 1.3 Rationale of the Maximum Entropy Method.- 1.4 Present and Future Research.- 2. Maximum Entropy Method MEM1 and Its Application in Spectral Analysis.- 2.1 Definition and Expressions of Entropy H1.- 2.1.1 Approach 1.- 2.1.2 Approach 2.- 2.1.3 Discussion.- 2.2 Formulation and Solution.- 2.2.1 Formulation.- 2.2.2 Solution.- 2.2.3 Discussion.- 2.3 Equivalents and Signal Model.- 2.3.1 ACF Extension Subject to the Nonnegativity Constraint.- 2.3.2 Principle of MCE.- 2.3.3 AR Process (Signal Model).- 2.3.4 Bayesian Method.- 2.3.5 Wiener Filter and Approximation Theoretic Approach.- 2.4 Algorithms and Numerical Example (Given ACF).- 2.4.1 Levinson’s Recursion for 1-D Noiseless Data.- 2.4.2 Lim-Malik Algorithm for 2-D Noiseless Data.- 2.4.3 Wernecke-D’Addario Algorithm for 2-D Noisy Data.- 2.4.4 Numerical Example.- 2.5 Algorithms and Numerical Example (Given Time Series).- 2.5.1 Burg Algorithm.- 2.5.2 Marple Algorithm.- 2.5.3 Other Fast Algorithms.- 2.5.4 Numerical Example.- 2.6 Order Selection.- 2.6.1 FPE Criterion.- 2.6.2 AIC Criterion.- 2.6.3 Other Criteria.- 2.6.4 Summary.- 3. Maximum Entropy Method MEM2 and Its Application in Image Restoration.- 3.1 Definition and Expressions of Entropy H2.- 3.1.1 MLM.- 3.1.2 Direct Definition Method.- 3.1.3 Discussion.- 3.2 Formulation and Implicit Solution.- 3.2.1 Formulation.- 3.2.2 Implicit Solution.- 3.2.3 Iterative Algorithm.- 3.2.4 Discussion.- 3.3 Explicit Solution.- 3.3.1 Explicit Solution.- 3.3.2 Discussion.- 3.3.3 Examples.- 3.4 Equivalents and Signal Model.- 3.4.1 ACF Extension Subject to the Nonnegativity Constraint.- 3.4.2 Principle of MCE.- 3.4.3 Exponential Process (Signal Model).- 3.4.4 Bayesian Method.- 3.4.5 MLM.- 3.5 R – ? Procedure.- 3.5.1 Statements of the MEM2 Problem.- 3.5.2 R – ? Procedure.- 3.5.3 Example.- 3.6 Algorithms and Numerical Examples (I).- 3.6.1 Frieden Algorithm.- 3.6.2 Gull-Daniell Algorithm.- 3.6.3 Revised GD Algorithm.- 3.6.4 Simplified Newton-Raphson Algorithm.- 3.6.5 Numerical Example.- 3.7 Algorithms and Numerical Examples (II).- 3.7.1 Skilling-Bryan Algorithm.- 3.7.2 Differential Equation Approach.- 3.8 Algorithms and Numerical Examples (III).- 3.8.1 MEM/MemSys5 Package.- 3.8.2 MEM Task in IRAF.- 3.8.3 Restoration with Variable Resolution.- 3.8.4 Numerical Examples.- 3.8.5 Other Algorithms.- 4. Analysis and Comparison of the Maximum Entropy Method.- 4.1 Generalized MEM.- 4.1.1 Formulation of GMEM.- 4.1.2 “Entropy” Expressions in GMEM.- 4.1.3 Properties of GMEM.- 4.2 Expressions of Entropy.- 4.3 Solution’s Properties.- 4.3.1 Existence.- 4.3.2 Uniqueness.- 4.3.3 Consistency.- 4.3.4 Statistical Properties.- 4.4 Resolution Enhancement and Data Extension (Experimental Results).- 4.4.1 Examples.- 4.4.2 Resolvability in 1-D Spectral Estimation.- 4.4.3 Resolvability in 2-D Spectral Estimation.- 4.4.4 Super resolut ion and Spectral Line Splitting.- 4.5 Resolution Enhancement and Data Extension (Theoretical Analysis).- 4.5.1 Data Extension in MEM1 and MEM2.- 4.5.2 Resolution Enhancement of MEM1 and MEM2.- 4.5.3 MEM1 and MEM2 Spectra at Low SNR.- 4.5.4 Line Splitting of MEM1.- 4.6 Peak Location and Relative Power Estimation (Experimental Results).- 4.6.1 Peak Location (Given ACF).- 4.6.2 Peak Location (Given Time Series).- 4.6.3 Relative Power Estimation (Given ACF).- 4.6.4 Summary and Comments.- 4.7 Peak Location and Relative Power Estimation (Theoretical Analysis).- 4.7.1 Interference Between Peaks Causes Peak Shifting.- 4.7.2 Explanation of the Peak Shifting in MEMI Spectra.- 4.7.3 Relative Power Estimation for MEMI.- 4.7.4 Summary for Sects. 4.4–4.7.- 4.8 Comments on the Three Schools of Thought on MEM.- 5. Applications of the Maximum Entropy Method in Mathematics and Physics.- 5.1 Solution of Moment Problems.- 5.1.1 General Theory.- 5.1.2 Numerical Methods.- 5.1.3 Noisy Moment Problems.- 5.1.4 Numerical Examples.- 5.2 Solution of Integral Equations.- 5.2.1 Conversion of Integral Equations to Moment Problems.- 5.2.2 Solution of Moment Problems by MEM.- 5.2.3 Numerical Examples.- 5.2.4 Discussion.- 5.3 Solution of Partial Differential Equations.- 5.3.1 Theory.- 5.3.2 Numerical Example.- 5.3.3 Discussion.- 5.4 Predictive Statistical Mechanics.- 5.4.1 Formulation and Solution.- 5.4.2 Useful Formulae.- 5.5 Distributions of Particles Among Energy Levels.- 5.5.1 Boltzmann Distribution.- 5.5.2 Fermi-Dirac and Bose-Einstein Distributions.- 5.6 Classical Statistical Ensembles.- 5.6.1 Microcanonical Ensemble.- 5.6.2 Canonical Ensemble.- 5.6.3 Grand Canonical Ensemble.- 5.7 Quantum Statistical Ensembles.- 5.7.1 Microcanonical Ensemble.- 5.7.2 Canonical Ensemble.- 5.7.3 Grand Canonical Ensemble.- Appendices.- A. Cepstral Analysis.- A.1 Cepstral Analysis System.- A.2 I/O Relationship.- A.3 Properties of the Complex Cepstrum.- A.4 I/O Relationship for Minimum-Phase Input.- B. Image Restoration.- B.1 Image Formation.- B.2 Image Restoration.- B.3 Relationship Between Image Restoration and Spectral Estimation.- References.

Blurb

The Maximum Entropy Method addresses the principle and applications of the powerful maximum entropy method (MEM), which has its roots in the principle of maximum entropy introduced into the field of statistical mechanics almost 40 years ago. This method has since been adopted in many areas of science and technology, such as spectral analysis, image restoration, mathematics, and physics. Readers of this monograph are lead to current research frontiers through the analysis and comparison of three schools of thought in MEM research. The step-by-step approach and the detailed examples make this an invaluable textbook for graduate students. The detailed practical algorithms will also appeal to scientists and engineers using this book as a reference work.