Conjugate Gradient Algorithms in Nonconvex Optimization

ISBN-13: 9783642099250 / Angielski / Miękka / 2010 / 478 str.

Radoslaw Pytlak

Opis

Conjugate direction methods were proposed in the early 1950s. When high speed digital computing machines were developed, attempts were made to lay the fo- dations for the mathematical aspects of computations which could take advantage of the ef?ciency of digital computers. The National Bureau of Standards sponsored the Institute for Numerical Analysis, which was established at the University of California in Los Angeles. A seminar held there on numerical methods for linear equationswasattendedbyMagnusHestenes, EduardStiefel andCorneliusLanczos. This led to the ?rst communication between Lanczos and Hestenes (researchers of the NBS) and Stiefel (of the ETH in Zurich) on the conjugate direction algorithm. The method is attributed to Hestenes and Stiefel who published their joint paper in 1952 101] in which they presented both the method of conjugate gradient and the conjugate direction methods including conjugate Gram Schmidt processes. A closelyrelatedalgorithmwasproposedbyLanczos 114]whoworkedonalgorithms for determiningeigenvalues of a matrix. His iterative algorithm yields the similarity transformation of a matrix into the tridiagonal form from which eigenvalues can be well approximated.Thethree-termrecurrencerelationofthe Lanczosprocedurecan be obtained by eliminating a vector from the conjugate direction algorithm scheme. Initially the conjugate gradient algorithm was called the Hestenes Stiefel Lanczos method 86]."

Recenzje

From the reviews:

"The book describes important algorithms for the numerical treatment of unconstrained nonlinear optimization problems with many variables. ... This monograph is suitable as a text for a graduate course in computational optimization. It is useful to anyone active in this field. ... This book is well written and well organized. The argument is clear. Lists of algorithms as well as tables and figures facilitate for the reader the search for desired information in the text. The reference list is comprehensive and contains 214 items." (Sven-Åke Gustafson, Mathematical Reviews, Issue 2009 i)

"It is a very nice written book which can be used by researchers in optimization, in the teaching for seminars and by students ... . Lists of figures, tables and algorithms make this book to a useful compendium for research and teaching. A lot of bibliographical hints with respect to a large reference list make the reader known with the historical development of CG-methods ... . appendices with elements of topology, analysis, linear algebra and numerics of linear algebra make it to a self-contained book." (Armin Hoffmann, Zentralblatt MATH, Vol. 1171, 2009)

Spis treści

Conjugate Direction Methods for Quadratic Problems.- Conjugate Gradient Methods for Nonconvex Problems.- Memoryless Quasi-Newton Methods.- Preconditioned Conjugate Gradient Algorithms.- Limited Memory Quasi-Newton Algorithms.- The Method of Shortest Residuals and Nondifferentiable Optimization.- The Method of Shortest Residuals for Differentiable Problems.- The Preconditioned Shortest Residuals Algorithm.- Optimization on a Polyhedron.- Conjugate Gradient Algorithms for Problems with Box Constraints.- Preconditioned Conjugate Gradient Algorithms for Problems with Box Constraints.- Preconditioned Conjugate Gradient Based Reduced-Hessian Methods.

Blurb

This up-to-date book is on algorithms for large-scale unconstrained and bound constrained optimization. Optimization techniques are shown from a conjugate gradient algorithm perspective.

Large part of the book is devoted to preconditioned conjugate gradient algorithms. In particular memoryless and limited memory quasi-Newton algorithms are presented and numerically compared to standard conjugate gradient algorithms.

The special attention is paid to the methods of shortest residuals developed by the author. Several effective optimization techniques based on these methods are presented.

Because of the emphasis on practical methods, as well as rigorous mathematical treatment of their convergence analysis, the book is aimed at a wide audience. It can be used by researches in optimization, graduate students in operations research, engineering, mathematics and computer science. Practitioners can benefit from numerous numerical comparisons of professional optimization codes discussed in the book.