Self-Organising Neural Networks: Independent Component Analysis and Blind Source Separation

ISBN-13: 9781852330668 / Angielski / Miękka / 1999 / 271 str.

Mark Girolami

Opis

The conception of fresh ideas and the development of new techniques for Blind Source Separation and Independent Component Analysis have been rapid in recent years. It is also encouraging, from the perspective of the many scientists involved in this fascinating area of research, to witness the growing list of successful applications of these methods to a diverse range of practical everyday problems. This growth has been due, in part, to the number of promising young and enthusiastic researchers who have committed their efforts to expanding the current body of knowledge within this field of research. The author of this book is among one of their number. I trust that the present book by Dr. Mark Girolami will provide a rapid and effective means of communicating some of these new ideas to a wide international audience and that in turn this will expand further the growth of knowledge. In my opinion this book makes an important contribution to the theory of Independent Component Analysis and Blind Source Separation. This opens a range of exciting methods, techniques and algorithms for applied researchers and practitioner engineers, especially from the perspective of artificial neural networks and information theory. It has been interesting to see how rapidly the scientific literature in this area has grown.

Spis treści

1. Introduction.- 1.1 Self-Organisation and Blind Signal Processing.- 1.2 Outline of Book Chapters.- 2. Background to Blind Source Separation.- 2.1 Problem Formulation.- 2.2 Entropy and Information.- 2.2.1 Entropy.- 2.2.2 Kullback-Leibler Entropy and Mutual Information.- 2.2.3 Invertible Probability Density Transformations.- 2.3 A Contrast Function for ICA.- 2.4 Cumulant Expansions of Probability Densities and Higher Order Statistics.- 2.4.1 Moment Generating and Cumulant Generating Functions.- 2.4.2 Properties of Moments and Cumulants.- 2.5 Gradient Based Function Optimisation.- 2.5.1 The Natural Gradient and Covariant Algorithms.- 3. Fourth Order Cumulant Based Blind Source Separation.- 3.1 Early Algorithms and Techniques.- 3.2 The Method of Contrast Minimisation.- 3.3 Adaptive Source Separation Methods.- 3.4 Conclusions.- 4. Self-Organising Neural Networks.- 4.1 Linear Self-Organising Neural Networks.- 4.1.1 Linear Hebbian Learning.- 4.1.2 Principal Component Analysis.- 4.1.3 Linear Anti-Hebbian Learning.- 4.2 Non-Linear Self-Organising Neural Networks.- 4.2.1. Non-Linear Anti-Hebbian Learning: The Herrault-Jutten Network.- 4.2.2 Information Theoretic Algorithms.- 4.2.3 Non-Linear Hebbian Learning Algorithms.- 4.2.3.1 Signal Representation Error Minimisation.- 4.2.3.2 Non-Linear Criterion Maximisation.- 4.3 Conclusions.- 5. The Non-Linear PCA Algorithm and Blind Source Separation.- 5.1 Introduction.- 5.2 Non-Linear PCA Algorithm and Source Separation.- 5.3 Non-Linear PCA Algorithm Cost Function.- 5.4 Non-Linear PCA Algorithm Activation Function.- 5.4.1 Asymptotic Stability Requirements.- 5.4.2 Stability Properties of the Compound Activation Function.- 5.4.3 Stability of Solution with Sub-Gaussian Sources.- 5.4.4 Simulation: Separation of Mixtures of Sub-Gaussian Sources.- 5.4.5 Stability of Solution with Super-Gaussian Sources.- 5.4.6 Simulation: Separation of Mixtures of Super-Gaussian Sources.- 5.4.7 Separation of Mixtures of Both Sub-and Super-Gaussian Sources.- 5.5 Conclusions.- 6. Non-Linear Feature Extraction and Blind Source Separation.- 6.1 Introduction.- 6.2 Structure Identification in Multivariate Data.- 6.3 Neural Network Implementation of Exploratory Projection Pursuit.- 6.4 Neural Exploratory Projection Pursuit and Blind Source Separation.- 6.5 Kurtosis Extrema.- 6.6 Finding Interesting and Independent Directions.- 6.7 Finding Multiple Interesting and Independent Directions Using Symmetric Feedback and Adaptive Whitening.- 6.7.1 Adaptive Spatial Whitening.- 6.7.2 Simulations.- 6.7.3 An Extended EPP Network with Non-Linear Output Connections.- 6.8 Finding Multiple Interesting and Independent Directions Using Hierarchic Feedback and Adaptive Whitening.- 6.9 Simulations.- 6.10 Adaptive BSS Using a Deflationary EPP Network.- 6.11 Conclusions.- 7. Information Theoretic Non-Linear Feature Extraction And Blind Source Separation.- 7.1 Introduction.- 7.2 Information Theoretic Indices for EPP.- 7.3 Maximum Negentropy Learning.- 7.3.1 Single Neuron Maximum Negentropy Learning.- 7.3.2 Multiple Output Neuron Maximum Negentropy Learning.- 7.3.3 Maximum Negentropy Learning and Infomax Equivalence.- 7.3.4 The Natural Gradient and Covariant Learning.- 7.4 General Maximum Negentropy Learning.- 7.5 Stability Analysis of Generalised Algorithm.- 7.6 Simulation Results.- 7.7 Conclusions.- 8. Temporal Anti-Hebbian Learning.- 8.1 Introduction.- 8.2 Blind Source Separation of Convolutive Mixtures.- 8.3 Temporal Linear Anti-Hebbian Model.- 8.4 Comparative Simulation.- 8.5 Review of Existing Work on Adaptive Separation of Convolutive Mixtures.- 8.6 Maximum Likelihood Estimation and Source Separation.- 8.7 Temporal Anti-Hebbian Learning Based on Maximum Likelihood Estimation.- 8.8 Comparative Simulations Using Varying PDF Models.- 8.9 Conclusions.- 9. Applications.- 9.1 Introduction.- 9.2 Industrial Applications.- 9.2.1 Rotating Machine Vibration Analysis.- 9.2.2 A Multi-Tag Frequency Identification System.- 9.3 Biomedical Applications.- 9.3.1 Detection of Sleep Spindles in EEG.- 9.4 ICA: A Data Mining Tool.- 9.5 Experimental Results.- 9.5.1 The Oil Pipeline Data.- 9.5.2 The Swiss Banknote Data.- 9.6 Conclusions.- References.