Pattern Classification: Neuro-Fuzzy Methods and Their Comparison

ISBN-13: 9781447110774 / Angielski / Miękka / 2012 / 327 str.

Shigeo Abe

Opis

Neural networks have a learning capability but analysis of a trained network is difficult. On the other hand, extraction of fuzzy rules is difficult but once they have been extracted, it is relatively easy to analyze the fuzzy system. This book solves the above problems by developing new learning paradigms and architectures for neural networks and fuzzy systems.
The book consists of two parts: Pattern Classification and Function Approximation. In the first part, based on the synthesis principle of the neural-network classifier: A new learning paradigm is discussed and classification performance and training time of the new paradigm for several real-world data sets are compared with those of the widely-used back-propagation algorithm; Fuzzy classifiers of different architectures based on fuzzy rules can be defined with hyperbox, polyhedral, or ellipsoidal regions. The book discusses the unified approach for training these fuzzy classifiers; The performance of the newly-developed fuzzy classifiers and the conventional classifiers such as nearest-neighbor classifiers and support vector machines are evaluated using several real-world data sets and their advantages and disadvantages are clarified.
In the second part: Function approximation is discussed extending the discussions in the first part; Performance of the function approximators is compared.
This book is aimed primarily at researchers and practitioners in the field of artificial intelligence and neural networks.

Spis treści

I. Pattern Classification.- 1. Introduction.- 1.1 Development of a Classification System.- 1.2 Optimum Features.- 1.3 Classifiers.- 1.3.1 Neural Network Classifiers.- 1.3.2 Conventional Fuzzy Classifiers.- 1.3.3 Fuzzy Classifiers with Learning Capability.- 1.4 Evaluation.- 1.5 Data Sets Used in the Book.- 2. Multilayer Neural Network Classifiers.- 2.1 Three-layer Neural Networks.- 2.2 Synthesis Principles.- 2.3 Training Methods.- 2.4 Training by the Back-propagation Algorithm.- 2.5 Training by Solving Inequalities.- 2.5.1 Setting of Target Values.- 2.5.2 Formulation of Training by Solving Inequalities.- 2.5.3 Determination of Weights by Solving Inequalities.- 2.6 Performance Evaluation.- 2.6.1 Iris Data.- 2.6.2 Numeral Data.- 2.6.3 Thyroid Data.- 2.6.4 Blood Cell Data.- 2.6.5 Hiragana Data.- 2.6.6 Discussions.- 3. Support Vector Machines.- 3.1 Support Vector Machines for Pattern Classification.- 3.1.1 Conversion to Two-class Problems.- 3.1.2 The Optimal Hyperplane.- 3.1.3 Mapping to a High-dimensional Space.- 3.2 Performance Evaluation.- 3.2.1 Iris Data.- 3.2.2 Numeral Data.- 3.2.3 Thyroid Data.- 3.2.4 Blood Cell Data.- 3.2.5 Hiragana Data.- 3.2.6 Discussions.- 4. Membership Functions.- 4.1 One-dimensional Membership Functions.- 4.1.1 Triangular Membership Functions.- 4.1.2 Trapezoidal Membership Functions.- 4.1.3 Bell-shaped Membership Functions.- 4.2 Multi-dimensional Membership Functions.- 4.2.1 Extension to Multi-dimensional Membership Functions.- 4.2.2 Rectangular Pyramidal Membership Functions.- 4.2.3 Truncated Rectangular Pyramidal Membership Functions.- 4.2.4 Polyhedral Pyramidal Membership Functions.- 4.2.5 Truncated Polyhedral Pyramidal Membership Functions.- 4.2.6 Bell-shaped Membership Functions.- 4.2.7 Relations between Membership Functions.- 5. Static Fuzzy Rule Generation.- 5.1 Classifier Architecture.- 5.2 Fuzzy Rules.- 5.2.1 Fuzzy Rules with Pyramidal Membership Functions.- 5.2.2 Polyhedral Fuzzy Rules.- 5.2.3 Ellipsoidal Fuzzy Rules.- 5.3 Class Boundaries.- 5.3.1 Fuzzy Rules with Pyramidal Membership Functions.- 5.3.2 Ellipsoidal Fuzzy Rules.- 5.3.3 Class Boundaries for the Iris Data.- 5.4 Training Architecture.- 5.4.1 Fuzzy Rule Generation by Preclustering.- 5.4.2 Fuzzy Rule Generation by Postclustering.- 6. Clustering.- 6.1 Fuzzy c-means Clustering Algorithm.- 6.2 The Kohonen Network.- 6.3 Minimum Volume Clustering Algorithm.- 6.4 Fuzzy Min-max Clustering Algorithm.- 6.5 Overlap Resolving Clustering Algorithm.- 6.5.1 Approximation of Overlapping Regions.- 6.5.2 Extraction of Data from the Overlapping Regions.- 6.5.3 Clustering Algorithm.- 7. Tuning of Membership Functions.- 7.1 Problem Formulation.- 7.2 Direct Methods.- 7.2.1 Tuning of Slopes.- 7.2.2 Tuning of Locations.- 7.2.3 Order of Tuning.- 7.3 Indirect Methods.- 7.3.1 Tuning of Slopes Using the Least-squares Method.- 7.3.2 Tuning by the Steepest Descent Method.- 7.4 Performance Evaluation.- 7.4.1 Performance Evaluation of the Fuzzy Classifier with Pyramidal Membership Functions.- 7.4.2 Performance Evaluation of the Fuzzy Classifier with Polyhedral Regions.- 7.4.3 Performance Evaluation of the Fuzzy Classifier with Ellipsoidal Regions.- 8. Robust Pattern Classification.- 8.1 Why Robust Classification Is Necessary?.- 8.2 Robust Classification.- 8.2.1 The First Stage.- 8.2.2 The Second Stage.- 8.2.3 Tuning Slopes near Class Boundaries.- 8.2.4 Upper and Lower Bounds Determined by Correctly Classified Data.- 8.2.5 Range of the Interclass Tuning Parameter that Resolves Misclassification.- 8.3 Performance Evaluation.- 8.3.1 Classification Performance without Outliers.- 8.3.2 Classification Performance with Outliers.- 9. Dynamic Fuzzy Rule Generation.- 9.1 Fuzzy Min-max Classifiers.- 9.1.1 Concept.- 9.1.2 Approximation of Input Regions.- 9.1.3 Fuzzy Rule Extraction.- 9.1.4 Performance Evaluation.- 9.2 Fuzzy Min-max Classifiers with Inhibition.- 9.2.1 Concept.- 9.2.2 Fuzzy Rule Extraction.- 9.2.3 Fuzzy Rule Inference.- 9.2.4 Performance Evaluation.- 10. Comparison of Classifier Performance.- 10.1 Evaluation Conditions.- 10.2 Iris Data.- 10.3 Numeral Data.- 10.4 Thyroid Data.- 10.5 Blood Cell Data.- 10.6 Hiragana Data.- 10.7 Discussions.- 11. Optimizing Features.- 11.1 Scaling Features.- 11.1.1 Scale and Translation Invariance of the Fuzzy Classifier with Pyramidal Membership Functions.- 11.1.2 Invariance of the Fuzzy Classifiers with Hyperbox Regions.- 11.1.3 Invariance of the Fuzzy Classifier with Ellipsoidal Regions.- 11.2 Feature Extraction.- 11.2.1 Principal Component Analysis.- 11.2.2 Discriminant Analysis.- 11.2.3 Neural-network-based Feature Extraction.- 11.3 Feature Selection.- 11.3.1 Criteria for Feature Selection.- 11.3.2 Monotonicity of the Selection Criteria.- 11.3.3 Selection Search.- 11.4 Performance Evaluation of Feature Selection.- 11.4.1 Evaluation Conditions.- 11.4.2 Selected Features.- 11.4.3 Iris Data.- 11.4.4 Numeral Data.- 11.4.5 Thyroid Data.- 11.4.6 Blood Cell Data.- 11.4.7 Discussions.- 12. Generation of Training and Test Data Sets.- 12.1 Resampling Techniques.- 12.1.1 Estimation of Classification Performance.- 12.1.2 Effect of Resampling on Classifier Performance.- 12.2 Division of Data by Pairing.- 12.3 Similarity Measures.- 12.3.1 Relative Difference of Center Norms.- 12.3.2 Relative Difference of Covariance Matrix Norms.- 12.4 Performance Evaluation.- 12.4.1 Evaluation Conditions.- 12.4.2 Blood Cell Data.- 12.4.3 Hiragana-50 Data.- 12.4.4 Discussions.- II. Function Approximation.- 13. Introduction.- 13.1 Function Approximators.- 13.1.1 Neural Networks.- 13.1.2 Conventional Fuzzy Function Approximators.- 13.1.3 Fuzzy Function Approximators with Learning Capability.- 14. Fuzzy Rule Representation and Inference.- 14.1 Fuzzy Rule Representation.- 14.2 Defuzzification Methods.- 15. Fuzzy Rule Generation.- 15.1 Fuzzy Rule Generation by Preclustering.- 15.1.1 Clustering of Input Space.- 15.1.2 Clustering of Input and Output Spaces.- 15.2 Fuzzy Rule Generation by Postclustering.- 15.2.1 Concept.- 15.2.2 Rule Generation for FACG.- 15.2.3 Rule Generation for FALC.- 15.3 Fuzzy Rule Tuning.- 15.3.1 Concept.- 15.3.2 Evaluation Functions.- 15.3.3 Tuning of Parameters.- 15.4 Performance Evaluation.- 15.4.1 Mackey-Glass Differential Equation.- 15.4.2 Water Purification Plant.- 15.4.3 Discussions.- 16. Robust Function Approximation.- 16.1 Introduction.- 16.2 Fuzzy Function Approximator with Ellipsoidal Regions.- 16.2.1 Fuzzy Rule Representation.- 16.2.2 Fuzzy Rule Generation.- 16.3 Robust Parameter Estimation.- 16.3.1 Robust Estimation by the Least-median-of-squares Method.- 16.3.2 Robust Estimation by the Least-squares Method with Partial Data.- 16.4 Performance Evaluation.- 16.4.1 Determination of the Multiplier.- 16.4.2 Performance Comparison.- 16.5 Discussions.- III. Appendices.- A. Conventional Classifiers.- A.1 Bayesian Classifiers.- A.2 Nearest Neighbor Classifiers.- A.2.1 Classifier Architecture.- A.2.2 Performance Evaluation.- B. Matrices.- B.1 Matrix Properties.- B.2 Least-squares Method and Singular Value Decomposition.- B.3 Covariance Matrix.- References.