Uncertainty Handling and Quality Assessment in Data Mining

ISBN-13: 9781852336554 / Angielski / Twarda / 2003 / 226 str.

Michalis Vazirigiannis; A. V. Nebylov; Michalis Vazirgiannis

Opis

Uncertainty Handling and Quality Assessment in Data Mining provides an introduction to the application of these concepts in Knowledge Discovery and Data Mining. It reviews the state-of-the-art in uncertainty handling and discusses a framework for unveiling and handling uncertainty. Coverage of quality assessment begins with an introduction to cluster analysis and a comparison of the methods and approaches that may be used. The techniques and algorithms involved in other essential data mining tasks, such as classification and extraction of association rules, are also discussed together with a review of the quality criteria and techniques for evaluating the data mining results. This book presents a general framework for assessing quality and handling uncertainty which is based on tested concepts and theories. This framework forms the basis of an implementation tool, 'Uminer' which is introduced to the reader for the first time. This tool supports the key data mining tasks while enhancing the traditional processes for handling uncertainty and assessing quality. Aimed at IT professionals involved with data mining and knowledge discovery, the work is supported with case studies from epi

Spis treści

Data Mining Process.- 2.1 Introduction to the Main Concepts of Data Mining.- 2.2 Knowledge and Data Mining.- 2.2.1 Knowledge Discovery in Database vs Data Mining.- 2.3 The Data Mining Process.- 2.3.1 Data Mining Requirements.- 2.4 Classification of Data Mining Methods.- 2.5 Overview of Data Mining Tasks.- 2.5.1 Clustering.- 2.5.1.1 Overview of Clustering Algorithms.- 2.5.1.2 Comparison of Clustering Algorithms.- 2.5.2 Classification.- 2.5.2.1 Bayesian Classification.- 2.5.2.2 Decision Trees.- 2.5.2.3 Neural Networks.- 2.5.2.4 Nearest Neighbor Classification.- 2.5.2.5 Support Vector Machines (SVMs).- 2.5.2.6 Fuzzy Classification approaches.- 2.5.3 Induction of classification rules.- 2.5.4 Association Rules.- 2.5.5 Sequential Patterns.- 2.5.6 Time Series Similarity.- 2.5.7 Visualization and Dimensionality Reduction.- 2.5.8 Regression.- 2.5.9 Summarization.- 2.6 Summary.- References.- Quality Assessment in Data Mining.- 3.1 Introduction.- 3.2 Data Pre-processing and Quality Assessment.- 3.3 Evaluation of Classification Methods.- 3.3.1 Classification Model Accuracy.- 3.3.1.1 Alternatives to the Accuracy Measure.- 3.3.2 Evaluating the Accuracy of Classification Algorithms.- 3.3.2.1 McNemar’s Test.- 3.3.2.2 A Test for the Difference of Two Proportions.- 3.3.2.3 The Resampled Paired t Test.- 3.3.2.4 The k-fold Cross-validated Paired t Test.- 3.3.3 Interestingness Measures of Classification Rules.- 3.3.3.1 Rule-Interest Function.- 3.3.3.2 Smyth and Goodman’s J-Measure.- 3.3.3.3 General Impressions.- 3.3.3.4 Gago and Bento’s Distance Metric.- 3.4 Association Rules.- 3.4.1 Association Rules Interestingness Measures.- 3.4.1.1 Coverage.- 3.4.1.2 Support.- 3.4.1.3 Confidence.- 3.4.1.4 Leverage.- 3.4.1.5 Lift.- 3.4.1.6 Rule Templates.- 3.4.1.7 Gray and Orlowska’s Interestingness.- 3.4.1.8 Dong and Li’s Interestingness.- 3.4.1.9 Peculiarity.- 3.4.1.10 Closed Association Rules Mining.- 3.5 Cluster Validity.- 3.5.1 Fundamental Concepts of Cluster Validity.- 3.5.2 External and Internal Validity Indices.- 3.5.2.1 Hypothesis Testing in Cluster Validity.- 3.5.2.2 External Criteria.- 3.5.2.3 Internal Criteria.- 3.5.3 Relative Criteria.- 3.5.3.1 Crisp Clustering.- 3.5.3.2 Fuzzy Clustering.- 3.5.4 Other Approaches for Cluster Validity.- 3.5.5 An Experimental Study on cluster validity.- 3.5.5.1 A Comparative Study.- 3.6 Summary.- References.- Uncertainty Handling in Data Mining.- 4.1 Introduction.- 4.2 Basic Concepts on Fuzzy Logic.- 4.2.1 Fuzzy Set Theory.- 4.2.2 Membership Functions.- 4.2.2.1 Hypertrapezoidal Fuzzy Membership Functions.- 4.2.2.2 Joint Degree of Membership.- 4.2.3 Fuzzy Sets and Information Measures.- 4.3 Basic Concepts on Probabilistic Theory.- 4.3.1 Uncertainty Quantified Probabi1istically.- 4.3.1.1 Bayesian Theorem.- 4.4 Probabilistic and Fuzzy Approaches.- 4.5 The EM Algorithm.- 4.5.1 General Description of EM Algorithm.- 4.6 Fuzzy Cluster Analysis.- 4.6.1 Fuzzy C-Means and its Variants.- 4.6.2 Fuzzy C-Means for Object-Data.- 4.6.3 Fuzzy C-Means (FCM) Alternatives.- 4.6.4 Applying Fuzzy C-Means Methodology to Relational Data.- 4.6.5 The Fuzzy C-Means Algorithm for Relational data.- 4.6.5.1 Comments on FCM for Relational Data.- 4.6.6 Noise Fuzzy Clustering Algorithm.- 4.6.7 Conditional Fuzzy C-Means Clustering.- 4.7 Fuzzy Classification Approaches.- 4.7.1 Fuzzy Decision Trees.- 4.7.1.1 Building a Fuzzy Decision Tree.- 4.7.1.2 Inference for Decision Assignment.- 4.7.2 Fuzzy Rules.- 4.8 Managing Uncertainty and Quality in the Classification Process.- 4.8.1 Framework Description.- 4.8.2 Mapping to the Fuzzy Domain.- 4.8.2.1 Classification Space (CS).- 4.8.2.2 Classification Value Space (CVS).- 4.8.3 Information Measures for Decision Support.- 4.8.3.1 Class Energy Metric.- 4.8.3.2 Attribute Energy Metric.- 4.8.4 Queries & Decision Support.- 4.8.5 Classification Scheme Quality Assessment.- 4.9 Fuzzy Association Rules.- 4.9.1 Defining Fuzzy Sets.- 4.9.2 Fuzzy Association Rule Definition.- 4.9.2.1 Fuzzy Support.- 4.9.2.2 Fuzzy Confidence.- 4.9.2.3 Fuzzy Correlation.- 4.9.3 Mining Fuzzy Association Rules Algorithms.- 4.10 Summary.- References.- UMiner: A Data Mining System Handling Uncertainty and Quality.- 5.1 Introduction.- 5.2 UMiner Development Approach.- 5.3 System Architecture.- 5.4 UMiner’s Data Mining Tasks.- 5.5 Demonstration.- 5.5.1 Clustering process.- 5.6 Summary.- References.- Case Studies.- 6.1 Extracting Association Rules for Medical Data Analysis.- 6.2 The Mining Process.- 6.2.1 Collection of Data.- 6.2.2 Data Cleaning and Pre-processing.- 6.2.3 Further Analysis of Extracted Association Rules.- 6.3 Cluster Analysis of Epidemiological Data.- References.