Multivariate Analysis

ISBN-13: 9781118738023 / Angielski / Twarda / 2024

KV Mardia

Spis treści

EpigraphPreface xiiiPreface to the Second Edition xiiiPreface to the First Edition xvAcknowledgements from First Edition xviiiNotation, abbreviations and key ideas xix1 Introduction 11.1 Objects and Variables 11.2 Some Multivariate Problems and Techniques 21.2.1 Generalizations of univariate techniques 21.2.2 Dependence and regression 21.2.3 Linear combinations 21.2.4 Assignment and dissection 51.2.5 Building configurations 61.3 The Data Matrix 61.4 Summary Statistics 71.4.1 The mean vector and covariance matrix 81.4.2 Measures of multivariate scatter 111.5 Linear Combinations 111.5.1 The scaling transformation 121.5.2 Mahalanobis transformation 121.5.3 Principal component transformation 121.6 Geometrical Ideas 131.7 Graphical Representation 141.7.1 Univariate scatters 141.7.2 Bivariate scatters 161.7.3 Harmonic curves 161.7.4 Parallel coordinates plot 191.8 Measures of Multivariate Skewness and Kurtosis 192 Basic Properties of Random Vectors 272.1 Cumulative Distribution Functions and Probability Density Functions 272.2 Population Moments 292.2.1 Expectation and correlation 292.2.2 Population mean vector and covariance matrix 292.2.3 Mahalanobis space 312.2.4 Higher moments 312.2.5 Conditional moments 332.3 Characteristic Functions 332.4 Transformations 352.5 The Multivariate Normal Distribution 362.5.1 Definition 362.5.2 Geometry 382.5.3 Properties 382.5.4 Singular multivariate normal distribution 432.5.5 The matrix normal distribution 442.6 Random Samples 452.7 Limit Theorems 473 Non-normal Distributions 533.1 Introduction 533.2 Some Multivariate Generalizations of Univariate Distributions 533.2.1 Direct generalizations 533.2.2 Common components 543.2.3 Stochastic generalizations 553.3 Families of Distributions 563.3.1 The exponential family 563.3.2 The spherical family 573.3.3 Elliptical distributions 603.3.4 Stable distributions 623.4 Insights into skewness and kurtosis 623.5 Copulas 643.5.1 The Gaussian Copula 663.5.2 The Clayton-Mardia copula 673.5.3 Archimedean Copulas 683.5.4 Fr´echet-H¨offding Bounds 694 Normal Distribution Theory 774.1 Characterization and Properties 774.1.1 The central role of multivariate normal theory 774.1.2 A definition by characterization 784.2 Linear Forms 794.3 Transformations of Normal Data Matrices 814.4 The Wishart Distribution 834.4.1 Introduction 834.4.2 Properties of Wishart matrices 834.4.3 PartitionedWishart matrices 864.5 The Hotelling T 2 Distribution 894.6 Mahalanobis Distance 924.6.1 The two-sample Hotelling T 2 statistic 924.6.2 A decomposition of Mahalanobis distance 934.7 Statistics Based on the Wishart Distribution 954.8 Other Distributions Related to the Multivariate Normal 995 Estimation 1115.1 Likelihood and Sufficiency 1115.1.1 The likelihood function 1115.1.2 Efficient scores and Fisher's information 1125.1.3 The Cram´er-Rao lower bound 1145.1.4 Sufficiency 1155.2 Maximum Likelihood Estimation 1165.2.1 General case 1165.2.2 Multivariate normal case 1175.2.3 Matrix normal distribution 1225.3 Robust Estimation of Location and Dispersion for Multivariate Distributions 1235.3.1 M-Estimates of location 1235.3.2 Minimum covariance determinant 1245.3.3 Multivariate trimming 1245.3.4 Stahel-Donoho estimator 1255.3.5 Minimum volume estimator 1255.3.6 Tyler's estimate of scatter 1275.4 Bayesian inference 1276 Hypothesis Testing 1376.1 Introduction 1376.2 The Techniques Introduced 1396.2.1 The likelihood ratio test (LRT) 1396.2.2 The union intersection test (UIT) 1436.3 The Techniques Further Illustrated 1466.3.1 One-sample hypotheses on mu 1466.3.2 One-sample hypotheses on _ 1486.3.3 Multi-sample hypotheses 1526.4 Simultaneous Confidence Intervals 1566.4.1 The one-sample Hotelling T 2 case 1566.4.2 The two-sample Hotelling T 2 case 1576.4.3 Other examples 1576.5 The Behrens-Fisher Problem 1576.6 Multivariate Hypothesis Testing: Some General Points 1586.7 Non-normal Data 1596.8 Mardia's Non-parametric Test for the Bivariate Two-sample Problem 1627 Multivariate Regression Analysis 1697.1 Introduction 1697.2 Maximum Likelihood Estimation 1707.2.1 Maximum likelihood estimators for B and _ 1707.2.2 The distribution of ^B and ^_ 1727.3 The General Linear Hypothesis 1737.3.1 The likelihood ratio test (LRT) 1737.3.2 The union intersection test (UIT) 1757.3.3 Simultaneous confidence intervals 1757.4 Design Matrices of Degenerate Rank 1767.5 Multiple Correlation 1787.5.1 The effect of the mean 1787.5.2 Multiple correlation coefficient 1787.5.3 Partial correlation coefficient 1807.5.4 Measures of correlation between vectors 1817.6 Least Squares Estimation 1827.6.1 Ordinary least squares (OLS) estimation 1827.6.2 Generalized least squares 1837.6.3 Application to multivariate regression 1837.6.4 Asymptotic consistency of least squares estimators 1847.7 Discarding of Variables 1847.7.1 Dependence analysis 1847.7.2 Interdependence analysis 1868 GraphicalModels 1958.1 Introduction 1958.2 Graphs and Conditional independence 1968.3 Gaussian Graphical Models 2018.3.1 Estimation 2028.3.2 Model selection 2078.4 Log-linear Graphical Models 2088.4.1 Notation 2098.4.2 Log-linear models 2108.4.3 Log-linear models with a graphical interpretation 2138.5 Directed and Mixed Graphs 2159 Principal Component Analysis 2219.1 Introduction 2219.2 Definition and Properties of Principal Components 2219.2.1 Population principal components 2219.2.2 Sample principal components 2249.2.3 Further properties of principal components 2259.2.4 Correlation structure 2299.2.5 The effect of ignoring some components 2299.2.6 Graphical representation of principal components 2329.2.7 Biplots 2329.3 Sampling Properties of Principal Components 2369.3.1 Maximum likelihood estimation for normal data 2369.3.2 Asymptotic distributions for normal data 2399.4 Testing Hypotheses about Principal Components 2429.4.1 Introduction 2429.4.2 The hypothesis that (_1 + * * * + _k)/(_1 + * * * + _p) = 2449.4.3 The hypothesis that (p . k) eigenvalues of _ are equal 2459.4.4 Hypotheses concerning correlation matrices 2469.5 Correspondence Analysis 2479.5.1 Contingency tables 2479.5.2 Gradient analysis 2539.6 Allometry-- the Measurement of Size and Shape 2559.7 Discarding of variables 2589.8 Principal Component Regression 2599.9 Projection Pursuit and Independent Component Analysis 2619.9.1 Projection pursuit 2619.9.2 Independent component analysis 2639.10 PCA in high dimensions 26610 Factor Analysis 27710.1 Introduction 27710.2 The Factor Model 27810.2.1 Definition 27810.2.2 Scale invariance 27910.2.3 Non-uniqueness of factor loadings 27910.2.4 Estimation of the parameters in factor analysis 28010.2.5 Use of the correlation matrix R in estimation 28110.3 Principal Factor Analysis 28210.4 Maximum Likelihood Factor Analysis 28410.5 Goodness of Fit Test 28710.6 Rotation of Factors 28810.6.1 Interpretation of factors 28810.6.2 Varimax rotation 28910.7 Factor Scores 29310.8 Relationships Between Factor Analysis and Principal Component Analysis 29410.9 Analysis of Covariance Structures 29511 Canonical Correlation Analysis 29911.1 Introduction 29911.2 Mathematical Development 30011.2.1 Population canonical correlation analysis 30011.2.2 Sample canonical correlation analysis 30411.2.3 Sampling properties and tests 30511.2.4 Scoring and prediction 30611.3 Qualitative Data and Dummy Variables 30711.4 Qualitative and Quantitative Data 30912 Discriminant Analysis and Statistical Learning 31712.1 Introduction 31712.2 Bayes' Discriminant Rule 31912.3 The error rate 32012.3.1 Probabilities of misclassification 32012.3.2 Estimation of error rate 32312.3.3 Confusion matrix 32412.4 Discrimination Using the Normal Distribution 32412.4.1 Population discriminant rules 32412.4.2 The sample discriminant rules 32612.4.3 Is discrimination worthwhile? 33412.5 Discarding of Variables 33412.6 Fisher's Linear Discriminant Function 33612.7 Nonparametric Distance-based Methods 33912.7.1 Nearest neighbor classifier 33912.7.2 Large sample behavior of the Nearest Neighbor Classifier 34112.7.3 Kernel classifiers 34412.8 Classification Trees 34612.8.1 Splitting criteria 34812.8.2 Pruning 35112.9 Logistic Discrimination 35412.9.1 Logistic regression model 35412.9.2 Estimation and inference 35612.9.3 Interpretation of the parameter estimates 35612.9.4 Extensions 36012.10Neural Networks 36012.10.1Motivation 36012.10.2Multi-layer perceptron 36112.10.3 Radial basis functions 36312.10.4 Support Vector Machines 36613 Multivariate Analysis of Variance 37913.1 Introduction 37913.2 Formulation of Multivariate One-way Classification 37913.3 The Likelihood Ratio Principle 38013.4 Testing Fixed Contrasts 38213.5 Canonical Variables and a Test of Dimensionality 38313.5.1 The problem 38313.5.2 The LR test (_ known) 38313.5.3 Asymptotic distribution of the likelihood ratio criterion 38513.5.4 The estimated plane 38613.5.5 The LR test (unknown _) 38713.5.6 The estimated plane (unknown _) 38713.5.7 Profile analysis 39313.6 The union intersection approach 39413.7 Two-way Classification 39514 Cluster Analysis and Unsupervised Learning 40514.1 Introduction 40514.2 Probabilistic membership models 40614.3 Parametric mixture models 41014.4 Partitioning Methods 41214.5 Hierarchical Methods 41814.5.1 Agglomerative algorithms 41814.5.2 Minimum spanning tree and single linkage 42114.5.3 Properties of different agglomerative algorithms 42314.6 Distances and Similarities 42514.6.1 Distances 42514.6.2 Similarity coefficients 43014.7 Grouped Data 43214.8 Mode Seeking 43414.9 Measures of agreement 43615 Multidimensional Scaling 44915.1 Introduction 44915.2 Classical solution 45115.2.1 Some theoretical results 45115.2.2 An algorithm for the classic MDS solution 45415.2.3 Similarities 45615.3 Duality Between Principal Coordinate Analysis and Principal ComponentAnalysis 45915.4 Optimal Properties of the Classical Solution and Goodness of Fit 46015.5 Seriation 46715.5.1 Description 46715.5.2 Horseshoe effect 46815.6 Non-metric methods 46915.7 Goodness of Fit Measure: Procrustes Rotation 47215.8 Multi-sample Problem and Canonical Variates 47516 High-dimensional Data 48116.1 Introduction 48116.2 ShrinkageMethods in Regression 48316.2.1 The multiple linear regression model 48316.2.2 Ridge regression 48416.2.3 Least absolute selection and shrinkage operator (LASSO) 48616.3 Principal Component Regression 48816.4 Partial Least Squares Regression 49016.4.1 Overview 49016.4.2 The PLS1 algorithm to construct the PLS loading matrix for p = 1response variable 49016.4.3 The PLS2 algorithm to construct the PLS loading matrix for p > 1response variables 49316.4.4 The predictor envelope model 49416.4.5 PLS regression 49416.4.6 Joint envelope models 49616.5 Functional Data 49816.5.1 Functional principal component analysis 49916.5.2 Functional linear regression models 503A Matrix Algebra 509A.1 Introduction 509A.2 Matrix Operations 512A.2.1 Transpose 512A.2.2 Trace 513A.2.3 Determinants and cofactors 513A.2.4 Inverse 515A.2.5 Kronecker products 516A.3 Further Particular Matrices and Types of Matrix 517A.3.1 Orthogonal matrices 517A.3.2 Equicorrelation matrix 518A.3.3 Centering matrix 519A.4 Vector Spaces, Rank, and Linear Equations 519A.4.1 Vector spaces 519A.4.2 Rank 521A.4.3 Linear equations 522A.5 Linear Transformations 523A.6 Eigenvalues and Eigenvectors 523A.6.1 General results 523A.6.2 Symmetric matrices 525A.7 Quadratic Forms and Definiteness 531A.8 Generalized Inverse 533A.9 Matrix Differentiation and Maximization Problems 535A.10 Geometrical Ideas 538A.10.1 n-dimensional geometry 538A.10.2 Orthogonal transformations 538A.10.3 Projections 539A.10.4 Ellipsoids 539B Univariate Statistics 543B.1 Introduction 543B.2 Normal Distribution 543B.3 Chi-squared Distribution 544B.4 F and Beta Variables 544B.5 t distribution 545B.6 Poisson distribution 546C R commands and data 547C.1 Basic R Commands Related to Matrices 547C.2 R Libraries and Commands Used in Exercises and Figures 548C.3 Data Availability 549D Tables 551References 560Index

O autorze

Kanti V. Mardia is a Senior Research Professor in the Department of Statistics at the University of Leeds, Leverhulme Emeritus Fellow, and Visiting Professor in the Department of Statistics, University of Oxford.John T. Kent and Charles C. Taylor are both Professors in the Department of Statistics, University of Leeds.