Understanding Computational Bayesian Statistics

ISBN-13: 9780470046098 / Angielski / Twarda / 2009 / 336 str.

William M. Bolstad

Opis

A hands-on introduction to computational statistics from a Bayesian point of view Providing a solid grounding in statistics while uniquely covering the topics from a Bayesian perspective, Understanding Computational Bayesian Statistics successfully guides readers through this new, cutting-edge approach. With its hands-on treatment of the topic, the book shows how samples can be drawn from the posterior distribution when the formula giving its shape is all that is known, and how Bayesian inferences can be based on these samples from the posterior. These ideas are illustrated on common statistical models, including the multiple linear regression model, the hierarchical mean model, the logistic regression model, and the proportional hazards model. The book begins with an outline of the similarities and differences between Bayesian and the likelihood approaches to statistics. Subsequent chapters present key techniques for using computer software to draw Monte Carlo samples from the incompletely known posterior distribution and performing the Bayesian inference calculated from these samples. Topics of coverage include:

• Direct ways to draw a random sample from the posterior by reshaping a random sample drawn from an easily sampled starting distribution
• The distributions from the one-dimensional exponential family
• Markov chains and their long-run behavior
• The Metropolis-Hastings algorithm
• Gibbs sampling algorithm and methods for speeding up convergence
• Markov chain Monte Carlo sampling

Using numerous graphs and diagrams, the author emphasizes a step-by-step approach to computational Bayesian statistics. At each step, important aspects of application are detailed, such as how to choose a prior for logistic regression model, the Poisson regression model, and the proportional hazards model. A related Web site houses R functions and Minitab macros for Bayesian analysis and Monte Carlo simulations, and detailed appendices in the book guide readers through the use of these software packages. Understanding Computational Bayesian Statistics is an excellent book for courses on computational statistics at the upper-level undergraduate and graduate levels. It is also a valuable reference for researchers and practitioners who use computer programs to conduct statistical analyses of data and solve problems in their everyday work.

Recenzje

"Understanding computational Bayesian statistics is an excellent book for courses on computational statistics at the advanced undergraduate and graduate levels. It is also a valuable reference for researchers and practitioners who use computer programs to conduct statistical analyses of data and solve problems in their everyday work." (Mathematical Reviews, 2011)

Spis treści

Preface xi

1 Introduction to Bayesian Statistics I

1.1 The Frequentist Approach to Statistics 1

1.2 The Bayesian Approach to Statistics 3

1.3 Comparing Likelihood and Bayesian Approaches to Statistics 6

1.4 Computational Bayesian Statistics 19

1.5 Purpose and Organization of This Book 20

2 Monte Carlo Sampling from the Posterior 25

2.1 Acceptance–Rejection–Sampling 27

2.2 Sampling–Importance–Resampling 33

2.3 Adaptive–Rejection–Sampling from a Log–Concave Distribution 35

2.4 Why Direct Methods Are Inefficient for High–Dimension Parameter Space 42

3. Bayesian Inference 47

3.1 Bayesian Inference from the Numerical Posterior 47

3.2 Bayesian Inference from Posterior Random Sample 54

4. Bayesian Statistics Using Conjugate Priors 61

4.1 One–Dimensional Exponential Family of Densities 61

4.2 Distributions for Count Data 62

4.3 Distributions for Waiting Times 69

4.4 Normally Distributed Observations with Known Variance 75

4.5 Normally Distributed Observations with Known Mean 78

4.6 Normally Distributed Observations with Unknown Mean and Variance 80

4.7 Multivariate Normal Observations with Known Covariance Matrix 85

4.8 Observations from Normal Linear Regression Model 87

Appendix: Proof of Poisson Process Theorem 97

5. Markov Chains 101

5.1 Stochastic Processes 102

5.2 Markov Chains 103

5.3 Time–Invariant Markov Chains with Finite State Space 104

5.4 Classification of States of a Markov Chain 109

5.5 Sampling from a Markov Chain 114

5.6 Time–Reversible Markov Chains and Detailed Balance 117

5.7 Markov Chains with Continuous State Space 120

6. Markov Chain Monte Carlo Sampling from Posterior 127

6.1 Metropolis–Hastings Algorithm for a Single Parameter 130

6.2 Metropolis–Hastings Algorithm for Multiple Parameters 137

6.3 Blockwise Metropolis–Hastings Algorithm 144

6.4 Gibbs Sampling 149

6.5 Summary 150

7 Statistical Inference from a Markov Chain Monte Carlo Sample 159

7.1 Mixing Properties of the Chain 160

7.2 Finding a Heavy–Tailed Matched Curvature Candidate Density 162

7.3 Obtaining An Approximate Random Sample For Inference 168

Appendix: Procedure for Finding the Matched

Curvature Candidate Density for a Multivariate Parameter 176

8 Logistic Regression 179

8.1 Logistic Regression Model 180

8.2 Computational Bayesian Approach to the Logistic Regression Model 184

8.3 Modelling with the Multiple Logistic Regression Model 192

9 Poisson Regression and Proportional Hazards Model 203

9.1 Poisson Regression Model 204

9.2 Computational Approach to Poisson Regression Model 207

9.3 The Proportional Hazards Model 214

9.4 Computational Bayesian Approach to Proportional Hazards Model 218

10 Gibbs Sampling and Hierarchical Models 235

10.1 Gibbs Sampling Procedure 236

10.2 The Gibbs Sampler for the Normal Distribution 237

10.3 Hierarchical Models and Gibbs Sampling 242

10.4 Modelling Related Populations with Hierarchical Models 244

Appendix: Proof That Improper Jeffrey′s Prior Distribution for the Hypervariance Can Lead to an
Improper Postenor 261

11 Going Forward with Markov Chain Monte Carlo 265

A Using the Included Minitab Macros 271

B Using the Included R Functions 289

References 307

Topic Index 313

O autorze

William M. Bolstad, PhD, is Senior Lecturer in the Department of Statistics at The University of Waikato (New Zealand). Dr. Bolstad′s research interests include Bayesian statistics, MCMC methods, recursive estimation techniques, multiprocess dynamic time series models, and forecasting. He is the author of Introduction to Bayesian Statistics, Second Edition, also published by Wiley.