Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes: Application to Reliability

ISBN-13: 9781848218390 / Angielski / Twarda / 2016 / 298 str.

Beno?te De Saporta; Fran?ois Dufour; Huilong Zhang

Opis

Mark H.A. Davis introduced the Piecewise-Deterministic Markov Process (PDMP) class of stochastic hybrid models in an article in 1984. Today it is used to model a variety of complex systems in the fields of engineering, economics, management sciences, biology, Internet traffic, networks and many more. Yet, despite this, there is very little in the way of literature devoted to the development of numerical methods for PDMDs to solve problems of practical importance, or the computational control of PDMPs. This book therefore presents a collection of mathematical tools that have been recently developed to tackle such problems. It begins by doing so through examples in several application domains such as reliability. The second part is devoted to the study and simulation of expectations of functionals of PDMPs. Finally, the third part introduces the development of numerical techniques for optimal control problems such as stopping and impulse control problems.

Spis treści

Preface ix

Introduction xi

Part 1. Piecewise Deterministic Markov Processes and Quantization 1

Chapter 1. Piecewise Deterministic Markov Processes 3

1.1. Introduction 3

1.2. Notation 4

1.3. Definition of a PDMP 5

1.4. Regularityassumptions 8

1.4.1. Lipschitz continuity along the flow 8

1.4.2. Regularity assumptions on the local characteristics 9

1.5. Time–augmentedprocess 11

1.6. EmbeddedMarkovchain 15

1.7. Stopping times 16

1.8. ExamplesofPDMPs 20

1.8.1. Poisson processwith trend 20

1.8.2. TCP 21

1.8.3. Air conditioningunit 22

1.8.4. Crack propagationmodel 23

1.8.5. Repairworkshopmodel 24

Chapter 2. Examples in Reliability 27

2.1. Introduction 27

2.2. Structure subject to corrosion 28

2.2.1. PDMPmodel 29

2.2.2. Deterministic time to reach the boundary 32

2.3. The heatedhold–uptank 33

2.3.1. Tank dynamics 34

2.3.2. PDMPmodel 36

Chapter 3. Quantization Technique 39

3.1. Introduction 39

3.2. Optimal quantization 40

3.2.1. Optimal quantization of a random variable 40

3.2.2. Optimal quantization of a Markovchain 42

3.3. SimulationofPDMPs 44

3.3.1. Simulation of time–dependent intensity 45

3.3.2. Simulation of trajectories 45

3.4. QuantizationofPDMPs 47

3.4.1. Scale of coordinates of the state variable 48

3.4.2. Cardinality of the mode variable 50

Part 2. Simulation of Functionals 53

Chapter 4. Expectation of Functionals 55

4.1. Introduction 55

4.2. Recursive formulation 57

4.2.1. Lipschitz continuity 58

4.2.2. Iterated operator 60

4.2.3. Approximationscheme 61

4.3. Lipschitz regularity 62

4.4. Rate of convergence 69

4.5. Time–dependent functionals 71

4.6. Deterministic time horizon 74

4.6.1. Direct estimation of the running cost term 74

4.6.2. Bounds of the boundary jump cost term 77

4.6.3. Bounds in the general case 79

4.7. Example 81

4.8. Conclusion 84

Chapter 5. Exit Time 87

5.1. Introduction 87

5.2. Problem setting 88

5.2.1. Distribution 90

5.2.2. Moments 91

5.2.3. Computationhorizon 92

5.3. Approximationschemes 92

5.4. Convergence 95

5.4.1. Distribution 95

5.4.2. Moments 100

5.5. Example 101

5.6. Conclusion 108

Chapter 6. Example in Reliability: Service Time 109

6.1. Mean thickness loss 109

6.2. Service time 112

6.2.1. Mean service time 114

6.2.2. Distribution of the service time 118

6.3. Conclusion 121

Part 3. Optimization 123

Chapter 7. Optimal Stopping 125

7.1. Introduction 125

7.2. Dynamic programming equation 128

7.3. Approximation of the value function 130

7.4. Lipschitz continuity properties 132

7.4.1. Lipschitz properties of J and K 132

7.4.2. Lipschitz properties of the value functions 135

7.5. Error estimation for the value function 138

7.5.1. Second term 140

7.5.2. Third term 141

7.5.3. Fourth term 147

7.5.4. Proof of theorem 7.1 148

7.6. Numerical construction of an [1]–optimal stopping time 149

7.7. Example 161

Chapter 8. Partially Observed Optimal Stopping Problem 165

8.1. Introduction 165

8.2. Problem formulation and assumptions 167

8.3. Optimal filtering 170

8.4. Dynamicprogramming 175

8.4.1. Preliminaryresults 176

8.4.2. Optimal stopping problem under complete observation 180

8.4.3. Dynamic programming equation 181

8.5. Numerical approximation by quantization 188

8.5.1. Lipschitz properties 189

8.5.2. Discretization scheme 195

8.5.3. Numerical construction of an [1]–optimal stopping time 205

8.6. Numerical example 211

Chapter 9. Example in Reliability: Maintenance Optimization 215

9.1. Introduction 215

9.2. Corrosionprocess 216

9.3. Air conditioningunit 219

9.4. The heatedhold–uptank 221

9.4.1. Problem setting and simulation 222

9.4.2. Numerical results and validation 224

9.5. Conclusion 228

Chapter 10. Optimal Impulse Control 231

10.1. Introduction 231

10.2. Impulse controlproblem233

10.3. Lipschitz–continuity properties 236

10.3.1. Lipschitz properties of the operators 236

10.3.2. Lipschitz properties of the operator L 239

10.4. Approximation of the value function 242

10.4.1. Time discretization 245

10.4.2. Approximation of the value functions on the control grid U 246

10.4.3. Approximation of the value function 255

10.4.4. Step–by–step description of the algorithm 259

10.4.5. Practical implementation 259

10.5. Example 262

10.6. Conclusion 264

Bibliography 269

Index 277

O autorze

Benoîte de Saporta is Professor in Applied Probabilities at the University of Montpellier 2 in France.

François Dufour is Professor at the University of Bordeaux in France.

Huilong Zhang is a lecturer at INRIA in Bordeaux, France.