Title Details: | |
Stochastic Models and Markov Chains |
|
Authors: |
Zazanis, Michael |
Subject: | MATHEMATICS AND COMPUTER SCIENCE > MATHEMATICS > PROBABILITY THEORY AND STOCHASTIC PROCESSES MATHEMATICS AND COMPUTER SCIENCE > MATHEMATICS > PROBABILITY THEORY AND STOCHASTIC PROCESSES > STOCHASTIC PROCESSES MATHEMATICS AND COMPUTER SCIENCE > MATHEMATICS > PROBABILITY THEORY AND STOCHASTIC PROCESSES > MARKOV PROCESSES |
Keywords: |
Markov Chains
Stochastic Processes Stochastic Models Applied Probability Poisson Process Renewal Process |
Description: | |
Abstract: |
This book is an introduction to stochastic processes and stochastic models at the undergraduate level. While the emphasis is placed in developing the theory and computational techniques, a number of applications is presented in detail. Two introductory chapters, in discrete and “continuous” probability are included. The emphasis in these chapters is placed on topics and techniques that will be useful in the study of stochastic processes. We begin this study with the simple random walk and discrete time Markov chains. Following what is becoming the standard approach in the literature, a purely probabilistic approach is used in their analysis. However, a whole separate chapter is dedicated to the algebraic treatment, including the fundamental theorem of Perron and Frobenius, because we believe that the interplay between probability and linear algebra, on one hand deepens the understanding of the concepts and, on the other, prepares the student for the computational aspects of the subject. Time reversibility in Markov chains is studied both as a fundamental concept and in terms of its computational implications. The Poisson process and its properties are studied in some depth (for an undergraduate account) whereas continuous time Markov chains are treated with emphasis on the intuitive part of the theory and on applications, in order to maintain the mathematical tools needed at a modest level. The discussion of renewal theory is extensive. It includes precise statements of Blackwell’s Theorem and the Key Renewal Theorem but not its proof. Furthermore, a whole chapter is dedicated to applications of the renewal equation to Risk Theory and mathematical population models. Unfortunately, space and time limitations prevented us from discussing regenerative processes. The presentation of Brownian motion is kept at a mathematically elementary level while the chapter on martingales includes both an elementary and a mathematically more advanced part, adequate for students with the appropriate preparation.
|
Linguistic Editors: |
Triantafillidou, Sofia |
Technical Editors: |
Papadogonas, Ioannis |
Graphic Editors: |
Zazanis, Michael |
Type: |
Undergraduate textbook |
Creation Date: | 27-03-2025 |
Item Details: | |
ISBN |
978-618-228-333-2 |
License: |
Attribution – NonCommercial – NoDerivatives 4.0 International (CC BY-NC-ND 4.0) |
DOI | http://doi.org/10.57713/kallipos-1082 |
Handle | http://hdl.handle.net/11419/14736 |
Bibliographic Reference: | Zazanis, M. (2025). Stochastic Models and Markov Chains [Undergraduate textbook]. Kallipos, Open Academic Editions. https://doi.org/10.57713/kallipos-1082 |
Language: |
Greek |
Consists of: |
1. Discrete Random Variables 2. Continuous Random Variables 3. Simple Random Walk 4. Discrete Time Markov Chains 5. Applications of Markov Chains in Discrete Time - Time Reversibility 6. Algebraic Techniques in Markov Chains 7. Martingales 8. Poisson Process 9. Continuous Time Markov Chains 10. Markovian Models in Queueing Theory 11. Renewal Processes 12. Applications of Renewal Theory - Risk Theory 13. Markov Decision Processes 14. Brownian Motion |
Number of pages |
442 |
Publication Origin: |
Kallipos, Open Academic Editions |
You can also view | |
User comments | |
There are no published comments available! | |