excursion into Markov chains

This textbook will present, in a rigorous way, the basic theory of the discrete-time and the continuous-time Markov chains, along with many examples and solved problems. For both the topics a simple model, the Random Walk and the Poisson Process respectively, will be used to anticipate and illustrate the most interesting concepts rigorously defined in thefollowing sections. A great attention will be paid to theapplications of the theory of the Markov chains and many classical aswell as new results will befaced in the book. This textbook is intended for a basic course on stochastic processes at an advanced undergraduate level and the background needed will be a first course in probability theory. A big emphasis is givento the computational approach andto simulations.