Here, we will consider a simple example of Markov process with implementation in R.
The following example is taken from Bodo Winter website.
A Markov process is characterized by (1) a finite set of states and (2) fixed transition probabilities between the states.
Let’s consider an example. Assume you have a classroom, with students who could be either in the state alert or in the state bored. And then, at any given time point, there’s a certain probability of an alert student becoming bored (say 0.