MCS, a student is in one of three Markov states:
0:sleeping, 1:eating, or 2:reading. (To complete this assignment, you probably
should review Problem 5.5 and its solution in the back of the MCS.) With
respect to sleeping, Problem 5.5 says
Sleeping is always followed by eating. Each time the student
goes to sleep (takes a nap), the average nap time is 4 hours.
Because sleeping (state 0) is a Markov state, at the end of each hour, the
next state is given by the transition probabilitiesP00,P01 andP02. Next,
since \sleeping is always followed by eating,” we knowP01> 0 butP02 = 0.
So, at the end of each hour, the student switches from sleeping to eating
with probabilityP01. Hence the time (in hours) spent sleeping is a geometric
random variable with success probabilityP01. This implies that the average
time spent sleeping is 1=P01. Since the problem statement says that the
average nap time is 4 hours, we learn that 1=P01 = 4, orP01 = 1=4. Since
P2j=0P0j = 1, these facts implyP00 = 3=4.
The time spent \sleeping” is also known as thesojourn time in the sleeping
state. In general for any Markov chain, since the system departs statei with
probability 1–Pii, the sojourn time in any statei is a geometric random
variable with expected value 1=(1–Pii).
This review of Problem 5.5 is just to introduce the following variation on
Problem 5.5:
Problem 1: Consider a student who is always either sleeping, eating or
reading, such that
• If the student is eating, the student continues to eat for another hour
with probability 0.1; otherwise, the student begins to read.
• Sleeping is always followed by eating. Each time the student goes to
sleep (takes a nap), the nap time isexactly 4 hours.
• After an hour of reading, another hour of reading is four times more
probable than an hour of eating. However, the probability of going to
sleep equals the sum of the probabilities of reading or eating.
Just as in Problem 5.5, the time step is still 1 hour. For this modified
problem:
(a) What is the Markov chain? Hint: The Markov chain needs to keep
track of how many hours the student has slept (or how many hours of
sleep the student has remaining.)
(b) What are the stationary probabilities?
(c) After a very long time, what is the probability the student is sleeping?
Once you solve this problem, now consider the following generalization:
0:sleeping, 1:eating, or 2:reading. (To complete this assignment, you probably
should review Problem 5.5 and its solution in the back of the MCS.) With
respect to sleeping, Problem 5.5 says
Sleeping is always followed by eating. Each time the student
goes to sleep (takes a nap), the average nap time is 4 hours.
Because sleeping (state 0) is a Markov state, at the end of each hour, the
next state is given by the transition probabilitiesP00,P01 andP02. Next,
since \sleeping is always followed by eating,” we knowP01> 0 butP02 = 0.
So, at the end of each hour, the student switches from sleeping to eating
with probabilityP01. Hence the time (in hours) spent sleeping is a geometric
random variable with success probabilityP01. This implies that the average
time spent sleeping is 1=P01. Since the problem statement says that the
average nap time is 4 hours, we learn that 1=P01 = 4, orP01 = 1=4. Since
P2j=0P0j = 1, these facts implyP00 = 3=4.
The time spent \sleeping” is also known as thesojourn time in the sleeping
state. In general for any Markov chain, since the system departs statei with
probability 1–Pii, the sojourn time in any statei is a geometric random
variable with expected value 1=(1–Pii).
This review of Problem 5.5 is just to introduce the following variation on
Problem 5.5:
Problem 1: Consider a student who is always either sleeping, eating or
reading, such that
• If the student is eating, the student continues to eat for another hour
with probability 0.1; otherwise, the student begins to read.
• Sleeping is always followed by eating. Each time the student goes to
sleep (takes a nap), the nap time isexactly 4 hours.
• After an hour of reading, another hour of reading is four times more
probable than an hour of eating. However, the probability of going to
sleep equals the sum of the probabilities of reading or eating.
Just as in Problem 5.5, the time step is still 1 hour. For this modified
problem:
(a) What is the Markov chain? Hint: The Markov chain needs to keep
track of how many hours the student has slept (or how many hours of
sleep the student has remaining.)
(b) What are the stationary probabilities?
(c) After a very long time, what is the probability the student is sleeping?
Once you solve this problem, now consider the following generalization:
