Markov chains

[last mesa]

want help with the b) part here

[hollow parcel]

1. Ask your question and show the work you've done so far. If you've posted a screenshot of a question, specify which part you need help with.
2. Wait patiently for a helper to come along.
3. Once someone helps you, say thank you and close the thread with:

   +close
   

4. Feel free to nominate the person for helper of the week in #helper-nominations
5. Do not ping the mods, unless someone is breaking the rules.
6. If you're happy with the help you got here, and the server overall, you can contribute financially as well:

[shell osprey]

Well, you would start with giving the definition of Yt.
Yt = max {|X1|,|X2|, ..., |Xt|}
This tracks the largest absolute value reached by the random walk up to time (t).

A stochastic process satisfies the Markov property if the future state depends only on the current state and not on the sequence of events that preceded it.

For this process, given the current maximum deviation Yt and the current position |Xt|, the next state Y_t+1 depends only on the Yt and |Xt + 1|

Let's sort of map out the associated Markov chain

The states of the chain can be represented as the possible values of Yt, which can be 0, 1, 2 , ... m ( where m is the maximum barrier).

From each state k (where k represents the current maximum deviation) :
--> if |Xt + 1| exceeds k, then Yt+1 = |Xt+1| (i.e., the next state can be k + 1 or higher)
--> If |Xt+1| is less than or equal to k, then Yt + 1 = k

For the transition diagram, I think it'd look something like this...