#Markov Chain/Process Problem

95 messages · Page 1 of 1 (latest)

limpid patrolBOT
#
  1. Ask your question and show the work you've done so far. If you've posted a screenshot of a question, specify which part you need help with.
  2. Wait patiently for a helper to come along.
  3. Once someone helps you, say thank you and close the thread with:
    +close
  4. Feel free to nominate the person for helper of the week in #helper-nominations
  5. Do not ping the mods, unless someone is breaking the rules. If there is a conflict amongst multiple helpers feel free to ping “Helper Mod”
  6. If you're happy with the help you got here, and the server overall, you can contribute financially as well:
random wing
#

what's your setup?

odd elk
random wing
#

well, for starters, how do you work out the probabilities of having meat the next day provided he didn't/did eat meat the day before?

#

it's a markov process, tomorrow's weather depends only on what happens today

odd elk
#

Not having is 5/6

#

but i'm confused how it relates to him not having it the day before

bold surge
#

We have two states, for simplicity let's call them meat & veggies.

When the previous state is meat, we have a 100% chance of going into veggies. When the previous states is veggies, we have a 1/6th chance of going into meat and 5/6 chance of going into veggies.

odd elk
#

So how would the matrix be set up?

bold surge
#

Well do you know what the sum of each column needs to be?

odd elk
#

The sum of each column always has to be 1 in a transition matrix

bold surge
#

Yes

odd elk
#

However, i'm not sure how it needs to be set up because it could be like this:

#

or it could be this:

bold surge
#

It could be anything as long as it is a stochastic matrix and you've defined the elements correctly

odd elk
#

It's the defining part which I am confused with

#

I'm not too sure what the columns and rows are actually representing or whether the rows even matter?

#

Because we don't know whether he had meat or not on the first day

bold surge
#

...this is not a problem for a markov process

#

At least for this question

odd elk
#

Like shouldn't I have 2 equations which translate into one of the matrices above?

bold surge
#

Write the conditional probability for the question and see what happens

odd elk
#

Hmm I'm supposed to use Markov for this tho

bold surge
#

Use the markov property

#

P(X_(n+1) = meat | X_n = meat) = 0, right?

odd elk
#

uh

#

sorry, what does X_n+1 stand for?

bold surge
#

The next step in the process

#

Alternatively of course you can calculate the limit of the matrix

odd elk
#

and then P(X_(n+1) = meat | X_n = no meat) = 1/6

#

and then P(X_(n+1) = no meat | X_n = no meat) = 5/6

#

but then where does the 1 come into play?

bold surge
#

P(X_(n+1) = no meat | X_n = meat) = 1

odd elk
#

gotcha yeah

#

okay so given this

#

now how do we determine what our matrix will be?

bold surge
#

Well now you've given the numerical values some labels, just label the matrix so you can be certain that your matrix indeed represents the process

odd elk
#

Do I set it up as an equation?

bold surge
#

Hint: diagonalize the matrix, then calculate the limit of the diagonal component (should be easy), and perform multiplication

odd elk
#

That's what I have trouble understanding

#

How will diagonalization give us the answer to this question?

bold surge
#

Ask yourself what is the question

#

Then ask what is special about diagonal matrices and their powers

odd elk
#

Applying this property?

bold surge
#

No

#

Just normal matrix diagonalization here

odd elk
#

yes but once matrix diagonalization is performed

#

we want the eigen vector associated with eigenvalue 1 no?

bold surge
#

To do what?

odd elk
#

because that tells us what the steady state is and gives us the answer?

#

so I started with this matrix:

#

Has eigenvalues λ_1 = 1 and λ_2 = -1/6

#

The eigenvectors are v1 = [6 1]^T and v1 = [-1 1]^T

bold surge
#

Yes

odd elk
#

ok and then

#

let's call this A

#

we now want to diagonalize matrix A correct?

bold surge
#

Well you took a shortcut and you already know the steady state

#

But you can find the steady state by diagonalization

odd elk
#

We want to diagonalize Matrix A as P^(-1) A P = D for some diagonal matrix D?

odd elk
bold surge
#

Hint: it's one of the eigenvectors of lambda 1

odd elk
#

and your hint is due to this property right?

bold surge
#

I have no idea what that property is to be honest

#

I just know the boring way to find the steady state

odd elk
#

lol let's proceed with the boring way

odd elk
bold surge
#

Okay so diagonalize the matrix then

odd elk
bold surge
#

Yes

#

For some diagonal matrix A

odd elk
#

A?

bold surge
#

Normally we write A = P^(-1) D P, for some diagonal matrix D

#

the diagonal matrix is sandwiched between the matrices P

#

and A is our original matrix

#

which we want to diagonalize

odd elk
#

oh okay, thanks!

bold surge
#

diagonalization meaning, representing it as a product like so

odd elk
#

Sure so let's proceed that way

#

So P is just the eigenvectors in the same order which is [6 -1
1 1 ]

#

and the diagonal matrix D is simple the eigevalues in the same order which is [1 0
0 -1/6 ]

bold surge
#

Yes

#

So now calculate the inverse of P

#

And for diagonal matrices we have the nice property, that

A^n = P^(-1) D^n P

#

So calculate D^infty = limit of D^n as n goes to infinity, then calculate the product

#

This is of course more labor intensive than using theorems from your textbook, but this will help you understand why those theorems work

odd elk
#

I'm also confused about another thing

#

Does the 5/6 imply the 1 and the 1/6 imply the 0 here?

odd elk
# bold surge Yes

ahhhh
so the first column is no meat today and the second column is meat today
and the first row is now meat tomorrow
and the second row is meat tomorrow
based off that we can find probabilities for example, meat today and no meet tomorrow would be given the the entry in (2,1) which is 1.

odd elk
#

+close