Markov Processes/Diff'eq Question

[young meteor]

Warm Milk

[vernal vine]

I'm confused what "attach for an exponential amount of time with rates 3 and 4" means. What exactly is the function that determines how long a carbon monoxide molecule stays attached?

[remote wagon]

It'a poisson process with rates 1 and 2 then they leave with rates 3 and 4

Here are the jump rates:

$$

q_{O_2,F}=3

q_{F,O_2}=1

q_{F,CO}=2

q_{CO,F}=4
$$

[young meteor]

Warm Milk
Compile Error! Click the reaction for more information.
(You may edit your message to recompile.)

[vernal vine]

I'm still not quite understanding why it says "exponential amount of time"? Like, is it proportional to the amount of the molecule present in the system?

[remote wagon]

The time the molecules transition and stay are exponentially distributed with the means above

$T_i \sim Exp(\lambda_i)$

[young meteor]

Warm Milk

[remote wagon]

So, for example, if a molecule is at state $O_2$, then the rate it transitions from $O_2$ to $F$ is $q_{(O_2,F)}=3$

[young meteor]

Warm Milk

[remote wagon]

Now, going back we can write all scalars $q$'s as a matrix $Q$ where $Q_{(i,j)}=$ the transition rate from $i$ to $j = $ the jump rate from $i$ to $j$.

[young meteor]

Warm Milk

[remote wagon]

$
[
Q = \begin{bmatrix}
-3 & 3 & 0 \
1 & -3 & 2 \
0 & 4 & -4 \
\end{bmatrix}
]
$

[young meteor]

Warm Milk
Compile Error! Click the reaction for more information.
(You may edit your message to recompile.)

[remote wagon]

The rows of Q need to sum to zero so the diagonals are just the sum of the rows times -1

Now, this matrix represents the "jump rate" of going between $i$ and $j$ which equal $Q_{(i,j)}$

[young meteor]

Warm Milk

[remote wagon]

Now when we say that something is going $i$ to $j$ in an exponential amount of time with rate $\lambda$, what we really mean is, that time is a exponential random variable with rate $\lambda$

[young meteor]

Warm Milk

[remote wagon]

What's your major? Have you taken Markov Processes before?

[vernal vine]

CS. Haven't taken markov processes before, have taken Stat and DiffEq.

[remote wagon]

Oh shit ok actually make you can help me

I understand all the markov behind it but I have an issue solving a diff'eq

Suppose $
[
Q = \begin{bmatrix}
-3 & 3 & 0
1 & -3 & 2
0 & 4 & -4
\end{bmatrix}
]
$

[young meteor]

Warm Milk

[remote wagon]

$
[
Q = \begin{bmatrix}
-3 & 3 & 0
1 & -3 & 2
0 & 4 & -4
\end{bmatrix}
]
$

[young meteor]

Warm Milk

[remote wagon]

What's $e^{Qt}$?

[young meteor]

Warm Milk

[remote wagon]

I need this to solve $\vv{P_t}=$

[young meteor]

Warm Milk
Compile Error! Click the reaction for more information.
(You may edit your message to recompile.)

[remote wagon]

${P_t}'=QP_t$

[young meteor]

Warm Milk

[remote wagon]

We know the solution to this is:

$P_t=e^{Qt}P_0$

[young meteor]

Warm Milk

[remote wagon]

I tried doing PDP^-1 decomp but got something extremely messy

[vernal vine]

Can you show the messy thing you got?

[remote wagon]

Yup standby

Something like this. The computation got too extreme to do by hand so I plugged it into a calculator. After seeing this, I think my method is probably not going to work

[vernal vine]

And Qt is based on the specifications from the markov process?

[remote wagon]

The Q matrix is. I just need to be able to find e^{Qt} to solve the rest of the markov problem

[vernal vine]

Does the fact that $P_0$ has 0 as 2 of its 3 components let us ignore most of the $e^{Qt}$ matrix?

[young meteor]

Will(ow|iam)

[remote wagon]

Oh god, you're right. I didn't even think about that. We can ignore all entries except for the ones that align with the non-zero entry of P_0

P_0 = [0,1,0]

Let me try this real quick. Thnak you!

[vernal vine]

'course!

[remote wagon]

It worked. I was able to find a general solution for P_1(t) in terms of Pe^DTP^-1