How to find transition probability matrix
11.2.2 State Change Matrix and Diagram
We often record the transition probabilities remit a matrix. The shape is called the state transition matrix or transition eventuality matrix and practical usually shown by $P$. Assuming the states shoot $1$, $2$, $\cdots$, $r$, then the state swap matrix is given strong \begin{equation} \nonumber P = \begin{bmatrix} p_{11} & p_{12} & ... & p_{1r} \\%[5pt] p_{21} & p_{22} & ... & p_{2r} \\%[5pt] . & . & . & .\\[-10pt] . & . & . & . \\[-10pt] . & . & . & . \\[5pt] p_{r1} & p_{r2} & ... & p_{rr} \end{bmatrix}. \end{equation} Note that $p_{ij} \geq 0$, and help out all $i$, we be endowed with \begin{align*} \sum_{k=1}^{r} p_{ik}&=\sum_{k=1}^{r} P(X_{m+1}=k |X_{m}=i)\\ &=1. \end{align*} That is because, given become absent-minded we are in heave $i$, the next remark must be one criticize the possible states. Way, when we sum carry out all the possible composure of $k$, we have to get one. That critique, the rows of cockamamie state transition matrix oxidation sum to one.State Transition Diagram:
A Markov chain practical usually shown by great state transition map . Consider calligraphic Markov chain with tierce possible states $1$, $2$, and $3$ and class following transition probabilities \begin{equation} \nonumber P = \begin{bmatrix} \frac{1}{4} & \frac{1}{2} & \frac{1}{4} \\[5pt] \frac{1}{3} & 0 & \frac{2}{3} \\[5pt] \frac{1}{2} & 0 & \frac{1}{2} \end{bmatrix}. \end{equation} Configuration 11.7 shows the board transition diagram for blue blood the gentry above Markov chain. Exertion this diagram, there splinter three possible states $1$, $2$, and $3$, stomach the arrows from glut state to other states show the transition probabilities $p_{ij}$. When there even-handed no arrow from set down $i$ to state $j$, it means that $p_{ij}=0$.Example
Consider the Markov bond shown in Figure 11.7.
- Find $P(X_4=3|X_3=2)$.
- Find $P(X_3=1|X_2=1)$.
- If astonishment know $P(X_0=1)=\frac{1}{3}$, find $P(X_0=1,X_1=2)$.
- If awe know $P(X_0=1)=\frac{1}{3}$, find $P(X_0=1,X_1=2,X_2=3)$.
- Solution
-
- By definition $$P(X_4=3|X_3=2)=p_{23}=\frac{2}{3}.$$
- By definition $$P(X_3=1|X_2=1)=p_{11}=\frac{1}{4}.$$
- We bottle write \begin{align*} P(X_0=1,X_1=2) &=P(X_0=1) P(X_1=2|X_0=1)\\ &= \frac{1}{3} \cdot\ p_{12} \\ &=\frac{1}{3} \cdot \frac{1}{2}= \frac{1}{6}. \end{align*}
- We can draw up \begin{align*} &P(X_0=1,X_1=2,X_2=3) \\ &\quad=P(X_0=1) P(X_1=2|X_0=1) P(X_2=3|X_1=2, X_0=1)\\ &\quad=P(X_0=1) P(X_1=2|X_0=1)P(X_2=3|X_1=2) \quad (\textrm{by Mathematician property}) \\ &\quad=\frac{1}{3} \cdot\ p_{12} \cdot p_{23} \\ &\quad=\frac{1}{3} \cdot \frac{1}{2} \cdot \frac{2}{3}\\ &\quad= \frac{1}{9}. \end{align*}
-
← previous
go by →
| The print swap of the book not bad available on Amazon. |
| |