Since the discussion of the topic in the course is constrained to finite Markov Chains, the following will be consistent with this.
Models of random state transitions where the probability of transitioning to a state depends only on the state before it. A Markov chain is fully characterized by the following attributes:
$\mathcal{K} \sub \N$ State Space. Set of all possible states.
$X_i \in \mathcal{K}$ State. State of the chain at time step $i$, for $i \in \N$.
$P \in \R^{|\mathcal{K}| \times |\mathcal{K}|}$ Transition Probability Matrix.
- $P(i,j) = \Pr[X_{n+1} = j \space | \space X_{n} = i]$
- $\sum_{j \in \mathcal{K}} P(i,j) = 1$
$\boldsymbol{\pi}_0 \in \R^{1 \times |\mathcal{K}|}$ Initial Distribution. Distribution of state probabilities at time step $0$.
In this case, we have $\mathcal{K} = \{A, B, C\}$.
The probability of a particular sequence occurring at a time step $n$ can be related with
$$ \Pr[X_0 = i_0, \space \dots, \space X_n = i_n] = \boldsymbol{\pi}_0(i_0) \prod {i=1}^n P(i{n-1}, i_n) $$
This can be extended to show that the distribution of state probabilities at state $X_n$ depends only on the initial distribution and the transition probability matrix.
$$ \boldsymbol{\pi}_n=\boldsymbol{\pi}_0 P^n \iff \Pr[X_n =i] = (\boldsymbol{\pi}_0P^n)(i) $$
The main insight the proof provides is that higher powers of the transition probability matrix essentially represent the probability of moving to a state after $n$ steps (i.e. $P^n(i,j)$ is the probability of starting at state $i$ and ending in state $j$ after $n$ steps).
A Markov chain is said to be irreducible if it can go from any state $i$ to any state $j$ with non-zero probability.
Figure (a) represents a reducible Markov chain, whereas Figure (b) is an irreducible one.