Skip to main content
MathidiaMathematics Academy
Stochastic processes
Lesson 36 of 75
Introduction

Discrete Markov chains

Objective. Use transition matrices and communicating classes.

  1. 1. Learn
  2. 2. Worked example
  3. 3. Practice
  4. 4. Feedback
  5. 5. Continue

Learn: the key idea

A discrete Markov chain is a sequence of random states where the probability of moving to the next state depends only on the current state, encoded in a transition matrix P whose rows sum to one. Entry Pᵢⱼ gives the probability of moving from state i to state j in one step, and the probability of being in each state after n steps is found from the initial distribution multiplied by Pⁿ. States that can reach each other with positive probability form communicating classes, and a chain is irreducible if all states communicate in one class. Example: for a two-state chain with P = [[0.5,0.5],[0.2,0.8]], starting in state 1 gives a 50% chance of state 2 after one step.

Worked example

For the Markov chain with transition matrix P = [[0.5, 0.5], [0.2, 0.8]] and initial distribution π₀ = (1, 0), find the distribution after one step.

0.5 … 0.5
Illustrated card showing 0.5 … 0.5
  1. 1The distribution after one step is π₁ = π₀P, treating π₀ as a row vector.
  2. 2Compute the first entry: 1×0.5 + 0×0.2 = 0.5.
  3. 3Compute the second entry: 1×0.5 + 0×0.8 = 0.5.
  4. 4So π₁ = (0.5, 0.5), matching the first row of P since the chain started entirely in state 1.

Answer: π₁ = (0.5, 0.5)

Concept mastery

Based on your past attempts at this lesson — the question types to practise again come first.

Complete a practice set to see your concept breakdown here.

Practice

Practice happens on its own screen, one question at a time. Answers stay hidden until you submit yours, and you can stop and pick up at the same question later.

Start practising

Common mistake. Students sometimes multiply the transition matrix by the state vector on the wrong side; for row-stochastic P, the distribution after one step is (row vector)×P, not P×(row vector), since rows represent 'from' states.

Your progress

Tick each step as you finish it. Your exact reading position is saved automatically, so Continue drops you back on the same line.

0 of 5 steps complete

Your place saves automatically as you read.

My dashboard