Steady State In A Markov Process

Abstract

It is well known that a Markov process whose transition matrix is regular approaches a steady-state distribution, or equilibrium distribution. To find these steady-state probabilities requires the solution of a system of linear homogenous equations. However, the matrix of this system is singular and thus the system has infinitely many solutions. This obstacle is overcome by replacing one of the equations of the linear homogenous system by the linear non-homogeneous equation that simply expresses the requirement that the steady-state probabilities sum to one. But which equation of the original system should be chosen to be the one replaced. This brief article demonstrates that any of the equations of the original linear system can be selected as the one to be replaced; no matter which one is selected for replacement; the revised linear system will have the same unique solution.