Steady-State Solutions of Markov Chains In this chapter, we restrict ourselves to the computation of the steady-state probability vector’ of ergo&c Markov chains. Most of the literature on solution techniques of Markov chains assumes ergodicity of the underlying model. A comprehensive source on algorithms for steady-state solution techniques is the book by Stewart [Stew94]. From Eq. (2.15) and Eq. (2.58), we have v = VP and 0 = nQ, respectively, as points of departure for the study of steady-sta