# Latest content was relocated to https://bintanvictor.wordpress.com. This old blog will be shutdown soon.

## Wednesday, May 30, 2012

### simple solution to a simple Markov chain

Suppose there are 10000 particles. In the steady state, a number of them (say U) are in Bull state, a number of them (say E) are in Bear state, and R are in Recession state. They add up to the total population of 10000. Let's find the values of U/E/R, i.e. in the steady state, how many particles will be in each state. Note unlike most MC problems, there's no absorption in this problem.

After one step, all the particles each decide its next state, according to the transition probabilities. 2.5% of those in Bull state would change to Recession, while 90% of them would remain  in Bull.

0.025 U + 0.05 E + 0.5 R = R ..... one of 3 equations. 3 equations and 3 unknowns. U is found to be 6250, E = 3125 and R = 625

But why the " ... = R" part? Answer is the steady state. After one step, if 0.025 U + 0.05 E + 0.5 R is 0.000001% more than R, then this is not steady state and the R population will increase at every step and it would capture all 10000 particles.