Distribution of state occupancies in a multistate Markov model

*To*: mathgroup at smc.vnet.net*Subject*: [mg104349] Distribution of state occupancies in a multistate Markov model*From*: mathlawguy <chandler.seth at gmail.com>*Date*: Wed, 28 Oct 2009 04:09:48 -0500 (EST)

This is half a math question and half a Mathematica question. The math part: Suppose I have some discrete Markov process. The process can have n states where n is finite but can definitely be more than 2. The process starts at time 0 in some state s[0]. I want to compute a formula for the probability that as of time t (t an integer and t>0), the Markov process will have been in state m, a total of q times, where obviously 0<=q<=n. I want an analytic PDF. So the math question is whether anyone has a reference that answers this question or has a good idea how to search for the answer to this problem. I keep seeing terms like sojourn time being thrown around. It seems like it it is related to what I am discussing, but I can't find a definition of the term. I have also run across something called the state occupancy random variable, which seems to be what I am discussing, but I can't find any reference that computes an actual PDF. Instead, I get means and variances. But I KNOW the distribution is NOT normal since q is always bounded [0,n]. The Mathematica part. Anyone have ideas on how to develop an answer to this question using Mathematica without using simulations? Extra credit: Suppose the Markov process is -- as mine actually is -- time inhomogeneous. The transition probabilities change with each blessed iteration. How then would one compute a distribution for q? Thanks, Seth J. Chandler Professor of Law University of Houston Law Center P.S. This issue is relevant to various computations in long term care insurance. If I can't find something approaching an analytic answer, I get a very involved numeric problem.