[Date Index] [Thread Index] [Author Index]

Distribution of state occupancies in a multistate Markov model

• To: mathgroup at smc.vnet.net
• Subject: [mg104349] Distribution of state occupancies in a multistate Markov model
• From: mathlawguy <chandler.seth at gmail.com>
• Date: Wed, 28 Oct 2009 04:09:48 -0500 (EST)

This is half a math question and half a Mathematica question.

The math part:

Suppose I have some discrete Markov process. The process can have n
states where n is finite but can definitely be more than 2. The
process starts at time 0 in some state s[0]. I want to compute a
formula for the probability that as of time t (t an integer and t>0),
the Markov process will have been in state m, a total of q times,
where obviously 0<=q<=n. I want an analytic PDF.

So the math question is whether anyone has a reference that answers
this question or has a good idea how to search for the answer to this
problem. I keep seeing terms like sojourn time being thrown around. It
seems like it it is related to what I am discussing, but I can't find
a definition of the term.  I have also run across something called the
state occupancy random variable, which seems to be what I am
discussing, but I can't find any reference that computes an actual
PDF. Instead, I get means and variances. But I KNOW the distribution
is NOT normal since q is always bounded [0,n].

The Mathematica part.

Anyone have ideas on how to develop an answer to this question using
Mathematica without using simulations?

Extra credit:

Suppose the Markov process is -- as mine actually is -- time
inhomogeneous. The transition probabilities change with each blessed
iteration. How then would one compute a distribution for q?

Thanks,

Seth J. Chandler
Professor of Law
University of Houston Law Center

P.S. This issue is relevant to various computations in long term care
insurance. If I can't find something approaching an analytic answer, I
get a very involved numeric problem.