Re: Logit Probit in Mathematica
- To: mathgroup at smc.vnet.net
- Subject: [mg32253] Re: [mg32243] Logit Probit in Mathematica
- From: "Johannes Ludsteck" <johannes.ludsteck at wiwi.uni-regensburg.de>
- Date: Wed, 9 Jan 2002 23:37:06 -0500 (EST)
- Organization: Universitaet Regensburg
- Sender: owner-wri-mathgroup at wolfram.com
In his mail, Sapsi asked:
How do i go about doing a logistic regression (i.e logit model) and do
i implement the probit model on Mathematica 2.0 and 4.0??
Answer:
Here is a simple implementation for the probit (the logit is
a straightforward variant).
First, define the log likelihood and the maximize it with
FindMinimum. Here y is the binary dependent variable, x is
a matrix of regressors and b the coefficient vector.
Unfortunately, FindMinimum is very slow. Compilation of
the likelihood function leads only to small increases in
speed.
ProbitLogLik=Function[{y,x,b},
Plus@@Log[MapThread[
If[#1\[Equal]0,1-#2,#2]&,
{y,CDF[NormalDistribution[0.,1.],x.b]}]]]
FindMinimum[
-ProbitLogLik[y,r,{b1,b2,b3,b4,b5}],
{b1,0},{b2,0},{b3,0},{b4,0},{b5,0},
Method->QuasiNewton]
If you need the variance-covariance matrix of the estimators,
you have to define the gradient and compute its outer product.
ProbitLogLikGrad[y_,x_,b_]:=
Block[{d=NormalDistribution[0,1],xb},
xb=Map[If[Abs[#]>4.8,4.8 Sign[#],#]&, x.b];
Plus@@MapThread[
#2 PDF[d,#3]/(#1-CDF[d,#3])&,{y,x,-xb}]]
Of course, you can make the procedure more comfortable
by defining a function which takes the starting values for
{b_1,...b_k} as a vector and then calls FindMinimum. This is
straightforward.
!!! !!! !!! !!!
HOWEVER, EVEN THOUGH I LOVE MATHEMATICA,
I WOULD SUGGEST TO DO THESE JOBS WITH A STANDARD
STATISTICS PACKAGE LIKE STATA OR GAUSS IF YOU HAVE LARGE DATASETS,
SINCE MATHEMATICA'S FINDMINIMUM IS NOT WELL SUITED FOR SUCH JOBS
(FOR EXAMPLE, STATA IS FASTER BY A FACTOR OF ABOUT 100-300!!!).
I futily hoped for a long time that
Wolfram Research programmers improve FindMinimum for
optimization procedures with large data sets. But I don't know
whether this is possible for a general purpose symbolic language
like Mathematica.
Best regards,
Johannes
<><><><><><><><><><><><><><><><><><>
Johannes Ludsteck
Institut fuer Volkswirtschaftslehre
Lehrstuhl Prof. Dr. Moeller
Universitaet Regensburg
Universitaetsstrasse 31
93053 Regensburg
Tel +49/0941/943-2741