Re: Optimization packages
- To: mathgroup at smc.vnet.net
- Subject: [mg29432] Re: [mg29417] Optimization packages
- From: "Mark Harder" <harderm at ucs.orst.edu>
- Date: Tue, 19 Jun 2001 05:35:47 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
Hossein,
I have used GlobalOptimization successfully for nonlinear regression for
the past year. If you contact Craig Loehle at his company. See below for
my response to someone who needed to optimize functions with >100 variables:
Art;
I have used GlobalOpt. in my work; however, not with nearly as many
variables as you need. I use the MultiStartMin (MSM) routine, which
(randomly?) selects starting variable values within a bounded volume you
set, then minimizes the objective function you provide for each starting
point. In my application, it is not obvious where good initial points would
be, although reasonable bounds on these can be assumed. Also, matrix models
can't be fit with the Mathematica NLR function without serious problems. I need a
global minimizer like MSM to give me starting points for a nonlinear fitting
procedure that seeks 2-6 variable solutions for the fit of a matrix-valued
model to matrix data. Naturally, multiple minima or neighborhoods around
them are reflected in the results. Once I get good initial estimates using
MSM, I can use the Mathematica FindMinimum function to refine the solution.
Unfortunately, confidence intervals are not provided by either MSM or
FindMinimum, so I will have to develop some sort of Monte Carlo resampling
to do this. The time required can be extensive, although timing will depend
on the form of the objective near its minima. In my case, my model is not
very sensitive to its parameters ( "parameters"="variables" in my usage), so
there are multiple minima which are broad or long basins which cause long
convergence times. Using ~100 starting points can take a couple of hours on
my 266MHz PentiumII, WinNT machine.
There is also a GlobalMinima routine that uses a "Grid Refinement"
approach. It is not appropriate for my problem; I forget why.
There is a Nonlinear Regression routine that incorporates constraints
and a choice of L2 or L1 norms; otherwise, it is very similar to Mathematica's
built-in NLR routine, i.e. it provides lots of statistics on the solutions,
but requires the data & expression arguments like Mathematica., which precludes use
of "black box" functions like mine, which take parameter vectors and return
error norms. At least, a gradient function does not have to be provided; a
numerical gradient calculation is provided (this is true for all the
GlobalOptimization routines.), which seems to be pretty robust.
To answer your other question, there is a MaxLikelihood function, and it
does return standard errors. However, it does not allow use of black-box
functions, since it is in the "data-expression" format used in the package's
NonlinearRegression routine. Furthermore, the documentation says it is
assumed that the model must be "algebraic". This term is a little obscure,
since the routine is most easily used when the function is one of a number
of distribution functions, which are not "algebraic" in my book.
I suggest that you contact Craig Loehle of Loehle Enterprises, who
developed the package. He has been very helpful to me in all phases of my
work with his package. I'm sure he will be willing to make recommendations
that are honest, if you describe your problem to him. I suggest that you
e-mail him (craigloehl at aol.com). He would be willing, I'm sure, to send you
an attatchment containing the manual (~100 p.). Also, it might be possible
to download a time-limited version to try out.
Also, are you aware of the book by M.Asghar Bhatti, "Practical
Optimization Methods: with Mathematica applications"? It is a textbook aimed
at optimization, per se, not data fitting, and its methods are not global.
It comes with a CD that includes all the routines described and developed in
the text.
-mark
-----Original Message-----
From: Hossein Kazemi <kazemi at som.umass.edu>
To: mathgroup at smc.vnet.net
Subject: [mg29432] [mg29417] Optimization packages
>I have been using the MultiplierMethod package from Mathsource for last 2
>years and have been happy with it most of the times. Now my needs have
>grown and have been checking into commercial packages; e.g., Global
>Optimization and Industrial Optimization. I would like to know if you have
>had any experience with these or other packages. Are they really better
>than MultiplierMethod?
>
>My needs involve nonlinear optimization with nonlinear equality an
>inequality constraints. The number of variables normally do not exceed 20.
>
>Thanks
>H. Kazemi
>Kazemi at som.umass.edu
>
>
>
>