Re: Optimization
- To: mathgroup at yoda.physics.unc.edu
- Subject: Re: Optimization
- From: keiper
- Date: Sat, 17 Apr 93 16:49:31 CDT
> Steepest descent is a terrible algorithm!
> Does anyone know, is FindMinimum really being this stupid?
Of course not! The sentence "FindMinimum works by following the path
of steepest descent from each point that it reaches." is meant to help
a user better understand what the problem is that is being solved. It
is generally true that the algorithm follows a path that roughly
corresponds to the path of steepest descent. That is all that the
statement is meant to convey. If you want to see exactly what path it
is following do something like
path = {}
FindMinimum[AppendTo[path, {x, y}]; f[x, y], {x, x0}, {y, y0}]
ListPlot[path, PlotJoined -> True]
A brief description of the various algorithms follows. More details can
be gotten from the advanced-course notes or the tutorial notes of the
1992 Mathematica Conference (available from Wolfram Research, Inc.) from
which the following is taken.
The basic method for finding the minimum has two parts. One part is a
one dimensional minimization algorithm that is based on one of Brent's
two minimization algorithms. Which of these algorithms is used depends
upon whether information about the derivative of the objective function
is to be used. If the objective function is a function of a single
variable this part is all that is needed. If there are several variables
involved another part of the algorithm is needed. This part of the algorithm
attempts to choose the direction to go from the current point to the minimum.
If this direction can be chosen correctly then searching in that direction
with the appropriate algorithm of Brent will find the desired minimum.
There are also two direction choosing algorithms: one using derivative
information and the using information accumulated as a "direction set".
Jerry B. Keiper
keiper at wri.com
Wolfram Research, Inc.