Re: Optimization

*To*: mathgroup at yoda.physics.unc.edu*Subject*: Re: Optimization*From*: keiper*Date*: Sat, 17 Apr 93 16:49:31 CDT

> Steepest descent is a terrible algorithm! > Does anyone know, is FindMinimum really being this stupid? Of course not! The sentence "FindMinimum works by following the path of steepest descent from each point that it reaches." is meant to help a user better understand what the problem is that is being solved. It is generally true that the algorithm follows a path that roughly corresponds to the path of steepest descent. That is all that the statement is meant to convey. If you want to see exactly what path it is following do something like path = {} FindMinimum[AppendTo[path, {x, y}]; f[x, y], {x, x0}, {y, y0}] ListPlot[path, PlotJoined -> True] A brief description of the various algorithms follows. More details can be gotten from the advanced-course notes or the tutorial notes of the 1992 Mathematica Conference (available from Wolfram Research, Inc.) from which the following is taken. The basic method for finding the minimum has two parts. One part is a one dimensional minimization algorithm that is based on one of Brent's two minimization algorithms. Which of these algorithms is used depends upon whether information about the derivative of the objective function is to be used. If the objective function is a function of a single variable this part is all that is needed. If there are several variables involved another part of the algorithm is needed. This part of the algorithm attempts to choose the direction to go from the current point to the minimum. If this direction can be chosen correctly then searching in that direction with the appropriate algorithm of Brent will find the desired minimum. There are also two direction choosing algorithms: one using derivative information and the using information accumulated as a "direction set". Jerry B. Keiper keiper at wri.com Wolfram Research, Inc.