Re: variable metric method automatic gradient yields Indeterminate

*To*: mathgroup at smc.vnet.net*Subject*: [mg70433] Re: [mg70397] variable metric method automatic gradient yields Indeterminate*From*: "Chris Chiasson" <chris at chiasson.name>*Date*: Mon, 16 Oct 2006 02:34:52 -0400 (EDT)*References*: <200610150419.AAA12698@smc.vnet.net> <F474721B-1771-4C59-8CE9-7174C99CA71A@mimuw.edu.pl>

I've already implemented the limit approach, but I think your approach is better. It would be able to handle problems where the indeterminate expression arose from a different variable other than the one corresponding to that position in the gradient. On 10/15/06, Andrzej Kozlowski <akoz at mimuw.edu.pl> wrote: > > On 15 Oct 2006, at 13:19, Chris Chiasson wrote: > > > A user-defined augmented lagrange multiplier method for NMinimize > > drives a (user defined) variable metric method for FindMinimum. The > > NMinimize routine ends up creating penalty functions that have > > "discontinuous" first order derivatives due to the presence of > > functions like Max. At the points of discontinuity in the derivative, > > the Indeterminate result usually ends up multiplied by zero. The limit > > of the derivative exists. > > > > for example, the function passed to FindMinimum is > > func=Max[0,-X[1]]^2+Max[0,-1+X[1]+X[2]]^2+(-1+X[1])^2+(-1+X[2])^2 > > > > its derivative with respect to X[1] is > > 2*(-1+Max[0,-1+X[1]+X[2]]*Piecewise[{{1,X[1]+X[2]>1}},0]+ > > Max[0,-X[1]]*Piecewise[{{-1,X[1]<0},{0,X[1]>0}},Indeterminate]+X[1]) > > > > Notice that when X[1] is zero, the Piecewise returns Indeterminate, > > which is multiplied by zero from the nearby Max function. However, > > 0*Indeterminate is still Indeterminate in Mathematica. When the > > derivative is evaluated at X[1]=zero, the answer returned is > > Indeterminate. This totally messes up the numerical routine. > > > > the limit of D[func,X[1]] as X[1]->0 is > > Piecewise[{{-2,X[2]<1}},2*(-2 +X[2])] > > > > I am tempted to check the gradient vector for Indeterminate results, > > look up the "corresponding" variable (heh, how do I really know which > > one is responsible?), and take the limit as that variable approaches > > the value I wanted to evaluate at. I don't know how well that will > > work in practice. > > > > So, does anyone have any suggestions? > > -- > > http://chris.chiasson.name/ > > > > > It interesting to note that the following two methods give different > looking answers: > > > p = Block[{Indeterminate}, D[func, X[1]] /. X[1] -> 0] > > > 2*Max[0, X[2] - 1]*Piecewise[{{1, X[2] > 1}}] - 2 > > > q = Limit[D[func, X[1]], X[1] -> 0] > > > Piecewise[{{-2, X[2] < 1}}, 2*(X[2] - 2)] > > but are, of course, equivalent: > > Simplify[p == q] > True > > (This is also meant as a suggestion of an alternative method, though > I would not recommend its unthinking use in other situations where > Indeterminate occurs). > > Andrzej Kozlowski > > > > > -- http://chris.chiasson.name/

**References**:**variable metric method automatic gradient yields Indeterminate***From:*"Chris Chiasson" <chris@chiasson.name>