Re: variable metric method automatic gradient yields Indeterminate
- To: mathgroup at smc.vnet.net
- Subject: [mg70426] Re: [mg70397] variable metric method automatic gradient yields Indeterminate
- From: Andrzej Kozlowski <akoz at mimuw.edu.pl>
- Date: Mon, 16 Oct 2006 02:34:28 -0400 (EDT)
- References: <200610150419.AAA12698@smc.vnet.net>
On 15 Oct 2006, at 13:19, Chris Chiasson wrote:
> A user-defined augmented lagrange multiplier method for NMinimize
> drives a (user defined) variable metric method for FindMinimum. The
> NMinimize routine ends up creating penalty functions that have
> "discontinuous" first order derivatives due to the presence of
> functions like Max. At the points of discontinuity in the derivative,
> the Indeterminate result usually ends up multiplied by zero. The limit
> of the derivative exists.
>
> for example, the function passed to FindMinimum is
> func=Max[0,-X[1]]^2+Max[0,-1+X[1]+X[2]]^2+(-1+X[1])^2+(-1+X[2])^2
>
> its derivative with respect to X[1] is
> 2*(-1+Max[0,-1+X[1]+X[2]]*Piecewise[{{1,X[1]+X[2]>1}},0]+
> Max[0,-X[1]]*Piecewise[{{-1,X[1]<0},{0,X[1]>0}},Indeterminate]+X[1])
>
> Notice that when X[1] is zero, the Piecewise returns Indeterminate,
> which is multiplied by zero from the nearby Max function. However,
> 0*Indeterminate is still Indeterminate in Mathematica. When the
> derivative is evaluated at X[1]=zero, the answer returned is
> Indeterminate. This totally messes up the numerical routine.
>
> the limit of D[func,X[1]] as X[1]->0 is
> Piecewise[{{-2,X[2]<1}},2*(-2 +X[2])]
>
> I am tempted to check the gradient vector for Indeterminate results,
> look up the "corresponding" variable (heh, how do I really know which
> one is responsible?), and take the limit as that variable approaches
> the value I wanted to evaluate at. I don't know how well that will
> work in practice.
>
> So, does anyone have any suggestions?
> --
> http://chris.chiasson.name/
>
It interesting to note that the following two methods give different
looking answers:
p = Block[{Indeterminate}, D[func, X[1]] /. X[1] -> 0]
2*Max[0, X[2] - 1]*Piecewise[{{1, X[2] > 1}}] - 2
q = Limit[D[func, X[1]], X[1] -> 0]
Piecewise[{{-2, X[2] < 1}}, 2*(X[2] - 2)]
but are, of course, equivalent:
Simplify[p == q]
True
(This is also meant as a suggestion of an alternative method, though
I would not recommend its unthinking use in other situations where
Indeterminate occurs).
Andrzej Kozlowski
- References:
- variable metric method automatic gradient yields Indeterminate
- From: "Chris Chiasson" <chris@chiasson.name>
- variable metric method automatic gradient yields Indeterminate