Re: simple two step optimization
- To: mathgroup at smc.vnet.net
- Subject: [mg36960] Re: [mg36913] simple two step optimization
- From: Brett Champion <brettc at wolfram.com>
- Date: Thu, 3 Oct 2002 00:16:49 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
On Wednesday, October 2, 2002, at 02:31 AM, Johannes Ludsteck wrote:
> Dear MathGroup Members,
>
> I want to minimize a function which returns the
> minimizing value (arg min) of another function.
>
> For a simple example consider the following
> function opt which returns the arg min of x-2.5(1+Erf[x-s]).
>
> opt[s_]:=Block[{x}, x/. Last[
> FindMinimum[x-2.5(1+Erf[x-s]), {x,1,3}]]]
>
> Now in a second step I want (again this is only
> a simple example for illustrative purposes) to minimize
> (opt[s]-2)^2 with respect to s.
>
> FindMininum has no problems with this.
>
> FindMinimum[(opt[s]-2)^2,{s,0.9,1.1}]
> {3.18689*^-23, {s -> 0.9816}}\)
>
> However, NMinimize surrenders(!!!). Typing
>
> <<NumericalMath`NMinimize`
> NMinimize[(opt[s]-2)^2,{s,0.9,1.1}]
> only leads to the error message
>
> FindMinimum::fmnum: Objective function
> 0.1 - 2.5 (1. +Erf[0.1 - 1. s]) is not real at {x} = {1.}.
>
> There is nothing wrong with minimand. It has exactly
> one minimum in the Interval[{0.9,1.1}].
>
> I guess the reason is that NMinimize calls opt[s]
> not with a numerical value for s. This causes the
> problem, since opt again calls FindMinimum.
> Why? Can someone explain the failure and tell me
> how to avoid this drawback? Wolfram Research boasts
> that NMinimize can handle any function...
>
> I hope that nobody will recommend me to use FindMinimum
> here instead. I know that the example here could of
> course be solved by FindMinimum, but my real world
> application can not.
>
> Best regards and thanks in advance,
> Johannes Ludsteck
>
Ok, here's what is happening, and how to get this to work.
NMinimize, unlike FindMinimum, doesn't hold any of it's arguments,
which means that when you run
NMinimize[(opt[s]-2)^2, {s, 0.9, 1.1}]
the first argument
(opt[s]-2)^2
is evaluated, which gives the FindMinimum and ReplaceAll messages, and
returns
x /. {x, 1, 3}.
The result is that the call above is essentially the same as evaluating
NMinimize[x/.{x,1,3}, {s, 0.9, 1.1}].
The solution is to prevent symbolic evaluation of your objective
function. Here is one way to do it, by only defining the function for
Real numbers.
opt[s_Real] := Block[{x}, x /. Last[
FindMinimum[x - 2.5(1 + Erf[x - s]), {x, 1, 3}]]]
<< NumericalMath`NMinimize`
NMinimize[(opt[s] - 2)^2, {s, 0.9, 1.1}]
This gives the solution
{4.4699427618145155*^-25, {s -> 0.9816321026885765}}
There are a couple examples where objective functions are written to
only take numeric input in the NMinimize documentation ("Finding
Multiple Optima, Method 2" and "Queens on a Chessboard" under the
"Further Examples" section.) This will frequently be needed when the
objective function does structural operations (such as First, Last,
ReplaceAll, Part, etc...)
Hope this helps,
Brett Champion
Wolfram Research