Re: Gradient in FindMininum
- To: mathgroup at smc.vnet.net
- Subject: [mg24049] Re: [mg24006] Gradient in FindMininum
- From: "Mark Harder" <harderm at ucs.orst.edu>
- Date: Wed, 21 Jun 2000 02:20:09 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
Johannes,
Getting FindMinimum (& NonlinearRegression) to work when gradients, and
for that matter, the target function, cannot be found symbolicly is
currently a real problem of mine, and I'm seeking professional counselling
for it (from Support at Wolfram, ;-) ). Your problem, I think , is tractable.
Taking your example,
In[317]:=
f[x_] := x^2 + NIntegrate[Exp[-t^2], {t, -Infinity, x}];
I tried to find a symbolic derivative &
In[318]:=
D[f[x], x]
NIntegrate::"nintp": "Encountered the non-number \!\(x\) at \!\({t}\) = \
\!\({t}\)."
Out[318]=
Indeterminate
Yes, it is indeterminate, but setting up FM[] with Method ->Gradient, and a
variable list specifying an
interval around 1. works:
In[319]:=
FindMinimum[f[x], {x, {.99, 1.01}}, Method -> Gradient]
Out[319]=
{0.666069, {x -> -0.419365}}.
But, is this what you want for your full-blown problem? i.e. What did you
mean by
>(minimization algorithms which don't use the gradient
>are impracticable.)
???
Now, there is a numerical derivatives routine in the
NumericMath`NLimits` package, called ND[]. It requires evaluation of a
symbolic function (which won't work for me: I need to use a Pseudoinverse,
which takes forever to evaluate symbolically, or SVD, which *only* works on
numeric arguments), & it complains about your function, then goes ahead &
returns what look like reasonable results, to me anyway:
In[322]:=
{ND[f[x], x, 1], ND[f[x], x, -.419365], ND[f[x], x, -1] }
NIntegrate::"nintp": "Encountered the non-number \!\(x\) at \!\({t}\) = \
\!\({t}\)."
NIntegrate::"nintp": "Encountered the non-number \!\(x\) at \!\({t}\) = \
\!\({t}\)."
NIntegrate::"nintp": "Encountered the non-number \!\(x\) at \!\({t}\) = \
\!\({t}\)."
General::"stop": "Further output of \!\(NIntegrate :: \"nintp\"\) will be \
suppressed during this calculation."
Out[322]={2.3678794389572717`, -4.789926799402722`*^-7, -1.6321205594031925`
So, if the "gradient-free" method of optimization I proposed above won't
work for you, perhaps you could construct a numeric gradient function with
ND[]. If that doesn't work, I'd really like to know how to solve your
complicated problem, too.
-mark harder
harderm at ucs.orst.edu
-----Original Message-----
From: Johannes Ludsteck <ludsteck at zew.de>
To: mathgroup at smc.vnet.net
Subject: [mg24049] [mg24006] Gradient in FindMininum
>Dear MathGroup Members,
>Unfortunately I got no answer when I sent the question below last
>week to the mathgroup mailing list. Since I think that the problem
>is not a very special one but a general problem of the way how
>Mathematica treats numerical integrals in the computation of
>gradients, I retry to get an answer.
>I want to minimize a complicated function which contains
>numerical integrals. Since the function is too complicated for a
>direct demonstration, I give a simple example which makes the
>structure of the problem clear:
>
>The (example) function to be minimized is:
>f[x_] := NIntegrate[g[t], {t, -Infinity, x}]
>
>(g is a known function; however symbolical integration is
>impossible).
>
>When I request numerical minimization of this function by typing
>
>FindMinimum[f[x],{x,1}]
>
>Mathematica gives me the following error message:
>
>FindMinimum::fmgl: Gradient {Indeterminate} is not a length 1
>list of real numbers at {x} = {1.}.
>
>Appearently, Mathematica is not able to find the gradient
>symbolically. A simple solution would be to define f using Integrate
>(without prefix N) and to wrap it with N[ ]:
>
>f[x_]:= N[ Integrate[g[t], {t,-Infinity, x}] ]
>
>However, since the function contains some hundred terms,
>evaluation of the function takes several minutes. (Mathematica then
>tries to find the integral symbolically before applying the numerical
>integration procedure.) This makes optimization impracticable.
>(the function I want to optimize has about 40 variables!).
>
>Are there any suggestions how to avoid computation of the gradient
>manually? (minimization algorithms which don't use the gradient
>are impracticable.)
>I.e. how can I tell Mathematica to use the first definition
>
>f[x_] := NIntegrate[g[t], {t, -Infinity, x}]
>
>for evaluation of the function and the second
>
>f[x_]:= N[ Integrate[g[t], {t,-Infinity, x}] ]
>
>for the computation of the gradient.
>
>Thank you
>
>P.S If you want to reproduce the error message, you can use a
>simple definition:
>
>f[x_]:= x^2 + NIntegrate[ Exp[-t^2], {t, -Infinity, x} ].
>
>
>
>
>Johannes Ludsteck
>Centre for European Economic Research (ZEW)
>Department of Labour Economics,
>Human Resources and Social Policy
>Phone (+49)(0)621/1235-157
>Fax (+49)(0)621/1235-225
>
>P.O.Box 103443
>D-68034 Mannheim
>GERMANY
>
>Email: ludsteck at zew.de
>