MathGroup Archive 2000

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Gradient in FindMininum

    Getting FindMinimum (& NonlinearRegression) to work when gradients, and
for that matter, the target function, cannot be found symbolicly is
currently a real problem of mine, and I'm seeking professional counselling
for it (from Support at Wolfram, ;-) ).  Your problem, I think , is tractable.
Taking your example,
f[x_] := x^2 + NIntegrate[Exp[-t^2], {t, -Infinity, x}];

I tried to find a symbolic derivative &
D[f[x], x]

NIntegrate::"nintp": "Encountered the non-number \!\(x\) at \!\({t}\) = \

Yes, it is indeterminate, but setting up FM[] with Method ->Gradient, and a
variable list specifying an
interval around 1. works:
FindMinimum[f[x], {x, {.99, 1.01}}, Method -> Gradient]

{0.666069, {x -> -0.419365}}.

But, is this what you want for your full-blown problem? i.e. What did you
mean by
>(minimization algorithms which don't use the gradient
>are impracticable.)
    Now, there is a numerical derivatives routine in the
NumericMath`NLimits` package, called ND[]. It requires evaluation of a
symbolic function (which won't work for me: I need to use a Pseudoinverse,
which takes forever to evaluate symbolically, or SVD, which *only* works on
numeric arguments), & it complains about your function, then goes ahead &
returns what look like reasonable results, to me anyway:
{ND[f[x], x, 1], ND[f[x], x, -.419365], ND[f[x], x, -1] }

NIntegrate::"nintp": "Encountered the non-number \!\(x\) at \!\({t}\) = \
NIntegrate::"nintp": "Encountered the non-number \!\(x\) at \!\({t}\) = \
NIntegrate::"nintp": "Encountered the non-number \!\(x\) at \!\({t}\) = \
General::"stop": "Further output of \!\(NIntegrate :: \"nintp\"\) will be \
suppressed during this calculation."

Out[322]={2.3678794389572717`, -4.789926799402722`*^-7, -1.6321205594031925`

So, if the "gradient-free" method of optimization I proposed above won't
work for you, perhaps you could construct a numeric gradient function with
ND[].  If that doesn't work, I'd really like to know how to solve your
complicated problem, too.
-mark harder
harderm at

-----Original Message-----
From: Johannes Ludsteck <ludsteck at>
To: mathgroup at
Subject: [mg24049] [mg24006] Gradient in FindMininum

>Dear MathGroup Members,
>Unfortunately I got no answer when I sent the question below last
>week to the mathgroup mailing list. Since I think that the problem
>is not a very special one but a general problem of the way how
>Mathematica treats numerical integrals in the computation of
>gradients, I retry to get an answer.
>I want to minimize a complicated function which contains
>numerical integrals. Since the function is too complicated for a
>direct demonstration, I give a simple example which makes the
>structure of the problem clear:
>The (example) function to be minimized is:
>f[x_] := NIntegrate[g[t], {t, -Infinity, x}]
>(g is a known function; however symbolical integration is
>When I request numerical minimization of this function by typing
>Mathematica gives me the following error message:
>FindMinimum::fmgl: Gradient {Indeterminate} is not a length 1
>list of real numbers at {x} = {1.}.
>Appearently, Mathematica is not able to find the gradient
>symbolically. A simple solution would be to define f using Integrate
>(without prefix N) and to wrap it with N[ ]:
>f[x_]:= N[  Integrate[g[t], {t,-Infinity, x}] ]
>However, since the function contains some hundred terms,
>evaluation of the function takes several minutes. (Mathematica then
>tries to find the integral symbolically before applying the numerical
>integration procedure.) This makes optimization impracticable.
>(the function I want to optimize has about 40 variables!).
>Are there any suggestions how to avoid computation of the gradient
>manually? (minimization algorithms which don't use the gradient
>are impracticable.)
>I.e. how can I tell Mathematica to use the first definition
>f[x_] := NIntegrate[g[t], {t, -Infinity, x}]
>for evaluation of the function and the second
>f[x_]:= N[ Integrate[g[t], {t,-Infinity, x}] ]
>for the computation of the gradient.
>Thank you
>P.S If you want to reproduce the error message, you can use a
>simple definition:
>f[x_]:= x^2 + NIntegrate[ Exp[-t^2], {t, -Infinity, x} ].
>Johannes Ludsteck
>Centre for European Economic Research (ZEW)
>Department of Labour Economics,
>Human Resources and Social Policy
>Phone (+49)(0)621/1235-157
>Fax (+49)(0)621/1235-225
>P.O.Box 103443
>D-68034 Mannheim
>Email: ludsteck at

  • Prev by Date: Precision example
  • Next by Date: Re: imposing side conditions on Solve
  • Previous by thread: RE: Gradient in FindMininum
  • Next by thread: Findminimum computation time (ignore my previous posting)