MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

Speeding up FindMinimum setup

  • To: mathgroup at
  • Subject: [mg86652] Speeding up FindMinimum setup
  • From: Art <grenander at>
  • Date: Sun, 16 Mar 2008 04:39:29 -0500 (EST)

I am having trouble with the time FindMinimum takes to make it's first
step in an optimization. I am doing something wrong but I am not sure
what. This is a repost of a previous question that I phrased poorly.

I am minimizing a convex f: R^n->R with derivative df: R^n->R^n using
FindMinimum. A toy example,

n = 1000;
f[k_] := k.k
df[k_] := 2k

I find the minimal k starting at random k0:

k0 = RandomReal[1, n];
ker = Array[k, n];

min = Reap[
  t0 = AbsoluteTime[];
  FindMinimum[f[ker], Transpose[{ker, k0}], Gradient -> df[ker],
    StepMonitor :> Sow[AbsoluteTime[] - t0]]

min[[2,1]] gives the times of the steps

(Above is a bad example as it converges in one step.)

For my particular convex f, FindMinimum takes a very long time before
it makes it's first step but then proceeds very quickly. And time
scales superlinearly with n. It takes so long that I have resorted to
a C Mathlink function that does BFGS or ConjugateGradient (from Press
et al., Numerical Recipes, v3) and calls f and df through Mathlink.

What is FindMinimum doing during this time before the first step?
Compiling f and df? It does this even when I precompile f and df and
specify Method. How do I specify my problem so it works faster? I
would like it to just start making optimization steps because it's
surprisingly efficient even with n large when it gets going.

Also, thanks to whoever wrote all the excellent tutorial documentation
for the optimization functions in Mathematica and the examples in


  • Prev by Date: RE: Re: Pi is not a real number - the News from Mathematica Mathgroup
  • Next by Date: Re: Re: ordering of multivariate polynomial terms
  • Previous by thread: Re: Exporting Graphs
  • Next by thread: Re: Speeding up FindMinimum setup