Speeding up FindMinimum setup
- To: mathgroup at smc.vnet.net
- Subject: [mg86652] Speeding up FindMinimum setup
- From: Art <grenander at gmail.com>
- Date: Sun, 16 Mar 2008 04:39:29 -0500 (EST)
I am having trouble with the time FindMinimum takes to make it's first
step in an optimization. I am doing something wrong but I am not sure
what. This is a repost of a previous question that I phrased poorly.
I am minimizing a convex f: R^n->R with derivative df: R^n->R^n using
FindMinimum. A toy example,
n = 1000;
f[k_] := k.k
df[k_] := 2k
I find the minimal k starting at random k0:
k0 = RandomReal[1, n];
ker = Array[k, n];
min = Reap[
t0 = AbsoluteTime[];
FindMinimum[f[ker], Transpose[{ker, k0}], Gradient -> df[ker],
StepMonitor :> Sow[AbsoluteTime[] - t0]]
];
min[[2,1]] gives the times of the steps
(Above is a bad example as it converges in one step.)
For my particular convex f, FindMinimum takes a very long time before
it makes it's first step but then proceeds very quickly. And time
scales superlinearly with n. It takes so long that I have resorted to
a C Mathlink function that does BFGS or ConjugateGradient (from Press
et al., Numerical Recipes, v3) and calls f and df through Mathlink.
What is FindMinimum doing during this time before the first step?
Compiling f and df? It does this even when I precompile f and df and
specify Method. How do I specify my problem so it works faster? I
would like it to just start making optimization steps because it's
surprisingly efficient even with n large when it gets going.
Also, thanks to whoever wrote all the excellent tutorial documentation
for the optimization functions in Mathematica and the examples in
UnconstrainedProblems.m.
Art.
- Follow-Ups:
- Re: Speeding up FindMinimum setup
- From: Carl Woll <carlw@wolfram.com>
- Re: Speeding up FindMinimum setup