convex FindMinimum for a large vector
- To: mathgroup at smc.vnet.net
- Subject: [mg85712] convex FindMinimum for a large vector
- From: Art <grenander at gmail.com>
- Date: Tue, 19 Feb 2008 07:10:27 -0500 (EST)
What is the best way to formulate the following minimization. I am
trying to estimate the parameters of a Poisson generalized linear
model (GLM) by minimizing the negative loglikelihood. I think I am not
formulating the FindMinimum call for a vector in a good way. Any help
would be appreciated.
n = 10000; (* number of data points *)
d = 500; (* dimensionality of parameter being estimated *)
f[x_] := E^x (* convex, log concave function from reals to reals *)
(* data = a list of non-negative integers of length n *)
(* input = a list of {real vectors of dimensionality d} of length n*)
theta = Array[k, d]; (* would like to solve for vector k[i] *)
logl = Sum[ data[[i]] Log[ f[theta . input[[i]]] ] - f[theta .
input[[i]]] , {i,1,n} ]
logl = logl /. Log[E^x_] ->x (* Mathematica doesn't simplify these *)
FindMinimum[-logl, theta]
This function surprisingly works for n < 1000 (not Method->"Newton"
though), but otherwise doesn't return for me and uses a lot of memory.
Is there a better way to do the above so that it can handle larger n?
In principle, it's the size of d that should cause problems, but not
n. FindFit has no problem with optimizations several times larger and
is extremely fast. I think computing symbolic derivatives and so forth
must balloon the number of terms involved here.
Thanks!
Art