Mathematica 9 is now available
Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

convex FindMinimum for a large vector

  • To: mathgroup at smc.vnet.net
  • Subject: [mg85712] convex FindMinimum for a large vector
  • From: Art <grenander at gmail.com>
  • Date: Tue, 19 Feb 2008 07:10:27 -0500 (EST)

What is the best way to formulate the following minimization. I am
trying to estimate the parameters of a Poisson generalized linear
model (GLM) by minimizing the negative loglikelihood. I think I am not
formulating the FindMinimum call for a vector in a good way. Any help
would be appreciated.

n = 10000;    (* number of data points *)
d = 500;        (* dimensionality of parameter being estimated *)

f[x_] := E^x  (* convex, log concave function from reals to reals *)

(* data = a list of non-negative integers of length n *)
(* input = a list of {real vectors of dimensionality d} of length n*)

theta = Array[k, d];    (* would like to solve for vector k[i] *)

logl = Sum[ data[[i]] Log[ f[theta . input[[i]]] ]  -  f[theta .
input[[i]]] , {i,1,n} ]
logl = logl /. Log[E^x_] ->x    (* Mathematica doesn't simplify these *)

FindMinimum[-logl, theta]

This function surprisingly works for n < 1000 (not Method->"Newton"
though), but otherwise doesn't return for me and uses a lot of memory.
Is there a better way to do the above so that it can handle larger n?
In principle, it's the size of d that should cause problems, but not
n. FindFit has no problem with optimizations several times larger and
is extremely fast. I think computing symbolic derivatives and so forth
must balloon the number of terms involved here.

Thanks!
Art


  • Prev by Date: Re: Summation question
  • Next by Date: Re: NDSolve[] and Differential Equations: Problem solving two similar
  • Previous by thread: Re: Plotting points of the function at increments.
  • Next by thread: Re: convex FindMinimum for a large vector