Re: Numerical Optimization of
- To: mathgroup at smc.vnet.net
- Subject: [mg44469] Re: Numerical Optimization of
- From: Brian Rogers <rogers at tanis.hss.caltech.edu>
- Date: Mon, 10 Nov 2003 19:55:55 -0500 (EST)
- Organization: California Institute of Technology, Pasadena
- References: <bonnhd$h8t$1@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
Thanks for the reply!
Here are more details of my program.
The function I care to optimize is NegLogLikelihood[lambda] (which I do
believe actually should be convex, but I'm not positive)--see below.
datamat is a matrix that contains my data.
publicbelief = PA = 0.5.
siga[] and sigb[], and postplus[] and postminus[] compute a Baye's Law
application that depends on a CDF with parameter \lambda.
Just to confirm, I'm actually not using a function with multidimensional
output here, so that is not the problem.
Plot[NegLogLikelihood[z],{z,8,12}] works fine, but
FindMinimum[NegLogLikelihood[z],{z,6,7}] and
NMinimize[NegLogLikelihood[z],{z}] do not--they run forever.
If I replace nMatch by the number 2 in NegLogLikelihood[], then the plot
appears almost instantly, and the optimization takes 36 sec of CPU time
(using Timing[]), which seems way too long. Could something be wrong with
the IF statement?
Thanks so much again.
-Brian
In[130]:=
(*likelihood function*)
NegLogLikelihood[lambda_]:=(
\[Lambda]=lambda;
like=0;
(*SetAttributes[like,HoldAll];*)
For[m=1,m\[LessEqual] nMatch,m++,
publicBelief=PA; (*A=0, B=1*)
For[s=1,s\[LessEqual]nSub,s++,
curSignal=datamat\[LeftDoubleBracket]m,s,1\[RightDoubleBracket];
curDec=datamat\[LeftDoubleBracket]m,s,2\[RightDoubleBracket];
PcurDec=
If[curSignal\[Equal]0,
If[curDec\[Equal]0,siga[publicBelief],1-siga[publicBelief]],
If[curDec\[Equal]0,sigb[publicBelief],1-sigb[publicBelief]]];
like-=Log[PcurDec];
publicBelief=
If[curDec\[Equal]0,postplus[publicBelief],postminus[publicBelief]];
];(*s*)
];(*m*)
Return[like];
)
At 03:20 PM 11/10/03, you wrote:
>Reply.
>
>"Bill Rowe" <readnewscix at mail.earthlink.net> wrote in message
>news:<bonnhd$h8t$1 at smc.vnet.net>...
> > On 11/9/03 at 6:01 AM, rogers at tanis.hss.caltech.edu (Brian Rogers) wrote:
> >
> > > I have a function f[x] of one variable that computes it's output by
> > > solving a system of equations and returning a list of three numbers.
> > > So f[x] looks like {f[x][[1]],f[x][[2]],f[x][[3]]}. When I plot
> > > f[x][[1]], I get a nice smooth strictly convex function that clearly
> > > has a global minimum around 6.7. But when I try to find the minimum
> > > I can't. Specifically, using NMinimize[f[x][[1]],{x}] and
> > > FindMinimum[f[x][[1]],{x,6}] both run forever without returning any
> > > errors or results.
> >
> > In principle, there should be no problems finding the minimum for a
> function returning using the
>syntax you've outlined above. The problem is undoubtedly related to the
>details of your function
>which you didn't supply. For me to offer any specific guidance, I need the
>details of the function.
> >
> > I will note that simply because Mathematica plots a function as
> strictly convex and smooth does
>not mean that it actually is strictly convex and smooth. For example
>consider plotting the function
> >
> > (x-1)^2 + Abs[x-1]^-.0001
> >
> > With Mathematica 5.0 on a Mac running OS 10.3, the plot looks strictly
> convex and smooth when
>plotted from x = 0 to x = 2. Yet it clearly has a singularity at x = 1.
>Remeber, Mathematica samples
>your function and makes assumptions about the behaviour of your function
>between the sampled points.
>Regardless of what plotting algorithm is chosen, it will always be
>possible to find examples where
>the resulting plot does not accurately reflect the true behaviour of the
>function.
> > --
> > To reply via email subtract one hundred and nine
> >
***************************************
Brian W. Rogers
California Institute of Technology
Mail Code 228-77
Pasadena, CA 91125
(626)395-2171
rogers at hss.caltech.edu
***************************************
On Mon, 10 Nov 2003, Bill Rowe wrote:
> On 11/9/03 at 6:01 AM, rogers at tanis.hss.caltech.edu (Brian Rogers) wrote:
>
> > I have a function f[x] of one variable that computes it's output by
> > solving a system of equations and returning a list of three numbers.
> > So f[x] looks like {f[x][[1]],f[x][[2]],f[x][[3]]}. When I plot
> > f[x][[1]], I get a nice smooth strictly convex function that clearly
> > has a global minimum around 6.7. But when I try to find the minimum
> > I can't. Specifically, using NMinimize[f[x][[1]],{x}] and
> > FindMinimum[f[x][[1]],{x,6}] both run forever without returning any
> > errors or results.
>
> In principle, there should be no problems finding the minimum for a function returning using the syntax you've outlined above. The problem is undoubtedly related to the details of your function which you didn't supply. For me to offer any specific guidance, I need the details of the function.
>
> I will note that simply because Mathematica plots a function as strictly convex and smooth does not mean that it actually is strictly convex and smooth. For example consider plotting the function
>
> (x-1)^2 + Abs[x-1]^-.0001
>
> With Mathematica 5.0 on a Mac running OS 10.3, the plot looks strictly convex and smooth when plotted from x = 0 to x = 2. Yet it clearly has a singularity at x = 1. Remeber, Mathematica samples your function and makes assumptions about the behaviour of your function between the sampled points. Regardless of what plotting algorithm is chosen, it will always be possible to find examples where the resulting plot does not accurately reflect the true behaviour of the function.
> --
> To reply via email subtract one hundred and nine
>
>
>