Re: Using LevenbergMarquardt Method with a complicated function

*To*: mathgroup at smc.vnet.net*Subject*: [mg53084] Re: Using LevenbergMarquardt Method with a complicated function*From*: algaba at alumni.uv.es (algaba)*Date*: Thu, 23 Dec 2004 07:58:14 -0500 (EST)*Sender*: owner-wri-mathgroup at wolfram.com

Seems it doesn't work. I have tried some examples to understand the proceeding and it worked, but when I wanted to implement it on the program it fails. I have tried to add the instruction R = Join[ResAlpha, ResDelta, ResAlphaComp, ResDeltaComp]; to specify the residual explicitly and use it in the minimization, but it gives me the error message FindMinimum::fmgz: Encountered a gradient which is effectively zero. The \ result returned may not be a minimum; it may be a maximum or a saddle point. It is true that mathematica now gives me some results that, in principle, appear to be good, but in the next step I see this is not the case: R is a null list and posterior work with data gives me indeterminates expressions. I think now the problem is that the residuals are not constants but can vary (in fact, the number of residuals can also be different each time we run the algorith), but I don't know how does it affect to mathematica. I have also tried to remove some summands or introduce the residuals in other several ways, such as creating a list before the execution, inside the function, just when implementing L-M method, etc. So, think I'm in the correct way, because I get some (but surely false) results. So, what's the next step? Thanks. --- algaba wrote: > Hi. > I have defined a very long function like this: > > ChiSquare[Per0_?NumericQ, Ppa0_?NumericQ, Ecc0_?NumericQ] := > > (some steps and definitions here) > > ChiSQ = Sum[ResAlpha[[i]]^2 + ResDelta[[i]]^2, {i, 1, Length[ > TExp]}] + Sum[ResAlphaComp[[i]]^2 + > ResDeltaComp[[i]]^2, {i, 1, Length[TComp]}]); > > which tries to find the Chi-Square of an array of data. Now, I want to > minimize it and I use FindMinimum, which works well. The problem > arises when I want to use the Levenberg-Marquardt method, which seems > to be better for this kind of functions (As you can see, it is a sum > of squares) But when I run Mathematica 5 it gives me the next error > message: > > FindMinimum::notlm: The objective function for the method > LevenbergMarquardt \ > must be in a least-squares form: Sum[f[i][x]^2,{i,1,n}] or Sum[w[i] \ > f[i][x]^2,{i,1,n}] with positive w[i]. > > I think the function accomplishes all the requirements. Why I get this > error? Is it maybe because of the long definition of the function? Is > it because Mathematica doesn't see this function as a sum of squares > but as a sequence of steps? > What can I do to solve this problem? I do want to use this method to > minimize the Chi-Square. Thanks. > Because the definition is set up to only evaluate with numerical values of the arguments, Mathematica cannot do the computations necessary to decompose the sum of squares. To use the Levenberg-Marquardt method, the function needs to be decomposed into a residual function r[X] such that f[X] = r[X].r[X]/2. The LevenbergMarquardt method has a method option that allows you to specify the residual explicitly for cases like this where you may not want the sum of squares evaluated symbolically. Here is a simple example that should give you an idea of how it works: In[1]:= SS[x_?NumberQ, y_?NumberQ] := (x - 1)^2 + 100 (y - x^2)^2 In[2]:= FindMinimum[SS[x,y],{{x,1},{y,-1}}, Method->"LevenbergMarquardt"] >From In[2]:= FindMinimum::notlm: The objective function for the method LevenbergMarquardt must be in a least-squares form: Sum[f[i][x]^2,{i,1,n}] or Sum[w[i] f[i][x]^2,{i,1,n}] with positive w[i]. More... Out[2]= FindMinimum[SS[x, y], {{x, 1}, {y, -1}}, Method -> LevenbergMarquardt] In[3]:= R[x_?NumberQ, y_?NumberQ] := Sqrt[2] {x-1, 10 (y -x^2)} In[4]:= FindMinimum[SS[x,y], {{x,1},{y,-1}}, Method->{"LevenbergMarquardt", "Residual"->R[x,y]}] Out[4]= {0., {x -> 1., y -> 1.}} Note you could leave out the Sqrt[2] and minimize 2 SS[x,y] which would be slightly more efficient. There is more explanation and examples in the Advanced Documentation for unconstrained optimization. Look in the Mathematica help browser under Advanced Documentation->Optimization->Unconstrained Optimization->Methods for Local Minimization->Gauss Newton Methods Rob Knapp Wolfram Research