Re: Using LevenbergMarquardt Method with a complicated function

*To*: mathgroup at smc.vnet.net*Subject*: [mg52745] Re: Using LevenbergMarquardt Method with a complicated function*From*: rknapp at wolfram.com*Date*: Sat, 11 Dec 2004 05:22:15 -0500 (EST)*References*: <c42fu1475r5h@legacy><cpavsq$j0p$1@smc.vnet.net>*Sender*: owner-wri-mathgroup at wolfram.com

algaba wrote: > Hi. > I have defined a very long function like this: > > ChiSquare[Per0_?NumericQ, Ppa0_?NumericQ, Ecc0_?NumericQ] := > > (some steps and definitions here) > > ChiSQ = Sum[ResAlpha[[i]]^2 + ResDelta[[i]]^2, {i, 1, Length[ > TExp]}] + Sum[ResAlphaComp[[i]]^2 + > ResDeltaComp[[i]]^2, {i, 1, Length[TComp]}]); > > which tries to find the Chi-Square of an array of data. Now, I want to > minimize it and I use FindMinimum, which works well. The problem > arises when I want to use the Levenberg-Marquardt method, which seems > to be better for this kind of functions (As you can see, it is a sum > of squares) But when I run Mathematica 5 it gives me the next error > message: > > FindMinimum::notlm: The objective function for the method > LevenbergMarquardt \ > must be in a least-squares form: Sum[f[i][x]^2,{i,1,n}] or Sum[w[i] \ > f[i][x]^2,{i,1,n}] with positive w[i]. > > I think the function accomplishes all the requirements. Why I get this > error? Is it maybe because of the long definition of the function? Is > it because Mathematica doesn't see this function as a sum of squares > but as a sequence of steps? > What can I do to solve this problem? I do want to use this method to > minimize the Chi-Square. Thanks. > Because the definition is set up to only evaluate with numerical values of the arguments, Mathematica cannot do the computations necessary to decompose the sum of squares. To use the Levenberg-Marquardt method, the function needs to be decomposed into a residual function r[X] such that f[X] = r[X].r[X]/2. The LevenbergMarquardt method has a method option that allows you to specify the residual explicitly for cases like this where you may not want the sum of squares evaluated symbolically. Here is a simple example that should give you an idea of how it works: In[1]:= SS[x_?NumberQ, y_?NumberQ] := (x - 1)^2 + 100 (y - x^2)^2 In[2]:= FindMinimum[SS[x,y],{{x,1},{y,-1}}, Method->"LevenbergMarquardt"] >From In[2]:= FindMinimum::notlm: The objective function for the method LevenbergMarquardt must be in a least-squares form: Sum[f[i][x]^2,{i,1,n}] or Sum[w[i] f[i][x]^2,{i,1,n}] with positive w[i]. More... Out[2]= FindMinimum[SS[x, y], {{x, 1}, {y, -1}}, Method -> LevenbergMarquardt] In[3]:= R[x_?NumberQ, y_?NumberQ] := Sqrt[2] {x-1, 10 (y -x^2)} In[4]:= FindMinimum[SS[x,y], {{x,1},{y,-1}}, Method->{"LevenbergMarquardt", "Residual"->R[x,y]}] Out[4]= {0., {x -> 1., y -> 1.}} Note you could leave out the Sqrt[2] and minimize 2 SS[x,y] which would be slightly more efficient. There is more explanation and examples in the Advanced Documentation for unconstrained optimization. Look in the Mathematica help browser under Advanced Documentation->Optimization->Unconstrained Optimization->Methods for Local Minimization->Gauss Newton Methods Rob Knapp Wolfram Research