Re: Filtering data from numerical minimization

*To*: mathgroup at smc.vnet.net*Subject*: [mg116415] Re: Filtering data from numerical minimization*From*: Ray Koopman <koopman at sfu.ca>*Date*: Mon, 14 Feb 2011 04:26:57 -0500 (EST)

On Sun, 13 Feb 2011 at 04:49:45 -0800 (PST), Sebastian <sebhofer at gmail.com> wrote: [Ray Koopman wrote:] >> ... >> It sounds like you're doing nonlinear regression and you have a fair >> number of dependent-variable outliers, on the order of 20%. The usual >> approach to such situations is to do some form of robust regression, >> that minimizes something other than the unweighted sum of squared >> residuals. There are many possibilities. Can you be more specific >> about your model? > > Thanks for your answer. I tried using the algorithm you suggested, > but a simple MovingMedian gives me "nicer" results. > I have to admit that I have no clue what you are talking about in your > second post (I'm not even sure if it really applies to my specific > problem, but I'm happy to learn about it, if it actually does!), > so I just try to clear one thing up: > In my original post I may have not been complete clear about the fact > that my function f is actually a deterministic, analytically given > function, which I try to minimize numerically. The "noise" is > introduced by the minimization which fails to find the correct optimal > value! So another way of solving my problems would be by improving the > way I do the minimization. I just thought that this may not be > possible. > Does this actually provide new information to you...? > I'm sorry, I'm a little lost here... > > Thanks for your effort! > Sebastian OK, I think I understand the situation now. You're not doing nonlinear regression, so my previous post was irrelevant and misleading. Ignore it. Try increasing WorkingPrecision, AccuracyGoal, and PrecisionGoal. Also, try a different Method. If that doesn't fix things, try using better starting intervals. This may take two passes thru the list 1...N. On the first pass, use your best a priori guess. On the second pass, take the results from n-1 and n+1 on the previous pass as the starting intervals for n. Take whichever results (pass 1 or 2) give a lower fmin. Iterate (pass 3,4,...) until it stabilizes.