Re: Filtering data from numerical minimization
- To: mathgroup at smc.vnet.net
- Subject: [mg116200] Re: Filtering data from numerical minimization
- From: Ray Koopman <koopman at sfu.ca>
- Date: Fri, 4 Feb 2011 06:58:31 -0500 (EST)
- References: <iig75a$rkb$1@smc.vnet.net>
On Feb 3, 10:42 pm, Sebastian <sebho... at gmail.com> wrote:
> I have a function f(n,x,y,z) that I want to numerically minimize with
> respect to x,y,z for a list of n=1...N. The function is pretty
> complicated and also contains other parameters, which are fixed as far
> as optimization is concerned. I use NMinimize to create a table for
> different values of n. This effectively leaves me with 4 lists (f_min,
> x_min,...), where f_min = f(x_min, y_min, z_min). I then plot these
> lists against the vector n.
>
> Depending on the other parameters the minimization sometimes works
> well, and sometimes it doesn't. The problem is the following: while
> the overall form of the curves is often easily recognizable by eye,
> there is some noise on top of it, i.e. some of the points (every 5th
> say) are just way off. As I don't think that f behaves that way, but
> rather NMinimize fails to find the correct value, I'd like to filter
> that noise. Filtering the points by hand, however, doesn't seem like a
> feasible solution. Does anyone know a good way to tackle this problem?
> I already tried running MovingMedian which works reasonably well but
> I'd like to keep the rest of the data untouched if possible.
>
> I'm happy for all suggestions!
> TIA, Sebastian
Try Tukey's "4253h,twice" -- t4253h below. I've wrapped the multiline
functions in parentheses so that I wouldn't have to worry about where
the CRs come.
med42[y_] := ( (* running medians of 4, recentered by
running medians of 2; endpoints copied *)
Join[{y[[1]]},
ListCorrelate[{.5,.5}, Join[{{.5,.5}.Take[y,2]},
Median/@Partition[y,4,1],
{{.5,.5}.Take[y,-2]}]],
{y[[-1]]}] );
med5[y_] := ( (* running medians of 5, endpoints copied *)
Join[{y[[1]], Median@Take[y,3]},
Median/@Partition[y,5,1],
{Median@Take[y,-3], y[[-1]]}] );
med3e[y_] := ( (* running medians of 3, endpoints adjusted *)
Join[{Median@{{3.,-2.}.Take[#, 2], y[[ 1]], #[[ 1]]}}, #,
{Median@{{-2.,3.}.Take[#,-2], y[[-1]], #[[-1]]}}]& [
Median /@ Partition[y,3,1] ] );
hann[y_] := ( (* 1:2:1-weighted running means; endpoints copied *)
Join[{y[[1]]}, ListCorrelate[{.25,.5,.25},y], {y[[-1]]}] );
s4253h[y_] := hann @ med3e @ med5 @ med42 @ y;
t4253h[y_] := # + s4253h[y - #] & [s4253h @ y];
y = {60,70,54,56,70,66,53,95,70,69,56,70,70,60,60,
60,50,50,48,59,50,60,70,54,46,57,57,51,51,59};
t4253h[y]
{60., 60.3594, 60.5781, 60.9375, 62.2109, 65.0078,
67.8438, 69.2813, 69.8438, 70.0508, 69.5313, 67.8711,
65.5625, 63.0078, 59.9336, 55.793, 51.8281, 50.2344,
50.2031, 52.0352, 55.6055, 57.375, 57.1328, 56.4063,
55.4141, 54.5781, 54.1875, 54.2148, 54.5742, 55.2031}