Re: Filtering data from numerical minimization
- To: mathgroup at smc.vnet.net
- Subject: [mg116200] Re: Filtering data from numerical minimization
- From: Ray Koopman <koopman at sfu.ca>
- Date: Fri, 4 Feb 2011 06:58:31 -0500 (EST)
- References: <iig75a$rkb$1@smc.vnet.net>
On Feb 3, 10:42 pm, Sebastian <sebho... at gmail.com> wrote: > I have a function f(n,x,y,z) that I want to numerically minimize with > respect to x,y,z for a list of n=1...N. The function is pretty > complicated and also contains other parameters, which are fixed as far > as optimization is concerned. I use NMinimize to create a table for > different values of n. This effectively leaves me with 4 lists (f_min, > x_min,...), where f_min = f(x_min, y_min, z_min). I then plot these > lists against the vector n. > > Depending on the other parameters the minimization sometimes works > well, and sometimes it doesn't. The problem is the following: while > the overall form of the curves is often easily recognizable by eye, > there is some noise on top of it, i.e. some of the points (every 5th > say) are just way off. As I don't think that f behaves that way, but > rather NMinimize fails to find the correct value, I'd like to filter > that noise. Filtering the points by hand, however, doesn't seem like a > feasible solution. Does anyone know a good way to tackle this problem? > I already tried running MovingMedian which works reasonably well but > I'd like to keep the rest of the data untouched if possible. > > I'm happy for all suggestions! > TIA, Sebastian Try Tukey's "4253h,twice" -- t4253h below. I've wrapped the multiline functions in parentheses so that I wouldn't have to worry about where the CRs come. med42[y_] := ( (* running medians of 4, recentered by running medians of 2; endpoints copied *) Join[{y[[1]]}, ListCorrelate[{.5,.5}, Join[{{.5,.5}.Take[y,2]}, Median/@Partition[y,4,1], {{.5,.5}.Take[y,-2]}]], {y[[-1]]}] ); med5[y_] := ( (* running medians of 5, endpoints copied *) Join[{y[[1]], Median@Take[y,3]}, Median/@Partition[y,5,1], {Median@Take[y,-3], y[[-1]]}] ); med3e[y_] := ( (* running medians of 3, endpoints adjusted *) Join[{Median@{{3.,-2.}.Take[#, 2], y[[ 1]], #[[ 1]]}}, #, {Median@{{-2.,3.}.Take[#,-2], y[[-1]], #[[-1]]}}]& [ Median /@ Partition[y,3,1] ] ); hann[y_] := ( (* 1:2:1-weighted running means; endpoints copied *) Join[{y[[1]]}, ListCorrelate[{.25,.5,.25},y], {y[[-1]]}] ); s4253h[y_] := hann @ med3e @ med5 @ med42 @ y; t4253h[y_] := # + s4253h[y - #] & [s4253h @ y]; y = {60,70,54,56,70,66,53,95,70,69,56,70,70,60,60, 60,50,50,48,59,50,60,70,54,46,57,57,51,51,59}; t4253h[y] {60., 60.3594, 60.5781, 60.9375, 62.2109, 65.0078, 67.8438, 69.2813, 69.8438, 70.0508, 69.5313, 67.8711, 65.5625, 63.0078, 59.9336, 55.793, 51.8281, 50.2344, 50.2031, 52.0352, 55.6055, 57.375, 57.1328, 56.4063, 55.4141, 54.5781, 54.1875, 54.2148, 54.5742, 55.2031}