       Re: Select, from table data

• To: mathgroup at smc.vnet.net
• Subject: [mg112993] Re: Select, from table data
• From: "Nasser M. Abbasi" <nma at 12000.org>
• Date: Sun, 10 Oct 2010 06:41:20 -0400 (EDT)
• References: <i8pgih\$fjn\$1@smc.vnet.net>

```On 10/9/2010 3:36 AM, Kurtis wrote:
> Easy fix here.
>
> I'm trying to drop outliers from 2D data originally collected in the
> time domain.  My method is to analyze the 2nd derivative of smoothed
> data and drop those deviating more than, say 3x the standard deviation
> from the mean. I need to make a new table of the smoothed data with
> these outliers dropped.  I'm attempting the Select function in the
> last line, i'm just not familiar enough with the syntax for analyzing
> the criteria in the 2nd column (y values) from the dataset.  Thanks!
>
> SetDirectory[NotebookDirectory[]];
> set = Import ["testMELT2.xls", {"Data", 1}];
> <<  Smooth` (* savitzsky-golay - like filter *)
>
> t = Table[set[[n]][], {n, 1, Length[set]}];

I am a little confused (not too much xls exposure), but it seems to me
that you have just imported one sheet (the first one), so set is just a
matrix. So to get the first column, why not just write

t=set[[All,1]]

?

> a260 = Table[set[[n]][], {n, 1, Length[set]}];

a260=set[[All,2]]

?

> Dataset1 = Table[{t[[n]], a260[[n]]}, {n, 1, Length[set]}];
>

Dataset1=t[[All,1;;2]]

?

(I'll leave the rest for someone else to figure out for now :)

>
> SmoothDatasetDERIV = Smooth[Dataset1, 13, 3, 2]; (* external package
> filtering over 13 points, polynomial order 3, f'')
> meanSECderiv = Mean[SmoothDatasetDERIV];
> meanY = meanSECderiv[]  (* pulling out Y values *)
> sd = StandardDeviation[SmoothDatasetDERIV];
> sdY = sd[]  (* pulling out Y values *)
>
> revision =
>   Select[{SmoothDatasetDERIV}, Function[Abs[meanY - #<= (3*sdY)&]]]
>
>
>
> Any help would be great, thanks!
>
>

--Nasser

```

• Prev by Date: Re: Further expanding d[xy]/dx in the output of a Dt calculation
• Next by Date: Re: Select, from table data
• Previous by thread: Select, from table data
• Next by thread: Re: Select, from table data