Re: Regress, Eliminate Outliers, Regress

• To: mathgroup at smc.vnet.net
• Subject: [mg29279] Re: [mg29270] Regress, Eliminate Outliers, Regress
• From: BobHanlon at aol.com
• Date: Mon, 11 Jun 2001 04:38:23 -0400 (EDT)
• Sender: owner-wri-mathgroup at wolfram.com

```In a message dated 2001/6/9 3:22:35 AM, Moranresearch at aol.com writes:

>I would like to do a simple linear regression on a data set, eliminate
>data
>points that are > 2 standard deviations away from the regression line,
>then
>repeat the regression on the reduced data set. Thank you. John
>

xmax = 10;

data = Table[{x=Random[Real, {0, xmax} ], 3*x+10+3(2Random[]-1)}, {15}];

Clear[x];

f1[x_] := Evaluate[Fit[data, {1, x}, x]];

f1[x]

3.0631275218404572*x + 10.073785162728084

sd = Sqrt[Plus@@((f1[#[[1]]]-#[[2]])^2& /@ data)/Length[data]];

threshold = 1.25 sd;  (*  your threshold is 2sd  *)

filteredData = Select[data, Abs[f1[#[[1]]]-#[[2]]] <= threshold&];

f2[x_] := Evaluate[Fit[filteredData, {1, x}, x]];

f2[x]

3.0696602316273487*x + 9.752061399750337

Plot[{f2[x], f1[x], f1[x]+threshold, f1[x]-threshold}, {x, 0, xmax},
PlotStyle ->
{{AbsoluteThickness[2], AbsoluteDashing[{2,8, 5, 8}],
RGBColor[1, 0, 0]},
{AbsoluteThickness[2], AbsoluteDashing[{5,8, 2, 8}],
RGBColor[0, 0, 1]},
{AbsoluteDashing[{2,8, 2, 8}]},
{AbsoluteDashing[{2,8, 2, 8}]}},
Prolog ->
{AbsolutePointSize[5], RGBColor[1, 0, 1],
Point /@ Complement[data, filteredData],
AbsolutePointSize[3], RGBColor[0, 0, 0], Point /@ filteredData},
ImageSize -> 500];

Bob Hanlon
Chantilly, VA  USA

```

• Prev by Date: Restting Probabilities